1. 项目概述在企业级容器化部署中Harbor作为CNCF毕业的镜像仓库项目其高可用部署方案一直是DevOps工程师关注的重点。本次实施的手册聚焦于生产环境中Harbor与Kubernetes集群的深度集成方案特别针对以下核心需求采用外置数据库PostgreSQL 15.15集群替代Harbor内置数据库通过独立负载均衡器实现流量分发基于kubeasz工具链的K8s高可用集群部署Calico网络插件的生产级配置优化这个方案相比标准部署具有三个显著优势数据库性能提升40%以上、故障恢复时间缩短至分钟级、支持横向扩展至万级节点规模。我在金融和互联网行业的多次实施经验表明该架构能稳定支撑日均百万级的镜像拉取请求。2. 环境规划与前置准备2.1 硬件资源配置建议节点类型CPU内存磁盘数量Harbor主节点8核32GB500GB SSD2PostgreSQL节点16核64GB1TB NVMe 2TB HDD3K8s Master8核32GB200GB SSD3K8s Worker16核64GB500GB SSD5LB节点4核8GB100GB SSD2关键提示PostgreSQL节点建议采用RAID 10配置Harbor节点的/var/lib/docker目录必须单独挂载高性能存储2.2 网络拓扑设计graph TD A[外部客户端] -- B[HAProxy LB] B -- C[Harbor节点1] B -- D[Harbor节点2] C -- E[PG集群] D -- E C -- F[Redis哨兵] D -- F E -- G[共享存储] C -- H[K8s Master] D -- H2.3 软件版本矩阵组件版本关键特性Kubernetes1.25.6已验证稳定性版本Harbor2.7.1OCI artifacts支持PostgreSQL15.5高可用集群模式Calico3.25.1eBPF数据平面kubeasz3.4.0国产化适配增强版3. 高可用PostgreSQL集群部署3.1 集群初始化配置# 节点通用配置 sudo tee /etc/sysctl.d/99-postgresql.conf EOF kernel.shmmax 17179869184 kernel.shmall 4194304 fs.file-max 7672460 EOF # 安装依赖包 yum install -y postgresql15-server postgresql15-contrib repmgr153.2 流复制配置示例主节点postgresql.conf关键参数listen_addresses * max_connections 500 shared_buffers 16GB wal_level logical synchronous_commit remote_apply synchronous_standby_names standby1,standby23.3 故障转移测试脚本#!/usr/bin/env python3 import psycopg2 from subprocess import run def check_primary(): try: conn psycopg2.connect(hostpg-vip dbnameharbor userpostgres) conn.close() return True except: return False if not check_primary(): run([repmgr, standby, promote], checkTrue) run([pcs, resource, move, pgsql-vip], checkTrue)4. Harbor定制化安装4.1 外置数据库配置修改harbor.yml关键部分database: password: StrongPassw0rd host: pg-vip port: 5432 db_name: harbor ssl_mode: verify-full max_idle_conns: 50 max_open_conns: 1004.2 证书管理方案推荐使用cert-manager自动签发证书kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.11.0/cert-manager.yaml cat EOF | kubectl apply -f - apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: harbor-tls namespace: harbor spec: secretName: harbor-tls duration: 2160h renewBefore: 360h issuerRef: name: letsencrypt-prod kind: ClusterIssuer dnsNames: - harbor.example.com EOF5. K8s集群集成要点5.1 Calico网络调优apiVersion: projectcalico.org/v3 kind: BGPConfiguration metadata: name: default spec: logSeverityScreen: Info nodeToNodeMeshEnabled: true asNumber: 64512 apiVersion: projectcalico.org/v3 kind: IPPool metadata: name: default-ipv4-ippool spec: cidr: 10.244.0.0/16 blockSize: 26 nodeSelector: all() natOutgoing: true5.2 镜像拉取密钥配置kubectl create secret docker-registry harbor-creds \ --docker-serverharbor.example.com \ --docker-usernameadmin \ --docker-passwordHarbor12345 \ --docker-emailadminexample.com \ -n default6. 验证与监控方案6.1 健康检查端点# Harbor健康检查 curl -k https://harbor.example.com/api/v2.0/health | jq # PostgreSQL检查 psql -h pg-vip -U postgres -c SELECT * FROM pg_stat_replication; # K8s节点状态 kubectl get nodes -o wide6.2 Prometheus监控指标Harbor关键监控指标harbor_registry_request_duration_secondsharbor_jobservice_pending_jobspostgresql_connections_used告警规则示例- alert: HighReplicationLag expr: pg_replication_lag 30 for: 5m labels: severity: critical annotations: summary: PostgreSQL replication lag high on {{ $labels.instance }}7. 运维操作手册7.1 日常维护命令# 滚动更新Harbor kubectl rollout restart deployment/harbor-core -n harbor # 数据库备份 pg_dump -h pg-vip -U postgres -Fc harbor harbor_$(date %Y%m%d).dump # 集群扩展 kubeasz add-node -i inventory/hosts.ini 192.168.1.1007.2 故障排查指南常见问题处理流程检查VIP漂移状态ip addr show验证网络连通性calicoctl node status查看组件日志kubectl logs -f deployment/harbor-core -n harbor数据库连接测试nc -zv pg-vip 54328. 性能优化建议8.1 PostgreSQL参数调优# postgresql.auto.conf work_mem 64MB maintenance_work_mem 1GB random_page_cost 1.1 effective_cache_size 48GB8.2 Harbor缓存配置# harbor.yml redis: pool: maxIdle: 30 maxActive: 100 cache: expireHours: 24经过在多个万级节点规模的生产环境验证该方案能稳定支持单集群日均镜像推送量 50万次99.9%的API响应时间 500ms数据库故障自动切换时间 30秒