RAG 主链路在本地跑通只是第一步。卡住 80% AI 工程师的是怎么变成对外能用的服务——不是技术难是工程杂。三件事要做模型路由复杂推理走 GPT-4o、客服短问答走 DeepSeek、月账单直接砍半用量统计每个用户每次花了多少钱月底看清单不用猜老板要查能秒出一键部署5 个服务 4 个中间件docker compose up -d5 分钟拉起一、先看我们要补哪两块回顾 Day87 的架构图红色部分是这一篇要补的┌─ rag-service ──────────────┐ │ Day88 已实现 │ gateway-ai ──→│ DocumentChunkService │ 本篇实现 │ RagSearchService │ │ RagChatController (SSE) │ └─────────────────────────────┘ │ └→ admin-service 本篇实现用量落库 仪表盘 ​ 其它服务document-service / conversation-svc / bff维持原状gateway-ai是 Day49 微服务化那篇讲的模型网关的具体落地admin-service是 Day79 用量统计的具体实现。两块独立部署互不影响主链路。二、多模型动态路由业务侧无感账单会说话2.1 为什么必须做路由我见过一个最离谱的账单某团队全员用 GPT-4o 写 SQL 解释月花费 12 万。后来把 70% 的请求切到 DeepSeek账单降到 1.8 万——模型选择是 AI 系统里 ROI 最高的优化手段。路由要解决三件事维度怎么决定例子场景简单客服/翻译/总结 vs 复杂推理客服走 mini代码审查走 GPT-4o成本单次调用预算0.001 美元以下用 DeepSeek可用性主模型挂了切备用DeepSeek 超时切通义千问2.2 模型路由实现// ai-platform-gateway-ai 模块 Component public class ModelRouter { ​ /** 模型客户端池key 是模型标识 */ private final MapString, ChatModel models new ConcurrentHashMap(); /** Micrometer 指标每选一次模型都打点Grafana 能看分布 */ private final MeterRegistry metrics; /** 各模型的实时熔断状态 */ private final CircuitBreakerRegistry breakers; ​ // 路由决策权重可放进 Nacos 配置动态调整 Value(${router.cost-weight:0.4}) private double costWeight; Value(${router.latency-weight:0.3}) private double latencyWeight; Value(${router.quality-weight:0.3}) private double qualityWeight; ​ public ModelRouter(MapString, ChatModel modelBeans, MeterRegistry metrics, CircuitBreakerRegistry breakers) { modelBeans.forEach((k, v) - models.put(k, v)); this.metrics metrics; this.breakers breakers; } ​ /** * 根据 hint 选一个模型。 * 这是 AI 系统里被调用最频繁的入口必须做到 0 阻塞、0 远程调用。 */ public ChatModel route(RouteHint hint) { String chosen decide(hint); metrics.counter(llm.route.chosen, model, chosen, scenario, hint.scenario()).increment(); return models.get(chosen); } ​ private String decide(RouteHint h) { // ① 熔断的模型直接排除 SetString available models.keySet().stream() .filter(name - breakers.circuitBreaker(name).getState() ! CircuitBreaker.State.OPEN) .collect(Collectors.toSet()); ​ // ② 按场景分流绝大多数请求走这里 String byScenario switch (h.scenario()) { case code-review, deep-reason, complex-rag - available.contains(gpt-4o) ? gpt-4o : qwen-long; case translation - available.contains(claude-sonnet) ? claude-sonnet : gpt-4o-mini; case summarize - h.inputLength() 500 ? gpt-4o-mini : claude-sonnet; case customer-service, general - h.budgetPerCall() 0.001 ? deepseek-chat : gpt-4o-mini; default - gpt-4o-mini; }; ​ if (available.contains(byScenario)) return byScenario; ​ // ③ 兜底场景命中但该模型熔断取第一个可用 return available.stream().findFirst().orElseThrow( () - new IllegalStateException(所有模型都不可用请检查 LLM 服务)); } ​ /** 用于 A/B 测试同一用户始终走同一模型避免体验抖动 */ public ChatModel routeAB(RouteHint hint, String experimentName) { if (experimentName null) return route(hint); int hash Math.abs(Objects.hash(hint.userId(), experimentName)); boolean toTreatment (hash 0x7F) 0x40; // 50% 流量 String model toTreatment ? deepseek-chat : gpt-4o-mini; metrics.counter(llm.ab.test, experiment, experimentName, arm, toTreatment ? treatment : control).increment(); return models.get(model); } }几个关键细节路由决策是本地计算不调远程纳秒级返回。不要去数据库查权重全放进内存或本地配置。熔断器用 Resilience4jOpen 状态持续 30 秒后自动切到 Half-Open 试探。DeepSeek 抽风时它能把流量秒切到备用。routeAB用userId哈希同一用户在同一实验里始终走同一模型否则他一会儿收到 DeepSeek 的答案一会儿收到 GPT 的体感混乱。2.3 模型客户端配置// ai-platform-gateway-ai/config/ModelConfig.java Configuration public class ModelConfig { ​ Bean(deepseek-chat) public ChatModel deepSeekChat(Value(${llm.deepseek.api-key}) String key) { return new DeepSeekChatModel( https://api.deepseek.com, key, deepseek-chat); } ​ Bean(gpt-4o-mini) public ChatModel gpt4oMini(Value(${llm.openai.api-key}) String key) { return new OpenAiChatModel( https://api.openai.com, key, gpt-4o-mini); } ​ Bean(gpt-4o) public ChatModel gpt4o(Value(${llm.openai.api-key}) String key) { return new OpenAiChatModel( https://api.openai.com, key, gpt-4o); } ​ Bean(qwen-long) public ChatModel qwenLong(Value(${llm.qwen.api-key}) String key) { return new QwenChatModel( https://dashscope.aliyuncs.com, key, qwen-long); } ​ /** 每个模型一个熔断器失败率 50% 持续 10 秒就熔断 */ Bean public CircuitBreakerRegistry circuitBreakerRegistry() { CircuitBreakerConfig cfg CircuitBreakerConfig.custom() .failureRateThreshold(50) .waitDurationInOpenState(Duration.ofSeconds(30)) .slidingWindowSize(20) .build(); return CircuitBreakerRegistry.of(cfg); } } application.yml 关键段 llm: deepseek: api-key: ${DEEPSEEK_API_KEY} base-url: https://api.deepseek.com openai: api-key: ${OPENAI_API_KEY} base-url: https://api.openai.com qwen: api-key: ${QWEN_API_KEY} base-url: https://dashscope.aliyuncs.com ​ router: cost-weight: 0.4 latency-weight: 0.3 quality-weight: 0.32.4 在 RAG 服务里使用路由Day88 的 RagChatController 改成注入 ModelRouter而不是写死 deepseek-chat RestController RequestMapping(/api/v1/rag) public class RagChatController { ​ Autowired private RagSearchService searchService; Autowired private ModelRouter modelRouter; Autowired private ConversationRepository conversationRepository; ​ GetMapping(value /ask, produces MediaType.TEXT_EVENT_STREAM_VALUE) public FluxServerSentEventString ask( RequestParam String question, RequestParam(required false) Long conversationId, RequestHeader(X-User-Id) Long userId) { ​ // 根据场景选择模型复杂 RAG 用 GPT-4o简单客服用 mini RouteHint hint new RouteHint( userId, complex-rag, question.length(), 0.005 // 单次预算 5 厘 ); ChatModel chosenModel modelRouter.route(hint); ​ // ... 后续 Day88 的检索 流式输出代码不变 // chatClient 改成用 chosenModel.stream(...) } }2.5 路由效果怎么验证光看代码不知道有没有用。两条线验证离线评测用 50 个真实业务问题让 4 个模型各跑一遍统计准确率/成本/延迟画成雷达图。每月回归一次新模型上线前必须有这张图。在线 A/B上线新模型走routeAB灰度 10% → 30% → 50% → 100%每个阶段至少跑 3 天对比业务指标追问率、人工接管率。别相信新模型肯定更好的嘴炮用数据说话。三、用量统计每分钱都有出处3.1 落库时机关键认知用量落库要在LLM 响应返回前同步落否则月底补账永远算不准。落库逻辑放在ModelRouter.route()返回的模型外面包一层Component RequiredArgsConstructor public class BillingChatModel implements ChatModel { ​ private final ChatModel delegate; private final UsageRecorder usageRecorder; private final MeterRegistry metrics; ​ Override public ChatResponse call(Prompt prompt) { long start System.currentTimeMillis(); try { ChatResponse resp delegate.call(prompt); // ① 成功记录用量 成本 recordUsage(prompt, resp, start, null); return resp; } catch (Throwable t) { // ② 失败记录失败指标prompt 侧 token 也得算按最大预估 recordUsage(prompt, null, start, t.getClass().getSimpleName()); throw t; } } ​ private void recordUsage(Prompt prompt, ChatResponse resp, long start, String errType) { // 这里的 token 计数OpenAI/DeepSeek 协议都会在 response.usage 里返回 Usage usage resp ! null ? resp.getMetadata().getUsage() : null; int promptTokens usage ! null ? usage.getPromptTokens() : estimateTokens(prompt); int completionTokens usage ! null ? usage.getCompletionTokens() : 0; String modelName delegate.getClass().getSimpleName(); long costCents computeCostCents(modelName, promptTokens, completionTokens); ​ // ① 异步落库不阻塞主链路 usageRecorder.recordAsync(UsageRecord.builder() .userId(currentUserId()) // 从 ThreadLocal / Context 拿 .model(modelName) .promptTokens(promptTokens) .completionTokens(completionTokens) .costCents(costCents) .latencyMs(System.currentTimeMillis() - start) .errType(errType) .createdAt(Instant.now()) .build()); ​ // ② Prometheus 指标 metrics.counter(llm.tokens.total, type, prompt, model, modelName).increment(promptTokens); metrics.counter(llm.tokens.total, type, completion, model, modelName).increment(completionTokens); metrics.counter(llm.cost.cents.total, model, modelName).increment(costCents); if (errType ! null) { metrics.counter(llm.call.error, model, modelName, type, errType).increment(); } } ​ /** 成本计算单位分避免浮点 */ private long computeCostCents(String model, int promptTok, int completionTok) { // 价格表2026 年中参考价按 ¥7/$1 折算成人民币分 return switch (model) { case deepseek-chat - (long) ((promptTok * 0.14 completionTok * 0.28) / 7); case gpt-4o-mini - (long) ((promptTok * 1.5 completionTok * 6.0) / 7); case gpt-4o - (long) ((promptTok * 75 completionTok * 225) / 7); case qwen-long - (long) ((promptTok * 0.5 completionTok * 2.0) / 7); default - 0L; }; } }3.2 异步落库与聚合Service RequiredArgsConstructor public class UsageRecorder { ​ private final KafkaTemplateString, UsageRecord kafka; /** 实时累计用于实时仪表盘小时粒度 */ private final RedisTemplateString, AtomicLong redis; /** 日聚合用于月报表 */ private final UsageDailyRepository dailyRepo; ​ /** * 异步落库主链路不等待。 * 失败时降级到本地日志运维定期捞回。 */ Async(usageExecutor) public void recordAsync(UsageRecord r) { try { kafka.send(llm-usage, String.valueOf(r.userId()), r); } catch (Exception e) { // Kafka 不可用降级到 Redis Stream redis.opsForStream().add(usage-fallback, toMap(r)); log.warn(usage kafka send failed, fallback to redis: {}, e.getMessage()); } } } ​ /** 消费端日聚合落库 */ Component RequiredArgsConstructor public class UsageDailyConsumer { ​ private final UsageDailyRepository repo; ​ KafkaListener(topics llm-usage, groupId usage-aggregator) public void consume(UsageRecord r) { // 同一用户同一模型同一天的记录累加 UsageDailyKey key new UsageDailyKey(r.userId(), r.model(), LocalDate.ofInstant(r.createdAt(), ZoneId.systemDefault())); repo.upsert(key, r.promptTokens(), r.completionTokens(), r.costCents(), 1); } }3.3 仪表盘接口与前端管理后台需要一个/api/v1/admin/usage/dashboard接口按天/按模型聚合GetMapping(/api/v1/admin/usage/dashboard) public UsageDashboard dashboard( RequestParam DateTimeFormat(iso DATE) LocalDate from, RequestParam DateTimeFormat(iso DATE) LocalDate to, RequestParam(required false) Long userId) { ​ return UsageDashboard.builder() .summary(dailyRepo.summary(from, to, userId)) // 总调用/总 token/总成本 .byModel(dailyRepo.aggregateByModel(from, to, userId)) .byDay(dailyRepo.dailySeries(from, to, userId)) .topUsers(dailyRepo.topUsersByCost(from, to, 10)) .build(); }返回示例前端 Echarts 直接画{ summary: { totalCalls: 12450, totalPromptTokens: 28900000, totalCompletionTokens: 5320000, totalCostCents: 186420 }, byModel: [ { model: deepseek-chat, calls: 8720, costCents: 42180 }, { model: gpt-4o-mini, calls: 3100, costCents: 86400 }, { model: gpt-4o, calls: 480, costCents: 51000 }, { model: qwen-long, calls: 150, costCents: 6840 } ], byDay: [ { date: 2026-08-26, costCents: 24580, calls: 1620 }, { date: 2026-08-27, costCents: 28140, calls: 1850 }, ... ] }前端 Vue3 Element Plus 画饼图 折线图参考 Day59-Day60 那套组件库。关键的运营指标成本 Top 10 用户内部知识库看到这列表就要找用户聊是不是有人刷接口单次调用成本分布发现异常贵请求Prompt 写太长循环调用按模型成本占比判断路由策略是否生效3.4 用户自查询接口每个用户能查自己的用量减少客诉GetMapping(/api/v1/me/usage) public UsageReport myUsage(AuthenticationPrincipal UserPrincipal user, RequestParam(defaultValue 30) int days) { LocalDate since LocalDate.now().minusDays(days); return UsageReport.builder() .totalCalls(usageRepo.countCalls(user.id(), since)) .totalCostCents(usageRepo.sumCost(user.id(), since)) .byModel(usageRepo.aggregateByModel(user.id(), since)) .dailySeries(usageRepo.dailySeries(user.id(), since)) .build(); }B 端用户嵌入企业控制台C 端用户放进账单明细。让用户能自己查比每天被问这个月怎么这么贵省心 10 倍。四、Docker Compose 一键拉起4.1 服务清单回顾 Day87 的架构我们要拉起 5 个 Spring Boot 服务 4 个中间件角色服务镜像/构建数据库PostgreSQL 16 pgvectorpgvector/pgvector:pg16缓存Redis 7redis:7-alpine消息队列RabbitMQ 3rabbitmq:3-management对象存储MinIOminio/minio检索兜底Elasticsearch 8elasticsearch:8.11.0应用document-service本地多阶段构建应用rag-service本地多阶段构建应用conversation-service本地多阶段构建应用gateway-ai本地多阶段构建应用admin-service本地多阶段构建反代Nginxnginx:alpine4.2 生产级 Compose 文件# docker/docker-compose.prod.yml version: 3.9 ​ services: # 基础设施 postgres: image: pgvector/pgvector:pg16 restart: always environment: POSTGRES_DB: kbplatform POSTGRES_USER: kb POSTGRES_PASSWORD_FILE: /run/secrets/pg_pwd secrets: [pg_pwd] volumes: - pgdata:/var/lib/postgresql/data - ./init.sql:/docker-entrypoint-initdb.d/init.sql:ro healthcheck: test: [CMD-SHELL, pg_isready -U kb] interval: 10s timeout: 5s retries: 5 deploy: resources: { limits: { cpus: 2, memory: 4G } } ​ redis: image: redis:7-alpine restart: always command: redis-server --appendonly yes --requirepass ${REDIS_PWD} volumes: - redisdata:/data healthcheck: test: [CMD, redis-cli, -a, ${REDIS_PWD}, ping] interval: 10s deploy: resources: { limits: { cpus: 1, memory: 2G } } ​ rabbitmq: image: rabbitmq:3-management restart: always environment: RABBITMQ_DEFAULT_USER: kb RABBITMQ_DEFAULT_PASS_FILE: /run/secrets/rabbit_pwd secrets: [rabbit_pwd] ports: [15672:15672] # 控制台生产可不开 deploy: resources: { limits: { cpus: 1, memory: 1G } } ​ minio: image: minio/minio restart: always command: server /data --console-address :9001 environment: MINIO_ROOT_USER_FILE: /run/secrets/minio_user MINIO_ROOT_PASSWORD_FILE: /run/secrets/minio_pwd secrets: [minio_user, minio_pwd] volumes: - miniodata:/data ports: [9001:9001] ​ elasticsearch: image: docker.elastic.co/elasticsearch/elasticsearch:8.11.0 restart: always environment: discovery.type: single-node xpack.security.enabled: false ES_JAVA_OPTS: -Xms2g -Xmx2g volumes: - esdata:/usr/share/elasticsearch/data deploy: resources: { limits: { cpus: 2, memory: 4G } } ​ # 应用服务 document-service: build: context: ../ai-platform-document dockerfile: Dockerfile restart: always depends_on: postgres: { condition: service_healthy } rabbitmq: { condition: service_started } minio: { condition: service_started } environment: SPRING_PROFILES_ACTIVE: prod DB_PASSWORD_FILE: /run/secrets/pg_pwd RABBIT_PASSWORD_FILE: /run/secrets/rabbit_pwd MINIO_PASSWORD_FILE: /run/secrets/minio_pwd secrets: [pg_pwd, rabbit_pwd, minio_pwd] deploy: resources: { limits: { cpus: 2, memory: 3G } } ​ rag-service: build: { context: ../ai-platform-rag, dockerfile: Dockerfile } restart: always depends_on: postgres: { condition: service_healthy } redis: { condition: service_healthy } elasticsearch: { condition: service_started } gateway-ai: { condition: service_started } environment: SPRING_PROFILES_ACTIVE: prod DB_PASSWORD_FILE: /run/secrets/pg_pwd REDIS_PASSWORD: ${REDIS_PWD} DEEPSEEK_API_KEY_FILE: /run/secrets/deepseek_key OPENAI_API_KEY_FILE: /run/secrets/openai_key QWEN_API_KEY_FILE: /run/secrets/qwen_key secrets: [pg_pwd, deepseek_key, openai_key, qwen_key] deploy: resources: { limits: { cpus: 2, memory: 4G } } ​ conversation-service: build: { context: ../ai-platform-conversation, dockerfile: Dockerfile } restart: always depends_on: postgres: { condition: service_healthy } redis: { condition: service_healthy } environment: SPRING_PROFILES_ACTIVE: prod DB_PASSWORD_FILE: /run/secrets/pg_pwd REDIS_PASSWORD: ${REDIS_PWD} secrets: [pg_pwd] deploy: resources: { limits: { cpus: 1, memory: 2G } } ​ gateway-ai: build: { context: ../ai-platform-gateway-ai, dockerfile: Dockerfile } restart: always depends_on: [postgres, redis] environment: SPRING_PROFILES_ACTIVE: prod DB_PASSWORD_FILE: /run/secrets/pg_pwd REDIS_PASSWORD: ${REDIS_PWD} DEEPSEEK_API_KEY_FILE: /run/secrets/deepseek_key OPENAI_API_KEY_FILE: /run/secrets/openai_key QWEN_API_KEY_FILE: /run/secrets/qwen_key secrets: [pg_pwd, deepseek_key, openai_key, qwen_key] deploy: resources: { limits: { cpus: 2, memory: 4G } } ​ admin-service: build: { context: ../ai-platform-admin, dockerfile: Dockerfile } restart: always depends_on: postgres: { condition: service_healthy } rabbitmq: { condition: service_started } environment: SPRING_PROFILES_ACTIVE: prod DB_PASSWORD_FILE: /run/secrets/pg_pwd RABBIT_PASSWORD_FILE: /run/secrets/rabbit_pwd secrets: [pg_pwd, rabbit_pwd] deploy: resources: { limits: { cpus: 1, memory: 2G } } ​ # 反向代理 nginx: image: nginx:alpine restart: always depends_on: - rag-service - admin-service volumes: - ./nginx.conf:/etc/nginx/nginx.conf:ro - ./certs:/etc/nginx/certs:ro ports: [80:80, 443:443] deploy: resources: { limits: { cpus: 0.5, memory: 256M } } ​ volumes: pgdata: {} redisdata: {} miniodata: {} esdata: {} ​ secrets: pg_pwd: { file: ./secrets/pg_pwd.txt } redis_pwd: { file: ./secrets/redis_pwd.txt } rabbit_pwd: { file: ./secrets/rabbit_pwd.txt } minio_user: { file: ./secrets/minio_user.txt } minio_pwd: { file: ./secrets/minio_pwd.txt } deepseek_key: { file: ./secrets/deepseek_key.txt } openai_key: { file: ./secrets/openai_key.txt } qwen_key: { file: ./secrets/qwen_key.txt }4.3 多阶段构建镜像体积砍 70%应用服务的Dockerfile以rag-service为例# 阶段一构建 FROM maven:3.9-eclipse-temurin-21 AS builder WORKDIR /build COPY pom.xml . RUN mvn -B -e dependency:go-offline COPY src ./src RUN mvn -B -e clean package -DskipTests ​ # 阶段二运行时 FROM eclipse-temurin:21-jre-alpine WORKDIR /app COPY --frombuilder /build/target/ai-platform-rag-*.jar app.jar # 非 root 运行 RUN addgroup -S app adduser -S app -G app USER app EXPOSE 8080 # 镜像启动时优雅停机 ENTRYPOINT [java, -XX:UseG1GC, \ -XX:MaxRAMPercentage75.0, \ -jar, /app/app.jar]为什么不用 Spring Boot 的 fat jar launcher虚拟线程JDK 21下手动指定 GC 参数更可控。4.4 Nginx 配置关键段# docker/nginx.conf upstream rag_upstream { server rag-service:8080; keepalive 32; } upstream admin_upstream { server admin-service:8080; keepalive 16; } ​ server { listen 443 ssl; server_name kb.example.com; ssl_certificate /etc/nginx/certs/fullchain.pem; ssl_certificate_key /etc/nginx/certs/privkey.pem; ​ # SSE 关键配置禁用缓冲、禁用超时 location /api/v1/rag/ask { proxy_pass http://rag_upstream; proxy_http_version 1.1; proxy_set_header Connection ; proxy_buffering off; # 关闭缓冲SSE 必需 proxy_cache off; proxy_read_timeout 300s; # 5 分钟硬上限 proxy_set_header X-Accel-Buffering no; } ​ location /api/v1/admin/ { proxy_pass http://admin_upstream; proxy_read_timeout 60s; } ​ location /api/v1/documents/ { proxy_pass http://rag_upstream; # 文档管理走 rag-service client_max_body_size 100m; # 大文件上传 } }SSE 三个魔鬼参数缺一不可proxy_buffering off—— 否则 Nginx 会把响应缓存完才转发前端看不到流式proxy_read_timeout 300s—— 默认 60 秒会断X-Accel-Buffering: no—— 兼容某些 Nginx 版本4.5 一键部署脚本#!/usr/bin/env bash # docker/deploy.sh set -euo pipefail ​ SERVER${1:?用法: ./deploy.sh userserver-ip} REMOTE_DIR/opt/knowledge-platform IMAGE_TAG${IMAGE_TAG:-$(git rev-parse --short HEAD)} ​ echo [1/5] 构建应用镜像 cd .. docker build -t ghcr.io/example/ai-platform-rag:${IMAGE_TAG} ./ai-platform-rag docker build -t ghcr.io/example/ai-platform-gateway-ai:${IMAGE_TAG} ./ai-platform-gateway-ai docker build -t ghcr.io/example/ai-platform-admin:${IMAGE_TAG} ./ai-platform-admin # ... 其它服务同理 ​ echo [2/5] 推送镜像到镜像仓库 docker push ghcr.io/example/ai-platform-rag:${IMAGE_TAG} # ... 其它服务 ​ echo [3/5] 同步配置到服务器 rsync -az --delete \ --exclude secrets/*.txt \ ./docker/ $SERVER:$REMOTE_DIR/docker/ ​ echo [4/5] 远端拉起服务 ssh $SERVER EOF cd $REMOTE_DIR/docker ​ # 注入密钥运维一次性执行 [ -f secrets/deepseek_key.txt ] || echo \$DEEPSEEK_API_KEY secrets/deepseek_key.txt chmod 600 secrets/*.txt ​ # 拉镜像 启动 docker compose -f docker-compose.prod.yml pull docker compose -f docker-compose.prod.yml up -d ​ # 等待健康检查 sleep 30 docker compose -f docker-compose.prod.yml ps EOF ​ echo [5/5] 验证 HEALTH$(curl -sf https://kb.example.com/actuator/health || echo FAIL) if [[ $HEALTH *UP* ]]; then echo ✅ 上线成功: $HEALTH else echo ❌ 健康检查失败请查日志: ssh $SERVER docker compose logs --tail200 exit 1 fi执行chmod x docker/deploy.sh ./docker/deploy.sh deploy10.0.0.55 分钟后https://kb.example.com/api/v1/rag/ask?questionhi就能用了。五、部署后的运维清单代码上线不是结束运维 24 小时才能算稳定。我给你三件必做5.1 监控告警Grafana 必备 6 个面板PromQL 一并给你面板PromQLLLM 调用 QPSsum(rate(llm_tokens_total[5m])) by (model)各模型成本占比sum by (model) (rate(llm_cost_cents_total[1h]))P99 响应延迟histogram_quantile(0.99, sum(rate(rag_latency_ms_bucket[5m])) by (le, model))错误率rate(llm_call_error_total[5m]) / rate(llm_call_total[5m])缓存命中率rate(semantic_cache_hit_total[5m]) / (rate(semantic_cache_hit_total[5m]) rate(semantic_cache_miss_total[5m]))单用户成本 Top10topk(10, sum by (user_id) (rate(llm_cost_cents_total[24h])))告警规则建议直接进 Alertmanagergroups: - name: ai-platform rules: - alert: HighErrorRate expr: rate(llm_call_error_total[5m]) / rate(llm_call_total[5m]) 0.05 for: 2m labels: { severity: P1 } annotations: { summary: LLM 错误率超过 5% } - alert: HighCost expr: increase(llm_cost_cents_total[1h]) 50000 labels: { severity: P2 } annotations: { summary: LLM 单小时花费超 ¥500 } - alert: SlowRag expr: histogram_quantile(0.99, rate(rag_latency_ms_bucket[5m])) 5000 for: 5m labels: { severity: P2 }5.2 数据备份# crontab 每天凌晨 3 点执行 0 3 * * * /opt/knowledge-platform/scripts/backup.sh ​ # scripts/backup.sh #!/usr/bin/env bash set -e BACKUP_DIR/data/backup/postgres/$(date %Y%m%d) mkdir -p $BACKUP_DIR docker exec postgres pg_dump -U kb kbplatform | gzip $BACKUP_DIR/db.sql.gz # 同步到 OSS ossutil cp $BACKUP_DIR/db.sql.gz oss://kb-backup/postgres/ -r # 保留 30 天 find /data/backup/postgres -mtime 30 -delete5.3 灰度发布新版本不要一把全量# 步骤 1单实例验证 docker compose -f docker-compose.prod.yml up -d --scale rag-service2 ​ # 步骤 2Nginx 切 10% 流量到新实例 # 通过 weight 控制 # upstream rag_upstream { # server rag-service-v1:8080 weight9; # server rag-service-v2:8080 weight1; # } ​ # 步骤 3观察 1 小时无异常逐步提比例 # 10% → 30% → 50% → 100% ​ # 步骤 4缩掉旧实例 docker compose -f docker-compose.prod.yml up -d --scale rag-service1 \ --no-deps rag-service-v1六、给团队的三条建议路由策略先 if-else三个月内别碰 ML。4 条 if-else 解决 90% 的路由决策。引入学习排序看似高级实际是给自己挖坑——冷启动数据不够、调参浪费时间、ROI 极低。等日调用量过 10 万、规则扛不住时再考虑。用量落库要全链路不能漏。任何一次 LLM 调用——包括 ReAct Agent 的 tool 调用、Embedding 计算、流式输出——都要落账。漏一个口子月底账单就对不上然后老板问你是不是有人在薅羊毛你只能沉默。BillingChatModel这种包装模式的好处是强制全量打点业务侧拿不到裸ChatModel。Docker Compose 是 80% 项目的最优解别急着 K8s。我见过太多团队Compose 没跑通就上 K8s结果上线 3 个月etcd 崩过 2 次、Operator 版本不兼容、运维工程师 24 小时待命。Compose 跑稳了单机扛不住了再上 K8s 不迟。Day58 Day63 那套 K8s 配置留着等量到了再翻出来。90 天专栏走到 Day89Day87 的架构图、Day88 的 RAG 代码、今天这篇的路由 统计 部署——一套完整可上线的 AI 知识库平台就齐了。代码不是写完就完能跑、能算账、能扛流量、能恢复——这才是工程。从代码到生产不是部署而是治理。Docker Compose 只是房子监控告警 / 备份 / 灰度 / 应急才是物业——我见过最多代码能跑但上不了线的项目都死在把部署当一次性任务。下篇预告Day90Java 后端工程师的 AI 时代竞争力地图。90 天系列收官。不写代码、不画架构聊三件事你下一步应该学什么、面试该怎么准备 AI 题、3 年后能拿什么 offer。一份写给听完了所有 89 篇的你的一份成长指南。