Loki Operator 对象存储短时令牌认证STS / Workload Identity实战指南【免费下载链接】lokiLike Prometheus, but for logs.项目地址: https://gitcode.com/GitHub_Trending/lok/loki导读LokiStack 默认通过静态云服务账号Client ID/Secret 或 Access Key访问对象存储但静态凭证存在轮换繁琐、泄露影响面大等安全短板。本文以 short_lived_tokens_authentication.md 增强提案为骨架系统讲解 Loki Operator 如何在 AWS、GCP、Azure 三大公有云上启用短时令牌认证STS / Workload Identity / Workload Identity Federation并深入源码揭示凭证模式判定、ServiceAccount Token 注入与 Secret 校验的底层实现。读完你将掌握CredentialMode三种取值的使用与自动检测规则、三大云厂商最小化存储 Secret 的字段构成以及从 IAM 资源创建到 LokiStack 声明式配置的完整落地步骤。术语说明本文使用 short-lived token authentication 作为统一术语对应三大云厂商的 IAM 能力AWS STSSecurity Token Service、Azure Workload Identity Federation、GCP Workload Identity Federation。一、背景与动机为什么要用短时令牌替代静态凭证LokiStack 的对象存储访问在增强提案之前仅支持静态云服务账号凭证。静态账号虽然创建简单、便于自动化管理但管理员必须人工处理一系列安全运维问题密钥轮换、账号到期后的重建、泄露后的影响面控制等。短时令牌认证的引入带来了三项核心能力自动化凭证签发与轮换每个 Kubernetes 集群使用一个 OIDC 授权服务器为 ServiceAccount 签发凭证按工作负载自动创建并轮换最小权限绑定托管集群上的每个工作负载必须绑定到特定 IAM 角色才能访问云厂商服务S3/GCS/Azure Storage短时令牌强制刷新OIDC 工作流只向每个工作负载签发短期有效令牌从而高频轮换凭证。该机制的落点在于每个 Kubernetes 工作负载只能通过自身的 ServiceAccount 向 IAM 受控资源发起访问请求每次合法请求都由已轮换的短时令牌加固。这样既保留了静态账号通过 IAM 统一管控访问的能力例如安全事件时可一键禁用又通过自动轮换把安全事件的影响面降到最低。目标与边界GoalsLokiStack 管理员只需提供最小且有效的对象存储 Secret即可在三大云厂商启用短时令牌认证管理员无需在集群内暴露凭证/身份令牌等敏感信息。Non-Goals不提供辅助性的、面向云厂商 IAM 资源预置的完整自动化工具链IAM 资源需管理员手工创建本文第三节给出具体命令。二、API 扩展CredentialMode字段与自动检测2.1 新的可选字段提案在ObjectStorageSecretSpec中引入可选字段CredentialMode。如果用户未显式提供Operator 会根据 Secret 字段或自身环境变量自动检测后者仅适用于 OpenShift 集群并将检测结果同步写入status.storage.credentialMode字段。从 lokistack_types.go 可以看到该字段的 API 定义// ObjectStorageSecretSpec is a secret reference containing name only, no namespace. type ObjectStorageSecretSpec struct { // Type of object storage that should be used Type ObjectStorageSecretType json:type // Name of a secret in the namespace configured for object storage secrets. Name string json:name // CredentialMode can be used to set the desired credential mode for authenticating with the object storage. // If this is not set, then the operator tries to infer the credential mode from the provided secret and its // own configuration. // // optional // kubebuilder:validation:Optional CredentialMode CredentialMode json:credentialMode,omitempty }2.2 三种凭证模式的语义CredentialMode类型通过kubebuilder枚举约束限定为static;token;token-cco三个取值其定义与语义见 lokistack_types.go 中CredentialMode类型声明// CredentialMode represents the type of authentication used for accessing the object storage. // // kubebuilder:validation:Enumstatic;token;token-cco type CredentialMode string const ( // CredentialModeStatic represents the usage of static, long-lived credentials stored in a Secret. // This is the default authentication mode and available for all supported object storage types. CredentialModeStatic CredentialMode static // CredentialModeToken represents the usage of short-lived tokens retrieved from a credential source. // In this mode the static configuration does not contain credentials needed for the object storage. // Instead, they are generated during runtime using a service, which allows for shorter-lived credentials and // much more granular control. This authentication mode is not supported for all object storage types. CredentialModeToken CredentialMode token // CredentialModeTokenCCO represents the usage of short-lived tokens retrieved from a credential source. // This mode is similar to CredentialModeToken, but instead of having a user-configured credential source, // it is configured by the environment and the operator relies on the Cloud Credential Operator to provide // a secret. This mode is only supported for certain object storage types in certain runtime environments. CredentialModeTokenCCO CredentialMode token-cco )三个取值的适用场景模式凭证来源适用场景staticSecret 中存放的长期静态凭证所有对象存储类型默认模式兼容现状token运行时通过凭证服务签发的短时令牌仅支持部分对象存储类型Azure/AWS/GCStoken-cco由环境配置依赖 OpenShift Cloud Credential OperatorCCO注入 Secret仅限 AWS-STS / Azure-WIF 托管的 OpenShift 集群等特定环境CredentialMode的核心作用是覆盖从对象存储 Secret 或 Operator 环境变量检测出的凭证类型。一个典型场景在 AWS-STS/Azure-WIF 托管的 OpenShift 集群上Operator 默认使用token-cco但如果用户想在同一集群把日志写入 Minio而非 AWS S3则可以显式指定static模式覆盖默认行为。2.3 源码视角凭证模式的自动检测与状态回写检测逻辑secrets.go 中的determineCredentialMode实现了完整的判定流程func determineCredentialMode(spec lokiv1.ObjectStorageSecretSpec, secret *corev1.Secret, fg configv1.FeatureGates) (lokiv1.CredentialMode, error) { if spec.CredentialMode ! { // Return user-defined credential mode if defined return spec.CredentialMode, nil } if fg.OpenShift.TokenCCOAuthEnv { // Default to token cco credential mode on a token-cco-auth installation return lokiv1.CredentialModeTokenCCO, nil } switch spec.Type { case lokiv1.ObjectStorageSecretAzure: if keyPresent(secret, storage.KeyAzureStorageClientID) { return lokiv1.CredentialModeToken, nil } case lokiv1.ObjectStorageSecretGCS: _, credentialType, err : extractGoogleCredentialSource(secret) if err ! nil { return , err } if credentialType gcpAccountTypeExternal { return lokiv1.CredentialModeToken, nil } case lokiv1.ObjectStorageSecretS3: if keyPresent(secret, storage.KeyAWSRoleArn) { return lokiv1.CredentialModeToken, nil } case lokiv1.ObjectStorageSecretSwift: // does only support static mode case lokiv1.ObjectStorageSecretAlibabaCloud: // does only support static mode default: return , fmt.Errorf(%w: %s, errSecretUnknownType, spec.Type) } return lokiv1.CredentialModeStatic, nil }可见检测优先级为用户显式指定 OpenShift token-cco 环境变量 Secret 字段指纹 默认 static。其中 GCS 模式的关键判据是key.json中type字段是否为external_accountextractGoogleCredentialSource会解析 JSON 中的credential_source.file与type见 secrets.go。Swift 与 AlibabaCloud 仅支持 static 模式。状态回写检测结果通过 status.go 写入stack.Status.Storage.CredentialModeStorage字段类型为lokiv1.CredentialMode见 lokistack.go用户可通过kubectl get lokistack -o yaml查看实际生效的凭证模式。2.4 CRD 的生成与展示上述 API 定义通过 Kubebuilder 标记生成为 CRD 清单见 operator/config/crd/bases/loki.grafana.com_lokistacks.yaml并在 OpenShift OLM Bundle 中同步发布如 operator/bundle/openshift/manifests/loki.grafana.com_lokistacks.yaml。Operator API 文档 operator/docs/operator/api.md 中也有对应说明。三、三大云厂商的 Secret 格式与 IAM 前置准备3.1 Azure Workload Identity Federation静态凭证 Secret现状data: environment: # The Azure Storage account environment container: # The Azure Storage account container account_name: # The Azure Storage account name account_key: # The Azure Storage account key短时令牌认证的最小 Secretdata: environment: # The Azure Storage account environment container: # The Azure Storage account container account_name: # The Azure Storage account name client_id: # The Azure Workload Identitys Client ID tenant_id: # The Azure Accounts Tenant ID holding the workload identity for LokiStack subscription_id: # The Azure Accounts Subscription ID holding the workload identity for LokiStack其中client_id、tenant_id、subscription_id正是determineCredentialMode检测 token 模式的指纹字段也是 configure.go 注入容器环境变量AZURE_CLIENT_ID、AZURE_TENANT_ID、AZURE_SUBSCRIPTION_ID、AZURE_FEDERATED_TOKEN_FILE的数据来源。前置条件创建托管身份与联合凭证在与 Kubernetes 集群相同的资源组中创建 Azure 托管身份Managed Identityaz identity create \ --name $IDENTITY_NAME \ --resource-group $RESOURCE_GROUP_NAME \ --location $LOCATION \ --subscription $SUBSCRIPTION_ID为Kubernetes accessing Azure resources场景创建两个联合凭证Federated Credentials——因为 Operator 会协调两个 ServiceAccount一个供所有 Loki Pod 共用另一个专供 Loki Ruler Pod 使用az identity federated-credential create \ --name openshift-logging-lokistack \ --identity-name $IDENTITY_NAME \ --resource-group $RESOURCE_GROUP_NAME \ --issuer $CLUSTER_ISSUER_URL \ --subject system:serviceaccount:$LOKISTACK_NS:$LOKISTACK_NAME \ --audiences $AUDIENCES az identity federated-credential create \ --name openshift-logging-lokistack-ruler \ --identity-name $IDENTITY_NAME \ --resource-group $RESOURCE_GROUP_NAME \ --issuer $CLUSTER_ISSUER_URL \ --subject system:serviceaccount:$LOKISTACK_NS:$LOKISTACK_NAME-ruler \ --audiences $AUDIENCES注意联邦凭证的 subject 必须是system:serviceaccount:NAMESPACE:SA_NAME形式。issuer 与 audiences 与承载 LokiStack 的 Kubernetes 集群相关audiences 可设为 Azure 默认值api://AzureADTokenExchange。将托管身份绑定到 Azure 角色Storage Blob Data Contributoraz role assignment create \ --assignee $MANAGED_IDENTITY_ID \ --role Storage Blob Data Contributor \ --scope /subscriptions/$SUBSCRIPTION_ID查询托管身份 ID 可使用az ad sp list --all --filter servicePrincipalType eq ManagedIdentity。源码佐证Azure 凭证校验secrets.go 的validateAzureCredentials按模式校验static 模式要求account_key存在且为合法 base64token 模式要求client_id、tenant_id、subscription_id三者齐备token-cco 模式则禁止在存储 Secret 中出现任何凭证字段account_key/client_id/tenant_id/subscription_id任一存在即报errAzureManagedIdentityNoOverride因为该模式下凭证由 CCO 注入、不允许用户覆盖。此外environment或endpoint_suffix二者必须设置其一且environment仅允许AzureGlobal/AzurePublicCloud/AzureChinaCloud/AzureGermanCloud/AzureUSGovernment五个取值secrets.go。3.2 AWS Secure Token ServiceSTS静态凭证 Secret现状data: bucketnames: # A comma-separated list of bucket names access_key_id: # The AWS static service accounts key ID access_key_secret: # The AWS static service accounts key secret endpoint: # The AWS endpoint URL.短时令牌认证的最小 Secretdata: bucketnames: # A comma-separated list of bucket names region: # A valid AWS region, e.g. us-east-1 role_arn: # The AWS IAM Role associated with a trust relationship to Lokistacks serviceaccountrole_arn是 token 模式检测的指纹字段而region在 token 模式下从可选变为必填secrets.go 的extractS3ConfigSecret中static 模式还需校验endpoint且要求access_key_id/access_key_secrettoken 模式只要求bucketnames与region。凭证注入方面configure.go 会注入AWS_ROLE_ARN、AWS_WEB_IDENTITY_TOKEN_FILE指向 ServiceAccount Token 文件路径、AWS_REGION三个环境变量。前置条件创建 IAM 角色与信任关系信任关系Trust relationship确保每个向 AWS STS 认证的 LokiStack 容器都以 ServiceAccount Token 作为身份凭据{ Version: 2012-10-17, Statement: [ { Effect: Allow, Principal: { Federated: arn:aws:iam::${AWS_ACCOUNT_ID}:oidc-provider/${OIDC_PROVIDER} }, Action: sts:AssumeRoleWithWebIdentity, Condition: { StringEquals: { ${OIDC_PROVIDER}:sub: [ system:serviceaccount:${LOKISTACK_NS}:${LOKISTACK_NAME} system:serviceaccount:${LOKISTACK_NS}:${LOKISTACK_NAME}-ruler ] } } } ] }注意subject 同样必须是system:serviceaccount:NAMESPACE:SA_NAME形式且需同时覆盖 Loki 与 Loki Ruler 两个 ServiceAccount。创建 AWS IAM 角色aws iam create-role \ --role-name my-lokistack-s3-access \ --assume-role-policy-document file:///tmp/trust.json \ --query Role.Arn \ --output text为该角色附加策略aws iam attach-role-policy \ --role-name my-lokistack-s3-access \ --policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess源码佐证S3 端点与 SSE 校验static 模式下validateS3Endpointsecrets.go要求 endpoint 为 http/https URL、不得带路径若 host 以.amazonaws.com结尾则必须匹配https://s3.{region}.amazonaws.com或 VPC 接口端点https://bucket.vpce-{id}.s3.{region}.vpce.amazonaws.com且与region一致。forcepathstyle可选字段仅接受true/false非 AWS 端点默认启用 path-style。此外还支持可选的sse_typeSSE-KMS/SSE-S3服务端加密配置SSE-KMS 时sse_kms_key_id必填secrets.go。3.3 GCP Workload Identity Federation静态凭证 Secret现状data: bucketname: # The GCS bucket name key.json: # The static serviceaccount json短时令牌认证的最小 Secretdata: audience: # The audience configured for Lokis k8s serviceacount bucketname: # The GCS bucket name key.json: # The serviceacount json file for type external_accountkey.json会被校验为以下格式特别是credential_source.file必须指向默认值/var/run/secrets/storage/serviceaccount/token该路径由 var.go 中的ServiceAccountTokenFilePath常量定义即 SA Token 卷挂载路径 /token{ type: external_account, audience: //iam.googleapis.com/projects/123456789/locations/global/workloadIdentityPools/test-pool/providers/test-provider, subject_token_type: urn:ietf:params:oauth:token-type:jwt, token_url: https://sts.googleapis.com/v1/token, service_account_impersonation_url: https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/test-service-account-42ssvtest-project.iam.gserviceaccount.com:generateAccessToken, credential_source: { file: /var/run/secrets/storage/serviceaccount/token, format: { type: text } } }源码侧secrets.go 的extractGCSConfigSecret在 token 模式下要求audience非空、key.json中credential_source.file必须等于ServiceAccountTokenFilePath否则报errGCPWrongCredentialSourceFiletoken-cco 模式下则禁止设置key.json。前置条件创建 GCP 服务账号与工作负载身份池绑定创建供 LokiStack 访问 GCP 资源的 GCP 服务账号gcloud iam service-accounts create $SERVICE_ACCOUNT_NAME \ --display-nameLoki Operator Account \ --project $PROJECT_ID为新建的 GCP 服务账号绑定最小角色集roles/iam.workloadIdentityUser与roles/storage.objectAdmingcloud projects add-iam-policy-binding $project_id \ --memberserviceAccount:$SERVICE_ACCOUNT_EMAIL \ --roleroles/iam.workloadIdentityUser\ --formatnone gcloud projects add-iam-policy-binding $project_id \ --memberserviceAccount:$SERVICE_ACCOUNT_EMAIL \ --roleroles/storage.objectAdmin \ --formatnone再为 LokiStack 的两个 ServiceAccountLoki 与 Loki Ruler分别建立主体绑定gcloud projects add-iam-policy-binding $PROJECT_ID \ --memberserviceAccount:$SERVICE_ACCOUNT_EMAIL \ --roleroles/iam.workloadIdentityUser \ --memberprincipal://iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/subject/system:serviceaccount:${LOKISTACK_NS}:${LOKISTACK_NAME} gcloud projects add-iam-policy-binding $PROJECT_ID \ --memberserviceAccount:$SERVICE_ACCOUNT_EMAIL \ --roleroles/storage.objectAdmin \ --memberprincipal://iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/subject/system:serviceaccount:${LOKISTACK_NS}:${LOKISTACK_NAME} gcloud projects add-iam-policy-binding $PROJECT_ID \ --memberserviceAccount:$SERVICE_ACCOUNT_EMAIL \ --roleroles/iam.workloadIdentityUser \ --memberprincipal://iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/subject/system:serviceaccount:${LOKISTACK_NS}:${LOKISTACK_NAME}-ruler gcloud projects add-iam-policy-binding $PROJECT_ID \ --memberserviceAccount:$SERVICE_ACCOUNT_EMAIL \ --roleroles/storage.objectAdmin \ --memberprincipal://iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/subject/system:serviceaccount:${LOKISTACK_NS}:${LOKISTACK_NAME}-ruler注意subject 需为system:serviceaccount:NAMESPACE:SA_NAME形式工作负载身份池workload identity pool必须与托管其他 Kubernetes 集群托管身份所用的池保持一致。为托管身份生成供 LokiStack 使用的凭证配置文件gcloud iam workload-identity-pools create-cred-config \ projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID \ --service-account$SERVICE_ACCOUNT_EMAIL \ --credential-source-file/var/run/secrets/serviceaccount/token \ --credential-source-typetext \ --output-file/tmp/google-application-credentials.json注意工作负载身份池及其关联的 OIDC Provider 必须与托管其他 Kubernetes 集群托管身份所用的保持一致。四、源码纵深短时令牌如何在 Pod 内生效4.1 凭证注入与 SA Token 投影卷当CredentialMode为token或token-cco时configure.go 的ensureObjectStoreCredentials会为容器注入令牌认证环境变量并挂载一个投影的 ServiceAccount Token 卷。该卷的核心配置configure.go 的saTokenVolumeServiceAccountToken: corev1.ServiceAccountTokenProjection{ ExpirationSeconds: ptr.To(saTokenExpiration), // 3600 秒1 小时 Path: corev1.ServiceAccountTokenKey, // token Audience: audience, },默认 audience 由 var.go 定义并可按存储类型在 Secret 中覆盖AWS默认sts.amazonaws.comAzure默认api://AzureADTokenExchangeGCP默认openshift各存储类型注入的环境变量configure.go云厂商注入环境变量数据来源AWSAWS_ROLE_ARN、AWS_WEB_IDENTITY_TOKEN_FILE、AWS_REGIONSecret 的role_arn/region SA Token 文件AzureAZURE_STORAGE_ACCOUNT_NAME、AZURE_CLIENT_ID、AZURE_TENANT_ID、AZURE_SUBSCRIPTION_ID、AZURE_FEDERATED_TOKEN_FILESecret 的account_name/client_id/tenant_id/subscription_id SA Token 文件GCSGOOGLE_APPLICATION_CREDENTIALS指向 Secret 中的key.jsonSecret 的key.json以 AWS 为例Loki 容器最终通过AWS_WEB_IDENTITY_TOKEN_FILE指向的 ServiceAccount Token配合AWS_ROLE_ARN完成sts:AssumeRoleWithWebIdentity换取临时凭证——这正是信任关系中Action所要求的动作。4.2 OpenShift 场景Cloud Credential Operator 集成在 AWS-STS / Azure-WIF 托管的 OpenShift 集群上Operator 以token-cco模式运行credentialsrequest.go 会创建/更新一个CredentialsRequest资源向 OpenShift cloud-credentials-operator 请求注入云凭证 SecrethasManagedCredentialModecredentialsrequest.go仅在CredentialMode未显式指定或显式为token-cco时才创建该请求若用户显式指定static/token则会删除已有 CredentialsRequest。此时存储 Secret 中不允许出现任何凭证字段Azure 侧由errAzureManagedIdentityNoOverride强制CCO 注入的 Secret 通过tokenCCOAuthConfigVolumeconfigure.go挂载供AWS_SHARED_CREDENTIALS_FILEAWS_SDK_LOAD_CONFIGtrue或GOOGLE_APPLICATION_CREDENTIALS等环境变量读取configure.go、configure.go。4.3 配置一致性保障Operator 会对存储 Secret 内容计算 SHA1 哈希secrets.go 的hashSecretData并将其写入storage.Options见 options.go 中的SecretName/SecretSHA1。Secret 变更会触发 Pod 配置的相应更新确保令牌认证配置与 Secret 内容始终一致。五、实战落地LokiStack 声明式配置示例在完成云厂商 IAM 资源预置后即可通过 LokiStack CR 的spec.storage.secret引用新格式的存储 Secret。以下是一个组合示例以 AWS STS 为例Azure/GCP 替换type与 Secret 字段即可apiVersion: loki.grafana.com/v1 kind: LokiStack metadata: name: my-lokistack namespace: openshift-logging spec: size: 1x.medium storage: schemas: - version: v13 effectiveDate: 2024-01-01 secret: type: s3 name: lokistack-s3-secret # 可选显式指定凭证模式省略时 Operator 依据 Secret 字段自动检测 credentialMode: token存储 Secret 最小化示例apiVersion: v1 kind: Secret metadata: name: lokistack-s3-secret namespace: openshift-logging stringData: bucketnames: lokistack-logs region: us-east-1 role_arn: arn:aws:iam::123456789012:role/my-lokistack-s3-access type: Opaque落地后的验证手段kubectl get lokistack my-lokistack -o yaml检查status.storage.credentialMode是否为token查看 Loki Pod 环境变量确认AWS_WEB_IDENTITY_TOKEN_FILE等令牌认证变量已注入检查 Pod 内/var/run/secrets/storage/serviceaccount/token文件存在且可读。前提与限制本文涉及的token-cco模式仅适用于 AWS-STS / Azure-WIF 托管的 OpenShift 集群依赖 Cloud Credential OperatorSwift 与 AlibabaCloud 仅支持static模式GCS 的 token 模式要求key.json的type为external_account。所有配置以当前仓库源码为准云厂商 IAM 侧的最终策略以各云厂商官方文档为准。六、演进历史该能力在 Loki Operator 中按云厂商逐步落地对应的上游实现记录见 operator/CHANGELOG.md 与提案中的 Implementation HistoryAWS STS 支持GCS Workload Identity Federation 支持Azure Workload Identity Federation 支持随后的统一重构将三个存储客户端的凭证处理收敛到CredentialMode框架之下即本文描述的统一检测与注入机制。相关测试覆盖可参考 configure_test.go验证各类存储类型在不同凭证模式下的环境变量与卷挂载断言与 secrets_test.go验证 Secret 字段校验与模式检测需要深入验证细节的读者可直接阅读这两份测试文件。结语短时令牌认证把 LokiStack 对象存储访问从长期静态凭证 人工轮换升级为ServiceAccount 身份绑定 自动轮换的短时令牌在不改变 IAM 统一管控能力的前提下显著缩小了安全事件的影响面。通过CredentialMode的显式配置或自动检测配合三大云厂商的最小化 Secret 与文中 IAM 预置命令即可在生产集群上完成从静态凭证到 STS / Workload Identity 的平滑迁移。【免费下载链接】lokiLike Prometheus, but for logs.项目地址: https://gitcode.com/GitHub_Trending/lok/loki创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考
