Collect Kubernetes Container Logs with Tencent Cloud CLS

Kubernetes changed how applications are packaged and scheduled, but it also changed how logs must be collected. Containers are lightweight and often stateless, Pods appear and disappear frequently, and a single node may run dozens or hundreds of containers. A static host-era collection rule is usually not enough.

Tencent Cloud CLS addresses the Kubernetes case by extending LogListener with container-environment components. Together, they form a Log-agent that can watch Kubernetes resources, calculate collection paths, update CLS-side configuration, and send container logs to CLS.

What makes Kubernetes log collection different

Challenge	Kubernetes-specific reason	Collection requirement
Many log forms	Kubernetes produces audit logs, event logs, stdout logs, and file logs.	Support multiple sources and parsing methods.
Dynamic runtime	Pods are created, destroyed, scaled, and rescheduled continuously.	Dynamically match containers and refresh paths.
Reliability pressure	Nodes and Pods can disappear during normal operations.	Pre-read files and continue collecting even when files are deleted.
Performance pressure	A large node can contain many containers.	Use high-performance, multi-threaded collection.
DevOps integration	Mature Kubernetes environments rely on CI/CD and declarative resources.	Support CRD-based configuration instead of only console clicks.

The design goal is simple: collection rules should follow Kubernetes dynamics instead of requiring operators to manually chase every runtime change.

Log-agent architecture

Log-agent coordinates CRDs, local Pod discovery, LogListener configuration, and CLS log topic updates.

The architecture contains three roles:

Component	Role
CLS service side	Creates and manages the Log-agent relationship and log-topic configuration.
Log-agent	Watches cluster CRDs and local Pods, then computes collection paths and updates CLS configuration.
LogListener	Collects log files and standard output based on generated configuration, then uploads logs to CLS.

The working flow is:

1. Create a machine group in CLS for the containers to be collected.
2. Generate LogListener configuration from the collection rule.
3. Start LogListener and pull collection configuration from CLS.
4. Create or update the collection CRD as business needs change.
5. Log-agent watches the CRD and updates the corresponding CLS log topic configuration.
6. Log-agent watches local Pod changes and recalculates collection paths.
7. LogListener collects, parses, and uploads the target logs.

TKE and EKS: what can be collected

TKE and EKS collection covers business logs, audit logs, and Kubernetes events.

For Tencent Cloud container environments, CLS supports both environment-variable configuration and CRD configuration. It can collect:

Log category	Examples
Business logs	Standard output, files inside containers, and files on the host.
Kubernetes audit logs	kube-apiserver audit records stored and searched through CLS.
Kubernetes events	Resource scheduling and runtime events for daily observation and troubleshooting.

TKE can enable log service with default collection and default dashboards for audit logs and events.

Business log collection types

When creating a collection rule for TKE or EKS business logs, choose the target data source first.

Collection type	When to use it
Standard output	The application writes logs to stdout, and container runtime such as Docker or Containerd manages the output. This is usually the simplest option.
Container file	The application writes logs inside the container filesystem.
Host file	The application writes logs to a mounted hostPath so raw files remain on the node after the container stops. This helps preserve logs when collection is abnormal.

Console configuration

Console-based collection rules can target namespace, Pod, container, and log source type.

The console path supports real-time configuration delivery and updates. Rules can filter by namespace, Pod, and container, then target the specific log source that should be collected.

This path is useful when operators need an interactive configuration flow or when the collection rule is managed outside the application deployment pipeline.

CRD configuration

LogConfig CRD carries the same collection intent as console configuration in a Kubernetes-native workflow.

CRD configuration is better aligned with automated Kubernetes operations. Define a LogConfig resource, and Log-agent updates the relevant CLS log topic and machine-group binding when the resource changes.

A practical LogConfig object usually needs to express:

apiVersion: cls.cloud.tencent.com/v1
kind: LogConfig
metadata:
  name: app-log-config
spec:
  clsDetail:
    logsetName: app-logset
    topicName: app-topic
  inputDetail:
    type: container_file
    containerFile:
      namespace: production
      podLabels:
        app: payment-service
      container: app
      logPath: /data/logs
      filePattern: "*.log"
  outputDetail:
    logType: minimalist_log

Use the real fields from your cluster and CLS topic. The important engineering pattern is that collection intent lives inside Kubernetes resource management, so CI/CD can create and update log rules with the application.

Audit logs

Audit-log dashboards summarize operations, CRUD distribution, resource types, active nodes, exceptions, and trends.

Kubernetes audit logs are JSON-structured records produced by kube-apiserver according to configurable audit policy. After log service is enabled in TKE, CLS can automatically collect and report audit logs.

The default audit view includes:

Audit view	Useful for
User operation count	Seeing who is changing cluster resources.
CRUD distribution	Understanding create, read, update, and delete patterns.
Resource-type distribution	Finding which Kubernetes resources are most active.
Active nodes	Tracking nodes involved in operations.
Exception access	Reviewing abnormal operation patterns.
Operation trends and list	Moving from aggregate trends to concrete audit records.

Kubernetes events

Event-log dashboards summarize event categories, abnormal reasons, node anomalies, trends, and event lists.

Kubernetes Events record cluster operation and resource scheduling behavior. They help operators observe resource changes and diagnose runtime problems.

The default event view includes:

Event view	Useful for
Event category distribution	Seeing the shape of cluster activity.
Abnormal reason statistics	Finding repeated failure causes.
Node anomaly statistics	Locating infrastructure-side problems.
Abnormal event trend	Tracking whether a problem is growing or recovering.
Event list	Jumping from a panel to concrete events.

Self-built and multi-cloud Kubernetes clusters

Self-built and multi-cloud Kubernetes clusters can send logs into the same CLS management plane.

CLS also supports Kubernetes clusters outside TKE and EKS, including self-built clusters and other cloud environments. The setup uses the same collection idea with a four-step flow:

1. Install LogListener in the self-built Kubernetes cluster.
2. Define the LogConfig resource type.
3. Define a LogConfig object.
4. Create the LogConfig object.

Hybrid Kubernetes access follows the same installation, CRD definition, LogConfig object, and creation flow.

The console and CRD configuration choices remain available, which makes the model useful for hybrid cloud log management.

Operational checklist

• Decide whether each workload writes stdout, container files, or host files.
• Prefer standard output where possible for simple application logs.
• Use hostPath-mounted host files when raw logs must survive container termination.
• Use CRD configuration when log rules should move through CI/CD.
• Enable audit logs and event logs when cluster-level troubleshooting matters.
• Verify that Log-agent watches CRD and local Pod changes correctly.
• Confirm that LogListener keeps collecting during Pod deletion and file deletion cases.

FAQ

Why does Kubernetes need a different log collection model?

Pods are dynamic, log paths can change, and a single node can run many containers. Collection rules need to follow Kubernetes resources rather than fixed host assumptions.

What is the role of Log-agent?

Log-agent connects Kubernetes state with CLS configuration. It watches CRDs and Pods, computes collection paths, and updates the log topic configuration used by LogListener.

When should CRD configuration be used?

Use CRD configuration when log collection should be part of the Kubernetes deployment workflow and managed through automation.

Can non-TKE clusters use this model?

Yes. Self-built and other-cloud Kubernetes clusters can install LogListener, define LogConfig, and send logs to CLS for unified management.