others-how to setup opentelemetry collector using docker locally ?

Sep 5, 2023

1. Purpose

In this post, I will show you how to setup opentelemetry collector using docker locally. Specifically, I will give you an example of the opentelemetry collector configuration for test.

2. Solution

2.1 What is opentelemetry?

OpenTelemetry (also known as OTel) is an open source observability framework consisting of a set of tools, APIs and SDKs that enables IT teams to detect, generate, collect and export remote monitoring data for analysis and understanding of software performance and behavior.

2.2 What is opentelemetry collector?

OpenTelemetry Collector is a high performance, scalable and reliable data collection pipeline for observability data. It receives telemetry data from various sources, performs processing and transformation into a common format, and then exports the data to various backends for storage and analysis

2.3 The example configuration of opentelemetry collector

Here is an example config.yaml for opentelemetry collector:

receivers:
  otlp:
    protocols:
      grpc:
      http:

processors:
  batch:

exporters:
  logging:
    verbosity: detailed
    sampling_initial: 5
    sampling_thereafter: 200

extensions:
  health_check:
  pprof:
  zpages:

service:
  extensions: [health_check, pprof, zpages]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging]
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging]

You can see that we have defined the following parts:

the receivers: define the source of metrics or logs or traces.
the processors: define how to process the source
the exporters: define how to send the processed results , for example, we can use logging to print the output of the processors
the extensions: for extension.
the service: the part which glue all the other parts of the file, you must put all your definitions into services to make it work.

Then the docker startup script:

docker run -v $(pwd)/config.yaml:/etc/otelcol-contrib/config.yaml otel/opentelemetry-collector-contrib:0.84.0

The above script will map the host’s $(pwd)/config.yaml file into the docker container’s /etc/otelcol-contrib/config.yaml.

Run the script, you will get this:

[root@local collector]# ./start.sh
2023-09-01T08:51:04.320Z    info    service/telemetry.go:84 Setting up own telemetry...
2023-09-01T08:51:04.320Z    info    service/telemetry.go:201    Serving Prometheus metrics  {"address": ":8888", "level": "Basic"}
2023-09-01T08:51:04.321Z    info    [email protected]/exporter.go:275    Development component. May change in the future.    {"kind": "exporter", "data_type": "traces", "name": "logging"}
2023-09-01T08:51:04.321Z    info    [email protected]/exporter.go:275    Development component. May change in the future.    {"kind": "exporter", "data_type": "metrics", "name": "logging"}
2023-09-01T08:51:04.321Z    info    [email protected]/exporter.go:275    Development component. May change in the future.    {"kind": "exporter", "data_type": "logs", "name": "logging"}
2023-09-01T08:51:04.323Z    info    service/service.go:138  Starting otelcol-contrib... {"Version": "0.84.0", "NumCPU": 16}
2023-09-01T08:51:04.323Z    info    extensions/extensions.go:31 Starting extensions...
2023-09-01T08:51:04.323Z    info    extensions/extensions.go:34 Extension is starting...    {"kind": "extension", "name": "health_check"}
2023-09-01T08:51:04.323Z    info    [email protected]/healthcheckextension.go:35 Starting health_check extension {"kind": "extension", "name": "health_check", "config": {"Endpoint":"0.0.0.0:13133","TLSSetting":null,"CORS":null,"Auth":null,"MaxRequestBodySize":0,"IncludeMetadata":false,"ResponseHeaders":null,"Path":"/","ResponseBody":null,"CheckCollectorPipeline":{"Enabled":false,"Interval":"5m","ExporterFailureThreshold":5}}}
2023-09-01T08:51:04.323Z    warn    [email protected]/warning.go:40  Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks    {"kind": "extension", "name": "health_check", "documentation": "https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks"}
2023-09-01T08:51:04.323Z    info    extensions/extensions.go:38 Extension started.  {"kind": "extension", "name": "health_check"}
2023-09-01T08:51:04.323Z    info    extensions/extensions.go:34 Extension is starting...    {"kind": "extension", "name": "pprof"}
2023-09-01T08:51:04.324Z    info    [email protected]/pprofextension.go:60 Starting net/http/pprof server  {"kind": "extension", "name": "pprof", "config": {"TCPAddr":{"Endpoint":"localhost:1777"},"BlockProfileFraction":0,"MutexProfileFraction":0,"SaveToFile":""}}
2023-09-01T08:51:04.324Z    info    extensions/extensions.go:38 Extension started.  {"kind": "extension", "name": "pprof"}
2023-09-01T08:51:04.324Z    info    extensions/extensions.go:34 Extension is starting...    {"kind": "extension", "name": "zpages"}
2023-09-01T08:51:04.324Z    info    [email protected]/zpagesextension.go:53   Registered zPages span processor on tracer provider {"kind": "extension", "name": "zpages"}
2023-09-01T08:51:04.324Z    info    [email protected]/zpagesextension.go:63   Registered Host's zPages    {"kind": "extension", "name": "zpages"}
2023-09-01T08:51:04.324Z    info    [email protected]/zpagesextension.go:75   Starting zPages extension   {"kind": "extension", "name": "zpages", "config": {"TCPAddr":{"Endpoint":"localhost:55679"}}}
2023-09-01T08:51:04.324Z    info    extensions/extensions.go:38 Extension started.  {"kind": "extension", "name": "zpages"}
2023-09-01T08:51:04.324Z    warn    [email protected]/warning.go:40  Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks    {"kind": "receiver", "name": "otlp", "data_type": "traces", "documentation": "https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks"}
2023-09-01T08:51:04.324Z    info    [email protected]/otlp.go:83 Starting GRPC server    {"kind": "receiver", "name": "otlp", "data_type": "traces", "endpoint": "0.0.0.0:4317"}
2023-09-01T08:51:04.324Z    warn    [email protected]/warning.go:40  Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks    {"kind": "receiver", "name": "otlp", "data_type": "traces", "documentation": "https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks"}
2023-09-01T08:51:04.324Z    info    [email protected]/otlp.go:101    Starting HTTP server    {"kind": "receiver", "name": "otlp", "data_type": "traces", "endpoint": "0.0.0.0:4318"}
2023-09-01T08:51:04.324Z    info    healthcheck/handler.go:132  Health Check state change   {"kind": "extension", "name": "health_check", "status": "ready"}
2023-09-01T08:51:04.324Z    info    service/service.go:161  Everything is ready. Begin running and processing data.

3. Summary

In this post, I demonstrated how to setup opentelemetry using docker locally , and I gave you an example collector config.yml for testing. That’s it, thanks for your reading.