OpenTelemetry Collector

The OpenTelemetry Collector offers a vendor-agnostic implementation of how to receive, process and export telemetry data. It removes the need to run, operate, and maintain multiple agents/collectors.

The OpenTelemetry Collector can be configured to receive logs from various sources and export them to OTEL compatible logging systems. This document explains how to configure the OpenTelemetry Collector to use the OTLP Receiver and the HTTP Exporter.

Parseable OTEL collector design

Pre-requisites

Configuration

We'll use the OTLP receiver and the OTEL HTTP exporter to send logs to Parseable.

receivers:
  otlp:
    protocols:
      grpc: null
      http: null
exporters:
  otlphttp:
    endpoint: '[PARSEABLE_URL]'
    headers:
      Authorization: '[BASIC_AUTH_TOKEN]'
      X-P-Log-Source: otel
      X-P-Stream: '[NAME_OF THE_STREAM_TO_SEND_LOGS_TO]'
      Content-Type: application/json
    encoding: json
    tls:
      insecure: true
processors:
  batch: null
service:
  pipelines:
    logs:
      receivers:
        - otlp
      processors:
        - batch
      exporters:
        - otlphttp

The configuration above sets up the OTLP receiver to receive logs and the OTEL HTTP exporter to send logs to Parseable. The exporter configuration includes the Parseable endpoint, authorization headers, and the log source and stream names.

Please ensure to change the [PARSEABLE_URL], [BASIC_AUTH_TOKEN], and [NAME_OF THE_STREAM_TO_SEND_LOGS_TO] placeholders with the actual values. The [PARSEABLE_URL] is the Parseable ingestor's endpoint to send logs to, like http://ingestor.demo.parseable.com. The [BASIC_AUTH_TOKEN] is the basic auth token to authenticate with Parseable. The [NAME_OF THE_STREAM_TO_SEND_LOGS_TO] is the name of the stream in Parseable to send logs to.

Start the OpenTelemetry Collector

After you have update the configuration file successfully (replaced the placeholders), start the OpenTelemetry Collector with the updated configuration file. For example, for a configuration file named config.yaml with OTEL Collector binary named otel, you can start the collector with the following command:

./otelcol --config ./config.yaml

Now the collector is running on HTTP port 4317 and gRPC port 4318. By default, Parseable's /v1/logs endpoint is active, so it will ingest the logs directly.

Updated on
Vector
Logstash