Migrating from collectd to OpenTelemetry

GitHub Enterprise Server is transitioning from the collectd metrics stack to OpenTelemetry. Collectd is retired in version 3.23, and upgrading to 3.23 disables it.

Migration timeline

Version	Default for new installations	Default for instances upgraded from earlier releases	Notes
3.18	Collectd	Collectd	OpenTelemetry is opt-in.
3.19 to 3.22	OpenTelemetry	Collectd	Migration to OpenTelemetry is possible before upgrading to 3.23.
3.23 and later	OpenTelemetry	OpenTelemetry	Collectd is retired and cannot be used. The upgrade disables it.

Preparing to migrate

Before you migrate, complete the following preparations:

• Test the migration on a staging instance before applying it in production.
• Identify the monitoring integrations that depend on collectd metrics, such as external dashboards and alerts.
• If you plan to run both metrics stacks during a transition period, review the caveats in Running both metrics stacks.

Migrating to OpenTelemetry

You can migrate to OpenTelemetry on instances running version 3.19 to 3.22. On version 3.18, OpenTelemetry is opt-in, so you can also enable it there. We recommend migrating before you upgrade to 3.23, so that you can verify your monitoring setup while you can still use collectd.

If you don't migrate, upgrading to version 3.23 or later does it for you: collectd is disabled and cannot be re-enabled, and OpenTelemetry metrics are enabled and become the only metrics stack.

Migration requires enabling OpenTelemetry metrics and disabling collectd. Both settings are available through the 管理控制台 or the command line.

Using the 管理控制台

1. 在 GitHub Enterprise Server 上的管理帐户中，在任一页面的右上角，单击“”。

2. 如果你尚未在“站点管理员”页上，请在左上角单击“站点管理员”。

3. 在“ 站点管理”边栏中，单击“管理控制台”****。

4. In the monitoring settings, select Enable OpenTelemetry.

5. Under Metrics - Collectd, select Disable collectd metrics.

6. 在“设置”边栏下，单击“保存设置”。

   注意

   保存 管理控制台 中的设置会重启系统服务，这可能会导致用户可察觉的停机时间。

7. 等待配置运行完毕。

Using the command line

1. 通过 SSH 连接到 你的 GitHub Enterprise Server 实例。 如果实例包含多个节点，例如，如果配置了高可用性或异地复制，则通过 SSH 连接到主节点。 如果使用群集，则可以通过 SSH 连接到任何节点。 将 HOSTNAME 替换为实例的主机名，或节点的主机名或 IP 地址。 有关详细信息，请参阅“访问管理 shell (SSH)”。

   Shell

   ssh -p 122 admin@HOSTNAME
   

2. To enable OpenTelemetry metrics and disable collectd, run the following commands.

   ghe-config observability.metrics.next-enabled true
   ghe-config observability.metrics.legacy-disabled true
   

3. 若要应用配置，请运行以下命令。

   注意

   在配置运行过程中，你的 GitHub Enterprise Server 实例 上的服务可能会重启，这可能会导致用户短暂停机。

   Shell

   ghe-config-apply
   

4. 等待配置运行完毕。

Running both metrics stacks

On version 3.18 to 3.22, running the collectd and OpenTelemetry stacks at the same time is possible but not recommended. This configuration has the following caveats:

• Some metric names may differ to remain compatible with the collectd format.
• Some OpenTelemetry metrics do not include labels.
• Increased resource utilization to deliver metrics to both solutions.

Collectd is retired in version 3.23, so running both stacks is not possible after upgrading.

Post-upgrade checks

After the configuration run completes, whether you migrated manually or upgraded to version 3.23 or later:

• Verify that the metrics are visible and collected in the Monitoring tab of the 管理控制台.
• Update monitoring integrations that depend on collectd metrics. For information about connecting external tools, see 使用 OpenTelemetry 设置外部监视.
• For information about analyzing metrics, see 使用 Grafana 分析 OpenTelemetry 指标.

Viewing historical collectd data

Since patches 3.22.3, 3.21.8, 3.20.10, and 3.19.14, the Monitoring tab includes collectd dashboards for viewing historical data. These dashboards display collectd metrics recorded up to the time the collectd stack was disabled.