Semantic conventions for system metrics

Status: Development

This document describes instruments and attributes for common system level metrics in OpenTelemetry. Consider the general metric semantic conventions when creating instruments not explicitly defined in the specification.

The system.* namespace SHOULD be exclusively used to report hosts’ metrics. The system.* namespace SHOULD only be used when the metrics are collected from within the target system. (physical servers, virtual machines etc). Metrics collected from technology-specific, well-defined APIs (e.g. Kubelet’s API or container runtimes) should be reported under their respective namespace (e.g. k8s., container.). Resource attributes related to a host, SHOULD be reported under the host.* namespace.

Warning Existing instrumentations and collector that are using v1.21.0 of this document (or prior):

  • SHOULD NOT adopt any breaking changes from document until the system semantic conventions are marked stable. Conventions include, but are not limited to, attributes, metric names, and unit of measure.
  • SHOULD introduce a control mechanism to allow users to opt-in to the new conventions once the migration plan is finalized.

General Metrics

Metric: system.uptime

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.uptimeGaugesThe time the system has been running [1]Development

[1]: Instrumentations SHOULD use a gauge with type double and measure uptime in seconds as a floating point number with the highest precision available. The actual accuracy would depend on the instrumentation and operating system.

Processor Metrics

Description: System level processor metrics captured under the namespace system.cpu.

Metric: system.cpu.physical.count

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.cpu.physical.countUpDownCounter{cpu}Reports the number of actual physical processor cores on the hardware [1]Development

[1]: Calculated by multiplying the number of sockets by the number of cores per socket

Metric: system.cpu.logical.count

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.cpu.logical.countUpDownCounter{cpu}Reports the number of logical (virtual) processor cores created by the operating system to manage multitasking [1]Development

[1]: Calculated by multiplying the number of sockets by the number of cores per socket, and then by the number of threads per core

Memory Metrics

Description: System level memory metrics capture under the namespace system.memory. This does not include paging/swap memory.

Metric: system.memory.usage

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.memory.usageUpDownCounterByReports memory in use by state. [1]Development

[1]: The sum over all system.memory.state values SHOULD equal the total memory available on the system, that is system.memory.limit.

AttributeTypeDescriptionExamplesRequirement LevelStability
system.memory.statestringThe memory statefree; cachedRecommendedDevelopment

system.memory.state has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
buffersbuffersDevelopment
cachedcachedDevelopment
freefreeDevelopment
usedusedDevelopment

Metric: system.memory.limit

This metric is opt-in.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.memory.limitUpDownCounterByTotal memory available in the system. [1]Development

[1]: Its value SHOULD equal the sum of system.memory.state over all states.

Metric: system.memory.shared

This metric is opt-in.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.memory.sharedUpDownCounterByShared memory used (mostly by tmpfs). [1]Development

[1]: Equivalent of shared from free command or Shmem from /proc/meminfo"

Metric: system.memory.utilization

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.memory.utilizationGauge1Development
AttributeTypeDescriptionExamplesRequirement LevelStability
system.memory.statestringThe memory statefree; cachedRecommendedDevelopment

system.memory.state has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
buffersbuffersDevelopment
cachedcachedDevelopment
freefreeDevelopment
usedusedDevelopment

Paging/Swap Metrics

Description: System level paging/swap memory metrics captured under the namespace system.paging.

Metric: system.paging.usage

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.paging.usageUpDownCounterByUnix swap or windows pagefile usageDevelopment
AttributeTypeDescriptionExamplesRequirement LevelStability
system.devicestringUnique identifier for the device responsible for managing paging operations./dev/dm-0RecommendedDevelopment
system.paging.statestringThe memory paging statefreeRecommendedDevelopment

system.paging.state has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
freefreeDevelopment
usedusedDevelopment

Metric: system.paging.utilization

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.paging.utilizationGauge1Development
AttributeTypeDescriptionExamplesRequirement LevelStability
system.devicestringUnique identifier for the device responsible for managing paging operations./dev/dm-0RecommendedDevelopment
system.paging.statestringThe memory paging statefreeRecommendedDevelopment

system.paging.state has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
freefreeDevelopment
usedusedDevelopment

Metric: system.paging.faults

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.paging.faultsCounter{fault}Development
AttributeTypeDescriptionExamplesRequirement LevelStability
system.paging.typestringThe memory paging typeminorRecommendedDevelopment

system.paging.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
majormajorDevelopment
minorminorDevelopment

Metric: system.paging.operations

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.paging.operationsCounter{operation}Development
AttributeTypeDescriptionExamplesRequirement LevelStability
system.paging.directionstringThe paging access directioninRecommendedDevelopment
system.paging.typestringThe memory paging typeminorRecommendedDevelopment

system.paging.direction has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
ininDevelopment
outoutDevelopment

system.paging.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
majormajorDevelopment
minorminorDevelopment

Disk Controller Metrics

Description: System level disk performance metrics captured under the namespace system.disk.

Metric: system.disk.io

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.disk.ioCounterByDevelopment
AttributeTypeDescriptionExamplesRequirement LevelStability
disk.io.directionstringThe disk IO operation direction.readRecommendedDevelopment
system.devicestringThe device identifier(identifier)RecommendedDevelopment

disk.io.direction has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
readreadDevelopment
writewriteDevelopment

Metric: system.disk.operations

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.disk.operationsCounter{operation}Development
AttributeTypeDescriptionExamplesRequirement LevelStability
disk.io.directionstringThe disk IO operation direction.readRecommendedDevelopment
system.devicestringThe device identifier(identifier)RecommendedDevelopment

disk.io.direction has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
readreadDevelopment
writewriteDevelopment

Metric: system.disk.io_time

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.disk.io_timeCountersTime disk spent activated [1]Development

[1]: The real elapsed time (“wall clock”) used in the I/O path (time from operations running in parallel are not counted). Measured as:

AttributeTypeDescriptionExamplesRequirement LevelStability
system.devicestringThe device identifier(identifier)RecommendedDevelopment

Metric: system.disk.operation_time

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.disk.operation_timeCountersSum of the time each operation took to complete [1]Development

[1]: Because it is the sum of time each request took, parallel-issued requests each contribute to make the count grow. Measured as:

  • Linux: Fields 7 & 11 from procfs-diskstats
  • Windows: “Avg. Disk sec/Read” perf counter multiplied by “Disk Reads/sec” perf counter (similar for Writes)
AttributeTypeDescriptionExamplesRequirement LevelStability
disk.io.directionstringThe disk IO operation direction.readRecommendedDevelopment
system.devicestringThe device identifier(identifier)RecommendedDevelopment

disk.io.direction has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
readreadDevelopment
writewriteDevelopment

Metric: system.disk.merged

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.disk.mergedCounter{operation}Development
AttributeTypeDescriptionExamplesRequirement LevelStability
disk.io.directionstringThe disk IO operation direction.readRecommendedDevelopment
system.devicestringThe device identifier(identifier)RecommendedDevelopment

disk.io.direction has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
readreadDevelopment
writewriteDevelopment

Metric: system.disk.limit

This metric is opt-in.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.disk.limitUpDownCounterByThe total storage capacity of the diskDevelopment
AttributeTypeDescriptionExamplesRequirement LevelStability
system.devicestringThe device identifier(identifier)RecommendedDevelopment

Filesystem Metrics

Description: System level filesystem metrics captured under the namespace system.filesystem.

Metric: system.filesystem.usage

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.filesystem.usageUpDownCounterByReports a filesystem’s space usage across different states. [1]Development

[1]: The sum of all system.filesystem.usage values over the different system.filesystem.state attributes SHOULD equal the total storage capacity of the filesystem, that is system.filesystem.limit.

AttributeTypeDescriptionExamplesRequirement LevelStability
system.devicestringIdentifier for the device where the filesystem resides./dev/sda; \network-driveRecommendedDevelopment
system.filesystem.modestringThe filesystem moderw, roRecommendedDevelopment
system.filesystem.mountpointstringThe filesystem mount path/mnt/dataRecommendedDevelopment
system.filesystem.statestringThe filesystem stateusedRecommendedDevelopment
system.filesystem.typestringThe filesystem typeext4RecommendedDevelopment

system.filesystem.state has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
freefreeDevelopment
reservedreservedDevelopment
usedusedDevelopment

system.filesystem.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
exfatexfatDevelopment
ext4ext4Development
fat32fat32Development
hfsplushfsplusDevelopment
ntfsntfsDevelopment
refsrefsDevelopment

Metric: system.filesystem.utilization

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.filesystem.utilizationGauge1Development
AttributeTypeDescriptionExamplesRequirement LevelStability
system.devicestringIdentifier for the device where the filesystem resides./dev/sda; \network-driveRecommendedDevelopment
system.filesystem.modestringThe filesystem moderw, roRecommendedDevelopment
system.filesystem.mountpointstringThe filesystem mount path/mnt/dataRecommendedDevelopment
system.filesystem.statestringThe filesystem stateusedRecommendedDevelopment
system.filesystem.typestringThe filesystem typeext4RecommendedDevelopment

system.filesystem.state has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
freefreeDevelopment
reservedreservedDevelopment
usedusedDevelopment

system.filesystem.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
exfatexfatDevelopment
ext4ext4Development
fat32fat32Development
hfsplushfsplusDevelopment
ntfsntfsDevelopment
refsrefsDevelopment

Metric: system.filesystem.limit

This metric is opt-in.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.filesystem.limitUpDownCounterByThe total storage capacity of the filesystemDevelopment
AttributeTypeDescriptionExamplesRequirement LevelStability
system.devicestringIdentifier for the device where the filesystem resides./dev/sda; \network-driveRecommendedDevelopment
system.filesystem.modestringThe filesystem moderw, roRecommendedDevelopment
system.filesystem.mountpointstringThe filesystem mount path/mnt/dataRecommendedDevelopment
system.filesystem.typestringThe filesystem typeext4RecommendedDevelopment

system.filesystem.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
exfatexfatDevelopment
ext4ext4Development
fat32fat32Development
hfsplushfsplusDevelopment
ntfsntfsDevelopment
refsrefsDevelopment

Network Metrics

Description: System level network metrics captured under the namespace system.network.

Metric: system.network.dropped

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.network.droppedCounter{packet}Count of packets that are dropped or discarded even though there was no error [1]Development

[1]: Measured as:

AttributeTypeDescriptionExamplesRequirement LevelStability
network.interface.namestringThe network interface name.lo; eth0RecommendedDevelopment
network.io.directionstringThe network IO operation direction.transmitRecommendedDevelopment

network.io.direction has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
receivereceiveDevelopment
transmittransmitDevelopment

Metric: system.network.packets

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.network.packetsCounter{packet}Development
AttributeTypeDescriptionExamplesRequirement LevelStability
network.io.directionstringThe network IO operation direction.transmitRecommendedDevelopment
system.devicestringThe device identifier(identifier)RecommendedDevelopment

network.io.direction has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
receivereceiveDevelopment
transmittransmitDevelopment

Metric: system.network.errors

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.network.errorsCounter{error}Count of network errors detected [1]Development

[1]: Measured as:

AttributeTypeDescriptionExamplesRequirement LevelStability
network.interface.namestringThe network interface name.lo; eth0RecommendedDevelopment
network.io.directionstringThe network IO operation direction.transmitRecommendedDevelopment

network.io.direction has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
receivereceiveDevelopment
transmittransmitDevelopment

Metric: system.network.io

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.network.ioCounterByDevelopment
AttributeTypeDescriptionExamplesRequirement LevelStability
network.interface.namestringThe network interface name.lo; eth0RecommendedDevelopment
network.io.directionstringThe network IO operation direction.transmitRecommendedDevelopment

network.io.direction has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
receivereceiveDevelopment
transmittransmitDevelopment

Metric: system.network.connections

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.network.connectionsUpDownCounter{connection}Development
AttributeTypeDescriptionExamplesRequirement LevelStability
network.connection.statestringThe state of network connection [1]close_waitRecommendedDevelopment
network.interface.namestringThe network interface name.lo; eth0RecommendedDevelopment
network.transportstringOSI transport layer or inter-process communication method. [2]tcp; udpRecommendedStable

[1] network.connection.state: Connection states are defined as part of the rfc9293

[2] network.transport: The value SHOULD be normalized to lowercase.

Consider always setting the transport when setting a port number, since a port number is ambiguous without knowing the transport. For example different processes could be listening on TCP port 12345 and UDP port 12345.


network.connection.state has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
close_waitclose_waitDevelopment
closedclosedDevelopment
closingclosingDevelopment
establishedestablishedDevelopment
fin_wait_1fin_wait_1Development
fin_wait_2fin_wait_2Development
last_acklast_ackDevelopment
listenlistenDevelopment
syn_receivedsyn_receivedDevelopment
syn_sentsyn_sentDevelopment
time_waittime_waitDevelopment

network.transport has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
pipeNamed or anonymous pipe.Stable
quicQUICDevelopment
tcpTCPStable
udpUDPStable
unixUnix domain socketStable

Aggregate System Process Metrics

Description: System level aggregate process metrics captured under the namespace system.process. For metrics at the individual process level, see process metrics.

Metric: system.process.count

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.process.countUpDownCounter{process}Total number of processes in each stateDevelopment
AttributeTypeDescriptionExamplesRequirement LevelStability
system.process.statusstringThe process state, e.g., Linux Process State CodesrunningRecommendedDevelopment

system.process.status has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
defunctdefunctDevelopment
runningrunningDevelopment
sleepingsleepingDevelopment
stoppedstoppedDevelopment

Metric: system.process.created

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.process.createdCounter{process}Total number of processes created over uptime of the hostDevelopment

system.{os}. - OS Specific System Metrics

Instrument names for system level metrics that have different and conflicting meaning across multiple OSes should be prefixed with system.{os}. and follow the hierarchies listed above for different entities like CPU, memory, and network.

For example, UNIX load average over a given interval is not well standardized and its value across different UNIX like OSes may vary despite being under similar load:

Without getting into the vagaries of every Unix-like operating system in existence, the load average more or less represents the average number of processes that are in the running (using the CPU) or runnable (waiting for the CPU) states. One notable exception exists: Linux includes processes in uninterruptible sleep states, typically waiting for some I/O activity to complete. This can markedly increase the load average on Linux systems.

(source of quote, linux source code)

An instrument for load average over 1 minute on Linux could be named system.linux.cpu.load_1m, reusing the cpu name proposed above and having an {os} prefix to split this metric across OSes.

Metric: system.linux.memory.available

NameInstrument TypeUnit (UCUM)DescriptionStability
system.linux.memory.availableUpDownCounterByAn estimate of how much memory is available for starting new applications, without causing swapping [1]Development

[1]: This is an alternative to system.memory.usage metric with state=free. Linux starting from 3.14 exports “available” memory. It takes “free” memory as a baseline, and then factors in kernel-specific values. This is supposed to be more accurate than just “free” memory. For reference, see the calculations here. See also MemAvailable in /proc/meminfo.

Metric: system.linux.memory.slab.usage

This metric is recommended.

NameInstrument TypeUnit (UCUM)DescriptionStability
system.linux.memory.slab.usageUpDownCounterByReports the memory used by the Linux kernel for managing caches of frequently used objects. [1]Development

[1]: The sum over the reclaimable and unreclaimable state values in linux.memory.slab.usage SHOULD be equal to the total slab memory available on the system. Note that the total slab memory is not constant and may vary over time. See also the Slab allocator and Slab in /proc/meminfo.

AttributeTypeDescriptionExamplesRequirement LevelStability
linux.memory.slab.statestringThe Linux Slab memory statereclaimable; unreclaimableRecommendedDevelopment

linux.memory.slab.state has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
reclaimablereclaimableDevelopment
unreclaimableunreclaimableDevelopment