Repository navigation
amdgpu: include KFD compute occupancy in per-process usage - #555
Merged
Merged
Conversation
KFD/HSA compute queues are not tracked by the amdgpu context manager, so their work never appears in the DRM fdinfo engine counters. ROCm/HIP processes can therefore be listed without a gpu_usage value even while they are actively using the GPU. DRM contexts used by OpenGL, Vulkan and OpenCL remain covered by fdinfo. Read the per-process KFD compute unit occupancy from /sys/class/kfd/kfd/proc/<pid>/stats_<gpu_id>/cu_occupancy, convert it to a percentage of the node's active compute units, and add it to the process gpu_usage. The KFD gpu_id is read from the matching topology node because the stats directory is keyed by it rather than by the node directory index. KFD-aware processes that the generic fdinfo sweep defaulted to graphical are marked as compute. The sysfs file is optional: kernels and ASICs without cu_occupancy support are skipped. Tested on gfx1030 (Radeon RX 6800 XT) with Linux 7.4.0-rc1: an active HIP process reports 64 of 72 CUs while its fdinfo contains no engine counters.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ROCm/HIP submit work through KFD compute queues, which are not tracked by the
amdgpu DRM context manager. Their work therefore does not appear in the
drm-engine-*fdinfo counters, so an AMD GPU process that only uses KFD couldbe listed without any
gpu_usage.This reads the per-process KFD compute-unit occupancy from
/sys/class/kfd/kfd/proc/<pid>/stats_<gpu_id>/cu_occupancyand adds it to theprocess GPU utilization. OpenGL, Vulkan and OpenCL DRM contexts remain covered
by fdinfo; the two sources are disjoint.
Changes
gpu_idand active CU count from the topology node matchingthe DRM render minor.
nearest.
sweep default them to graphical.
cu_occupancy.Kernel support is documented in the README: CDNA since 5.11/6.2/6.6, RDNA 3/4
since 7.3, RDNA 1/2 since 7.4.
Testing
cmake --build buildctest— 28/28 passcu_occupancyreports theprocess's occupied CUs while its fdinfo has no engine counters.