Changes
Page history
Create Uncoordinated multi process GPU usage
authored
Apr 09, 2023
by
Robert Schade
Show whitespace changes
Inline
Side-by-side
Uncoordinated-multi-process-GPU-usage.md
0 → 100644
View page @
a4092cdb
*
Description: When multiple processes use the same GPU in an uncoordinated way this can lead to a dramatic reduction of efficiency.
*
Criterion:
*
nv_compute_processes>1
*
Works for shared jobs with existing metrics:
*
yes, if a GPU is allocated to at most one compute job
*
Possible false positives:
*
*
Possible cures/workarounds:
*
check MPI-rank-GPU affinity/visibility
*
deploy MPS (https://docs.nvidia.com/deploy/mps/index.html)