Background on power measure¶
This section provide general information on monitoring power on computers.
In short, RAPL measure the cpu, nvidia-smi the gpu, and other parts (fans, harddrive) are not accessible.
General considerations¶
The main sources of energy consumption in a computer are the gpu, the cpu and memory. However, other sources are non negligible, among them, the fans, the optical drives, motherboards, and hard drives. Moreover, the consumption will vary according to external factors, such as the temperature, and obviously, the type of programs run by the user.
Different sources of energy consumption. Green boxes are measured by our library (indicative figures)¶
A generic solution with an easy deployment has to use the inbuilt sensors and available software models, and thus is restricted to what is provided by the constructors. As most of the monitoring tools, we use RAPL for the CPU, and nvidia-smi for the GPU.
Preliminaries¶
The unit to measure energy is the Joule, it is the energy transferred to an object when a force of one newton acts on that object in the direction of the force’s motion through a distance of one metre (1 newton-metre or Nm). The watt is the unit to measure power. In other words:
1 watt = 1 Joule per second = The energy required to lift a medium-sized tomato up 1 metre (wikipedia)
CPU and RAPL¶
The Running Average Power Limit (RAPL) reports the accumulated energy consumption of the cpu, the ram mechanism, and a few other devices (but NOT the nvidia GPU). It is present since the Sandy bridge architecture in 2011 and from Haswell is supported by integrated voltage regulators in addition to power models ( [Hackenberg2015] ). There has been considerable study to validate its use for software monitoring ( [Khan2018] ).
It is divided into different physically meaningfull domains:
RAPL power domains (from [Khan2018] )¶
Power Plane 0 : the cores alias the CPU
Power Plane 1 : uncore : memory controller and cache, and an integrated on chip gpu is present (this is not the external nvidia GPU).
DRAM : energy consumption of the RAM
Psys : System on Chip energy consumption
The rapl interface writes these values in module specific registers located in /dev/cpu/*/msr. These values are updated every 1ms. Although reading from these files is possible, our code relies on the powercap linux tool which updates the energy consumption for the different domains in /sys/class/powercap/intel-rapl.
Note
RAPL does not measure every piece of hardware in the computer. Tipycally fans and harddrive are not recorded. A difference of 30% between the RAPL power and the results from a powermeter is not uncommon.
More readings on RAPL:
The official documentation is the Intel® 64 and IA-32 Architectures Software Developer Manual, Volume 3: System Programming Guide. But it is not trivial for most data scientists.
GPU and nvidia-smi¶
GPU are good for single instruction multiple data situations where they can parallelize on their numerous streaming processors
Regarding power measurements, things are simple with nvidia-smi, unfortunately, part of the reason for this is because we have much less information than RAPL. from the man page of nvidia-smi : The last measured power draw for the entire board, in watts. Only available if power management is supported. Please note that for boards without INA sensors, this refers to the power draw for the GPU and not for the entire board.
Still, nvidia provides us with the percentage of core memory used by each program for each GPU (% Streaming Multiprcessor). We use this information to assign the total power of the GPUs to the different programs running on it. Now see below for considerations about measuring multiple programs. Fortunately for analysts (but is it good for efficiency?), most of the IA experiments are conducted such that one GPU is exclusively allocated to one experiment.
Measuring multiple programs¶
In general, RAPL and nvidia provide us a power measurement for the whole board or entire sockets. Thus, to take into account the energy consumed from each program, we adopt the approach implemented in the experiment impact tracker and multiply the RAPL value by the percentage of cpu and memory used.
However, the energy consumed by programs is not exactly additive. For instance, the amount of Joules spent by two programs running on parallel and in sequence won’t be the same. This is due to a fix energy waste, for instance coming from the idle state, and because the relation between the usage of the computer and the energy consumed is not linear, for instance, due to a time delay between the change of core use and the energy usage. One approach, not implemented here, is to have a model, eventually build from profiling, which predicts the consumption based on the cpu clock time and gpu usage of the program.
In conclusion, we recommend to benchmark a program when it is running alone on the machine*.
Note
*Actually, others causes can have an influence such as the wear, the temperature of the room and so on. See [heinrich2017], Figure 11 for an inventory of all parameters that may influence the behavior of the system)
