May 31, 2011

Getting NVIDIA GPU Usage in C++

Strangely enough, NVAPI has no functions to get GPU usage/load. It turns out that there are some secret functions in nvapi.dll. You can use QueryInterface function to retrieve them by specifying the memory address of the function. Here's the code for Windows.

//
// Getting Nvidia GPU Usage
//
// Reference: Open Hardware Monitor (http://code.google.com/p/open-hardware-monitor)
//

#include <windows.h>
#include <iostream>

// magic numbers, do not change them
#define NVAPI_MAX_PHYSICAL_GPUS   64
#define NVAPI_MAX_USAGES_PER_GPU  34

// function pointer types
typedef int *(*NvAPI_QueryInterface_t)(unsigned int offset);
typedef int (*NvAPI_Initialize_t)();
typedef int (*NvAPI_EnumPhysicalGPUs_t)(int **handles, int *count);
typedef int (*NvAPI_GPU_GetUsages_t)(int *handle, unsigned int *usages);

int main()
{   
    HMODULE hmod = LoadLibraryA("nvapi.dll");
    if (hmod == NULL)
    {
        std::cerr << "Couldn't find nvapi.dll" << std::endl;
        return 1;
    }

    // nvapi.dll internal function pointers
    NvAPI_QueryInterface_t      NvAPI_QueryInterface     = NULL;
    NvAPI_Initialize_t          NvAPI_Initialize         = NULL;
    NvAPI_EnumPhysicalGPUs_t    NvAPI_EnumPhysicalGPUs   = NULL;
    NvAPI_GPU_GetUsages_t       NvAPI_GPU_GetUsages      = NULL;

    // nvapi_QueryInterface is a function used to retrieve other internal functions in nvapi.dll
    NvAPI_QueryInterface = (NvAPI_QueryInterface_t) GetProcAddress(hmod, "nvapi_QueryInterface");

    // some useful internal functions that aren't exported by nvapi.dll
    NvAPI_Initialize = (NvAPI_Initialize_t) (*NvAPI_QueryInterface)(0x0150E828);
    NvAPI_EnumPhysicalGPUs = (NvAPI_EnumPhysicalGPUs_t) (*NvAPI_QueryInterface)(0xE5AC921F);
    NvAPI_GPU_GetUsages = (NvAPI_GPU_GetUsages_t) (*NvAPI_QueryInterface)(0x189A1FDF);

    if (NvAPI_Initialize == NULL || NvAPI_EnumPhysicalGPUs == NULL ||
        NvAPI_EnumPhysicalGPUs == NULL || NvAPI_GPU_GetUsages == NULL)
    {
        std::cerr << "Couldn't get functions in nvapi.dll" << std::endl;
        return 2;
    }

    // initialize NvAPI library, call it once before calling any other NvAPI functions
    (*NvAPI_Initialize)();

    int          gpuCount = 0;
    int         *gpuHandles[NVAPI_MAX_PHYSICAL_GPUS] = { NULL };
    unsigned int gpuUsages[NVAPI_MAX_USAGES_PER_GPU] = { 0 };

    // gpuUsages[0] must be this value, otherwise NvAPI_GPU_GetUsages won't work
    gpuUsages[0] = (NVAPI_MAX_USAGES_PER_GPU * 4) | 0x10000;

    (*NvAPI_EnumPhysicalGPUs)(gpuHandles, &gpuCount);

    // print GPU usage every second
    for (int i = 0; i < 100; i++)
    {
        (*NvAPI_GPU_GetUsages)(gpuHandles[0], gpuUsages);
        int usage = gpuUsages[3];
        std::cout << "GPU Usage: " << usage << std::endl;
        Sleep(1000);
    }

    return 0;
}

8 comments:

  1. I was wondering how this code would work for multiple cards. There is the gpuCount variable but I don't seem to understand how it is being used.

    ReplyDelete
    Replies
    1. Replacing the for-loop as follows worked for me:

      // print GPU usage every second
      int usages[NVAPI_MAX_PHYSICAL_GPUS];
      for (int i = 0; i < 100; i++)
      {
      for(int j=0; j < gpuCount; j++)
      {
      (*NvAPI_GPU_GetUsages)(gpuHandles[j], gpuUsages);
      usages[j] = gpuUsages[3];
      std::cout << "GPU Usage" << j << ": " << usages[j] << std::endl;
      }
      Sleep(1000);
      }

      Delete
  2. Hi, this is very useful to me :) thanks!

    btw, how about the video memory usage? can we get the system total video memory usage (for both of dedicated video memory and shared video memory) and the process's video memory usage?

    ReplyDelete
  3. No need to query nvapi.dll dynamically with hardcoded offsets. Everything is there in nvapi.h, just link nvapi.lib and you're good to go.

    The relevant function is NvAPI_GPU_GetDynamicPstatesInfoEx, which returns a NV_GPU_DYNAMIC_PSTATES_INFO_EX. This structure contains the load percentages in its field "utilization".

    ReplyDelete
  4. Will the addresses change in future driver updates?

    ReplyDelete
  5. Hi Chang,
    Nice article
    Since you have mentioned OHM, I thought of checking one issue which I faced sometime back.
    On Windows server 2012, NvAPI_EnumPhysicalGPUs returned device not found. Could you please help me how to go about this issue?

    ReplyDelete
  6. Thank you for your article.
    Trying get fan in precentage. Found it in nvapi.dll NvAPI_GPU_GetCoolerSettings 0xDA141340

    NvAPI_GPU_GetCoolerSettings = (NvAPI_GPU_GetCoolerSettings_t) (*NvAPI_QueryInterface)(0xDA141340);

    Can you help me with this?

    ReplyDelete