• NCSDK C API 2.x
• Overview
• Enumerations
  • ncDeviceHwVersion_t
  • ncDeviceOption_t
  • ncDeviceState_t
  • ncFifoDataType_t
  • ncFifoOption_t
  • ncFifoState_t
  • ncFifoType_t
  • ncGlobalOption_t
  • ncGraphOption_t
  • ncGraphState_t
  • ncLogLevel_t
  • ncStatus_t
• Structures
  • ncDeviceHandle_t
  • ncFifoHandle_t
  • ncGraphHandle_t
  • ncTensorDescriptor_t
• Global Functions
  • ncGlobalGetOption()
  • ncGlobalSetOption()
• Device Functions
  • ncDeviceClose()
  • ncDeviceCreate()
  • ncDeviceDestroy()
  • ncDeviceGetOption()
  • ncDeviceOpen()
  • ncDeviceSetOption()
• Fifo Functions
  • ncFifoAllocate()
  • ncFifoCreate()
  • ncFifoDestroy()
  • Fifo.ncFifoGetOption()
  • ncFifoReadElem()
  • ncFifoRemoveElem()
  • ncFifoSetOption()
  • ncFifoWriteElem()
• Graph Functions
  • ncGraphAllocate()
  • ncGraphAllocateWithFifos()
  • ncGraphAllocateWithFifosEx()
  • ncGraphCreate()
  • ncGraphDestroy()
  • ncGraphGetOption()
  • ncGraphQueueInference()
  • ncGraphQueueInferenceWithFifoElem()
  • ncGraphSetOption()

Intel® Movidius™ Neural Compute SDK C API v2

The Intel® Movidius™ Neural Compute SDK (Intel® Movidius™ NCSDK) comes with a C language API (API) that enables developers to create applications in C or C++ that utilize hardware-accelerated Deep Neural Networks via neural compute devices like the Intel® Movidius™ Neural Compute Stick.

The C API is provided as a header file (mvnc.h) and an associated library file (libmvnc.so), both of which are placed on the development computer when the NCSDK is installed.

Documentation

C API Overview

1. Import the NCAPI

Import the mvnc.h header and anything else that you need.

#include <stdio.h>
#include <stdlib.h>
#include <mvnc.h>

int main(void) {
    // Every NCAPI function will return a status code that should normally be NC_OK
    ncStatus_t retCode;
...

You can get and set application-level information and options with ncGlobalGetOption() and ncGlobalSetOption().

2. Set up a neural compute device

The ncDeviceHandle_t struct is used to represent a neural compute device throughout the NCAPI. The NCAPI provides functions to communicate with the device.

Initialize a device handle with ncDeviceCreate().

    // Create a device handle for the first device found (index 0)
    struct ncDeviceHandle_t* deviceHandle;
    retCode = ncDeviceCreate(0, &deviceHandle);
    if (retCode != NC_OK) {
        printf("Error [%d]: Could not create a neural compute device handle.\n", retCode);
        exit(-1);
    }

Boot and initialize the neural compute device and open communication with ncDeviceOpen().

    // Boot the device and open communication
    retCode = ncDeviceOpen(deviceHandle);
    if (retCode != NC_OK) {  
        printf("Error [%d]: Could not open the neural compute device.\n", retCode);
        exit(-1);
    }

You can get information about the device using ncDeviceGetOption() for options in the ncDeviceOption_t enumeration.

*Note: If you are using more than one neural compute device, you must create and open a separate device handle for each.

3. Set up a network graph and associated FIFO queues for the device

The NCSDK requires a neural network graph file compiled with the mvNCCompile NCSDK tool. Many network models from TensorFlow* and Caffe are supported. See Configuring Your Network for the Intel® Movidius™ Neural Compute SDK for more information about preparing your network model for use with the NCSDK.

When you have a compiled graph, load the graph file data to a buffer.

    // Load the graph file into a buffer
    // void* graphFileBuffer = ...
    // unsigned int graphFileLength = ...

The ncGraphHandle_t struct is used to represent a neural network graph throughout the NCAPI. The NCAPI provides functions to perform inferences using the graph.

Initialize the graph handle with a name string. The name string can be anything you like up to NC_MAX_NAME_SIZE characters, or just an empty string. You can read this name later with ncGraphGetOption().

    // Initialize a graph handle
    struct ncGraphHandle_t* graphHandle;
    retCode = ncGraphCreate("graph1", &graphHandle);
    if (retCode != NC_OK) {
        printf("Error [%d]: Could not create a graph handle.\n", retCode);
        // Do clean up...
        exit(-1);
    }

Graph input and output is done with FIFO (first-in, first-out) queues. The ncFifoHandle_t struct is used to represent a FIFO throughout the NCAPI. The NCAPI provides functions for managing FIFO queues.

Create input and output FIFO queues for the graph using default FIFO options and allocate the graph to the device with ncGraphAllocateWithFifos().

    // Allocate the graph to the device and create input and output FIFOs with default options
    struct ncFifoHandle_t* inFifoHandle;
    struct ncFifoHandle_t* outFifoHandle;
    retCode = ncGraphAllocateWithFifos(deviceHandle, graphHandle, graphFileBuffer, 
                                       graphFileLength, &inFifoHandle, &outFifoHandle);
    if (retCode != NC_OK) {
        printf("Error [%d]: Could not allocate graph with FIFOs.\n", retCode);
        // Do clean up...
        exit(-1);
    }
    free(graphFileBuffer);

You can also use ncGraphAllocateWithFifosEx() to explicitly set FIFO options for type, data type, and size. See the ncFifoHandle_t documentation for more information on creating and allocating FIFOs.

You can get information about a graph using ncGraphGetOption() for options in the ncGraphOption_t enumeration. You can get information about a FIFO using ncFifoGetOption() for options in the ncFifoOption_t enumeration.

*Note: You must create and allocate a graph handle for each network graph file that you wish to use. A neural compute device can have more than one graph allocated to it, but each graph can only be allocated to a single device.

4. Get an input tensor

The way that you obtain and pre-process your input tensor will depend on your individual application. OpenCV and GStreamer are popular libraries for loading and manipulating images from file or a camera stream.

Input tensor data must be the data type specified by the NC_RW_FIFO_DATA_TYPE option for the input FIFO. The default is 32-bit floating point, but FIFOs can also be configured to store 16-bit floating point data. See the ncFifoDataType_t.html enumeration.

    // Load an input tensor and do pre-processing...
    // float* imageBuffer = ...
    // unsigned int imageBufferLength = ...

The NCSDK includes example applications that demonstrate image tensor pre-processing for several networks.

5. Perform an inference

Use ncGraphQueueInferenceWithFifoElem() to write the input tensor to your input FIFO and queue it for inference. When the inference is complete the input tensor will be removed from the input FIFO and the result tensor will be placed in the output FIFO. The third parameter is the size of the input tensor in bytes. This must match the expected size for the network. The fourth parameter can be any data that you wish to have associated with this particular tensor when you read the inference result, such as the original tensor or a window handle, or NULL.

    // Write the tensor to the input FIFO and queue an inference
    retCode = ncGraphQueueInferenceWithFifoElem(
                graphHandle, inFifoHandle, outFifoHandle, imageBuffer, &imageBufferLength, NULL);
    if (retCode != NC_OK) {
        printf("Error [%d]: Could not write to the input FIFO and queue an inference.\n", retCode);
        // Do clean up...
        exit(-1);
    }

If the input FIFO is full, this function call will block until there is room to write to the FIFO. You can check how many elements are in the input and output FIFOs with ncFifoGetOption() for NC_RO_FIFO_WRITE_FILL_LEVEL and NC_RO_FIFO_READ_FILL_LEVEL, respectively. Note that the inference will take some amount of time to complete, depending on network model speed and device communication latency, so you may need to wait to see updated levels.

After the inference is complete, you can get the inference result with ncFifoReadElem(). The size of the output tensor will depend on the network definition as well as the output FIFO’s data type. You can calculate the size of the output tensor yourself or use ncFifoGetOption() for NC_RO_FIFO_ELEMENT_DATA_SIZE to get the correct size.

    // Get the size of the output tensor
    unsigned int outFifoElemSize = 0; 
    unsigned int optionSize = sizeof(outFifoElemSize);
    retCode = ncFifoGetOption(outFifoHandle,  NC_RO_FIFO_ELEMENT_DATA_SIZE,
                              &outFifoElemSize, &optionSize);
    if (retCode != NC_OK) {
        printf("Error [%d]: Could not get the output FIFO element data size.\n", retCode);
        // Do clean up...
        exit(-1);
    }
                    
    // Get the output tensor
    float* resultData = (float*) malloc(outFifoElemSize);
    void* userParam;  // this will be set to point to the user-defined data that you passed into ncGraphQueueInferenceWithFifoElem() with this tensor  
    retCode = ncFifoReadElem(outFifoHandle, (void*)resultData, &outFifoElemSize, &userParam);
    if (retCode != NC_OK) {
        printf("Error [%d]: Could not read the result from the ouput FIFO.\n", retCode);
        // Do clean up...
        exit(-1);
    }

You can then use the output result as intended for your particular network model.

6. Clean up

Before closing communication with the device, use ncGraphDestroy() and ncFifoDestroy() to set graph and FIFO handles to NULL and clean up associated memory. The FIFOs must be empty before being destroyed. Then use ncDeviceClose() to close the device and ncDeviceDestroy() to set the device handle to NULL and clean up associated memory.

    // Clean up the FIFOs
    ncFifoDestroy(&inFifoHandle);
    ncFifoDestroy(&outFifoHandle);
    
    // Clean up the graph
    ncGraphDestroy(&graphHandle);

    // Close and clean up the device
    ncDeviceClose(deviceHandle);
    ncDeviceDestroy(&deviceHandle);
    
}