Migrating C/C++ Applications from NCAPI v1 to NCAPI v2

NCAPI v2 Documentation	NCAPI v1 Documentation
C API v2	C API v1

FIFO queues

In the Intel® Movidius™ Neural Compute SDK (Intel® Movidius NCSDK) v1, mvncLoadTensor() was used to load an input tensor to a neural compute device and trigger an inference. mvncGetResult() was used to read the output inference result.

NCAPI v2 introduced FIFO queues for input and output. This allows multiple inputs to be queued for inference and allows multiple graphs to read from the same input queue. FIFO usage is required by the NCAPI v2.

Typical FIFO usage through the API is as follows:

Create pointers to ncFifoHandle_t structs for input and output FIFOs.
Allocate a graph and initialize and allocate the associated FIFOs with ncGraphAllocateWithFifos() or ncGraphAllocateWithFifosEx().
Add input tensors to the input FIFO and queue an inference with ncGraphQueueInferenceWithFifoElem().
Read the inference result from the output FIFO with ncFifoReadElem().
After usage, free FIFO resources with ncFifoDestroy().

See the Example Code Comparison section below for example FIFO usage.

Input/output data types

In NCAPI v1, input data for inferences had to be 16 bit floating point data type.

In NCAPI v2, input data for inferences can be 16 bit or 32 bit floating point data type. The default data type is FP32. The data type is set for each FIFO using ncFifoSetOption() with ncFifoDataType_t or when creating FIFOs with ncGraphAllocateWithFifosEx().

Note: If the input FIFO ncFifoDataType_t is configured to FP32, the API will convert the data to the FP16 data type automatically when performing inferences. If the output FIFO ncFifoDataType_t is configured to 32FP, the API will convert the output back to the FP32 data type.

Other changes

Most function names and signatures were updated. Please reference the NCAPI v2 documentation for correct usage.

Equivalent API Calls

Enumerations

NCAPI v1	NCAPI v2	Notes
mvncStatus	ncStatus_t
mvncDeviceOptions	ncDeviceOption_t	There are new options in NCAPI v2.
mvncGlobalOptions	ncGlobalOption_t	There are new options in NCAPI v2.
mvncGraphOptions	ncGraphOption_t	There are new options in NCAPI v2.
n/a	ncFifoOption_t	This is a new enumeration in NCAPI v2 containing options for FIFO queues.
n/a	ncDeviceState_t	This is a new enumeration in NCAPI v2 containing possible device states.
n/a	ncGraphState_t	This is a new enumeration in NCAPI v2 containing possible graph states.
n/a	ncFifoState_t	This is a new enumeration in NCAPI v2 containing possible FIFO states.
n/a	ncFifoType_t	This is a new enumeration in NCAPI v2 containing possible FIFO types (i.e. read-only, write-only).
n/a	ncFifoDataType_t	This is a new enumeration in NCAPI v2 containing possible FIFO data types.
n/a	ncDeviceHwVersion_t	This is a new enumeration in NCAPI v2 containing device hardware versions.
n/a	ncLogLevel_t	This is a new enumeration in NCAPI v2 containing application logging levels.

Structs

NCAPI v1	NCAPI v2	Notes
n/a	struct ncDeviceHandle_t	This is a new struct in NCAPI v2 that is used to point to a device.
n/a	struct ncGraphHandle_t	This is a new struct in NCAPI v2 that is used to point to a graph.
n/a	struct ncFifoHandle_t	This is a new struct in NCAPI v2 that is used to point to a FIFO.
n/a	struct ncTensorDescriptor_t	This is a new struct in NCAPI v2 that describes graph inputs and outputs.

Functions

NCAPI v1	NCAPI v2	Notes
mvncGetGlobalOption()	ncGlobalGetOption()
mvncSetGlobalOption()	ncGlobalSetOption()
mvncGetDeviceName()	n/a	In NCAPI v2, select a device by zero-based index when calling ncDeviceCreate(). After creating the device handle, you get can get the device name with ncDeviceGetOption().
mvncOpenDevice()	ncDeviceCreate() + ncDeviceOpen()	In NCAPI v2 you must create a pointer to a ncDeviceHandle_t struct and pass it to ncDeviceCreate() and then ncDeviceOpen() before using it in any other functions.
mvncCloseDevice()	ncDeviceClose() + ncDeviceDestroy()
mvncGetDeviceOption()	ncDeviceGetOption()
mvncSetDeviceOption()	ncDeviceSetOption()	Some device options may need to be set before calling ncDeviceOpen().
mvncAllocateGraph()	ncGraphCreate() + ncGraphAllocate() or ncGraphCreate() + ncGraphAllocateWithFifos() or ncGraphCreate() + ncGraphAllocateWithFifosEx()	In NCAPI v2 you must create a pointer to a ncGraphHandle_t struct and pass it to ncGraphCreate() before using it in any other functions.
mvncDeallocateGraph()	ncGraphDestroy()
mvncGetGraphOption()	ncGraphGetOption()
mvncSetGraphOption()	ncGraphSetOption()	Some graph options may need to be set before calling ncGraphAllocate().
mvncLoadTensor()	ncFifoWriteElem() + ncGraphQueueInference() or ncGraphQueueInferenceWithFifoElem()	This replaces NCAPI v1’s mvncLoadTensor() when used in conjunction with ncGraphQueueInference(). The convenience function ncGraphQueueInferenceWithFifoElem() can be used to combine both NCAPI v2 calls into one.
mvncGetResult()	ncFifoReadElem()	The output is read from the output FIFO specified in the call to ncGraphQueueInference().
n/a	ncFifoCreate()	In NCAPI v2 you must create a pointer to an ncFifoHandle_t struct and pass it to ncFifoCreate() and then ncFifoAllocate() before using it in any other functions.
n/a	ncFifoAllocate()
n/a	ncFifoDestroy()
n/a	ncFifoSetOption()	All FIFO options must be set before calling ncFifoAllocate().
n/a	ncFifoGetOption()

Example Code Comparison

NCAPI v1:

// NCAPI v1
#include <stdio.h>
#include <stdlib.h>
#include <mvnc.h>

int main() {
    // You can check that this return code is MVNC_OK after each API function call
    // This is omitted in this example for better readability
    mvncStatus retCode;
    
    // Initialize and open a device
    const unsigned int NAME_SIZE = 100;
    char deviceName[NAME_SIZE];
    void* deviceHandle;
    retCode = mvncGetDeviceName(0, deviceName, NAME_SIZE);
    retCode = mvncOpenDevice(deviceName, &deviceHandle);
    
    // Read a graph from file at some GRAPH_FILEPATH
    // void* graphBuffer = ...
    // unsigned int graphBufferLength = ...
    
    // Allocate the graph to the device
    void* graphHandle
    retCode = mvncAllocateGraph(deviceHandle, &graphHandle, graphBuffer, graphBufferLength);
    free(graphBuffer)
    
    // Read and pre-process input (16 bit floating point data type required)
    // void* imageBuffer = ...
    // unsigned int imageBufferLength = ...
    
    # Load the image to the device
    retCode = mvncLoadTensor(graphHandle, imageBuffer, imageBufferLength, NULL);
    free(imageBuffer);
    
    // Get the results from the device
    void* resultData;
    void* userParam;
    unsigned int resultDataLength;
    retCode = mvncGetResult(graphHandle, &resultData, &resultDataLength, &userParam);
    
    // Do something with the results...
    
    // Clean up
    retCode = mvncDeallocateGraph(graphHandle);
    graphHandle = NULL;
    retCode = mvncCloseDevice(devHandle);
    deviceHandle = NULL;
}

NCAPI v2:

// NCAPI v2
#include <stdio.h>
#include <stdlib.h>
#include <mvnc.h>

int main() {
    // You can check that this return code is equal to NC_OK after each API function call
    // This is omitted in this example for better readability
    ncStatus_t retCode;
    unsigned int optionSize;

    // Initialize and open a device
    struct ncDeviceHandle_t* deviceHandle;
    retCode = ncDeviceCreate(0, &deviceHandle);
    retCode = ncDeviceOpen(deviceHandle);

    // Load a graph from file
    // void* graphBuffer = ...
    // unsigned int graphBufferLength = ...

    // Initialize and allocate the graph to the device
    struct ncGraphHandle_t* graphHandle;
    retCode = ncGraphCreate("graph1", &graphHandle);
    retCode = ncGraphAllocate(deviceHandle, graphHandle, graphBuffer, graphBufferLength);
    free(graphBuffer);
       
    // Create an input FIFO
    struct ncFifoHandle_t* inputFIFO;
    retCode = ncFifoCreate("input1", NC_FIFO_HOST_WO, &inputFIFO);
    struct ncTensorDescriptor_t inputDescriptor;
    optionSize = sizeof(inputDescriptor);
    ncGraphGetOption(graphHandle, NC_RO_GRAPH_INPUT_TENSOR_DESCRIPTORS, &inputDescriptor, &optionSize);
    retCode = ncFifoAllocate(inputFIFO, deviceHandle, &inputDescriptor, 2);

    // Create an output FIFO    
    struct ncFifoHandle_t* outputFIFO;
    retCode = ncFifoCreate("output1", NC_FIFO_HOST_RO, &outputFIFO);
    struct ncTensorDescriptor_t outputDescriptor;
    optionSize = sizeof(inputDescriptor);
    ncGraphGetOption(graphHandle, NC_RO_GRAPH_OUTPUT_TENSOR_DESCRIPTORS, &outputDescriptor, &optionSize);
    retCode = ncFifoAllocate(outputFIFO, deviceHandle, &outputDescriptor, 2);

    // Read and preprocess input...
    // float* imageBuffer = ...
    // unsigned int imageBufferLength = ...

    // Write the image to the input FIFO    
    retCode = ncFifoWriteElem(inputFIFO, imageBuffer, &imageBufferLength, 0);
    free(imageBuffer);
    
    // Queue the inference
    retCode = ncGraphQueueInference(graphHandle, &inputFIFO, 1, &outputFIFO, 1);

    // Get the results from the output FIFO
    void* result;
    unsigned int fifoOutputSize = 0;
    unsigned int optionDataLen = sizeof(fifoOutputSize);
    ncFifoGetOption(outFifoHandlePtr, NC_RO_FIFO_ELEMENT_DATA_SIZE, &fifoOutputSize, &optionDataLen);
    result = malloc(fifoOutputSize);    
    retCode = ncFifoReadElem(outputFIFO, result, &fifoOutputSize, NULL);

    // Do something with the results...
    free(result);
    
    // Clean up
    retCode = ncFifoDestroy(&inputFIFO);
    retCode = ncFifoDestroy(&outputFIFO);
    retCode = ncGraphDestroy(&graphHandle);
    retCode = ncDeviceClose(deviceHandle);
    retCode = ncDeviceDestroy(&deviceHandle);
}

NCAPI v2 with convenience functions:

// NCAPI v2
#include <stdio.h>
#include <stdlib.h>
#include <mvnc.h>

int main() {
    // You can check that this return code is equal to NC_OK after each API function call
    // This is omitted in this example for better readability
    ncStatus_t retCode;

    // Initialize and open a device
    struct ncDeviceHandle_t* deviceHandle;
    retCode = ncDeviceCreate(0, &deviceHandle);
    retCode = ncDeviceOpen(deviceHandle);

    // Load a graph from file
    // void* graphBuffer = ...
    // unsigned int graphBufferLength = ...
    
    // Initialize the graph
    struct ncGraphHandle_t* graphHandle;
    retCode = ncGraphCreate("graph1", &graphHandle);
       
    // CONVENIENCE FUNCTION: Allocate the graph to the device and create input/output FIFOs with default options
    // Alternatively, use ncGraphAllocateWithFifosEx() to do this with explicit FIFO options
    struct ncFifoHandle_t* inputFIFO;
    struct ncFifoHandle_t* outputFIFO;
    retCode = ncGraphAllocateWithFifos(deviceHandle, graphHandle, graphBuffer, graphBufferLength, &inputFIFO, &outputFIFO);
    free(graphBuffer);

    // Read and preprocess input...
    // float* imageBuffer = ...
    // unsigned int imageBufferLength = ...

    // Write the image to the input FIFO and queue the inference
    retCode = ncGraphQueueInferenceWithFifoElem(
                    graphHandle, &inputFIFO, &outputFIFO,
                    imageBuffer, &imageBufferLength, 0);    
    
    // Get the results from the output FIFO
    void* result;
    unsigned int fifoOutputSize = 0;
    unsigned int optionDataLen = sizeof(fifoOutputSize);
    ncFifoGetOption(outFifoHandlePtr, NC_RO_FIFO_ELEMENT_DATA_SIZE, &fifoOutputSize, &optionDataLen);
    result = malloc(fifoOutputSize);    
    retCode = ncFifoReadElem(outputFIFO, result, &fifoOutputSize, NULL);

    // Do something with the results...
    free(result);

    // Clean up
    retCode = ncFifoDestroy(&inputFIFO);
    retCode = ncFifoDestroy(&outputFIFO);
    retCode = ncGraphDestroy(&graphHandle);
    retCode = ncDeviceClose(deviceHandle);
    retCode = ncDeviceDestroy(&deviceHandle);
}