Migrating Python Applications from NCAPI v1 to NCAPI v2
Complete Python API documentation can be found here:
| NCAPI v2 Documentation | NCAPI v1 Documentation |
|---|---|
| Python API v2 | Python API v1 |
FIFO queues
In the Intel® Movidius™ Neural Compute SDK (Intel® Movidius NCSDK) v1, the Graph class was used to load an input tensor to a neural compute device and to read the output inference result.
NCAPI v2 introduced the Fifo class for input/output FIFO queues. This allows multiple inputs to be queued for inference and allows multiple graphs to read from the same input queue. Fifo usage is required by the NCAPI v2.
Typical Fifo usage through the NCAPI is as follows:
- Allocate a graph and create and allocate input and output Fifos with Graph.allocate_with_fifos().
- Add input tensors to the input FIFO and queue an inference with Graph.queue_inference_with_fifo_elem().
- Read the inference result from the output Fifo with Fifo.read_elem().
- After usage, free Fifo resources with Fifo.destroy().
See the Example Code Comparison section below for example Fifo usage.
Input/output data types
In NCAPI v1, input data for inferences had to be 16 bit floating point data type.
In NCAPI v2, input data for inferences can be 16 bit or 32 bit floating point data type. The default data type is FP32. The data type is set for each Fifo using Fifo.set_option() with FifoDataType.
Note: If the input FifoDataType is configured to FP32, the API will convert the data to the FP16 data type automatically when performing inferences. If the output FifoDataType is configured to 32FP, the API will convert the output back to the FP32 data type.
fifo.set_option(mvncapi.FifoOption.RW_DATA_TYPE, mvncapi.FifoDataType.FP32)
Other changes
Most class and function names and signatures were updated. Please reference the NCAPI v2 documentation for correct usage.
Equivalent API Calls
Enumerations
| NCAPI v1 | NCAPI v2 | Notes |
|---|---|---|
| Status | Status | |
| DeviceOption | DeviceOption | There are new options in NCAPI v2. |
| GlobalOption | GlobalOption | There are new options in NCAPI v2. |
| GraphOption | GraphOption | There are new options in NCAPI v2. |
| n/a | FifoOption | This is a new enumeration in NCAPI v2 containing options for the new Fifo class. |
| n/a | DeviceState | This is a new enumeration in NCAPI v2 containing possible Device states. |
| n/a | GraphState | This is a new enumeration in NCAPI v2 containing possible Graph states. |
| n/a | FifoState | This is a new enumeration in NCAPI v2 containing possible Fifo states. |
| n/a | FifoType | This is a new enumeration in NCAPI v2 containing possible Fifo types (i.e. read-only, write-only, etc.). |
| n/a | FifoDataType | This is a new enumeration in NCAPI v2 containing possible Fifo data types. |
| n/a | DeviceHwVersion | This is a new enumeration in NCAPI v2 containing neural compute device hardware versions. |
| n/a | LogLevel | This is a new enumeration in NCAPI v2 containing application logging levels. |
Structures
| NCAPI v1 | NCAPI v2 | Notes |
|---|---|---|
| n/a | TensorDescriptor | This is a Structure class that describes graph inputs and outputs. Use Graph.get_option() for GraphOption.INPUT_TENSOR_DESCRIPTORS and GraphOption.OUTPUT_TENSOR_DESCRIPTORS to get this data. |
Global Functions
| NCAPI v1 | NCAPI v2 | Notes |
|---|---|---|
| EnumerateDevices() | enumerate_devices() | |
| SetGlobalOption() | global_set_option() | |
| GetGlobalOption() | global_get_option() |
Classes
Device
| NCAPI v1 | NCAPI v2 | Notes |
|---|---|---|
| Device | Device | |
| Device.OpenDevice() | Device.open() | |
| Device.AllocateGraph() | Graph() + Graph.allocate() or Graph() + Graph.allocate_with_fifos() |
In NCAPI v2 two calls are required to first initialize a Graph object and then allocate the graph to the device. Graph.allocate_with_fifos() is a convenience function that creates and returns input and output Fifos in addition to calling Graph.allocate(). |
| Device.CloseDevice() | Device.close() + Device.destroy() | |
| Device.GetDeviceOption() | Device.get_option() | |
| Device.SetDeviceOption() | Device.set_option() |
Graph
| NCAPI v1 | NCAPI v2 | Notes |
|---|---|---|
| Graph | Graph | In NCAPI v2 Graph objects must be initialized before being passed to the Graph.allocate() function. |
| Graph.GetResult() | Fifo.read_elem() | In NCAPI v2 inference results are read from an output Fifo that was passed to the Graph.queue_inference() method. |
| Graph.LoadTensor() | Fifo.write_elem() + Graph.queue_inference() or Graph.queue_inference_with_fifo_elem() |
In NCAPI v2 input must be written to a properly initialized and created Fifo that will be passed to the Graph.queue_inference() method for inference. Graph.queue_inference_with_fifo_elem() is a convenience function that combines Fifo.write_elem() and Graph.queue_inference() into one call. |
| Graph.DeallocateGraph() | Graph.destroy() | |
| Graph.GetGraphOption() | Graph.get_option() | |
| Graph.SetGraphOption() | Graph.set_option() |
Fifo (new to NCAPI2)
| NCAPI v1 | NCAPI v2 | Notes |
|---|---|---|
| n/a | Fifo | See the Fifo documentation or the Example Code Comparison section below for more complete examples of Fifo usage. |
| n/a | Fifo.allocate() | This must be called before attempting to read/write to/from the Fifo. |
| n/a | Fifo.destroy() | When done using the Fifo, this must be called to free associated resources. |
| n/a | Fifo.read_elem() | This replaces NCAPI v1’s Graph.GetResult(). |
| n/a | Fifo.write_elem() | This replaces NCAPI v1’s Graph.LoadTensor() when used in conjunction with Graph.queue_inference(). The convenience function Graph.queue_inference_with_fifo_elem() can be used to combine both NCAPI v2 calls into one. |
| n/a | Fifo.get_option() | See FifoOption / FifoType / FifoDataType. |
| n/a | Fifo.set_option() | See FifoOption / FifoType / FifoDataType. |
Example Code Comparison
This section contains basic usage examples for NCAPI v1, NCAPI v2, and NCAPI v2 with convenience functions for comparison.
NCAPI v1:
"""NCAPI v1"""
from mvnc import mvncapi
# Open a device
device_list = mvncapi.EnumerateDevices()
device = mvncapi.Device(device_list[0])
device.OpenDevice()
# Load a graph from file at some GRAPH_FILEPATH
GRAPH_FILEPATH = './graph'
with open(GRAPH_FILEPATH, mode='rb') as f:
graph_buffer = f.read()
# Allocate the graph to the device
graph = device.AllocateGraph(graph_buffer)
# Read and pre-process input (16 bit floating point data type required)
input_tensor = ...
# Load the image to the device and trigger an inference
graph.LoadTensor(input_tensor, 'user object')
# Get the results from the device
output, userobj = graph.GetResult()
# Do something with the results...
# Clean up
graph.DeallocateGraph()
device.CloseDevice()
NCAPI v2:
"""NCAPI v2"""
from mvnc import mvncapi
# Initialize and open a device
device_list = mvncapi.enumerate_devices()
device = mvncapi.Device(device_list[0])
device.open()
# Initialize a graph from file at some GRAPH_FILEPATH
GRAPH_FILEPATH = './graph'
with open(GRAPH_FILEPATH, mode='rb') as f:
graph_buffer = f.read()
graph = mvncapi.Graph('graph1')
# Allocate the graph to the device
graph.allocate(device, graph_buffer)
# Get the graphTensorDescriptor structs (they describe expected graph input/output)
input_descriptors = graph.get_option(mvncapi.GraphOption.INPUT_TENSOR_DESCRIPTORS)
output_descriptors = graph.get_option(mvncapi.GraphOption.OUTPUT_TENSOR_DESCRIPTORS)
# Create input/output Fifos
input_fifo = mvncapi.Fifo('input1', mvncapi.FifoType.HOST_WO)
output_fifo = mvncapi.Fifo('output1', mvncapi.FifoType.HOST_RO)
input_fifo.allocate(device, input_descriptors[0], 2)
output_fifo.allocate(device, output_descriptors[0], 2)
# Read and pre-process input (data type must match input Fifo data type)
input_tensor = ...
# Write the image to the input queue
input_fifo.write_elem(input_tensor, None)
# Queue the inference
graph.queue_inference(input_fifo, output_fifo)
# Get the results from the output queue
output, user_obj = output_fifo.read_elem()
# Do something with the results...
# Clean up
input_fifo.destroy()
output_fifo.destroy()
graph.destroy()
device.close()
device.destroy()
NCAPI v2 with convenience functions:
"""NCAPI v2"""
from mvnc import mvncapi
# Initialize and open a device
device_list = mvncapi.enumerate_devices()
device = mvncapi.Device(device_list[0])
device.open()
# Initialize a graph from file at some GRAPH_FILEPATH
GRAPH_FILEPATH = './graph'
with open(GRAPH_FILEPATH, mode='rb') as f:
graph_buffer = f.read()
graph = mvncapi.Graph('graph1')
# CONVENIENCE FUNCTION:
# Allocate the graph to the device and create input/output Fifos with default options in one call
input_fifo, output_fifo = graph.allocate_with_fifos(device, graph_buffer)
# Read and pre-process input (data type must match input Fifo data type)
input_tensor = ...
# CONVENIENCE FUNCTION:
# Write the image to the input queue and queue the inference in one call
graph.queue_inference_with_fifo_elem(input_fifo, output_fifo, input_tensor, None)
# Get the results from the output queue
output, user_obj = output_fifo.read_elem()
# Do something with the results...
# Clean up
input_fifo.destroy()
output_fifo.destroy()
graph.destroy()
device.close()
device.destroy()