SNPE Execution Provider
The SNPE Execution Provider for ONNX Runtime enables hardware accelerated execution on Qualcomm Snapdragon CPU, the Qualcomm AdrenoTM GPU, or the Hexagon DSP. This execution provider makes use of the Qualcomm Snapdragon Neural Processing Engine SDK.
This execution provider uses the AOT converted DLC code as an embedded node in the ONNX model file.
Contents
Install Pre-requisites
Download the SNPE toolkit from the Qualcomm Developer Network for Android/Linux or Windows
SNPE Version Requirements
The SNPE version used with the ONNX Runtime SNPE Execution Provider must match the version used to generate the quantized SNPE-DLC file.
Build
For build instructions, please see the BUILD page.
Configuration Options
The SNPE Execution Provider supports a number of options to set the SNPE Runtime configuration for executing the model. The provider_option_keys, provider_options_values and num_keys enable different options for the application. Each provider_options_keys accepts values as shown below:
provider_options_values for provider_options_keys = "runtime" | Description |
|---|---|
| CPU or CPU_FLOAT32 | Using SnapDragon CPU with 32 bit data storage and math |
| DSP or DSP_FIXED8_TF | Using Hexagon DSP with 8bit fixed point Tensorflow style format data storage and 8bit fixed point Tensorflow style format math |
| GPU or GPU_FLOAT32_16_HYBRID | Using Adreno GPU with 16 bit data storage and 32 bit math |
| GPU_FLOAT16 | GPU with 16 bit data storage and 16 bit math |
| AIP_FIXED_TF or AIP_FIXED8_TF | Using Snapdragon AIX+HVX with 8bit fixed point Tensorflow style format data storage and 8bit fixed point Tensorflow style format math |
provider_options_values for provider_options_keys = "buffer_type" | Description |
|---|---|
| ITensor | Represents a tensor with n-dimensional data |
| TF8 | User defined buffer with 8-bit quantized value |
| TF16 | User defined buffer with 16-bit quantized value |
| UINT8 | User defined buffer with unsigned int value |
| FLOAT | User defined buffer with float value |
Usage
C++
Ort::Env env = Ort::Env{ORT_LOGGING_LEVEL_ERROR, "Default"};
std::unordered_map<std::string, std::string> snpe_options;
snpe_options["runtime"] = "DSP";
snpe_options["buffer_type"] = "FLOAT";
Ort::SessionOptions session_options;
session_options.AppendExecutionProvider("SNPE", snpe_options);
Ort::Session session(env, model_path, session_options);
The C API details are here.
Inference example
Image classification with Inception v3 in CPP using SNPE Execution Provider
Image classification with VGG16 in C# using SNPE Execution Provider