QNN Environment Setup¶

Overview¶

This section describes how to set up the QNN development environment, following the official QNN documentation. For more details, see: QNN Linux Setup.

Prerequisites¶

The QNN backend relies on two main SDKs:

Qualcomm QNN SDK: Required for QNN backend compilation
Hexagon SDK: Required for QNN custom operator(LLaMAOpPackage in mllm) compilation

Version Requirements¶

QNN: Linux v2.34+
Hexagon SDK: Linux 6.x (For Hexagon SDK 5.x, refer to v1 branch for correct makefile Hexagon SDK Tool version)

Warning

Some accounts may not have permission to access the Hexagon SDK and may need to contact Qualcomm for support.

SDK Download and Installation¶

QNN SDK Installation¶

Download the QNN SDK from the official Qualcomm website
Unzip the downloaded file
Set the environment variable QNN_SDK_ROOT to point to the unzipped directory

Hexagon SDK Installation¶

The Hexagon SDK is Qualcomm’s official development environment for programming and optimizing applications on the Hexagon DSP — the core processor architecture used in Snapdragon chips for efficient, low-power computation.

By installing and sourcing the Hexagon SDK, developers can build the custom op package, which is the LLaMAOpPackage in this project, enabling HVX capabilities.

To install the Hexagon SDK, follow these steps:

Download the Hexagon SDK using QPM (Qualcomm Package Manager)
Install the SDK following the QPM instructions

Note

If you encounter ‘Login Failed’ when using qpm-cli, check Qualcomm’s agreements at <https://www.qualcomm.com/agreements>_ and update your account agreements.

Environment Setup¶

After downloading and installing both SDKs, set up the environment by running the following commands:

# Set up QNN SDK environment
source <path-to-qnn-sdk>/bin/envsetup.sh

# Set up Hexagon SDK environment
source <path-to-hexagon-sdk>/setup_sdk_env.source

Environment Variables Verification¶

After setting up the environment, verify that the following environment variables are correctly set:

echo $QNN_SDK_ROOT      # Should point to /path/to/your/qnn/sdk
echo $HEXAGON_SDK_ROOT  # Should point to /path/to/your/hexagon/sdk

Note

These environment variables are essential for the QNN op package compilation process.

Op Package Compilation¶

To use QNN offload, both CPU and HTP QNN op packages are required. The following steps will build the QNN op packages needed by the project.

Prerequisites for Compilation¶

Ensure the following environment variables are set:

QNN_SDK_ROOT
HEXAGON_SDK_ROOT
ANDROID_NDK_ROOT

Compilation Commands¶

cd mllm/src/backends/qnn/LLaMAOpPackageHtp/LLaMAPackage/
make htp_aarch64 && make htp_v75

This will build the necessary QNN op packages for both AArch64 and HVX v75 targets.

Development Tips¶

LSP Configuration for HVX Development¶

To enable Language Server Protocol (LSP) support for HVX development, configure clangd to use the Hexagon toolchain:

Create or edit .vscode/settings.json in your project root
Add the following configuration:

{
  "clangd.path": "$HEXAGON_SDK_ROOT/tools/HEXAGON_Tools/8.7.06/Tools/bin/hexagon-clangd"
}

Generating Compilation Database¶

To generate the compile_commands.json file for the Op package:

cd mllm/src/backends/qnn/LLaMAOpPackageHtp/LLaMAPackage/
compiledb make htp_v75 -C .

This compilation database is useful for IDE features like code completion and error highlighting.

Next Steps¶

After completing the environment setup, you can proceed to:

Model conversion and quantization
Building the project with QNN backend
Running QNN-accelerated models

For detailed instructions on these steps, refer to the respective documentation sections.