diff --git a/docs/common/ai/qualcomm/_aotgan_qai-app-builder.mdx b/docs/common/ai/qualcomm/_aotgan_qai-app-builder.mdx index f1efc1b11..e586c1800 100644 --- a/docs/common/ai/qualcomm/_aotgan_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_aotgan_qai-app-builder.mdx @@ -3,8 +3,8 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | +| 设备 | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -20,134 +20,71 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` ### 运行脚本 -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- 进入上游示例目录 - + - - - - -- 准备输入图片，这里以以下图片为输入示例 + ```bash + cd qai-appbuilder/samples + ``` - {" "} + -
- - input image -
+- 准备输入数据（示例自带输入文件时可直接运行；也可按脚本参数指定） -
- - mask image -
+
+ + input image +
- 执行推理 ```bash - python3 aotgan/aotgan.py + python3 ComputerVision/Image_Editing/aotgan/aotgan.py --chipset 9075 ``` - ```bash - $ python3 aotgan/aotgan.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 136.6ms [WARNING] Time: Read model file to memory. 21.49 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +- 运行结果示意 + +
+ + mask image +
+
+ + output image +
- 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 205.5ms [WARNING] Time: contextCreateFromBinary. 68.83 - - 205.6ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 207.5ms [WARNING] Time: model_initialize aotgan 207.45 - - 415.4ms [WARNING] Time: model_inference aotgan 154.66 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmpjvwp3wzi.PNG' - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 801.6ms [WARNING] Time: model_destroy aotgan 67.93 - - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmp0z61xxdt.PNG' - ``` +:::tip +首次运行会通过 Qualcomm® AI Hub 自动下载模型，耗时取决于网络。也可在 `samples` 目录使用启动器： - {" "} +```bash +python3 run_inference.py --list +python3 run_inference.py --model aotgan --args "--chipset 9075" +``` -
- - output image -
+::: diff --git a/docs/common/ai/qualcomm/_beit_qai-app-builder.mdx b/docs/common/ai/qualcomm/_beit_qai-app-builder.mdx index 578c8484c..4b9a01239 100644 --- a/docs/common/ai/qualcomm/_beit_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_beit_qai-app-builder.mdx @@ -3,8 +3,8 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | +| 设备 | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -20,107 +20,51 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` ### 运行脚本 -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- 进入上游示例目录 - + - - - + ```bash + cd qai-appbuilder/samples + ``` -- 准备输入图片，这里以以下图片为输入示例 + - {" "} +- 准备输入数据（示例自带输入文件时可直接运行；也可按脚本参数指定） -
- - input image -
+
+ + input image +
- 执行推理 ```bash - python3 beit/beit.py + python3 ComputerVision/Image_Classification/beit/beit.py --chipset 9075 ``` - ```bash - $ python3 beit/beit.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 179.0ms [WARNING] Time: Read model file to memory. 60.05 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 284.7ms [WARNING] Time: contextCreateFromBinary. 105.62 - - 284.8ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 288.1ms [WARNING] Time: model_initialize beit 288.01 - - 365.3ms [WARNING] Time: model_inference beit 17.72 - - Top 5 predictions for image: - - "Samoyed", 0.6322221756 - "Pomeranian", 0.1098636836 - "Keeshond", 0.0410530604 - "Japanese Chin", 0.0093044275 - "Chow Chow", 0.0058453484 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +:::tip +首次运行会通过 Qualcomm® AI Hub 自动下载模型，耗时取决于网络。也可在 `samples` 目录使用启动器： - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 483.8ms [WARNING] Time: model_destroy beit 113.14 - ``` +```bash +python3 run_inference.py --list +python3 run_inference.py --model beit --args "--chipset 9075" +``` - 打印结果可以看到 `Samoyed` 置信度最高，这与输入图片内容吻合。 +::: diff --git a/docs/common/ai/qualcomm/_convnext_base_qai-app-builder.mdx b/docs/common/ai/qualcomm/_convnext_base_qai-app-builder.mdx deleted file mode 100644 index 42f562dfb..000000000 --- a/docs/common/ai/qualcomm/_convnext_base_qai-app-builder.mdx +++ /dev/null @@ -1,140 +0,0 @@ -此文档讲述使用 [QAI AppBuilder](../qai-appbuilder) Python API 利用 Qualcomm® Hexagon™ Processor (NPU) 推理 [ConvNext-Base](https://aihub.qualcomm.com/models/convnext_base) -目标识别模型。 - -**示例支持设备** - -| 设备 | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## 安装 QAI AppBuilder - -:::tip - -1. 请根据 [**QAI AppBuilder 安装方法**](../qai-appbuilder) 安装 QAI AppBuilder。 - -2. 请根据 [**配置 QAIRT 环境变量**](../qai-appbuilder#配置-qairt-环境变量) 配置 QAIRT 环境变量。 - ::: - -## 运行示例 - -### 安装依赖 - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### 运行脚本 - -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- 准备输入图片，这里以以下图片为输入示例 - - {" "} - -
- - input image -
- -- 执行推理 - - - - ```bash - python3 convnext_base/convnext_base.py - ``` - - - - ```bash - $ python3 convnext_base/convnext_base.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/convnext_base - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 193.9ms [WARNING] Time: Read model file to memory. 71.69 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 269.5ms [WARNING] Time: contextCreateFromBinary. 75.47 - - 269.5ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 270.9ms [WARNING] Time: model_initialize convnext_base 270.85 - - 348.3ms [WARNING] Time: model_inference convnext_base 30.17 - - Top 5 predictions for image: - - Samoyed 0.8726790547 - Pomeranian 0.0192712061 - keeshond 0.0150661934 - Japanese spaniel 0.0032834315 - Eskimo dog 0.0020474677 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 365.5ms [WARNING] Time: model_destroy convnext_base 14.44 - ``` - - 打印结果可以看到 `Samoyed` 置信度最高，这与输入图片内容吻合。 diff --git a/docs/common/ai/qualcomm/_convnext_tiny_qai-app-builder.mdx b/docs/common/ai/qualcomm/_convnext_tiny_qai-app-builder.mdx deleted file mode 100644 index 0a8cb4b20..000000000 --- a/docs/common/ai/qualcomm/_convnext_tiny_qai-app-builder.mdx +++ /dev/null @@ -1,140 +0,0 @@ -此文档讲述使用 [QAI AppBuilder](../qai-appbuilder) Python API 利用 Qualcomm® Hexagon™ Processor (NPU) 推理 [ConvNext-Tiny](https://aihub.qualcomm.com/models/convnext_tiny) -目标识别模型。 - -**示例支持设备** - -| 设备 | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## 安装 QAI AppBuilder - -:::tip - -1. 请根据 [**QAI AppBuilder 安装方法**](../qai-appbuilder) 安装 QAI AppBuilder。 - -2. 请根据 [**配置 QAIRT 环境变量**](../qai-appbuilder#配置-qairt-环境变量) 配置 QAIRT 环境变量。 - ::: - -## 运行示例 - -### 安装依赖 - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### 运行脚本 - -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- 准备输入图片，这里以以下图片为输入示例 - - {" "} - -
- - input image -
- -- 执行推理 - - - - ```bash - python3 convnext_tiny/convnext_tiny.py - ``` - - - - ```bash - $ python3 convnext_tiny/convnext_tiny.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/convnext_tiny - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 132.1ms [WARNING] Time: Read model file to memory. 15.88 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 176.1ms [WARNING] Time: contextCreateFromBinary. 44.01 - - 176.2ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 176.6ms [WARNING] Time: model_initialize convnext_tiny 176.53 - - 245.2ms [WARNING] Time: model_inference convnext_tiny 11.95 - - Top 5 predictions for image: - - Samoyed 0.637629509 - Pomeranian 0.1382825226 - keeshond 0.0217001885 - Arctic fox 0.0138142202 - white wolf 0.0115938596 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 263.1ms [WARNING] Time: model_destroy convnext_tiny 15.29 - ``` - - 打印结果可以看到 `Samoyed` 置信度最高，这与输入图片内容吻合。 diff --git a/docs/common/ai/qualcomm/_depth_anything_qai-app-builder.mdx b/docs/common/ai/qualcomm/_depth_anything_qai-app-builder.mdx index 2387a2d08..5faf10b89 100644 --- a/docs/common/ai/qualcomm/_depth_anything_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_depth_anything_qai-app-builder.mdx @@ -3,8 +3,8 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | +| 设备 | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -20,114 +20,61 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless matplotlib ``` ### 运行脚本 -- 进入示例目录 - - - - - - +- 进入上游示例目录 - ```bash - cd ai-engine-direct-helper/samples/python - ``` + - - - - - + ```bash + cd qai-appbuilder/samples + ``` -- 准备输入图片，这里以以下图片为输入示例 + - {" "} +- 准备输入数据（示例自带输入文件时可直接运行；也可按脚本参数指定） -
- - input image -
+
+ + input image +
- 执行推理 ```bash - python3 depth_anything/depth_anything.py + python3 ComputerVision/Depth_Estimation/depth_anything/depth_anything.py --chipset 9075 ``` - ```bash - $ python3 depth_anything/depth_anything.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 167.0ms [WARNING] Time: Read model file to memory. 37.39 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +- 运行结果示意 - 0.0ms [WARNING] This META does not have Alloc2 Support +
+ + output image +
- 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 329.2ms [WARNING] Time: contextCreateFromBinary. 162.14 - - 329.3ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 332.1ms [WARNING] Time: model_initialize depth_anything 332.05 - - 576.7ms [WARNING] Time: model_inference depth_anything 145.90 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 877.6ms [WARNING] Time: model_destroy depth_anything 85.31 +:::tip +首次运行会通过 Qualcomm® AI Hub 自动下载模型，耗时取决于网络。也可在 `samples` 目录使用启动器： - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmp_3i4qa54.PNG' - ``` +```bash +python3 run_inference.py --list +python3 run_inference.py --model depth_anything --args "--chipset 9075" +``` -
- - output image -
+::: diff --git a/docs/common/ai/qualcomm/_easy_ocr_qai-app-builder.mdx b/docs/common/ai/qualcomm/_easy_ocr_qai-app-builder.mdx index a82d98dd9..11173451a 100644 --- a/docs/common/ai/qualcomm/_easy_ocr_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_easy_ocr_qai-app-builder.mdx @@ -3,8 +3,8 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | +| 设备 | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -20,93 +20,78 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless easyocr ``` ### 运行脚本 -- 进入示例目录 - - +- 进入上游示例目录 - + - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` + ```bash + cd qai-appbuilder/samples + ``` - + - +- 准备输入数据（示例自带 `ch_en.png` / `english.png`；也可通过 `--Image_Path` 指定） - +
+ + input image +
-- 准备输入图片，这里以以下图片为输入示例 +:::warning +在 AIRbox Q900（QCS9075）上实测：上游 `easy_ocr.py` 里 `DataLoader(..., num_workers=1)` 会 **fork 子进程**，导致 QNN HTP context 被释放，识别阶段报错： - {" "} +```text +RuntimeError: Cannot perform 'Inference': context 'EasyOCRRecognizer' has been released. +``` -
- - input image -
+运行前请将 `Multimodal/Image_To_Text/easy_ocr/easy_ocr.py` 中所有 `num_workers=int(workers)` 改为 **`num_workers=0`**（或把 `recognizer_get_text` 的 `workers` 默认值改为 `0`）。 +::: - 执行推理 ```bash - python3 easy_ocr/easy_ocr.py + python3 Multimodal/Image_To_Text/easy_ocr/easy_ocr.py --chipset 9075 ``` - ```bash - $ python3 easy_ocr/easy_ocr.py - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - Calling EasyOCR_Detector::Inference on NPU - Calling EasyOCR_Recognizer::Inference on NPU - Calling EasyOCR_Recognizer::Inference on NPU - Calling EasyOCR_Recognizer::Inference on NPU - Calling EasyOCR_Recognizer::Inference on NPU - Calling EasyOCR_Recognizer::Inference on NPU - Calling EasyOCR_Recognizer::Inference on NPU - Calling EasyOCR_Recognizer::Inference on NPU - Calling EasyOCR_Recognizer::Inference on NPU - 白日依山尽 - 黄河入海流 - 欲穷千里目 - 更上一层楼 - The sun beyond the mountain glows, - The Yellow River seawards flows_ - You can enjoy a grander sight, - By climbing to a greater height。 - - Demo image display is disabled by default for remote servers. To override, set `QAIHM_ALWAYS_DISPLAY_OUTPUT=1` in your environment. - - Saving image to /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/samples/python/build/image.png - Logs will be sent to the system's default channel - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - ``` +成功时终端会打印识别到的中英文文本，例如（默认中英对照输入图）： + +```text +白日依山尽 +黄河入海流 +欲穷千里目 +更上一层楼 +The sun beyond the mountain glows, +The Yellow River seawards flows_ +You can enjoy a grander sight, +By climbing to a greater height。 +Saving image to .../samples/build/image.png +``` - 识别结果 +:::tip +首次运行会通过 Qualcomm® AI Hub 下载 Detector / Recognizer（含中英）模型，体积较大。也可在 `samples` 目录使用启动器（仍需先完成 `num_workers=0` 修改）： - ```bash - 白日依山尽 - 黄河入海流 - 欲穷千里目 - 更上一层楼 - The sun beyond the mountain glows, - The Yellow River seawards flows_ - You can enjoy a grander sight, - By climbing to a greater height。 - ``` +```bash +python3 run_inference.py --list +python3 run_inference.py --model easy_ocr --args "--chipset 9075" +``` + +::: diff --git a/docs/common/ai/qualcomm/_efficientnet_b0_qai-app-builder.mdx b/docs/common/ai/qualcomm/_efficientnet_b0_qai-app-builder.mdx deleted file mode 100644 index 57413e03b..000000000 --- a/docs/common/ai/qualcomm/_efficientnet_b0_qai-app-builder.mdx +++ /dev/null @@ -1,140 +0,0 @@ -此文档讲述使用 [QAI AppBuilder](../qai-appbuilder) Python API 利用 Qualcomm® Hexagon™ Processor (NPU) 推理 [EfficientNet-B0](https://aihub.qualcomm.com/models/efficientnet_b0) -目标识别模型。 - -**示例支持设备** - -| 设备 | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## 安装 QAI AppBuilder - -:::tip - -1. 请根据 [**QAI AppBuilder 安装方法**](../qai-appbuilder) 安装 QAI AppBuilder。 - -2. 请根据 [**创建 ADSP 环境变量**](../qai-appbuilder#创建-adsp-环境变量) 配置 ADSP 环境变量。 - ::: - -## 运行示例 - -### 安装依赖 - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### 运行脚本 - -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- 准备输入图片，这里以以下图片为输入示例 - - {" "} - -
- - input image -
- -- 执行推理 - - - - ```bash - python3 efficientnet_b0/efficientnet_b0.py - ``` - - - - ```bash - $ python3 efficientnet_b0/efficientnet_b0.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/efficientnet_b0 - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 122.3ms [WARNING] Time: Read model file to memory. 5.33 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 143.3ms [WARNING] Time: contextCreateFromBinary. 20.87 - - 143.3ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 143.7ms [WARNING] Time: model_initialize efficientnet_b0 143.64 - - 214.6ms [WARNING] Time: model_inference efficientnet_b0 6.08 - - Top 5 predictions for image: - - Samoyed 0.6965298057 - Pomeranian 0.1052966565 - keeshond 0.0131986532 - Eskimo dog 0.0075131184 - Arctic fox 0.0056676473 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 235.9ms [WARNING] Time: model_destroy efficientnet_b0 18.66 - ``` - - 打印结果可以看到 `Samoyed` 置信度最高，这与输入图片内容吻合。 diff --git a/docs/common/ai/qualcomm/_efficientnet_b4_qai-app-builder.mdx b/docs/common/ai/qualcomm/_efficientnet_b4_qai-app-builder.mdx deleted file mode 100644 index 6389a7721..000000000 --- a/docs/common/ai/qualcomm/_efficientnet_b4_qai-app-builder.mdx +++ /dev/null @@ -1,141 +0,0 @@ -此文档讲述使用 [QAI AppBuilder](../qai-appbuilder) Python API 利用 Qualcomm® Hexagon™ Processor (NPU) 推理 [EfficientNet-B4](https://aihub.qualcomm.com/models/efficientnet_b4) -目标识别模型。 - -**示例支持设备** - -| 设备 | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## 安装 QAI AppBuilder - -:::tip - -1. 请根据 [**QAI AppBuilder 安装方法**](../qai-appbuilder) 安装 QAI AppBuilder。 - -2. 请根据 [**创建 ADSP 环境变量**](../qai-appbuilder#创建-adsp-环境变量) 配置 ADSP 环境变量。 - ::: - -## 运行示例 - -### 安装依赖 - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### 运行脚本 - -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- 准备输入图片，这里以以下图片为输入示例 - - {" "} - -
- - input image -
- -- 执行推理 - - - - ```bash - python3 efficientnet_b4/efficientnet_b4.py - ``` - - - - ```bash - $ python3 efficientnet_b4/efficientnet_b4.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/efficientnet_b4 - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 134.2ms [WARNING] Time: Read model file to memory. 15.52 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 197.4ms [WARNING] Time: contextCreateFromBinary. 63.07 - - 197.4ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 197.8ms [WARNING] Time: model_initialize efficientnet_b4 197.73 - - 271.4ms [WARNING] Time: model_inference efficientnet_b4 12.60 - - Top 5 predictions for image: - - Samoyed 0.9568610787 - Pomeranian 0.0066686491 - keeshond 0.0054634647 - chow 0.0010403089 - Eskimo dog 0.0007608762 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 287.8ms [WARNING] Time: model_destroy efficientnet_b4 13.65 - - ``` - - 打印结果可以看到 `Samoyed` 置信度最高，这与输入图片内容吻合。 diff --git a/docs/common/ai/qualcomm/_efficientnet_v2s_qai-app-builder.mdx b/docs/common/ai/qualcomm/_efficientnet_v2s_qai-app-builder.mdx deleted file mode 100644 index b11c0c413..000000000 --- a/docs/common/ai/qualcomm/_efficientnet_v2s_qai-app-builder.mdx +++ /dev/null @@ -1,140 +0,0 @@ -此文档讲述使用 [QAI AppBuilder](../qai-appbuilder) Python API 利用 Qualcomm® Hexagon™ Processor (NPU) 推理 [EfficientNet-V2-s](https://aihub.qualcomm.com/models/efficientnet_v2_s) -目标识别模型。 - -**示例支持设备** - -| 设备 | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## 安装 QAI AppBuilder - -:::tip - -1. 请根据 [**QAI AppBuilder 安装方法**](../qai-appbuilder) 安装 QAI AppBuilder。 - -2. 请根据 [**创建 ADSP 环境变量**](../qai-appbuilder#创建-adsp-环境变量) 配置 ADSP 环境变量。 - ::: - -## 运行示例 - -### 安装依赖 - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### 运行脚本 - -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- 准备输入图片，这里以以下图片为输入示例 - - {" "} - -
- - input image -
- -- 执行推理 - - - - ```bash - python3 efficientnet_v2_s/efficientnet_v2_s.py - ``` - - - - ```bash - $ python3 efficientnet_v2_s/efficientnet_v2_s.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/efficientnet_v2_s - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 129.0ms [WARNING] Time: Read model file to memory. 11.18 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 158.9ms [WARNING] Time: contextCreateFromBinary. 29.89 - - 159.0ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 159.4ms [WARNING] Time: model_initialize efficientnet_v2_s 159.34 - - 227.4ms [WARNING] Time: model_inference efficientnet_v2_s 10.06 - - Top 5 predictions for image: - - Samoyed 0.7607400417 - keeshond 0.0230328161 - Pomeranian 0.0204458293 - white wolf 0.0193288065 - Arctic fox 0.0062915389 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 239.8ms [WARNING] Time: model_destroy efficientnet_v2_s 9.40 - ``` - - 打印结果可以看到 `Samoyed` 置信度最高，这与输入图片内容吻合。 diff --git a/docs/common/ai/qualcomm/_face_attrib_net_qai-app-builder.mdx b/docs/common/ai/qualcomm/_face_attrib_net_qai-app-builder.mdx index d87d10c01..337243073 100644 --- a/docs/common/ai/qualcomm/_face_attrib_net_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_face_attrib_net_qai-app-builder.mdx @@ -3,8 +3,8 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | +| 设备 | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -20,253 +20,54 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` ### 运行脚本 -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- 进入上游示例目录 - + - - - + ```bash + cd qai-appbuilder/samples + ``` -- 准备输入图片，这里以以下图片为输入示例 + - {" "} +- 准备输入数据（示例自带输入文件时可直接运行；也可按脚本参数指定） -
- - input image -
+
+ + input image +
- 执行推理 ```bash - python3 face_attrib_net/face_attrib_net.py --image ./face_attrib_net/test.jpeg + python3 ComputerVision/Face_Recognition/face_attrib_net/face_attrib_net.py --chipset 9075 ``` - ```bash - $ python3 face_attrib_net/face_attrib_net.py --image ./face_attrib_net/test.jpeg - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 138.8ms [WARNING] Time: Read model file to memory. 17.12 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 171.9ms [WARNING] Time: contextCreateFromBinary. 33.00 - - 172.0ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 173.7ms [WARNING] Time: model_initialize face_attrib_net 173.70 - - 206.7ms [WARNING] Time: model_inference face_attrib_net 2.68 - - Model outputs are saved at: /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/samples/python/face_attrib_net/build/output.json - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 304.2ms [WARNING] Time: model_destroy face_attrib_net 91.15 - ``` +:::tip +首次运行会通过 Qualcomm® AI Hub 自动下载模型，耗时取决于网络。也可在 `samples` 目录使用启动器： -- 识别结果 +```bash +python3 run_inference.py --list +python3 run_inference.py --model face_attrib_net --args "--chipset 9075" +``` - ```json - { - "id_feature": [ - -131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -110208.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, -65472.00390625, - -131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, -131008.0078125, 85824.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -9096.0009765625, -131008.0078125, 131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, 73472.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - -87232.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, -109696.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -31408.001953125, -131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -50720.00390625, - 131008.0078125, 131008.0078125, 95104.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - -131008.0078125, -68864.0078125, -131008.0078125, -28160.001953125, - 131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, -629.5000610351562, 131008.0078125, - -131008.0078125, 131008.0078125, 116608.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, -102656.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - -28240.001953125, 131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, -39840.00390625, -131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -40992.00390625, -131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, 111232.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, 73920.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, 131008.0078125, - 131008.0078125, 65184.00390625, -131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -129472.0078125, -131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125 - ], - "liveness_feature": [ - 10344.0009765625, 22816.001953125, -4112.00048828125, -13632.0009765625, - 8156.00048828125, -15176.0009765625, 7100.00048828125, 19952.001953125, - 7816.00048828125, 17936.001953125, -2642.000244140625, -3474.000244140625, - -1823.0001220703125, 4340.00048828125, 8304.0009765625, 6308.00048828125, - 25936.001953125, -5192.00048828125, 7112.00048828125, 3862.000244140625, - 12288.0009765625, 2606.000244140625, -11192.0009765625, -8352.0009765625, - 8132.00048828125, 19664.001953125, -2890.000244140625, 1288.0001220703125, - -7612.00048828125, -19664.001953125, 2094.000244140625, -10056.0009765625 - ], - "eye_closeness": [131008.0078125, -131008.0078125], - "glasses": [76608.0078125, -79488.0078125], - "mask": [-86208.0078125, 67264.0078125], - "sunglasses": [0.9985352158546448, 0.0] - } - ``` +::: diff --git a/docs/common/ai/qualcomm/_facemap_3dmm_qai-app-builder.mdx b/docs/common/ai/qualcomm/_facemap_3dmm_qai-app-builder.mdx index 488820ddf..cc01d1f39 100644 --- a/docs/common/ai/qualcomm/_facemap_3dmm_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_facemap_3dmm_qai-app-builder.mdx @@ -3,8 +3,8 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | +| 设备 | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -20,122 +20,66 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless scikit-image onnxruntime ``` ### 运行脚本 -- 进入示例目录 - - - - - - +- 进入上游示例目录 - ```bash - cd ai-engine-direct-helper/samples/python - ``` + - - - - - + ```bash + cd qai-appbuilder/samples + ``` -- 准备输入图片，这里以以下图片为输入示例 + - {" "} +- 准备输入数据（示例自带输入文件时可直接运行；也可按脚本参数指定） -
- - input image -
+
+ + input image +
- 执行推理 ```bash - python3 facemap_3dmm/facemap_3dmm.py + python3 ComputerVision/Face_Recognition/facemap_3dmm/facemap_3dmm.py --chipset 9075 ``` - ```bash - $ python3 facemap_3dmm/facemap_3dmm.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 120.2ms [WARNING] Time: Read model file to memory. 9.63 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +在 AIRbox Q900 上，示例可能优先加载 ONNX float 模型（需 `onnxruntime`），成功后生成 `output.jpg`，并打印类似 `ONNX model inference time: ... ms`。 - 0.0ms [WARNING] This META does not have Alloc2 Support +- 运行结果示意 - 0.0ms [WARNING] This META does not have Alloc2 Support +
+ + output image +
- 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 138.7ms [WARNING] Time: contextCreateFromBinary. 18.41 - - 138.8ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 140.3ms [WARNING] Time: model_initialize facemap_3dmm 140.23 - - 182.0ms [WARNING] Time: model_inference facemap_3dmm 1.43 - - /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/samples/python/facemap_3dmm/facemap_3dmm.py:145: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at /pytorch/torch/csrc/utils/tensor_new.cpp:253.) - _output = torch.tensor(output) - Saving image to /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/samples/python/facemap_3dmm/output.jpg - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 706.9ms [WARNING] Time: model_destroy facemap_3dmm 72.10 - - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmpkpcumu3y.PNG' - ``` +:::tip +首次运行会通过 Qualcomm® AI Hub 自动下载模型，耗时取决于网络。也可在 `samples` 目录使用启动器： - {" "} +```bash +python3 run_inference.py --list +python3 run_inference.py --model facemap_3dmm --args "--chipset 9075" +``` -
- - output image -
+::: diff --git a/docs/common/ai/qualcomm/_fcn-resnet50_qai-app-builder.mdx b/docs/common/ai/qualcomm/_fcn-resnet50_qai-app-builder.mdx deleted file mode 100644 index f67e4be3d..000000000 --- a/docs/common/ai/qualcomm/_fcn-resnet50_qai-app-builder.mdx +++ /dev/null @@ -1,142 +0,0 @@ -此文档讲述使用 [QAI AppBuilder](../qai-appbuilder) Python API 利用 Qualcomm® Hexagon™ Processor (NPU) 推理 [FCN-ResNet50](https://aihub.qualcomm.com/compute/models/fcn_resnet50) -图像分割模型。 - -**示例支持设备** - -| 设备 | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## 安装 QAI AppBuilder - -:::tip - -1. 请根据 [**QAI AppBuilder 安装方法**](../qai-appbuilder) 安装 QAI AppBuilder。 - -2. 请根据 [**创建 ADSP 环境变量**](../qai-appbuilder#创建-adsp-环境变量) 配置 ADSP 环境变量。 - ::: - -## 运行示例 - -### 安装依赖 - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### 运行脚本 - -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- 准备输入图片，这里以以下图片为输入示例 - -
- - input image -
- -- 执行推理 - - - - ```bash - python3 fcn_resnet50/fcn_resnet50.py --image ./input.jpg - ``` - - - - ```bash - $ python3 fcn_resnet50/fcn_resnet50.py --image ./input.jpg - Current file directory: /mnt/ssd/git_clone/ai-engine-direct-helper/samples/linux/python/fcn_resnet50 - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 127.2ms [WARNING] Time: Read model file to memory. 12.72 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 185.9ms [WARNING] Time: contextCreateFromBinary. 58.58 - - 186.0ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 187.1ms [WARNING] Time: model_initialize fcn_resnet50 187.09 - - 385.4ms [WARNING] Time: model_inference fcn_resnet50 112.79 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 463.2ms [WARNING] Time: model_destroy fcn_resnet50 10.20 - ``` - -- 输出结果 - -
- - output image -
diff --git a/docs/common/ai/qualcomm/_googlenet_qai-app-builder.mdx b/docs/common/ai/qualcomm/_googlenet_qai-app-builder.mdx index 49ff3fabc..29a5766c3 100644 --- a/docs/common/ai/qualcomm/_googlenet_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_googlenet_qai-app-builder.mdx @@ -3,11 +3,11 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | -| Dragon Q6A | QCS6490 | +| 设备 | SoC | +| -------------------- | -------- | +| Dragon Q6A | QCS6490 | | Dragon Q8B | SC8280XP | -| Fogwise® AIRbox Q900 | QCS9075 | +| Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -22,133 +22,95 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` ### 运行脚本 -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` - - - - - - - -- 准备输入图片，这里以以下图片为输入示例 - - {" "} - -
- - input image -
+- 进入上游示例目录 -- 执行推理 - - + ```bash - python3 googlenet/googlenet.py + cd qai-appbuilder/samples ``` - - - ```bash - $ python3 googlenet/googlenet.py - 0.0ms [WARNING] Initializing HtpProvider + - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support +执行推理（Linux 需指定 `--chipset`；首次运行会自动下载测试图 `dog.jpg`、标签与模型） - 0.0ms [WARNING] This META does not have Alloc2 Support +:::tip +若自动下载的 `dog.jpg` 为空或损坏，可用本地有效图片覆盖，或通过 `--image /path/to/image.jpg` 指定输入。 +::: - 0.0ms [WARNING] This META does not have Alloc2 Support + + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 157.6ms [WARNING] Time: Read model file to memory. 8.91 +```bash +python3 ComputerVision/Image_Classification/googlenet/googlenet.py --chipset 6490 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + + - 0.0ms [WARNING] This META does not have Alloc2 Support +:::tip +SC8280XP 与 QCS6490 同为 Hexagon V68，下载模型时使用 `--chipset 6490`。 +::: - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Image_Classification/googlenet/googlenet.py --chipset 6490 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + + - 180.6ms [WARNING] Time: contextCreateFromBinary. 22.97 + - 180.7ms [WARNING] Time: UnmapViewOfFile. 0.00 +```bash +python3 ComputerVision/Image_Classification/googlenet/googlenet.py --chipset 9075 +``` - 181.6ms [WARNING] Time: model_initialize googlenet 181.55 + - 255.7ms [WARNING] Time: model_inference googlenet 2.67 + + - Top 5 predictions for image: +### 预期结果 - Samoyed 0.948613584 - West Highland White Terrier 0.0060515446 - Alaskan tundra wolf 0.0052576731 - Pomeranian 0.0047871661 - Chow Chow 0.0035575195 - 0.0ms [WARNING] This META does not have Alloc2 Support +在 Dragon Q6A（`--chipset 6490`）上，默认测试图输出类似： - 0.0ms [WARNING] This META does not have Alloc2 Support +```text +Top 5 predictions for image: - 0.0ms [WARNING] This META does not have Alloc2 Support +Samoyed 0.9282982349 +West Highland White Terrier 0.0102122389 +Pomeranian 0.0064557223 +Pyrenean Mountain Dog 0.0059806537 +Chow Chow 0.0051328284 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support +日志中可能出现 HTP / FastRPC 相关 WARNING，推理成功时可忽略。 - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 353.7ms [WARNING] Time: model_destroy googlenet 93.43 +:::tip +也可使用启动器： - ``` +```bash +python3 run_inference.py --model googlenet --args "--chipset 6490" +``` - 打印结果可以看到 `Samoyed` 置信度最高，这与输入图片内容吻合。 +::: diff --git a/docs/common/ai/qualcomm/_inception_v3_qai-app-builder.mdx b/docs/common/ai/qualcomm/_inception_v3_qai-app-builder.mdx index ebfc1059b..3521fcb7c 100644 --- a/docs/common/ai/qualcomm/_inception_v3_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_inception_v3_qai-app-builder.mdx @@ -3,11 +3,11 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | -| Dragon Q6A | QCS6490 | +| 设备 | SoC | +| -------------------- | -------- | +| Dragon Q6A | QCS6490 | | Dragon Q8B | SC8280XP | -| Fogwise® AIRbox Q900 | QCS9075 | +| Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -22,132 +22,101 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` ### 运行脚本 -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` +- 进入上游示例目录 - + - - - - - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` - - - - - - - -- 准备输入图片，这里以以下图片为输入示例 + ```bash + cd qai-appbuilder/samples + ``` - {" "} + -
- - input image -
+执行推理（Linux 需指定 `--chipset`；可选用示例自带 `input.jpg`） -- 执行推理 + - + - ```bash - python3 inception_v3/inception_v3.py --image ./inception_v3/input.jpg - ``` + - +```bash +python3 ComputerVision/Image_Classification/inception_v3/inception_v3.py --chipset 6490 +``` - ```bash - $ python3 inception_v3/inception_v3.py --image ./inception_v3/input.jpg - 0.0ms [WARNING] Initializing HtpProvider + - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +:::tip +SC8280XP 与 QCS6490 同为 Hexagon V68，下载模型时使用 `--chipset 6490`。 +::: - 0.0ms [WARNING] This META does not have Alloc2 Support + - 221.6ms [WARNING] Time: Read model file to memory. 17.28 +```bash +python3 ComputerVision/Image_Classification/inception_v3/inception_v3.py --chipset 6490 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Image_Classification/inception_v3/inception_v3.py --chipset 9075 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 258.5ms [WARNING] Time: contextCreateFromBinary. 36.82 + - 258.5ms [WARNING] Time: UnmapViewOfFile. 0.00 +可选指定图片： - 260.2ms [WARNING] Time: model_initialize inceptionV3 260.14 +```bash +python3 ComputerVision/Image_Classification/inception_v3/inception_v3.py --chipset 6490 --image ComputerVision/Image_Classification/inception_v3/input.jpg +``` - 332.1ms [WARNING] Time: model_inference inceptionV3 3.50 +### 预期结果 - Top 5 predictions for image: +在 Dragon Q6A（`--chipset 6490`）上，默认输入图输出类似： - Samoyed 0.9998983145 - Arctic fox 0.000054331 - white wolf 0.0000345349 - Eskimo dog 0.0000035142 - Pomeranian 0.0000030651 - 0.0ms [WARNING] This META does not have Alloc2 Support +```text +Top 5 predictions for image: - 0.0ms [WARNING] This META does not have Alloc2 Support +Samoyed 0.999714911 +white wolf 0.0001396882 +Arctic fox 0.0001165446 +Pomeranian 0.000006424 +Great Pyrenees 0.000006424 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support +日志中可能出现 HTP / FastRPC 相关 WARNING，推理成功时可忽略。 - 0.0ms [WARNING] This META does not have Alloc2 Support +:::tip +也可使用启动器： - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 443.3ms [WARNING] Time: model_destroy inceptionV3 105.74 - ``` +```bash +python3 run_inference.py --model inception_v3 --args "--chipset 6490" +``` - 打印结果可以看到 `Samoyed` 置信度最高，这与输入图片内容吻合。 +::: diff --git a/docs/common/ai/qualcomm/_lama_dilated_qai-app-builder.mdx b/docs/common/ai/qualcomm/_lama_dilated_qai-app-builder.mdx index 0020996bd..1e949af3b 100644 --- a/docs/common/ai/qualcomm/_lama_dilated_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_lama_dilated_qai-app-builder.mdx @@ -3,8 +3,8 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | +| 设备 | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -20,133 +20,71 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` ### 运行脚本 -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- 进入上游示例目录 - + - - - - -- 准备输入图片，这里以以下图片为输入示例 + ```bash + cd qai-appbuilder/samples + ``` - {" "} + -
- - input image -
+- 准备输入数据（示例自带输入文件时可直接运行；也可按脚本参数指定） -
- - mask image -
+
+ + input image +
- 执行推理 ```bash - python3 lama_dilated/lama_dilated.py + python3 ComputerVision/Image_Editing/lama_dilated/lama_dilated.py --chipset 9075 ``` - ```bash - $ python3 lama_dilated/lama_dilated.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 167.7ms [WARNING] Time: Read model file to memory. 39.91 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +- 运行结果示意 + +
+ + mask image +
+
+ + output image +
- 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 255.5ms [WARNING] Time: contextCreateFromBinary. 87.63 - - 255.5ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 257.3ms [WARNING] Time: model_initialize lamadilated 257.28 - - 406.4ms [WARNING] Time: model_inference lamadilated 103.83 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmp0t30nvk9.PNG' - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmpd98hfol8.PNG' - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 978.3ms [WARNING] Time: model_destroy lamadilated 103.66 - ``` +:::tip +首次运行会通过 Qualcomm® AI Hub 自动下载模型，耗时取决于网络。也可在 `samples` 目录使用启动器： - {" "} +```bash +python3 run_inference.py --list +python3 run_inference.py --model lama_dilated --args "--chipset 9075" +``` -
- - output image -
+::: diff --git a/docs/common/ai/qualcomm/_levit_qai-app-builder.mdx b/docs/common/ai/qualcomm/_levit_qai-app-builder.mdx deleted file mode 100644 index 6badd8593..000000000 --- a/docs/common/ai/qualcomm/_levit_qai-app-builder.mdx +++ /dev/null @@ -1,140 +0,0 @@ -此文档讲述使用 [QAI AppBuilder](../qai-appbuilder) Python API 利用 Qualcomm® Hexagon™ Processor (NPU) 推理 [EfficientNet-V2-s](https://aihub.qualcomm.com/models/efficientnet_v2_s) -目标识别模型。 - -**示例支持设备** - -| 设备 | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## 安装 QAI AppBuilder - -:::tip - -1. 请根据 [**QAI AppBuilder 安装方法**](../qai-appbuilder) 安装 QAI AppBuilder。 - -2. 请根据 [**创建 ADSP 环境变量**](../qai-appbuilder#创建-adsp-环境变量) 配置 ADSP 环境变量。 - ::: - -## 运行示例 - -### 安装依赖 - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### 运行脚本 - -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- 准备输入图片，这里以以下图片为输入示例 - - {" "} - -
- - input image -
- -- 执行推理 - - - - ```bash - python3 levit/levit.py - ``` - - - - ```bash - $ python3 levit/levit.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/levit - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 120.8ms [WARNING] Time: Read model file to memory. 4.10 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 147.6ms [WARNING] Time: contextCreateFromBinary. 26.69 - - 147.6ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 148.5ms [WARNING] Time: model_initialize levit 148.48 - - 226.3ms [WARNING] Time: model_inference levit 5.23 - - Top 5 predictions for image: - - Samoyed 0.6532489061 - Pomeranian 0.0412057079 - acorn 0.013004588 - wallaby 0.007310994 - keeshond 0.0034880731 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 239.0ms [WARNING] Time: model_destroy levit 10.01 - ``` - - 打印结果可以看到 `Samoyed` 置信度最高，这与输入图片内容吻合。 diff --git a/docs/common/ai/qualcomm/_mediapipe_hand_qai-app-builder.mdx b/docs/common/ai/qualcomm/_mediapipe_hand_qai-app-builder.mdx index 2deb98ef3..bd23d734b 100644 --- a/docs/common/ai/qualcomm/_mediapipe_hand_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_mediapipe_hand_qai-app-builder.mdx @@ -3,8 +3,8 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | +| 设备 | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -20,153 +20,79 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models pygame +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless pygame ``` -### 运行脚本 - -- 进入示例目录 +脚本还会在 `~/.qaihm` 下查找 [MediaPipePyTorch](https://github.com/zmurez/MediaPipePyTorch)（提供 `blazepalm` 等模块）。若缺失可： - + - +```bash +mkdir -p ~/.qaihm +git clone --depth 1 https://github.com/zmurez/MediaPipePyTorch.git ~/.qaihm/MediaPipePyTorch +``` - + - ```bash - cd ai-engine-direct-helper/samples/python - ``` +### 运行脚本 - +- 进入上游示例目录 - + - + ```bash + cd qai-appbuilder/samples + ``` -- 准备输入图片，这里以以下图片为输入示例 + - {" "} +- 准备输入数据（无摄像头界面时需 GUI；无界面环境请用静态图片） -
- - input image -
+
+ + input image +
- 执行推理 ```bash - python3 mediapipe_hand/mediapipe_hand.py --imagefile ./mediapipe_hand/hand.jpg + python3 ComputerVision/Pose_Estimation/mediapipe_hand/mediapipe_hand.py \ + --chipset 9075 \ + --imagefile /path/to/hand_image.png \ + --displayPredict False \ + --DemoMode False ``` - ```bash - $ python3 mediapipe_hand/mediapipe_hand.py --imagefile ./mediapipe_hand/hand.jpg - /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/.venv/lib/python3.12/site-packages/pygame/pkgdata.py:25: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. - from pkg_resources import resource_stream, resource_exists - pygame 2.6.1 (SDL 2.28.4, Python 3.12.3) - Hello from the pygame community. https://www.pygame.org/contribute.html - Press ESC key to exit if you are opening camera app. - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 110.2ms [WARNING] Time: Read model file to memory. 2.70 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 133.4ms [WARNING] Time: contextCreateFromBinary. 23.04 - - 133.4ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 133.9ms [WARNING] Time: model_initialize hand_detector 133.82 +- 运行结果示意：目录下生成 `output.jpg` - 191.3ms [WARNING] Time: Read model file to memory. 3.24 +
+ + output image +
- 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 204.0ms [WARNING] Time: contextCreateFromBinary. 12.63 - - 204.1ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 205.1ms [WARNING] Time: model_initialize landmark_detector 17.40 - - 242.5ms [WARNING] Time: model_inference hand_detector 2.86 - - 255.1ms [WARNING] Time: model_inference landmark_detector 2.10 - - /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/samples/python/mediapipe_hand/mediapipe_hand.py:906: DeprecationWarning: 'mode' parameter is deprecated and will be removed in Pillow 13 (2026-10-15) - out_image = Image.fromarray(pred_image[0], "RGB") - - Demo image display is disabled by default for remote servers. To override, set `QAIHM_ALWAYS_DISPLAY_OUTPUT=1` in your environment. - - Saving image to /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/samples/python/build/image.png - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 314.1ms [WARNING] Time: model_destroy hand_detector 11.84 - - Logs will be sent to the system's default channel - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 389.4ms [WARNING] Time: model_destroy landmark_detector 75.05 - ``` +:::tip +首次运行会通过 Qualcomm® AI Hub 自动下载 `handdetector` 与 `landmarkdetector` 模型。无界面时请使用 `--imagefile`，并推荐安装 `opencv-python-headless` 以避免 Qt 插件初始化失败。 - {" "} +```bash +python3 run_inference.py --list +python3 run_inference.py --model mediapipe_hand --args "--chipset 9075 --imagefile /path/to/hand_image.png --displayPredict False --DemoMode False" +``` -
- - output image -
+::: diff --git a/docs/common/ai/qualcomm/_nomic_embed_text_qai-app-builder.mdx b/docs/common/ai/qualcomm/_nomic_embed_text_qai-app-builder.mdx index 5409de135..5dfe77097 100644 --- a/docs/common/ai/qualcomm/_nomic_embed_text_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_nomic_embed_text_qai-app-builder.mdx @@ -3,8 +3,8 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | +| 设备 | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -20,219 +20,46 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless transformers ``` ### 运行脚本 -- 进入示例目录 - - - - - - +- 进入上游示例目录 - ```bash - cd ai-engine-direct-helper/samples/python - ``` + - - - + ```bash + cd qai-appbuilder/samples + ``` - + - 执行推理 ```bash - python3 nomic_embed_text/nomic_embed_text.py + python3 Multimodal/Text_Generation/nomic_embed_text/nomic_embed_text.py --chipset 9075 ``` - ```bash - $ python3 nomic_embed_text/nomic_embed_text.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 226.8ms [WARNING] Time: Read model file to memory. 107.40 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 324.1ms [WARNING] Time: contextCreateFromBinary. 97.18 - - 324.1ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 327.1ms [WARNING] Time: model_initialize nomic_embed_text 327.00 - - 343.6ms [WARNING] Time: model_inference nomic_embed_text 10.11 - - Embeddings: - [ 2.41241474e-02 -4.50897263e-03 -2.15209976e-01 -4.53796424e-02 - 1.46770489e-03 4.48608436e-02 -2.05802941e-03 1.30081186e-02 - -2.21405048e-02 -6.17675819e-02 1.77621914e-05 7.71484450e-02 - 5.78918494e-02 5.52978553e-02 2.69470233e-02 -4.53491248e-02 - 3.90319861e-02 -8.25195387e-02 -5.13305701e-02 1.16806040e-02 - -2.47650165e-02 -5.70983924e-02 6.58798264e-03 -6.29806565e-03 - 2.16308609e-01 9.33837984e-03 -3.27453651e-02 1.02417000e-01 - 2.17590351e-02 3.63349938e-03 1.28631601e-02 -1.32064829e-02 - 1.50604257e-02 2.56042499e-02 6.51855543e-02 -3.80249061e-02 - 1.70040142e-03 3.19480919e-03 1.28250131e-02 1.43127451e-02 - 2.92968773e-03 -4.25720261e-03 8.76617525e-03 -5.45043983e-02 - 7.24487379e-02 -1.21612558e-02 -1.37939462e-02 -1.17721567e-02 - 6.06994666e-02 -1.56402607e-02 -3.29895057e-02 -1.77764904e-03 - -3.09295673e-02 3.49731483e-02 3.03497333e-02 5.01403846e-02 - 6.08215369e-02 2.45971698e-02 1.79595966e-02 1.87683124e-02 - 6.84204176e-02 7.29980543e-02 2.01873798e-02 6.91528395e-02 - 1.96990985e-02 -3.41796912e-02 -7.67231046e-04 8.52050856e-02 - 5.60913123e-02 -1.63269062e-02 7.55615309e-02 -2.16979999e-02 - 4.34875526e-02 1.93786640e-02 6.58798264e-03 1.62200946e-02 - 2.09045429e-02 -3.43017615e-02 -4.82482947e-02 -7.37915114e-02 - 5.18493690e-02 -9.25903395e-02 1.05407722e-01 -4.20837440e-02 - 1.24588022e-02 -4.14123572e-02 -1.32904062e-02 7.17163133e-03 - -4.33959998e-02 3.15551795e-02 6.09130897e-02 2.07519550e-02 - -2.68402118e-02 4.03747596e-02 -4.18090858e-02 1.57775898e-02 - -4.78744553e-03 -6.70623826e-03 -4.66308631e-02 -3.60717811e-02 - -9.71222017e-03 4.33349656e-03 -2.19726581e-02 2.30560321e-02 - 7.20214918e-02 4.43115272e-02 -4.73022498e-02 -7.61718825e-02 - -4.11987342e-02 -4.75769080e-02 -3.17077674e-02 3.72924842e-02 - -7.02667283e-03 2.29034442e-02 -1.76544208e-02 5.59234666e-03 - 3.74450721e-02 -8.04443434e-02 3.09295673e-02 7.23877028e-02 - -8.33129957e-02 2.06451435e-02 8.11576901e-04 4.22363319e-02 - 1.62200946e-02 4.13513221e-02 -8.30688551e-02 -3.15246619e-02 - 2.56729149e-03 -3.37829627e-02 -2.52990741e-02 -3.06549091e-02 - -1.01547251e-02 -4.36706580e-02 4.28466834e-02 9.04083345e-03 - -2.00805683e-02 -3.41491736e-02 4.34875526e-02 1.84631366e-02 - 3.17993201e-02 2.14386005e-02 1.50070200e-02 -5.69458045e-02 - -4.06494178e-02 -2.87017841e-02 6.92749098e-02 -3.66516151e-02 - 8.38756619e-04 -4.78744553e-03 -1.58081073e-02 2.99072284e-02 - 2.27508564e-02 3.61633338e-02 1.66473407e-02 -3.07617206e-02 - -5.39855994e-02 -9.80377290e-03 5.27343787e-02 -1.64947528e-02 - 2.93540978e-03 -4.39453162e-02 -8.68530348e-02 1.49841318e-02 - 7.54928635e-03 -9.79003981e-02 4.23812913e-03 3.79333533e-02 - 4.14428748e-02 2.45056171e-02 8.48388765e-03 -6.26831129e-02 - -3.14636268e-02 9.82284662e-04 -2.68707294e-02 -4.19921912e-02 - -1.14517221e-02 -6.64062575e-02 3.82385291e-02 7.20977830e-03 - 7.65991285e-02 -5.41992225e-02 2.29949970e-02 3.59191932e-02 - -2.84881610e-02 -3.38439979e-02 1.17034921e-02 9.36126802e-03 - -4.04663123e-02 -3.29284705e-02 -6.25610426e-02 1.85699482e-02 - -8.16650465e-02 -2.84881610e-02 -3.53088416e-02 -3.37829627e-02 - 1.52816782e-02 6.61850022e-03 5.45043983e-02 -5.33142127e-02 - -3.93066444e-02 -1.22985849e-02 -1.91650409e-02 -7.98034761e-03 - 7.43389188e-04 9.42993239e-02 4.79126014e-02 2.72216816e-02 - 5.44128455e-02 3.80859412e-02 1.03332527e-01 -6.76269606e-02 - -3.86047401e-02 -4.93774451e-02 5.10788010e-03 5.16414701e-04 - 3.11431903e-02 -4.71801795e-02 2.41699237e-02 2.67791767e-02 - 1.48315439e-02 -3.46374549e-02 5.94177283e-02 -1.06201181e-02 - 6.31713942e-02 -2.00958271e-02 -3.12347431e-02 1.44042978e-02 - -1.09252937e-01 2.44712853e-03 -5.78308143e-02 -2.55584735e-02 - 3.06243915e-02 2.87017841e-02 1.12235553e-04 6.24465989e-03 - 2.92663593e-02 4.35180701e-02 -2.05383319e-02 -6.28051832e-02 - -5.83190955e-02 7.96508864e-02 5.04760779e-02 6.17218064e-03 - -8.22753981e-02 3.06701679e-02 -4.98352088e-02 1.56021127e-02 - 1.25732431e-02 -1.12838754e-02 -3.18603553e-02 -2.19268817e-02 - -1.43661508e-02 3.89404334e-02 4.02221717e-02 -3.65600623e-02 - -5.61523475e-02 1.40838632e-02 -3.44238319e-02 -3.15856971e-02 - 4.85534705e-02 -1.01867683e-01 4.33654822e-02 -5.83190955e-02 - -4.94995154e-02 -1.06124887e-02 -6.00891151e-02 1.51214609e-02 - 6.64672926e-02 -6.98852614e-02 -3.58772301e-03 2.17895526e-02 - 3.07617206e-02 3.03955097e-02 -1.13906870e-02 -7.06481980e-03 - 2.38037133e-03 9.26208589e-03 -4.83398475e-02 9.26971529e-03 - -3.26232947e-02 -1.14822397e-02 -8.05664137e-02 -4.54406776e-02 - -1.70135517e-02 4.29992713e-02 5.15136756e-02 3.87268104e-02 - -2.23083515e-02 -2.96630878e-02 3.32946815e-02 1.15509043e-02 - 6.46972703e-03 1.03027351e-01 -5.15136756e-02 1.44805918e-02 - 6.10961951e-02 1.43203745e-02 -7.97119215e-02 -1.64337177e-02 - -3.32641639e-02 2.47497577e-02 3.29284705e-02 -4.26025428e-02 - -2.39105243e-02 1.45950327e-02 1.55792246e-02 -4.04357947e-02 - 7.18383864e-02 2.07824726e-02 -8.57544020e-02 6.94580153e-02 - -9.28344801e-02 4.58908128e-03 -5.43823279e-02 2.66723651e-02 - -1.87835712e-02 5.48095740e-02 9.75341871e-02 2.11181659e-02 - -4.49218787e-02 -5.65490760e-02 -5.21545447e-02 -6.97631910e-02 - 3.87878455e-02 1.12686167e-02 3.94897498e-02 5.50537147e-02 - 2.33612079e-02 -4.33959998e-02 2.44903583e-02 1.21383676e-02 - -4.91638221e-02 -2.73742694e-02 -4.21752967e-02 3.28674354e-02 - -1.45111093e-02 -3.08837909e-02 -1.08566293e-02 8.66088942e-02 - 2.14996357e-02 -4.25720261e-03 5.08308469e-04 -4.74548377e-02 - -2.05230732e-02 -5.14221229e-02 -8.78906343e-03 -5.35965012e-03 - -1.34201059e-02 1.50451669e-02 -1.49536142e-02 4.24499549e-02 - -2.27355976e-02 3.18298377e-02 -5.84125577e-04 -3.09143085e-02 - 7.52258347e-03 6.65283278e-02 2.72979755e-02 -2.58026142e-02 - 2.94303917e-03 3.52859526e-04 3.31420936e-02 9.76562593e-03 - -4.97131385e-02 2.02178978e-03 3.74450721e-02 2.48146080e-03 - 3.79333533e-02 8.08715913e-03 -3.27148475e-02 -2.69317646e-02 - -2.42157001e-02 4.19006385e-02 2.75268573e-02 4.73632850e-02 - -1.00097664e-01 -6.15234412e-02 1.02996835e-02 -2.12249774e-02 - -4.45251502e-02 -6.92138746e-02 4.54406776e-02 -2.92358417e-02 - 2.97927880e-03 6.73828200e-02 4.59899940e-02 2.00805683e-02 - 1.33285532e-02 1.31225598e-03 -7.61032151e-03 -8.88061617e-03 - 2.07061786e-02 1.19972241e-03 -1.21154794e-02 1.90429706e-02 - 5.27343787e-02 2.87017841e-02 -3.32031287e-02 1.87835712e-02 - -2.16217060e-02 -2.12478661e-03 -8.14056490e-03 -1.14501961e-01 - 1.14440927e-02 3.71933007e-03 -2.95410175e-02 -2.03399677e-02 - 3.63159217e-02 -1.47781381e-02 4.47082557e-02 -1.11846933e-02 - 1.22222910e-02 -3.87878455e-02 -7.10296677e-03 2.98004169e-02 - 3.20129432e-02 -6.11572303e-02 -3.82995643e-02 1.98974628e-02 - 8.17871168e-02 6.12487830e-02 3.95202674e-02 3.47900428e-02 - -3.25927772e-02 9.42993239e-02 5.86547889e-02 5.72814979e-02 - 1.37329111e-02 -9.53979567e-02 -1.60675067e-02 4.14123572e-02 - 2.33764667e-02 3.53698768e-02 -4.33044471e-02 -3.56750526e-02 - -5.64098416e-04 -2.61383075e-02 1.24282846e-02 -3.62586998e-03 - 4.86450233e-02 -3.13415565e-02 -5.13610877e-02 -1.89971942e-02 - 1.85394306e-02 1.38549820e-01 1.28051773e-01 -5.66711463e-02 - -4.99572791e-02 -5.78689622e-03 -1.12075815e-02 5.44433631e-02 - 5.10864295e-02 1.31454477e-02 1.12670906e-01 -8.23974684e-02 - 5.42907752e-02 -1.43814096e-02 -6.16455125e-03 2.03704853e-02 - 4.37622108e-02 4.82177772e-02 -4.18701209e-02 -1.38397226e-02 - 1.66168232e-02 -8.55712965e-02 1.35803232e-02 -2.05383319e-02 - 2.05688495e-02 2.71606464e-02 -3.46984901e-02 -3.33557166e-02 - 3.99475135e-02 -5.79528846e-02 -6.28051832e-02 2.44750995e-02 - -3.88717675e-03 2.40898156e-03 -6.31332444e-03 2.38952655e-02 - 1.12915048e-02 2.56347680e-03 -5.83190955e-02 -4.41894569e-02 - 1.88140888e-02 5.84411658e-02 2.09198017e-02 4.80041541e-02 - -9.94873140e-03 5.40771522e-02 9.94110201e-03 -6.61850022e-03 - 5.13305701e-02 2.78015155e-02 -3.58276404e-02 8.48388765e-03 - -3.97338904e-02 2.51159687e-02 -2.02331562e-02 -2.45513935e-02 - 1.50146494e-02 1.08795175e-02 -1.56173715e-02 3.21655311e-02 - 7.26699876e-03 -1.08947763e-02 5.22460975e-02 -1.05224617e-01 - -2.36206073e-02 2.08435077e-02 -5.32531776e-02 4.02221717e-02] - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 468.5ms [WARNING] Time: model_destroy nomic_embed_text 120.04 +成功时终端会打印 embedding 向量（浮点数组）。首次运行会下载模型并可能拉取 Hugging Face tokenizer 相关文件。 - ``` +:::tip +也可在 `samples` 目录使用启动器： + +```bash +python3 run_inference.py --list +python3 run_inference.py --model nomic_embed_text --args "--chipset 9075" +``` + +::: diff --git a/docs/common/ai/qualcomm/_openai_clip_qai-app-builder.mdx b/docs/common/ai/qualcomm/_openai_clip_qai-app-builder.mdx index 33dab2904..5d777b28f 100644 --- a/docs/common/ai/qualcomm/_openai_clip_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_openai_clip_qai-app-builder.mdx @@ -3,8 +3,8 @@ CLIP 模型。 **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | +| 设备 | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -20,131 +20,74 @@ CLIP 模型。 ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers openai-clip +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless "openai-clip==1.0.1" ftfy "setuptools<81" ``` ### 运行脚本 -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- 进入上游示例目录 - + - - - + ```bash + cd qai-appbuilder/samples + ``` -- 准备输入图片，这里以以下图片为输入示例 + - {" "} +- 准备输入数据（示例自带输入文件时可直接运行；也可按脚本参数指定） -
- - image1 image2 image3 -
+
+ + image1 image2 image3 +
- 执行推理 ```bash - python3 openai_clip/openai_clip.py --text camping under the stars + python3 Multimodal/Image_Classification/openai_clip/openai_clip.py --chipset 9075 --text "mountain" ``` - ```bash - $ python3 openai_clip/openai_clip.py - /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/.venv/lib/python3.12/site-packages/clip/clip.py:6: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. - from pkg_resources import packaging - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 232.1ms [WARNING] Time: Read model file to memory. 121.58 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +成功时终端会为各图打印与文本的相似度分数，例如： - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 361.8ms [WARNING] Time: contextCreateFromBinary. 129.68 - - 361.9ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 365.4ms [WARNING] Time: model_initialize openai_clip 365.30 - - 1453.9ms [WARNING] Time: model_inference openai_clip 25.61 - - 1478.6ms [WARNING] Time: model_inference openai_clip 24.52 - - 1504.1ms [WARNING] Time: model_inference openai_clip 25.45 - - Searching images by prompt: camping under the stars - Image with name: image2.jpg has a similarity score=[19.906252] - Image with name: image1.jpg has a similarity score=[16.578127] - Image with name: image3.jpg has a similarity score=[26.828127] - Displaying the most relevant image - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +```text +Image with name: image2.jpg has a similarity score=[[18.687502]] +Image with name: image1.jpg has a similarity score=[[16.109377]] +Image with name: image3.jpg has a similarity score=[[34.031254]] +``` - /usr/bin/xdg-open: 882: www-browser: not found - 0.0ms [WARNING] This META does not have Alloc2 Support +（具体分数随输入图与文本变化；分数最高者与查询文本更匹配。） - /usr/bin/xdg-open: 882: links2: not found - 0.0ms [WARNING] This META does not have Alloc2 Support +- 运行结果示意 - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmpiq2mgemq.PNG' - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 7951.5ms [WARNING] Time: model_destroy openai_clip 122.86 - ``` +
+ + image3 +
- 结果显示与 `camping under the stars` 最匹配的图片是 `image3.jpg` +:::tip +`openai-clip` 依赖较旧的 `pkg_resources` API，请将 `setuptools` 固定到 81 以下。首次运行会通过 Qualcomm® AI Hub 自动下载模型。也可直接运行脚本（推荐）；若用启动器，注意 `--args` 中带空格的参数需正确引号嵌套： - {" "} +```bash +python3 run_inference.py --list +python3 Multimodal/Image_Classification/openai_clip/openai_clip.py --chipset 9075 --text "mountain" +``` -
- - image3 -
+::: diff --git a/docs/common/ai/qualcomm/_openpose_qai-app-builder.mdx b/docs/common/ai/qualcomm/_openpose_qai-app-builder.mdx index 2d837a833..7989ae302 100644 --- a/docs/common/ai/qualcomm/_openpose_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_openpose_qai-app-builder.mdx @@ -3,8 +3,8 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | +| 设备 | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -20,119 +20,64 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers openai-clip +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless scipy ``` ### 运行脚本 -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- 进入上游示例目录 - + - - - + ```bash + cd qai-appbuilder/samples + ``` -- 准备输入图片，这里以以下图片为输入示例 + - {" "} +- 准备输入数据（示例自带输入文件时可直接运行；也可按脚本参数指定） -
- - input image -
+
+ + input image +
- 执行推理 ```bash - python3 openpose/openpose.py + python3 ComputerVision/Pose_Estimation/openpose/openpose.py --chipset 9075 ``` - ```bash - $ python3 openpose/openpose.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 161.4ms [WARNING] Time: Read model file to memory. 49.87 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +- 运行结果示意 - 0.0ms [WARNING] This META does not have Alloc2 Support +
+ + output image +
- 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 214.6ms [WARNING] Time: contextCreateFromBinary. 53.04 - - 214.6ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 216.5ms [WARNING] Time: model_initialize openpose 216.47 - - 273.6ms [WARNING] Time: model_inference openpose 24.91 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 929.9ms [WARNING] Time: model_destroy openpose 57.19 - - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmpfvxac2nz.PNG' - ``` +:::tip +首次运行会通过 Qualcomm® AI Hub 自动下载模型，耗时取决于网络。也可在 `samples` 目录使用启动器： - {" "} +```bash +python3 run_inference.py --list +python3 run_inference.py --model openpose --args "--chipset 9075" +``` -
- - output image -
+::: diff --git a/docs/common/ai/qualcomm/_qai-appbuilder.mdx b/docs/common/ai/qualcomm/_qai-appbuilder.mdx index 01da87a8a..be13d972d 100644 --- a/docs/common/ai/qualcomm/_qai-appbuilder.mdx +++ b/docs/common/ai/qualcomm/_qai-appbuilder.mdx @@ -1,4 +1,4 @@ -[**Quick AI Application Builder**](https://github.com/quic/ai-engine-direct-helper) (QAI AppBuilder) 可帮助开发者轻松使用基于 [Qualcomm® AI Runtime SDK ](qairt-sdk#qairt) +[**Quick AI Application Builder**](https://github.com/qualcomm/qai-appbuilder) (QAI AppBuilder) 可帮助开发者轻松使用基于 [Qualcomm® AI Runtime SDK](qairt-sdk#qairt) 在搭载 Qualcomm® Hexagon™ Processor (NPU) 的 Qualcomm® SoC 平台上部署 AI 模型和设计 AI 应用。 它将模型部署 API 封装成一组简化的接口，用于将模型加载到 NPU 并执行推理。QAI AppBuilder 大大降低了开发者部署模型的复杂性并且提供多个 demo 让开发者参考设计自己的 AI 应用。 @@ -11,16 +11,16 @@ **支持设备** -| 设备 | SoC | -| -------------------- | ----- | -| Dragon Q6A | QCS6490 | +| 设备 | SoC | +| -------------------- | -------- | +| Dragon Q6A | QCS6490 | | Dragon Q8B | SC8280XP | -| Fogwise® AIRbox Q900 | QCS9075 | +| Fogwise® AIRbox Q900 | QCS9075 | ## 安装方法 :::tip -可 [**使用 wheel 包安装**](#使用-wheel-包安装) 或 [**源码编译**](#源码编译) 进行安装 +推荐 [**使用 wheel 包安装**](#使用-wheel-包安装)。如需自行编译，可参考 [**源码编译**](#源码编译)。 ::: ### 克隆仓库 @@ -28,7 +28,7 @@ ```bash -git clone -b radxa-dev https://github.com/ZIFENG278/ai-engine-direct-helper.git && cd ai-engine-direct-helper +git clone https://github.com/qualcomm/qai-appbuilder.git && cd qai-appbuilder ``` @@ -38,7 +38,7 @@ git clone -b radxa-dev https://github.com/ZIFENG278/ai-engine-direct-helper.git ##### 创建虚拟环境 :::tip -安装 python 库前需要创建虚拟环境进行环境隔 +安装 Python 库前需要创建虚拟环境进行环境隔离。 ::: @@ -51,483 +51,307 @@ source .venv/bin/activate - - -```bash -pip3 install https://github.com/ZIFENG278/ai-engine-direct-helper/releases/download/radxa-dev-2.38.0/qai_appbuilder-2.38.0-cp312-cp312-linux_aarch64.whl -``` +##### 安装 qai_appbuilder - - -### 源码编译 +从 [**GitHub Release**](https://github.com/qualcomm/qai-appbuilder/releases) 下载与 Python 版本匹配的 Linux aarch64 wheel，例如 Python 3.12： ```bash -git submodule update --init --recursive +pip3 install https://github.com/qualcomm/qai-appbuilder/releases/download/v2.48.40/qai_appbuilder-2.48.40-cp312-cp312-manylinux_2_39_aarch64.whl ``` -#### 下载 QAIRT SDK - :::tip -请根据 [**下载 QAIRT SDK 压缩包**](./qairt-install#下载-qairt-sdk-压缩包) 下载 QAIRT SDK 并解压 +从 v2.0.0 起，Python wheel 已内置运行所需的 Qualcomm® AI Runtime 库（如 `libQnnHtp.so`、Hexagon skel 等），一般无需再单独安装完整 QAIRT SDK 即可做 Python 推理。 ::: -##### 配置 QAIRT 环境变量 - - - - - - ```bash - export PRODUCT_SOC=6490 DSP_ARCH=68 - ``` - - - - - - - - - ```bash - export PRODUCT_SOC=9075 DSP_ARCH=73 - ``` - - - - - - - - ```bash - export PRODUCT_SOC=8280 DSP_ARCH=68 - ``` - - - - - - +验证安装： ```bash -cd qairt/2.42.0.251225 -source bin/envsetup.sh -export ADSP_LIBRARY_PATH=$QNN_SDK_ROOT/lib/hexagon-v${DSP_ARCH}/unsigned -export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$QNN_SDK_ROOT/lib/aarch64-oe-linux-gcc11.2 +python3 -c "import qai_appbuilder; print(qai_appbuilder.__file__)" ``` -#### 编译 qai_appbuilder +### 源码编译 -##### 创建虚拟环境 +如需从源码编译，请先安装构建依赖，并准备 [**QAIRT SDK**](./qairt-install#下载-qairt-sdk-压缩包)。 -:::tip -安装 python 库前需要创建虚拟环境进行环境隔 -::: +#### 安装构建依赖 ```bash -sudo apt install python3-venv python3-dev cmake gcc g++ +sudo apt update +sudo apt install -y cmake build-essential python3-dev python3-venv +git clone https://github.com/qualcomm/qai-appbuilder.git --recursive +cd qai-appbuilder python3 -m venv .venv source .venv/bin/activate +pip3 install wheel==0.45.1 setuptools==80.9.0 pybind11==2.13.6 build==1.4.0 ``` -##### 编译 qai_appbuilder wheel +#### 配置 QAIRT 环境变量 + +请先完成 QAIRT SDK 解压。将 `` 替换为实际 SDK 路径： ```bash -pip3 install setuptools -cd ai-engine-direct-helper -python3 setup.py bdist_wheel +export QNN_SDK_ROOT= +export QAI_TOOLCHAINS=aarch64-oe-linux-gcc11.2 ``` -编译完成会在 `dist` 目录下生成一个 `qai_appbuilder-2.38.0-cp312-cp312-linux_aarch64.whl - -` 文件 - -#### 安装 qai_appbuilder - -##### 安装 qai_appbuilder wheel +#### 编译并安装 qai_appbuilder wheel ```bash -cd dist -pip3 install ./qai_appbuilder-2.38.0-cp312-cp312-linux_aarch64.whl +python3 -m build -w +python3 -m pip install dist/qai_appbuilder-*.whl ``` -### 创建 QAIRT SDK LIB 软链接 - -请在 `ai-engine-direct-helper/samples/python` 下创建 `qai_libs` 软链接，用于链接 QAIRT SDK 里的 lib 目录 - - - - - :::tip -如果没有下载完整的 QAIRT_SDK 可以安装 `qcom-qairt` 后进行软链接 - -qcom-qairt 是一个体积缩小版的 QAIRT-SDK，仅安装所需的文件 +wheel 版本号会随所链接的 QAIRT SDK 版本变化。官方构建说明见 [**BUILD.md**](https://github.com/qualcomm/qai-appbuilder/blob/main/BUILD.md)。 ::: - - - - - ```bash - sudo apt install qcom-qnn-sdk-v68 qcom-genie-sdk-v68 - ``` - - - - - - - - - ```bash - sudo apt install qcom-qnn-sdk-v73 qcom-genie-sdk-v73 - ``` - - - - - - - - ```bash - sudo apt install qcom-qnn-sdk-v68 qcom-genie-sdk-v68 - ``` - +### 创建 ADSP 环境变量 +在 Linux 设备上运行 NPU 推理前，需要设置 `ADSP_LIBRARY_PATH`，指向包含 Hexagon skel（如 `libQnnHtpV68Skel.so` / `libQnnHtpV73Skel.so`）的目录。 - +#### 使用 wheel 内置库（推荐） - +安装 wheel 后，可直接指向包内 `libs` 目录： ```bash -ln -s /usr/lib/aarch64-linux-gnu samples/python/qai_libs +export ADSP_LIBRARY_PATH=$(python3 -c "import os, qai_appbuilder; print(os.path.join(os.path.dirname(qai_appbuilder.__file__), 'libs'))") ``` - - - +#### 使用系统 QAIRT 包 -:::tip -如果用户已经下载了完整版本的 QAIRT-SDK， 可以链接 QAIRT-SDK 下的 `lib/aarch64-oe-linux-gcc11.2/` -::: +`qairt-libs`、`qairt-dsp-binaries` 等系统包来自高通 apt 软件源（`ppa:ubuntu-qcom-iot/qcom-ppa`）。该软件源**默认未配置**，需先手动添加后再安装： ```bash -ln -s qairt/2.42.0.251225/lib/aarch64-oe-linux-gcc11.2/ samples/python/qai_libs +sudo apt update && sudo apt install -y software-properties-common +sudo add-apt-repository -y ppa:ubuntu-qcom-iot/qcom-ppa +sudo apt update +sudo apt install -y qairt-libs qairt-dsp-binaries ``` - - - - -### 创建 ADSP 环境变量 - - - - - -:::tip -如果安装 `qcom-qairt` 包的用户，可以使用以下命令创建 `ADSP_LIBRARY_PATH` 环境变量 -::: +若系统已安装 `qairt-dsp-binaries` 等包，也可将 `ADSP_LIBRARY_PATH` 指向对应平台的 DSP 目录，例如 QCS6490： ```bash -export ADSP_LIBRARY_PATH=/usr/lib/aarch64-linux-gnu +export ADSP_LIBRARY_PATH=/usr/share/qcom/qcm6490/Thundercomm/RB3gen2/dsp/cdsp ``` - - - - :::tip -如果安装了完整的 QAIRT-SDK 的用户，可以使用以下命令创建 `ADSP_LIBRARY_PATH` 环境变量 +可在 `~/.bashrc` 中写入上述 `export`，并执行 `source ~/.bashrc` 使其长期生效。 ::: - - - - - ```bash - export PRODUCT_SOC=6490 DSP_ARCH=68 - ``` - - +## 应用例子 - +:::tip +上游示例已按任务类型重新组织目录。Linux 上运行示例脚本时，需要通过 `--chipset` 指定芯片，以便自动下载对应模型： - +- QCS6490 / SC8280XP：`--chipset 6490` +- QCS9075：`--chipset 9075` - - ```bash - export PRODUCT_SOC=9075 DSP_ARCH=73 - ``` - +更多示例说明见上游 [**samples/README.md**](https://github.com/qualcomm/qai-appbuilder/blob/main/samples/README.md)。也可参考本地 [**Demos 示例**](./qai-appbuilder-demo)。 +::: - +### 安装示例依赖 - +在已激活的虚拟环境中安装示例常用依赖： - - ```bash - export PRODUCT_SOC=8280 DSP_ARCH=68 - ``` - + +```bash +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless +``` - + - +### 进入示例目录 ```bash -cd qairt/2.42.0.251225 -export ADSP_LIBRARY_PATH=$(pwd)/lib/hexagon-v${DSP_ARCH}/unsigned +cd qai-appbuilder/samples ``` - - - +### 运行示例 -## 应用例子 - - - - -:::tip -QCS6490 Demos 运行示例可参考 [**Demos 示例**](./qai-appbuilder-demo) -::: +可直接运行示例脚本，也可使用交互式启动器： ```bash -cd ai-engine-direct-helper/samples/linux/python +python3 run_inference.py --list +python3 run_inference.py --model inception_v3 --args "--chipset 6490" ``` -- 目标识别 - - | 模型 | 运行方法 | - | ----------------- | ------------------------------------------------ | - | convnext_base | `python3 convnext_base/convnext_base.py` | - | convnext_tiny | `python3 convnext_tiny/convnext_tiny.py` | - | efficientnet_b0 | `python3 efficientnet_b0/efficientnet_b0.py` | - | efficientnet_b4 | `python3 efficientnet_b4/efficientnet_b4.py` | - | efficientnet_v2_s | `python3 efficientnet_v2_s/efficientnet_v2_s.py` | - | googlenet | `python3 googlenet/googlenet.py` | - | inception_v3 | `python3 inception_v3/inception_v3.py` | - | levit | `python3 levit/levit.py` | - | regnet | `python3 regnet/regnet.py` | - | shufflenet_v2 | `python3 shufflenet_v2/shufflenet_v2.py` | - | squeezenet1_1 | `python3 squeezenet1_1/squeezenet1_1.py` | - | vit | `python3 vit/vit.py` | - | wideresnet50 | `python3 wideresnet50/wideresnet50.py` | - -- 图像分割 - - | 模型 | 运行方法 | - | ------------ | -------------------------------------- | - | fcn_resnet50 | `python3 fcn_resnet50/fcn_resnet50.py` | + + + +- 图像分类 + + | 模型 | 运行方法 | + | ------------ | ----------------------------------------------------------------------------------------- | + | googlenet | `python3 ComputerVision/Image_Classification/googlenet/googlenet.py --chipset 6490` | + | inception_v3 | `python3 ComputerVision/Image_Classification/inception_v3/inception_v3.py --chipset 6490` | - 目标检测 - | 模型 | 运行方法 | - | ---------- | ---------------------------------- | - | yolov8_det | `python3 yolov8_det/yolov8_det.py` | + | 模型 | 运行方法 | + | ---------- | --------------------------------------------------------------------------------- | + | yolov8_det | `python3 ComputerVision/Object_Detection/yolov8_det/yolov8_det.py --chipset 6490` | - 图像超分辨率 - | 模型 | 运行方法 | - | ------------------------ | -------------------------------------------------------------- | - | quicksrnetmedium | `python3 quicksrnetmedium/quicksrnetmedium.py` | - | real_esrgan_general_x4v3 | `python3 real_esrgan_general_x4v3/real_esrgan_general_x4v3.py` | - | real_esrgan_x4plus | `python3 real_esrgan_x4plus/real_esrgan_x4plus.py` | - | sesr_m5 | `python3 sesr_m5/sesr_m5.py` | - | xlsr | `python3 xlsr/xlsr.py` | + | 模型 | 运行方法 | + | ------------------------ | ----------------------------------------------------------------------------------------------------------------- | + | quicksrnetmedium | `python3 ComputerVision/Super_Resolution/quicksrnetmedium/quicksrnetmedium.py --chipset 6490` | + | real_esrgan_general_x4v3 | `python3 ComputerVision/Super_Resolution/real_esrgan_general_x4v3/real_esrgan_general_x4v3.py --chipset 6490` | + | real_esrgan_x4plus | `python3 ComputerVision/Super_Resolution/real_esrgan_x4plus/real_esrgan_x4plus.py --chipset 6490 --bin --no_show` | :::tip -SC8280XP Demos 运行示例可参考 [**Demos 示例**](./qai-appbuilder-demo) +上游 Linux 示例脚本目前通过 `--chipset 6490` / `--chipset 9075` 选择模型包。SC8280XP 与 QCS6490 同为 Hexagon V68，下载模型时使用 `--chipset 6490`。QCS6490 上 `real_esrgan_x4plus` 需加 `--bin`。 ::: - - -```bash -cd ai-engine-direct-helper/samples/linux/python -``` - - +- 图像分类 -- 目标识别 - - | 模型 | 运行方法 | - | ----------------- | ------------------------------------------------ | - | convnext_base | `python3 convnext_base/convnext_base.py` | - | convnext_tiny | `python3 convnext_tiny/convnext_tiny.py` | - | efficientnet_b0 | `python3 efficientnet_b0/efficientnet_b0.py` | - | efficientnet_b4 | `python3 efficientnet_b4/efficientnet_b4.py` | - | efficientnet_v2_s | `python3 efficientnet_v2_s/efficientnet_v2_s.py` | - | googlenet | `python3 googlenet/googlenet.py` | - | inception_v3 | `python3 inception_v3/inception_v3.py` | - | levit | `python3 levit/levit.py` | - | regnet | `python3 regnet/regnet.py` | - | shufflenet_v2 | `python3 shufflenet_v2/shufflenet_v2.py` | - | squeezenet1_1 | `python3 squeezenet1_1/squeezenet1_1.py` | - | vit | `python3 vit/vit.py` | - | wideresnet50 | `python3 wideresnet50/wideresnet50.py` | - -- 图像分割 - - | 模型 | 运行方法 | - | ------------ | -------------------------------------- | - | fcn_resnet50 | `python3 fcn_resnet50/fcn_resnet50.py` | + | 模型 | 运行方法 | + | ------------ | ----------------------------------------------------------------------------------------- | + | googlenet | `python3 ComputerVision/Image_Classification/googlenet/googlenet.py --chipset 6490` | + | inception_v3 | `python3 ComputerVision/Image_Classification/inception_v3/inception_v3.py --chipset 6490` | - 目标检测 - | 模型 | 运行方法 | - | ---------- | ---------------------------------- | - | yolov8_det | `python3 yolov8_det/yolov8_det.py` | + | 模型 | 运行方法 | + | ---------- | --------------------------------------------------------------------------------- | + | yolov8_det | `python3 ComputerVision/Object_Detection/yolov8_det/yolov8_det.py --chipset 6490` | - 图像超分辨率 - | 模型 | 运行方法 | - | ------------------------ | -------------------------------------------------------------- | - | quicksrnetmedium | `python3 quicksrnetmedium/quicksrnetmedium.py` | - | real_esrgan_general_x4v3 | `python3 real_esrgan_general_x4v3/real_esrgan_general_x4v3.py` | - | real_esrgan_x4plus | `python3 real_esrgan_x4plus/real_esrgan_x4plus.py` | - | sesr_m5 | `python3 sesr_m5/sesr_m5.py` | - | xlsr | `python3 xlsr/xlsr.py` | + | 模型 | 运行方法 | + | ------------------------ | ----------------------------------------------------------------------------------------------------------------- | + | quicksrnetmedium | `python3 ComputerVision/Super_Resolution/quicksrnetmedium/quicksrnetmedium.py --chipset 6490` | + | real_esrgan_general_x4v3 | `python3 ComputerVision/Super_Resolution/real_esrgan_general_x4v3/real_esrgan_general_x4v3.py --chipset 6490` | + | real_esrgan_x4plus | `python3 ComputerVision/Super_Resolution/real_esrgan_x4plus/real_esrgan_x4plus.py --chipset 6490 --bin --no_show` | -:::tip -QCS9075 Demos 运行示例可参考 [**Demos 示例**](./qai-appbuilder-demo) -::: - - - -```bash -cd ai-engine-direct-helper/samples/python -``` - - - - 图像生成 / 图像修复 - | 模型 | 运行方法 | - | --------------------- | ----------------------------------------------------------------------------------------- | - | stable_diffusion_v2_1 | `python3 stable_diffusion_v2_1/stable_diffusion_v2_1.py --prompt "the prompt string ..."` | - | stable_diffusion_v1_5 | `python3 stable_diffusion_v1_5/stable_diffusion_v1_5.py --prompt "the prompt string ..."` | - | lama_dilated | `python3 lama_dilated/lama_dilated.py` | - | aotgan | `python3 aotgan/aotgan.py` | + | 模型 | 运行方法 | + | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | + | stable_diffusion_v2_1 | `python3 GenerativeAI/Image_Generation/stable_diffusion_v2_1/stable_diffusion_v2_1.py --chipset 9075 --prompt "the prompt string ..."` | + | stable_diffusion_v1_5 | `python3 GenerativeAI/Image_Generation/stable_diffusion_v1_5/stable_diffusion_v1_5.py --chipset 9075 --prompt "the prompt string ..."` | + | lama_dilated | `python3 ComputerVision/Image_Editing/lama_dilated/lama_dilated.py --chipset 9075` | + | aotgan | `python3 ComputerVision/Image_Editing/aotgan/aotgan.py --chipset 9075` | - 图像超分辨率 - | 模型 | 运行方法 | - | ------------------------ | -------------------------------------------------------------- | - | real_esrgan_x4plus | `python3 real_esrgan_x4plus/real_esrgan_x4plus.py` | - | real_esrgan_general_x4v3 | `python3 real_esrgan_general_x4v3/real_esrgan_general_x4v3.py` | - | quicksrnetmedium | `python3 quicksrnetmedium/quicksrnetmedium.py` | + | 模型 | 运行方法 | + | ------------------------ | ------------------------------------------------------------------------------------------------------------- | + | real_esrgan_x4plus | `python3 ComputerVision/Super_Resolution/real_esrgan_x4plus/real_esrgan_x4plus.py --chipset 9075` | + | real_esrgan_general_x4v3 | `python3 ComputerVision/Super_Resolution/real_esrgan_general_x4v3/real_esrgan_general_x4v3.py --chipset 9075` | + | quicksrnetmedium | `python3 ComputerVision/Super_Resolution/quicksrnetmedium/quicksrnetmedium.py --chipset 9075` | - 图像识别 - | 模型 | 运行方法 | - | ------------ | -------------------------------------- | - | inception_v3 | `python3 inception_v3/inception_v3.py` | - | beit | `python3 beit/beit.py` | - | googlenet | `python3 googlenet/googlenet.py` | + | 模型 | 运行方法 | + | ------------ | ----------------------------------------------------------------------------------------- | + | inception_v3 | `python3 ComputerVision/Image_Classification/inception_v3/inception_v3.py --chipset 9075` | + | beit | `python3 ComputerVision/Image_Classification/beit/beit.py --chipset 9075` | + | googlenet | `python3 ComputerVision/Image_Classification/googlenet/googlenet.py --chipset 9075` | - 姿态识别 - | 模型 | 运行方法 | - | -------------- | ------------------------------------------ | - | openpose | `python3 openpose/openpose.py` | - | mediapipe_hand | `python3 mediapipe_hand/mediapipe_hand.py` | + | 模型 | 运行方法 | + | -------------- | ---------------------------------------------------------------------------------------- | + | openpose | `python3 ComputerVision/Pose_Estimation/openpose/openpose.py --chipset 9075` | + | mediapipe_hand | `python3 ComputerVision/Pose_Estimation/mediapipe_hand/mediapipe_hand.py --chipset 9075` | - 图像分割 / 深度估计 - | 模型 | 运行方法 | - | ----------------- | ------------------------------------------------ | - | unet_segmentation | `python3 unet_segmentation/unet_segmentation.py` | - | depth_anything | `python3 depth_anything/depth_anything.py` | + | 模型 | 运行方法 | + | ----------------- | ---------------------------------------------------------------------------------------------------- | + | unet_segmentation | `python3 ComputerVision/Semantic_Segmentation/unet_segmentation/unet_segmentation.py --chipset 9075` | + | depth_anything | `python3 ComputerVision/Depth_Estimation/depth_anything/depth_anything.py --chipset 9075` | - 目标检测 - | 模型 | 运行方法 | - | ---------- | ---------------------------------- | - | yolov8_det | `python3 yolov8_det/yolov8_det.py` | + | 模型 | 运行方法 | + | ---------- | --------------------------------------------------------------------------------- | + | yolov8_det | `python3 ComputerVision/Object_Detection/yolov8_det/yolov8_det.py --chipset 9075` | - 三维建模 - | 模型 | 运行方法 | - | --------------- | -------------------------------------------- | - | facemap_3dmm | `python3 facemap_3dmm/facemap_3dmm.py` | - | face_attrib_net | `python3 face_attrib_net/face_attrib_net.py` | + | 模型 | 运行方法 | + | --------------- | ------------------------------------------------------------------------------------------- | + | facemap_3dmm | `python3 ComputerVision/Face_Recognition/facemap_3dmm/facemap_3dmm.py --chipset 9075` | + | face_attrib_net | `python3 ComputerVision/Face_Recognition/face_attrib_net/face_attrib_net.py --chipset 9075` | - 音频模型 - | 模型 | 运行方法 | - | --------------- | -------------------------------------------- | - | yamnet | `python3 yamnet/yamnet.py` | - | whisper_base_en | `python3 whisper_base_en/whisper_base_en.py` | - | whisper_tiny_en | `python3 whisper_tiny_en/whisper_tiny_en.py` | + | 模型 | 运行方法 | + | --------------- | ------------------------------------------------------------------------------------ | + | yamnet | `python3 audio/Audio_Classification/yamnet/yamnet.py --chipset 9075` | + | whisper_base_en | `python3 audio/Speech_Recognition/whisper_base_en/whisper_base_en.py --chipset 9075` | + | whisper_tiny_en | `python3 audio/Speech_Recognition/whisper_tiny_en/whisper_tiny_en.py --chipset 9075` | - 文本 / 多模态嵌入 - | 模型 | 运行方法 | - | ---------------- | ------------------------------------------------------ | - | nomic_embed_text | `python3 nomic_embed_text/nomic_embed_text.py` | - | openai_clip | `python3 openai_clip/openai_clip.py --text "mountain"` | + | 模型 | 运行方法 | + | ---------------- | ----------------------------------------------------------------------------------------------------- | + | nomic_embed_text | `python3 Multimodal/Text_Generation/nomic_embed_text/nomic_embed_text.py --chipset 9075` | + | openai_clip | `python3 Multimodal/Image_Classification/openai_clip/openai_clip.py --chipset 9075 --text "mountain"` | + | qwen_vl | 需先下载模型包；`python3 Multimodal/qwen_vl/qwen_vl.py --model qwen2 --path ./qwen2vl2b`（见示例页） | - 三维动作识别 - | 模型 | 运行方法 | - | --------- | -------------------------------- | - | resnet_3d | `python3 resnet_3d/resnet_3d.py` | + | 模型 | 运行方法 | + | --------- | ----------------------------------------------------------------------------------- | + | resnet_3d | `python3 ComputerVision/Video_Classification/resnet_3d/resnet_3d.py --chipset 9075` | - OCR（文字识别） - | 模型 | 运行方法 | - | -------- | ------------------------------ | - | easy_ocr | `python3 easy_ocr/easy_ocr.py` | + | 模型 | 运行方法 | + | -------- | ---------------------------------------------------------------------- | + | easy_ocr | `python3 Multimodal/Image_To_Text/easy_ocr/easy_ocr.py --chipset 9075` | diff --git a/docs/common/ai/qualcomm/_qairt-sdk.mdx b/docs/common/ai/qualcomm/_qairt-sdk.mdx index 6b83b6534..47ac320b0 100644 --- a/docs/common/ai/qualcomm/_qairt-sdk.mdx +++ b/docs/common/ai/qualcomm/_qairt-sdk.mdx @@ -48,11 +48,11 @@ QAIRT 基于不同的系统与不同的推理后端，有以下 3 种模型文 #### SoC 架构对照表 -| SoC | dsp_arch | soc_id | -| ----- | -------- | ------ | -| QCS6490 | v68 | 35 | +| SoC | dsp_arch | soc_id | +| -------- | -------- | ------ | +| QCS6490 | v68 | 35 | | SC8280XP | v68 | 37 | -| QCS9075 | v73 | 77 | +| QCS9075 | v73 | 77 | {/* Additional QAIRT references are maintained below. */} @@ -86,7 +86,7 @@ QAIRT 基于不同的系统与不同的推理后端，有以下 3 种模型文 ### QAI-APPBUILDER -[**Quick AI Application Builder**](https://github.com/quic/ai-engine-direct-helper) (QAI AppBuilder) 可帮助开发者轻松使用基于 [Qualcomm® AI Runtime SDK ](qairt-sdk#qairt) +[**Quick AI Application Builder**](https://github.com/qualcomm/qai-appbuilder) (QAI AppBuilder) 可帮助开发者轻松使用基于 [Qualcomm® AI Runtime SDK](qairt-sdk#qairt) 在搭载 Qualcomm® Hexagon™ Processor (NPU) 的 Qualcomm® SoC 平台上部署 AI 模型和设计 AI 应用。 它将模型部署 API 封装成一组简化的接口，用于将模型加载到 NPU 并执行推理。QAI AppBuilder 大大降低了开发者部署模型的复杂性并且提供多个 demo 让开发者参考设计自己的 AI 应用。 @@ -99,7 +99,7 @@ QAIRT 基于不同的系统与不同的推理后端，有以下 3 种模型文 - [**QAI AppBuilder**](./qai-appbuilder) -- [**QAI AppBuilder 仓库**](https://github.com/quic/ai-engine-direct-helper) +- [**QAI AppBuilder 仓库**](https://github.com/qualcomm/qai-appbuilder) ### QAI-Hub diff --git a/docs/common/ai/qualcomm/_quicksrnetmedium_qai-app-builder.mdx b/docs/common/ai/qualcomm/_quicksrnetmedium_qai-app-builder.mdx index 3f011af36..57c1499a7 100644 --- a/docs/common/ai/qualcomm/_quicksrnetmedium_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_quicksrnetmedium_qai-app-builder.mdx @@ -3,11 +3,11 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | -| Dragon Q6A | QCS6490 | +| 设备 | SoC | +| -------------------- | -------- | +| Dragon Q6A | QCS6490 | | Dragon Q8B | SC8280XP | -| Fogwise® AIRbox Q900 | QCS9075 | +| Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -15,153 +15,92 @@ 1. 请根据 [**QAI AppBuilder 安装方法**](../qai-appbuilder) 安装 QAI AppBuilder。 -2. 请根据 [**配置 QAIRT 环境变量**](../qai-appbuilder#配置-qairt-环境变量) 配置 QAIRT 环境变量。 +2. 请根据 [**创建 ADSP 环境变量**](../qai-appbuilder#创建-adsp-环境变量) 配置 ADSP 环境变量。 ::: ## 运行示例 ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers openai-clip +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` ### 运行脚本 -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` - - - - - - - -- 准备输入图片，这里以以下图片为输入示例 - - {" "} - -
- - input image -
- -- 执行推理 +- 进入上游示例目录 - + ```bash - python3 quicksrnetmedium/quicksrnetmedium.py + cd qai-appbuilder/samples ``` - + - ```bash - $ python3 quicksrnetmedium/quicksrnetmedium.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/quicksrnetmedium - Init - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +执行推理（Linux 需指定 `--chipset`；首次运行会自动下载 `input.png` 与模型） - 0.0ms [WARNING] This META does not have Alloc2 Support + + - 0.0ms [WARNING] This META does not have Alloc2 Support - - 112.4ms [WARNING] Time: Read model file to memory. 0.12 + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Super_Resolution/quicksrnetmedium/quicksrnetmedium.py --chipset 6490 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + + - 0.0ms [WARNING] This META does not have Alloc2 Support +:::tip +SC8280XP 与 QCS6490 同为 Hexagon V68，下载模型时使用 `--chipset 6490`。 +::: - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Super_Resolution/quicksrnetmedium/quicksrnetmedium.py --chipset 6490 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 118.0ms [WARNING] Time: contextCreateFromBinary. 5.51 + + - 118.0ms [WARNING] Time: UnmapViewOfFile. 0.00 + - 118.3ms [WARNING] Time: model_initialize quicksrnetmedium 118.21 +```bash +python3 ComputerVision/Super_Resolution/quicksrnetmedium/quicksrnetmedium.py --chipset 9075 +``` - 174.0ms [WARNING] Time: model_inference quicksrnetmedium 25.10 + - 0.0ms [WARNING] This META does not have Alloc2 Support + + - 0.0ms [WARNING] This META does not have Alloc2 Support +### 预期结果 - 0.0ms [WARNING] This META does not have Alloc2 Support +在 Dragon Q6A 上推理成功后，会在示例目录生成： - 0.0ms [WARNING] This META does not have Alloc2 Support +```text +ComputerVision/Super_Resolution/quicksrnetmedium/output.png +``` - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 353.0ms [WARNING] Time: model_destroy quicksrnetmedium 18.10 +无图形界面（无 `DISPLAY`）时可能打印 `Error: no DISPLAY environment variable specified`，只要 `output.png` 已生成且进程退出码为 0，可忽略该提示。 - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmp751e_f48.PNG' - ``` +:::tip +也可使用启动器： - {" "} +```bash +python3 run_inference.py --model quicksrnetmedium --args "--chipset 6490" +``` -
- - output image -
+::: diff --git a/docs/common/ai/qualcomm/_qwen_vl_qai-app-builder.mdx b/docs/common/ai/qualcomm/_qwen_vl_qai-app-builder.mdx new file mode 100644 index 000000000..80db87c91 --- /dev/null +++ b/docs/common/ai/qualcomm/_qwen_vl_qai-app-builder.mdx @@ -0,0 +1,90 @@ +此文档讲述使用 [QAI AppBuilder](../qai-appbuilder) 运行上游 [Qwen VL](https://github.com/qualcomm/qai-appbuilder/tree/main/samples/Multimodal/qwen_vl) 多模态视觉语言示例（Linux aarch64；Gradio UI 或脚本内 `Inference`）。 + +**示例支持设备** + +| 设备 | SoC | +| -------------------- | ------- | +| Fogwise® AIRbox Q900 | QCS9075 | + +## 安装 QAI AppBuilder + +:::tip + +1. 请根据 [**QAI AppBuilder 安装方法**](../qai-appbuilder) 安装 QAI AppBuilder。 + +2. 请根据 [**创建 ADSP 环境变量**](../qai-appbuilder#创建-adsp-环境变量) 配置 ADSP 环境变量。 + ::: + +## 运行示例 + +### 安装依赖 + +在已激活的虚拟环境中安装示例依赖： + + + +```bash +pip3 install "transformers==4.57.0" qwen-vl-utils gradio Pillow torch torchvision opencv-python-headless +``` + + + +:::tip +`qwen_vl.py` 会 import Qwen3 相关模块，因此需要 **transformers ≥ 4.57**（含 `transformers.models.qwen3_vl`）。另需 `qwen-vl-utils`。 +::: + +### 准备模型 + +该示例**不会**通过 `--chipset` / AI Hub 自动下载 Qwen 权重。请按上游 README 获取预量化包，例如 Qwen2-VL 2B： + + + +```bash +cd qai-appbuilder/samples/Multimodal/qwen_vl +wget -c https://www.aidevhome.com/data/adh2/models/suggested/qwen2vl2b.zip +unzip -o qwen2vl2b.zip +# 解压后目录示例：./qwen2vl2b/（含 veg.serialized.bin、weight_sharing_model_1_of_1.serialized.bin、tokenizer.json 等） +``` + + + +压缩包约 1.8 GB，解压后约 2.6 GB，请预留磁盘空间。 + +### 运行脚本 + +- 进入示例目录并设置 ADSP（与安装页相同） + + + + ```bash + cd qai-appbuilder/samples/Multimodal/qwen_vl + export ADSP_LIBRARY_PATH=$(python3 -c "import os, qai_appbuilder; print(os.path.join(os.path.dirname(qai_appbuilder.__file__), 'libs'))") + ``` + + + +- 启动 Gradio Web UI（默认 `0.0.0.0:7861`） + + + + ```bash + python3 qwen_vl.py --model qwen2 --path ./qwen2vl2b --host 0.0.0.0 --port 7861 + ``` + + + +浏览器访问 `http://<设备IP>:7861`，上传图片并输入问题即可。 + +- 无界面自检（可选）：在已加载模型后调用 `Inference` 做一次问答。AIRbox Q900 上对示例图 `vlm_demo.png` 提问 _Describe this image in one sentence._ 时，曾得到类似： + +```text +A webpage with a form for uploading files. +``` + +:::warning +`run_inference.py --model qwen_vl` **不能**代替模型下载；且入口需要 `--path` 指向模型目录，**没有** `--chipset 9075` 参数。 +::: + +:::tip +若系统存在 TBB 库，部分环境可按上游 README 设置 `LD_PRELOAD` 指向 `libtbb.so`；Q900 上未设置亦可完成一次推理。 +::: diff --git a/docs/common/ai/qualcomm/_real-esrgan-general_x4v3_qai-app-builder.mdx b/docs/common/ai/qualcomm/_real-esrgan-general_x4v3_qai-app-builder.mdx index fb0441805..65fb91702 100644 --- a/docs/common/ai/qualcomm/_real-esrgan-general_x4v3_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_real-esrgan-general_x4v3_qai-app-builder.mdx @@ -3,11 +3,11 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | -| Dragon Q6A | QCS6490 | +| 设备 | SoC | +| -------------------- | -------- | +| Dragon Q6A | QCS6490 | | Dragon Q8B | SC8280XP | -| Fogwise® AIRbox Q900 | QCS9075 | +| Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -22,141 +22,89 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` ### 运行脚本 -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` +- 进入上游示例目录 - + - - - - - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` - - - - - - - -- 准备输入图片，这里以以下图片为输入示例 + ```bash + cd qai-appbuilder/samples + ``` - {" "} + -
- - input image -
+执行推理（Linux **必须**指定 `--chipset`，否则可能无法下载到对应模型） -- 执行推理 + - + - ```bash - python3 real_esrgan_general_x4v3/real_esrgan_general_x4v3.py - ``` - - - - ```bash - $ python3 real_esrgan_general_x4v3/real_esrgan_general_x4v3.py - 0.0ms [WARNING] Initializing HtpProvider + - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Super_Resolution/real_esrgan_general_x4v3/real_esrgan_general_x4v3.py --chipset 6490 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 248.6ms [WARNING] Time: Read model file to memory. 4.00 +:::tip +SC8280XP 与 QCS6490 同为 Hexagon V68，下载模型时使用 `--chipset 6490`。 +::: - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Super_Resolution/real_esrgan_general_x4v3/real_esrgan_general_x4v3.py --chipset 6490 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Super_Resolution/real_esrgan_general_x4v3/real_esrgan_general_x4v3.py --chipset 9075 +``` - 324.8ms [WARNING] Time: contextCreateFromBinary. 76.10 + - 324.9ms [WARNING] Time: UnmapViewOfFile. 0.00 + - 326.0ms [WARNING] Time: model_initialize realesrgan 325.97 + - 539.0ms [WARNING] Time: model_inference realesrgan 176.50 +### 预期结果 - 0.0ms [WARNING] This META does not have Alloc2 Support +在 Dragon Q6A 上推理成功后，会在示例目录生成： - 0.0ms [WARNING] This META does not have Alloc2 Support +```text +ComputerVision/Super_Resolution/real_esrgan_general_x4v3/output.jpg +``` - 0.0ms [WARNING] This META does not have Alloc2 Support +无图形界面时可能出现 `Error: no DISPLAY environment variable specified`，只要输出文件已生成且退出码为 0，可忽略。 - 0.0ms [WARNING] This META does not have Alloc2 Support +:::tip +也可使用启动器： - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmp3vvli4p1.PNG' - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 3871.6ms [WARNING] Time: model_destroy realesrgan 101.28 - ``` +```bash +python3 run_inference.py --model real_esrgan_general_x4v3 --args "--chipset 6490" +``` -
- - output image -
+::: diff --git a/docs/common/ai/qualcomm/_real-esrgan-x4plus_qai-app-builder.mdx b/docs/common/ai/qualcomm/_real-esrgan-x4plus_qai-app-builder.mdx index a97c39e35..326c2a52e 100644 --- a/docs/common/ai/qualcomm/_real-esrgan-x4plus_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_real-esrgan-x4plus_qai-app-builder.mdx @@ -3,11 +3,11 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | -| Dragon Q6A | QCS6490 | +| 设备 | SoC | +| -------------------- | -------- | +| Dragon Q6A | QCS6490 | | Dragon Q8B | SC8280XP | -| Fogwise® AIRbox Q900 | QCS9075 | +| Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -22,141 +22,105 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` ### 运行脚本 -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` +- 进入上游示例目录 - + - - - - - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` - - - - - - - -- 准备输入图片，这里以以下图片为输入示例 + ```bash + cd qai-appbuilder/samples + ``` - {" "} + -
- - input image -
+- 准备输入数据（示例自带输入文件时可直接运行；也可按脚本参数指定） -- 执行推理 +
+ + input image +
- +执行推理 - ```bash - python3 real_esrgan_x4plus/real_esrgan_x4plus.py - ``` +:::warning +在 QCS6490 / SC8280XP 上，**不要使用默认 float DLC**。默认 `.dlc` 会报 `The SocModel doesn't support FP16` 并导致加载失败。请使用 **`--bin`** 下载预编译 HTP context binary。 +::: - + + - ```bash - $ python3 real_esrgan_x4plus/real_esrgan_x4plus.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Super_Resolution/real_esrgan_x4plus/real_esrgan_x4plus.py --chipset 6490 --bin --no_show +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + + - 152.5ms [WARNING] Time: Read model file to memory. 44.96 +:::tip +SC8280XP 与 QCS6490 同为 Hexagon V68，下载模型时使用 `--chipset 6490`，并加 `--bin`。 +::: - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Super_Resolution/real_esrgan_x4plus/real_esrgan_x4plus.py --chipset 6490 --bin --no_show +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Super_Resolution/real_esrgan_x4plus/real_esrgan_x4plus.py --chipset 9075 --no_show +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 631.4ms [WARNING] Time: contextCreateFromBinary. 478.81 +:::tip +QCS9075 上 Hub 默认下发 `.bin`（HTP context）即可运行。`--no_show` 用于无头环境。 +::: - 631.4ms [WARNING] Time: UnmapViewOfFile. 0.00 + + - 633.3ms [WARNING] Time: model_initialize realesrgan 633.22 +`--no_show` 用于无图形界面的 SSH/无头环境，避免弹出看图窗口。 - 2532.0ms [WARNING] Time: model_inference realesrgan 1851.89 +### 预期结果 - 0.0ms [WARNING] This META does not have Alloc2 Support +推理成功后会生成： - 0.0ms [WARNING] This META does not have Alloc2 Support +```text +ComputerVision/Super_Resolution/real_esrgan_x4plus/output.png +``` - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmpczsph22w.PNG' - 0.0ms [WARNING] This META does not have Alloc2 Support +日志中可见类似 `Inference completed` 与输出路径信息。 - 0.0ms [WARNING] This META does not have Alloc2 Support +:::tip +也可使用启动器： - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 8196.8ms [WARNING] Time: model_destroy realesrgan 164.68 - ``` +```bash +python3 run_inference.py --model real_esrgan_x4plus --args "--chipset 6490 --bin --no_show" +``` -
- - output image -
+::: diff --git a/docs/common/ai/qualcomm/_regnet_qai-app-builder.mdx b/docs/common/ai/qualcomm/_regnet_qai-app-builder.mdx deleted file mode 100644 index ba281a851..000000000 --- a/docs/common/ai/qualcomm/_regnet_qai-app-builder.mdx +++ /dev/null @@ -1,140 +0,0 @@ -此文档讲述使用 [QAI AppBuilder](../qai-appbuilder) Python API 利用 Qualcomm® Hexagon™ Processor (NPU) 推理 [RegNet](https://aihub.qualcomm.com/models/regnet) -目标识别模型。 - -**示例支持设备** - -| 设备 | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## 安装 QAI AppBuilder - -:::tip - -1. 请根据 [**QAI AppBuilder 安装方法**](../qai-appbuilder) 安装 QAI AppBuilder。 - -2. 请根据 [**创建 ADSP 环境变量**](../qai-appbuilder#创建-adsp-环境变量) 配置 ADSP 环境变量。 - ::: - -## 运行示例 - -### 安装依赖 - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### 运行脚本 - -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- 准备输入图片，这里以以下图片为输入示例 - - {" "} - -
- - input image -
- -- 执行推理 - - - - ```bash - python3 regnet/regnet.py - ``` - - - - ```bash - $ python3 regnet/regnet.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/regnet - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 123.3ms [WARNING] Time: Read model file to memory. 9.22 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 143.9ms [WARNING] Time: contextCreateFromBinary. 20.52 - - 144.0ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 144.5ms [WARNING] Time: model_initialize regnet 144.44 - - 212.3ms [WARNING] Time: model_inference regnet 4.75 - - Top 5 predictions for image: - - Samoyed 0.7615189552 - Pomeranian 0.1356044859 - keeshond 0.0481537655 - white wolf 0.0241472572 - Eskimo dog 0.0072157839 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 223.8ms [WARNING] Time: model_destroy regnet 8.95 - ``` - - 打印结果可以看到 `Samoyed` 置信度最高，这与输入图片内容吻合。 diff --git a/docs/common/ai/qualcomm/_resnet3d_qai-app-builder.mdx b/docs/common/ai/qualcomm/_resnet3d_qai-app-builder.mdx index fc46c0207..8bd8746db 100644 --- a/docs/common/ai/qualcomm/_resnet3d_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_resnet3d_qai-app-builder.mdx @@ -3,8 +3,8 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | +| 设备 | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -20,121 +20,60 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers av +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` ### 运行脚本 -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- 进入上游示例目录 - + - - - - -- 准备输入图片，这里以以下视频为输入示例 + ```bash + cd qai-appbuilder/samples + ``` - {" "} + - {" "} +- 准备输入数据（示例自带输入文件时可直接运行；也可按脚本参数指定） -
- -

- input video -

-
+
+ +

+ input video +

+
- 执行推理 ```bash - python3 resnet_3d/resnet_3d.py + python3 ComputerVision/Video_Classification/resnet_3d/resnet_3d.py --chipset 9075 ``` - ```bash - $ python3 resnet_3d/resnet_3d.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 157.9ms [WARNING] Time: Read model file to memory. 35.20 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 213.3ms [WARNING] Time: contextCreateFromBinary. 55.38 - - 213.4ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 215.6ms [WARNING] Time: model_initialize resnet_3d 215.54 - - /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/.venv/lib/python3.12/site-packages/torchvision/io/_video_deprecation_warning.py:5: UserWarning: The video decoding and encoding capabilities of torchvision are deprecated from version 0.22 and will be removed in version 0.24. We recommend that you migrate to TorchCodec, where we'll consolidate the future decoding/encoding capabilities of PyTorch: https://github.com/pytorch/torchcodec - warnings.warn( - 3691.5ms [WARNING] Time: model_inference resnet_3d 27.89 - - Top 5 predictions: - surfing water, kitesurfing, windsurfing, jetskiing, water sliding - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 3795.4ms [WARNING] Time: model_destroy resnet_3d 92.13 - ``` +:::tip +首次运行会通过 Qualcomm® AI Hub 自动下载模型，耗时取决于网络。也可在 `samples` 目录使用启动器： - 识别结果 +```bash +python3 run_inference.py --list +python3 run_inference.py --model resnet_3d --args "--chipset 9075" +``` - ```bash - Top 5 predictions: - surfing water, kitesurfing, windsurfing, jetskiing, water sliding - ``` +::: diff --git a/docs/common/ai/qualcomm/_sd1_5_qai-app-builder.mdx b/docs/common/ai/qualcomm/_sd1_5_qai-app-builder.mdx index 2f34e6e9c..43d38de56 100644 --- a/docs/common/ai/qualcomm/_sd1_5_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_sd1_5_qai-app-builder.mdx @@ -3,8 +3,8 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | +| 设备 | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -20,89 +20,61 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers diffusers +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless transformers diffusers ``` ### 运行脚本 -- 进入示例目录 - - +- 进入上游示例目录 - + - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` + ```bash + cd qai-appbuilder/samples + ``` - + - +- 准备输入数据（示例自带输入文件时可直接运行；也可按脚本参数指定） - +
+ + output image +
- 执行推理 ```bash - python3 stable_diffusion_v1_5/stable_diffusion_v1_5.py --prompt spectacular view of northern lights from Alaska + python3 GenerativeAI/Image_Generation/stable_diffusion_v1_5/stable_diffusion_v1_5.py --chipset 9075 --prompt "a cat" ``` - ```bash - $ python3 stable_diffusion_v1_5/stable_diffusion_v1_5.py --prompt spectacular view of northern lights from Alaska - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - Step 0 Running... - Step 1 Running... - Step 2 Running... - Step 3 Running... - Step 4 Running... - Step 5 Running... - Step 6 Running... - Step 7 Running... - Step 8 Running... - Step 9 Running... - Step 10 Running... - Step 11 Running... - Step 12 Running... - Step 13 Running... - Step 14 Running... - Step 15 Running... - Step 16 Running... - Step 17 Running... - Step 18 Running... - Step 19 Running... - /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/samples/python/stable_diffusion_v1_5/stable_diffusion_v1_5.py:255: DeprecationWarning: 'mode' parameter is deprecated and will be removed in Pillow 13 (2026-10-15) - output_image = Image.fromarray(output_image, mode="RGB") - Image saved to '/mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/samples/python/stable_diffusion_v1_5/images/2025_10_30_03_55_02_6303777164_512.jpg' - time consumes for inference 5.059798002243042(s) - Logs will be sent to the system's default channel - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmpyor75jks.PNG' - Logs will be sent to the system's default channel - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - ``` +成功后会在示例目录下生成图片，例如： - {" "} +```text +GenerativeAI/Image_Generation/stable_diffusion_v1_5/images/_512.jpg +``` + +AIRbox Q900 上首次运行会下载 Text Encoder / UNet / VAE（合计约 1 GB 级），网络较慢时可能需数十分钟到数小时。 + +:::tip +依赖 `transformers` 与 `diffusers`。推荐直接运行脚本（避免 `run_inference.py --args` 对含空格 prompt 的引号问题）： + +```bash +python3 GenerativeAI/Image_Generation/stable_diffusion_v1_5/stable_diffusion_v1_5.py --chipset 9075 --prompt "a cat" +``` -
- - output image -
+::: diff --git a/docs/common/ai/qualcomm/_sd2_1_qai-app-builder.mdx b/docs/common/ai/qualcomm/_sd2_1_qai-app-builder.mdx index f8e6de1f6..0a99f013a 100644 --- a/docs/common/ai/qualcomm/_sd2_1_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_sd2_1_qai-app-builder.mdx @@ -3,8 +3,8 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | +| 设备 | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -20,89 +20,61 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers diffusers +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless transformers diffusers ``` ### 运行脚本 -- 进入示例目录 - - +- 进入上游示例目录 - + - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` + ```bash + cd qai-appbuilder/samples + ``` - + - +- 准备输入数据（示例自带输入文件时可直接运行；也可按脚本参数指定） - +
+ + output image +
- 执行推理 ```bash - python3 stable_diffusion_v2_1/stable_diffusion_v2_1.py --prompt spectacular view of northern lights from Alaska + python3 GenerativeAI/Image_Generation/stable_diffusion_v2_1/stable_diffusion_v2_1.py --chipset 9075 --prompt "a cat" ``` - ```bash - python3 stable_diffusion_v2_1/stable_diffusion_v2_1.py --prompt spectacular view of northern lights from Alaska - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - Step 0 Running... - Step 1 Running... - Step 2 Running... - Step 3 Running... - Step 4 Running... - Step 5 Running... - Step 6 Running... - Step 7 Running... - Step 8 Running... - Step 9 Running... - Step 10 Running... - Step 11 Running... - Step 12 Running... - Step 13 Running... - Step 14 Running... - Step 15 Running... - Step 16 Running... - Step 17 Running... - Step 18 Running... - Step 19 Running... - /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/samples/python/stable_diffusion_v2_1/stable_diffusion_v2_1.py:275: DeprecationWarning: 'mode' parameter is deprecated and will be removed in Pillow 13 (2026-10-15) - output_image = Image.fromarray(output_image, mode="RGB") - Image saved to '/mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/samples/python/stable_diffusion_v2_1/images/2025_10_30_05_46_28_2081127356_512.jpg' - time consumes for inference 4.454082250595093(s) - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - Logs will be sent to the system's default channel - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmp0ci7lglx.PNG' - Logs will be sent to the system's default channel - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - ``` +成功后会在示例目录下生成图片，例如： - {" "} +```text +GenerativeAI/Image_Generation/stable_diffusion_v2_1/images/_512.jpg +``` + +AIRbox Q900 上首次运行会下载 Text Encoder / UNet / VAE 三份模型（合计约 1.2 GB 级），网络较慢时可能需数十分钟到数小时。 + +:::tip +依赖 `transformers` 与 `diffusers`。推荐直接运行脚本（避免 `run_inference.py --args` 对含空格 prompt 的引号问题）： + +```bash +python3 GenerativeAI/Image_Generation/stable_diffusion_v2_1/stable_diffusion_v2_1.py --chipset 9075 --prompt "a cat" +``` -
- - output image -
+::: diff --git a/docs/common/ai/qualcomm/_sesr_m5_qai-app-builder.mdx b/docs/common/ai/qualcomm/_sesr_m5_qai-app-builder.mdx deleted file mode 100644 index 758ccc1a0..000000000 --- a/docs/common/ai/qualcomm/_sesr_m5_qai-app-builder.mdx +++ /dev/null @@ -1,147 +0,0 @@ -此文档讲述使用 [QAI AppBuilder](../qai-appbuilder) Python API 利用 Qualcomm® Hexagon™ Processor (NPU) 推理 [SESR-M5](https://aihub.qualcomm.com/models/sesr_m5) -图像超分模型。 - -**示例支持设备** - -| 设备 | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## 安装 QAI AppBuilder - -:::tip - -1. 请根据 [**QAI AppBuilder 安装方法**](../qai-appbuilder) 安装 QAI AppBuilder。 - -2. 请根据 [**创建 ADSP 环境变量**](../qai-appbuilder#创建-adsp-环境变量) 配置 ADSP 环境变量。 - ::: - -## 运行示例 - -### 安装依赖 - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### 运行脚本 - -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- 准备输入图片，这里以以下图片为输入示例 - - {" "} - -
- - input image -
- -- 执行推理 - - - - ```bash - python3 sesr_m5/sesr_m5.py - ``` - - - - ```bash - $ python3 sesr_m5/sesr_m5.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/sesr_m5 - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 113.8ms [WARNING] Time: Read model file to memory. 0.39 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 119.7ms [WARNING] Time: contextCreateFromBinary. 5.87 - - 119.8ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 119.9ms [WARNING] Time: model_initialize sesr_m5 119.89 - - 152.4ms [WARNING] Time: model_inference sesr_m5 11.00 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 330.6ms [WARNING] Time: model_destroy sesr_m5 18.65 - - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmpxodzub9w.PNG' - ``` - -
- - output image -
diff --git a/docs/common/ai/qualcomm/_shufflenet_v2_qai-app-builder.mdx b/docs/common/ai/qualcomm/_shufflenet_v2_qai-app-builder.mdx deleted file mode 100644 index a3c4a830a..000000000 --- a/docs/common/ai/qualcomm/_shufflenet_v2_qai-app-builder.mdx +++ /dev/null @@ -1,140 +0,0 @@ -此文档讲述使用 [QAI AppBuilder](../qai-appbuilder) Python API 利用 Qualcomm® Hexagon™ Processor (NPU) 推理 [Shufflenet-v2](https://aihub.qualcomm.com/models/shufflenet_v2) -目标识别模型。 - -**示例支持设备** - -| 设备 | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## 安装 QAI AppBuilder - -:::tip - -1. 请根据 [**QAI AppBuilder 安装方法**](../qai-appbuilder) 安装 QAI AppBuilder。 - -2. 请根据 [**创建 ADSP 环境变量**](../qai-appbuilder#创建-adsp-环境变量) 配置 ADSP 环境变量。 - ::: - -## 运行示例 - -### 安装依赖 - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### 运行脚本 - -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- 准备输入图片，这里以以下图片为输入示例 - - {" "} - -
- - input image -
- -- 执行推理 - - - - ```bash - python3 shufflenet_v2/shufflenet_v2.py - ``` - - - - ```bash - $ python3 shufflenet_v2/shufflenet_v2.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/shufflenet_v2 - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 115.6ms [WARNING] Time: Read model file to memory. 0.91 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 125.6ms [WARNING] Time: contextCreateFromBinary. 10.02 - - 125.7ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 125.9ms [WARNING] Time: model_initialize shufflenet_v2 125.87 - - 199.9ms [WARNING] Time: model_inference shufflenet_v2 2.19 - - Top 5 predictions for image: - - Samoyed 0.8289067745 - wallaby 0.0915796161 - Pomeranian 0.0265247617 - Great Pyrenees 0.0101179369 - hare 0.0050830133 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 219.4ms [WARNING] Time: model_destroy shufflenet_v2 17.56 - ``` - - 打印结果可以看到 `Samoyed` 置信度最高，这与输入图片内容吻合。 diff --git a/docs/common/ai/qualcomm/_squeezenet1_1_qai-app-builder.mdx b/docs/common/ai/qualcomm/_squeezenet1_1_qai-app-builder.mdx deleted file mode 100644 index 7a2e02bff..000000000 --- a/docs/common/ai/qualcomm/_squeezenet1_1_qai-app-builder.mdx +++ /dev/null @@ -1,140 +0,0 @@ -此文档讲述使用 [QAI AppBuilder](../qai-appbuilder) Python API 利用 Qualcomm® Hexagon™ Processor (NPU) 推理 [SqueezeNet-1.1](https://aihub.qualcomm.com/models/squeezenet1_1) -目标识别模型。 - -**示例支持设备** - -| 设备 | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## 安装 QAI AppBuilder - -:::tip - -1. 请根据 [**QAI AppBuilder 安装方法**](../qai-appbuilder) 安装 QAI AppBuilder。 - -2. 请根据 [**创建 ADSP 环境变量**](../qai-appbuilder#创建-adsp-环境变量) 配置 ADSP 环境变量。 - ::: - -## 运行示例 - -### 安装依赖 - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### 运行脚本 - -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- 准备输入图片，这里以以下图片为输入示例 - - {" "} - -
- - input image -
- -- 执行推理 - - - - ```bash - python3 squeezenet1_1/squeezenet1_1.py - ``` - - - - ```bash - $ python3 squeezenet1_1/squeezenet1_1.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/squeezenet1_1 - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 120.9ms [WARNING] Time: Read model file to memory. 2.75 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 132.8ms [WARNING] Time: contextCreateFromBinary. 11.81 - - 133.0ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 133.2ms [WARNING] Time: model_initialize squeezenet1_1 133.16 - - 225.5ms [WARNING] Time: model_inference squeezenet1_1 2.50 - - Top 5 predictions for image: - - Samoyed 0.8289067745 - wallaby 0.0915796161 - Pomeranian 0.0265247617 - Great Pyrenees 0.0101179369 - hare 0.0050830133 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 249.2ms [WARNING] Time: model_destroy squeezenet1_1 20.91 - ``` - - 打印结果可以看到 `Samoyed` 置信度最高，这与输入图片内容吻合。 diff --git a/docs/common/ai/qualcomm/_unet_seg_qai-app-builder.mdx b/docs/common/ai/qualcomm/_unet_seg_qai-app-builder.mdx index cd7225aae..edaa0c900 100644 --- a/docs/common/ai/qualcomm/_unet_seg_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_unet_seg_qai-app-builder.mdx @@ -3,8 +3,8 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | +| 设备 | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -20,126 +20,64 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` ### 运行脚本 -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- 进入上游示例目录 - + - - - + ```bash + cd qai-appbuilder/samples + ``` -- 准备输入图片，这里以以下图片为输入示例 + - {" "} +- 准备输入数据（示例自带输入文件时可直接运行；也可按脚本参数指定） -
- - input image -
+
+ + input image +
- 执行推理 ```bash - python3 unet_segmentation/unet_segmentation.py + python3 ComputerVision/Semantic_Segmentation/unet_segmentation/unet_segmentation.py --chipset 9075 ``` - ```bash - $ python3 unet_segmentation/unet_segmentation.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 153.4ms [WARNING] Time: Read model file to memory. 35.26 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +- 运行结果示意 - 0.0ms [WARNING] This META does not have Alloc2 Support +
+ + output image +
- 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 264.9ms [WARNING] Time: contextCreateFromBinary. 111.40 - - 265.0ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 267.2ms [WARNING] Time: model_initialize unet_segmentation 267.13 - - 627.1ms [WARNING] Time: model_inference unet_segmentation 246.67 - - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmp99xq_ell.PNG' - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /usr/bin/xdg-open: 882: www-browser: not found - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 2383.4ms [WARNING] Time: model_destroy unet_segmentation 67.34 - - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmp0hq03xtm.PNG' - ``` +:::tip +首次运行会通过 Qualcomm® AI Hub 自动下载模型，耗时取决于网络。也可在 `samples` 目录使用启动器： - {" "} +```bash +python3 run_inference.py --list +python3 run_inference.py --model unet_segmentation --args "--chipset 9075" +``` -
- - output image -
+::: diff --git a/docs/common/ai/qualcomm/_vit_qai-app-builder.mdx b/docs/common/ai/qualcomm/_vit_qai-app-builder.mdx deleted file mode 100644 index 97982c3b2..000000000 --- a/docs/common/ai/qualcomm/_vit_qai-app-builder.mdx +++ /dev/null @@ -1,140 +0,0 @@ -此文档讲述使用 [QAI AppBuilder](../qai-appbuilder) Python API 利用 Qualcomm® Hexagon™ Processor (NPU) 推理 [VIT](https://aihub.qualcomm.com/models/vit) -目标识别模型。 - -**示例支持设备** - -| 设备 | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## 安装 QAI AppBuilder - -:::tip - -1. 请根据 [**QAI AppBuilder 安装方法**](../qai-appbuilder) 安装 QAI AppBuilder。 - -2. 请根据 [**创建 ADSP 环境变量**](../qai-appbuilder#创建-adsp-环境变量) 配置 ADSP 环境变量。 - ::: - -## 运行示例 - -### 安装依赖 - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### 运行脚本 - -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- 准备输入图片，这里以以下图片为输入示例 - - {" "} - -
- - input image -
- -- 执行推理 - - - - ```bash - python3 vit/vit.py - ``` - - - - ```bash - $ python3 vit/vit.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/vit - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 148.6ms [WARNING] Time: Read model file to memory. 32.50 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 351.2ms [WARNING] Time: contextCreateFromBinary. 202.52 - - 351.2ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 351.9ms [WARNING] Time: model_initialize vit 351.87 - - 560.6ms [WARNING] Time: model_inference vit 159.35 - - Top 5 predictions for image: - - Samoyed 0.7157644629 - Pomeranian 0.0566752702 - keeshond 0.0083217919 - Eskimo dog 0.0057556415 - Japanese spaniel 0.003955483 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 585.7ms [WARNING] Time: model_destroy vit 20.11 - ``` - - 打印结果可以看到 `Samoyed` 置信度最高，这与输入图片内容吻合。 diff --git a/docs/common/ai/qualcomm/_whisper_base_en_qai-app-builder.mdx b/docs/common/ai/qualcomm/_whisper_base_en_qai-app-builder.mdx index 801ea5dfd..922b23ce1 100644 --- a/docs/common/ai/qualcomm/_whisper_base_en_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_whisper_base_en_qai-app-builder.mdx @@ -3,8 +3,8 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | +| 设备 | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -20,46 +20,38 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib openai-whisper audio2numpy samplerate transformers qai_hub_models==0.30.2 +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless audio2numpy ``` ### 运行脚本 -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- 进入上游示例目录 - + - - - + ```bash + cd qai-appbuilder/samples + ``` -- 准备输入音频，这里以以下音频为输入示例 + - {" "} +- 准备输入数据（示例自带输入文件时可直接运行；也可按脚本参数指定） -
+

input audio

@@ -71,181 +63,23 @@ pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matp ```bash - python3 whisper_base_en/whisper_base_en.py + python3 audio/Speech_Recognition/whisper_base_en/whisper_base_en.py --chipset 9075 ``` - ```bash - $ python3 whisper_base_en/whisper_base_en.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 184.8ms [WARNING] Time: Read model file to memory. 71.89 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 242.3ms [WARNING] Time: contextCreateFromBinary. 57.39 - - 242.3ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 244.9ms [WARNING] Time: model_initialize whisper_decoder 244.80 - - 282.6ms [WARNING] Time: Read model file to memory. 37.32 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 636.8ms [WARNING] Time: contextCreateFromBinary. 354.14 - - 636.9ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 638.6ms [WARNING] Time: model_initialize whisper_encoder 393.56 - - 1135.2ms [WARNING] Time: model_inference whisper_encoder 440.09 - - time consumes for encoder 0.4419412612915039(s) - Decoder Inference k_cache_cross type shape (6, 8, 64, 1500) type float32 - Decoder Inference v_cache_cross type shape (6, 8, 1500, 64) type float32 - start decode sample_len 224 - 1470.7ms [WARNING] Time: model_inference whisper_decoder 323.97 - - time consumes for decoder 0.3242673873901367(s) - 1796.0ms [WARNING] Time: model_inference whisper_decoder 322.57 - - time consumes for decoder 0.3229062557220459(s) - 2111.1ms [WARNING] Time: model_inference whisper_decoder 313.66 +默认输入为示例自带的 `jfk.wav`。成功时终端会打印转写结果，例如： - time consumes for decoder 0.31383657455444336(s) - 2425.6ms [WARNING] Time: model_inference whisper_decoder 313.01 - - time consumes for decoder 0.3132438659667969(s) - 2740.2ms [WARNING] Time: model_inference whisper_decoder 313.24 - - time consumes for decoder 0.3134174346923828(s) - 3055.0ms [WARNING] Time: model_inference whisper_decoder 313.45 - - time consumes for decoder 0.3136253356933594(s) - 3369.6ms [WARNING] Time: model_inference whisper_decoder 313.31 - - time consumes for decoder 0.31349658966064453(s) - 3684.4ms [WARNING] Time: model_inference whisper_decoder 313.44 - - time consumes for decoder 0.3136255741119385(s) - 3999.4ms [WARNING] Time: model_inference whisper_decoder 313.63 - - time consumes for decoder 0.3138108253479004(s) - 4314.0ms [WARNING] Time: model_inference whisper_decoder 313.20 - - time consumes for decoder 0.31337976455688477(s) - 4628.7ms [WARNING] Time: model_inference whisper_decoder 313.41 - - time consumes for decoder 0.313596248626709(s) - 4943.4ms [WARNING] Time: model_inference whisper_decoder 313.37 - - time consumes for decoder 0.3135509490966797(s) - 5258.1ms [WARNING] Time: model_inference whisper_decoder 313.29 - - time consumes for decoder 0.31348109245300293(s) - 5572.8ms [WARNING] Time: model_inference whisper_decoder 313.35 - - time consumes for decoder 0.31352949142456055(s) - 5887.5ms [WARNING] Time: model_inference whisper_decoder 313.35 - - time consumes for decoder 0.3135380744934082(s) - 6201.4ms [WARNING] Time: model_inference whisper_decoder 312.50 - - time consumes for decoder 0.31267857551574707(s) - 6515.8ms [WARNING] Time: model_inference whisper_decoder 313.02 - - time consumes for decoder 0.31319570541381836(s) - 6830.7ms [WARNING] Time: model_inference whisper_decoder 313.52 - - time consumes for decoder 0.31371450424194336(s) - 7145.6ms [WARNING] Time: model_inference whisper_decoder 313.57 - - time consumes for decoder 0.31376171112060547(s) - 7459.9ms [WARNING] Time: model_inference whisper_decoder 313.03 - - time consumes for decoder 0.3132154941558838(s) - 7774.9ms [WARNING] Time: model_inference whisper_decoder 313.60 - - time consumes for decoder 0.3137829303741455(s) - 8089.4ms [WARNING] Time: model_inference whisper_decoder 313.16 - - time consumes for decoder 0.31334543228149414(s) - 8404.7ms [WARNING] Time: model_inference whisper_decoder 313.92 - - time consumes for decoder 0.31411004066467285(s) - 8719.6ms [WARNING] Time: model_inference whisper_decoder 313.62 - - time consumes for decoder 0.31380605697631836(s) - 9034.5ms [WARNING] Time: model_inference whisper_decoder 313.51 - - time consumes for decoder 0.31369900703430176(s) - 9349.9ms [WARNING] Time: model_inference whisper_decoder 314.01 - - time consumes for decoder 0.31418848037719727(s) - 9664.7ms [WARNING] Time: model_inference whisper_decoder 313.53 - - time consumes for decoder 0.3137087821960449(s) - 9979.7ms [WARNING] Time: model_inference whisper_decoder 313.63 - - time consumes for decoder 0.3138093948364258(s) - Transcription: And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country. - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 10095.0ms [WARNING] Time: model_destroy whisper_decoder 13.25 - - Logs will be sent to the system's default channel - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 10198.2ms [WARNING] Time: model_destroy whisper_encoder 103.03 +```text +Transcription: And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country. +``` - ``` +:::tip +首次运行会通过 Qualcomm® AI Hub 自动下载 encoder/decoder 模型。解码阶段依赖 `tiktoken` 词表；若设备访问 `openaipublic.blob.core.windows.net` 失败，可先在可联网环境预热 `tiktoken.get_encoding("gpt2")` 缓存。 - 识别结果 +```bash +python3 run_inference.py --list +python3 run_inference.py --model whisper_base_en --args "--chipset 9075" +``` - ```bash - Transcription: And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country. - ``` +::: diff --git a/docs/common/ai/qualcomm/_whisper_tiny_en_qai-app-builder.mdx b/docs/common/ai/qualcomm/_whisper_tiny_en_qai-app-builder.mdx index 849986e32..da048bf9c 100644 --- a/docs/common/ai/qualcomm/_whisper_tiny_en_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_whisper_tiny_en_qai-app-builder.mdx @@ -3,8 +3,8 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | +| 设备 | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -20,46 +20,38 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib openai-whisper audio2numpy samplerate transformers qai_hub_models==0.30.2 +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless audio2numpy ``` ### 运行脚本 -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- 进入上游示例目录 - + - - - + ```bash + cd qai-appbuilder/samples + ``` -- 准备输入音频，这里以以下音频为输入示例 + - {" "} +- 准备输入数据（示例自带输入文件时可直接运行；也可按脚本参数指定） -
+

input audio

@@ -71,151 +63,23 @@ pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matp ```bash - python3 whisper_tiny_en/whisper_tiny_en.py + python3 audio/Speech_Recognition/whisper_tiny_en/whisper_tiny_en.py --chipset 9075 ``` - ```bash - $ python3 whisper_tiny_en/whisper_tiny_en.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 175.0ms [WARNING] Time: Read model file to memory. 51.68 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 224.3ms [WARNING] Time: contextCreateFromBinary. 49.16 - - 224.3ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 226.7ms [WARNING] Time: model_initialize whisper_decoder 226.65 - - 245.7ms [WARNING] Time: Read model file to memory. 18.37 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 311.9ms [WARNING] Time: contextCreateFromBinary. 66.16 - - 312.0ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 313.2ms [WARNING] Time: model_initialize whisper_encoder 86.17 - - 567.2ms [WARNING] Time: model_inference whisper_encoder 199.78 - - Decoder Inference k_cache_cross type shape (4, 6, 64, 1500) type float32 - Decoder Inference v_cache_cross type shape (4, 6, 1500, 64) type float32 - start decode sample_len 224 - 728.7ms [WARNING] Time: model_inference whisper_decoder 159.73 - - 887.8ms [WARNING] Time: model_inference whisper_decoder 157.66 - - 1046.9ms [WARNING] Time: model_inference whisper_decoder 157.76 +默认输入为示例自带的 `jfk.wav`。成功时终端会打印转写结果，例如： - 1205.3ms [WARNING] Time: model_inference whisper_decoder 157.02 - - 1365.0ms [WARNING] Time: model_inference whisper_decoder 158.39 - - 1523.8ms [WARNING] Time: model_inference whisper_decoder 157.56 - - 1682.7ms [WARNING] Time: model_inference whisper_decoder 157.58 - - 1841.0ms [WARNING] Time: model_inference whisper_decoder 157.04 - - 2000.0ms [WARNING] Time: model_inference whisper_decoder 157.82 - - 2158.6ms [WARNING] Time: model_inference whisper_decoder 157.36 - - 2317.7ms [WARNING] Time: model_inference whisper_decoder 157.76 - - 2476.8ms [WARNING] Time: model_inference whisper_decoder 157.78 - - 2635.7ms [WARNING] Time: model_inference whisper_decoder 157.58 - - 2794.4ms [WARNING] Time: model_inference whisper_decoder 157.39 - - 2953.2ms [WARNING] Time: model_inference whisper_decoder 157.47 - - 3111.9ms [WARNING] Time: model_inference whisper_decoder 157.46 - - 3270.2ms [WARNING] Time: model_inference whisper_decoder 157.10 - - 3429.0ms [WARNING] Time: model_inference whisper_decoder 157.66 - - 3588.0ms [WARNING] Time: model_inference whisper_decoder 157.72 - - 3747.3ms [WARNING] Time: model_inference whisper_decoder 157.92 - - 3906.3ms [WARNING] Time: model_inference whisper_decoder 157.68 - - 4065.3ms [WARNING] Time: model_inference whisper_decoder 157.71 - - 4224.5ms [WARNING] Time: model_inference whisper_decoder 157.89 - - 4383.8ms [WARNING] Time: model_inference whisper_decoder 158.02 - - 4542.9ms [WARNING] Time: model_inference whisper_decoder 157.74 - - 4702.4ms [WARNING] Time: model_inference whisper_decoder 158.21 - - 4861.6ms [WARNING] Time: model_inference whisper_decoder 157.92 - - 5021.1ms [WARNING] Time: model_inference whisper_decoder 158.13 - - Transcription: And so my fellow Americans ask not what your country can do for you ask what you can do for your country. - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 5134.7ms [WARNING] Time: model_destroy whisper_decoder 14.94 - - Logs will be sent to the system's default channel - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +```text +Transcription: And so my fellow Americans ask not what your country can do for you, ask what you can do for your country. +``` - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 5208.4ms [WARNING] Time: model_destroy whisper_encoder 73.58 - ``` +:::tip +首次运行会通过 Qualcomm® AI Hub 自动下载 encoder/decoder 模型。解码阶段依赖 `tiktoken` 词表；若设备访问 `openaipublic.blob.core.windows.net` 失败，可先在可联网环境预热 `tiktoken.get_encoding("gpt2")` 缓存。 - 识别结果 +```bash +python3 run_inference.py --list +python3 run_inference.py --model whisper_tiny_en --args "--chipset 9075" +``` - ```bash - Transcription: And so my fellow Americans ask not what your country can do for you ask what you can do for your country. - ``` +::: diff --git a/docs/common/ai/qualcomm/_wideresnet50_qai-app-builder.mdx b/docs/common/ai/qualcomm/_wideresnet50_qai-app-builder.mdx deleted file mode 100644 index e03ce5993..000000000 --- a/docs/common/ai/qualcomm/_wideresnet50_qai-app-builder.mdx +++ /dev/null @@ -1,140 +0,0 @@ -此文档讲述使用 [QAI AppBuilder](../qai-appbuilder) Python API 利用 Qualcomm® Hexagon™ Processor (NPU) 推理 [WideResNet50](https://aihub.qualcomm.com/models/wideresnet50) -目标识别模型。 - -**示例支持设备** - -| 设备 | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## 安装 QAI AppBuilder - -:::tip - -1. 请根据 [**QAI AppBuilder 安装方法**](../qai-appbuilder) 安装 QAI AppBuilder。 - -2. 请根据 [**创建 ADSP 环境变量**](../qai-appbuilder#创建-adsp-环境变量) 配置 ADSP 环境变量。 - ::: - -## 运行示例 - -### 安装依赖 - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### 运行脚本 - -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- 准备输入图片，这里以以下图片为输入示例 - - {" "} - -
- - input image -
- -- 执行推理 - - - - ```bash - python3 wideresnet50/wideresnet50.py - ``` - - - - ```bash - $ python3 wideresnet50/wideresnet50.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/wideresnet50 - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 146.7ms [WARNING] Time: Read model file to memory. 30.54 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 179.0ms [WARNING] Time: contextCreateFromBinary. 32.18 - - 179.0ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 179.5ms [WARNING] Time: model_initialize wideresnet50 179.49 - - 243.2ms [WARNING] Time: model_inference wideresnet50 9.66 - - Top 5 predictions for image: - - Samoyed 0.8442133665 - keeshond 0.0415370725 - Pomeranian 0.0302517358 - white wolf 0.0258170888 - chow 0.0072637959 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 258.1ms [WARNING] Time: model_destroy wideresnet50 11.75 - ``` - - 打印结果可以看到 `Samoyed` 置信度最高，这与输入图片内容吻合。 diff --git a/docs/common/ai/qualcomm/_xlsr_qai-app-builder.mdx b/docs/common/ai/qualcomm/_xlsr_qai-app-builder.mdx deleted file mode 100644 index ca6cf7d40..000000000 --- a/docs/common/ai/qualcomm/_xlsr_qai-app-builder.mdx +++ /dev/null @@ -1,152 +0,0 @@ -此文档讲述使用 [QAI AppBuilder](../qai-appbuilder) Python API 利用 Qualcomm® Hexagon™ Processor (NPU) 推理 [XLSR](https://aihub.qualcomm.com/models/xlsr) -图像超分模型。 - -**示例支持设备** - -| 设备 | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## 安装 QAI AppBuilder - -:::tip - -1. 请根据 [**QAI AppBuilder 安装方法**](../qai-appbuilder) 安装 QAI AppBuilder。 - -2. 请根据 [**创建 ADSP 环境变量**](../qai-appbuilder#创建-adsp-环境变量) 配置 ADSP 环境变量。 - ::: - -## 运行示例 - -### 安装依赖 - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers openai-clip -``` - - - -### 运行脚本 - -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- 准备输入图片，这里以以下图片为输入示例 - - {" "} - -
- - input image -
- -- 执行推理 - - - - ```bash - python3 xlsr/xlsr.py - ``` - - - - ```bash - $ python3 xlsr/xlsr.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/xlsr - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 113.4ms [WARNING] Time: Read model file to memory. 0.41 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 121.2ms [WARNING] Time: contextCreateFromBinary. 7.83 - - 121.3ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 121.5ms [WARNING] Time: model_initialize xlsr 121.46 - - 164.3ms [WARNING] Time: model_inference xlsr 7.48 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 300.2ms [WARNING] Time: model_destroy xlsr 6.65 - - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmp5i2_qhdq.PNG' - ``` - - {" "} - -
- - output image -
diff --git a/docs/common/ai/qualcomm/_yamnet_qai-app-builder.mdx b/docs/common/ai/qualcomm/_yamnet_qai-app-builder.mdx index 298d8e45d..932ad31ef 100644 --- a/docs/common/ai/qualcomm/_yamnet_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_yamnet_qai-app-builder.mdx @@ -3,8 +3,8 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | +| 设备 | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -20,46 +20,38 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision resampy soundfile +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless soxr soundfile resampy ``` ### 运行脚本 -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- 进入上游示例目录 - + - - - + ```bash + cd qai-appbuilder/samples + ``` -- 准备输入图片，这里以以下图片为输入示例 + - {" "} +- 准备输入数据（示例自带输入文件时可直接运行；也可按脚本参数指定） -
+

input audio

@@ -71,66 +63,24 @@ pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision resa ```bash - python3 yamnet/yamnet.py + python3 audio/Audio_Classification/yamnet/yamnet.py --chipset 9075 ``` - ```bash - $ python3 yamnet/yamnet.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 113.7ms [WARNING] Time: Read model file to memory. 4.55 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +默认输入为示例自带的 `input.wav`。成功时终端会打印 Top 5 类别，例如： - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 127.6ms [WARNING] Time: contextCreateFromBinary. 13.42 - - 128.0ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 129.0ms [WARNING] Time: model_initialize yamnet 128.97 - - 145.8ms [WARNING] Time: model_inference yamnet 0.80 - - Top 5 predictions: - Whistling | Alarm | Whistle | Field recording | Rattle (instrument) - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +```text +Top 5 predictions: +Whistling | Alarm | Whistle | Field recording | Rattle (instrument) +``` - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 229.1ms [WARNING] Time: model_destroy yamnet 79.87 - ``` +:::tip +首次运行会通过 Qualcomm® AI Hub 自动下载模型，并依赖 `soxr` 等音频库。也可在 `samples` 目录使用启动器： - 识别结果 +```bash +python3 run_inference.py --list +python3 run_inference.py --model yamnet --args "--chipset 9075" +``` - ```bash - Top 5 predictions: - Whistling | Alarm | Whistle | Field recording | Rattle (instrument) - ``` +::: diff --git a/docs/common/ai/qualcomm/_yolov8_det_qai-app-builder.mdx b/docs/common/ai/qualcomm/_yolov8_det_qai-app-builder.mdx index d75c440a2..879677d87 100644 --- a/docs/common/ai/qualcomm/_yolov8_det_qai-app-builder.mdx +++ b/docs/common/ai/qualcomm/_yolov8_det_qai-app-builder.mdx @@ -3,11 +3,11 @@ **示例支持设备** -| 设备 | SoC | -| -------------------- | ----- | -| Dragon Q6A | QCS6490 | +| 设备 | SoC | +| -------------------- | -------- | +| Dragon Q6A | QCS6490 | | Dragon Q8B | SC8280XP | -| Fogwise® AIRbox Q900 | QCS9075 | +| Fogwise® AIRbox Q900 | QCS9075 | ## 安装 QAI AppBuilder @@ -22,142 +22,159 @@ ### 安装依赖 +在已激活的虚拟环境中安装示例依赖： + ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` ### 运行脚本 -- 进入示例目录 - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - +- 进入上游示例目录 - + - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - + ```bash + cd qai-appbuilder/samples + ``` - + - +- 准备输入数据（示例自带输入文件时可直接运行；也可按脚本参数指定） - +
+ + input image +
- ```bash - cd ai-engine-direct-helper/samples/python - ``` +### 修正输出顺序（QCS6490 / SC8280XP / QCS9075 必做） - +在 Dragon Q6A（`--chipset 6490`）与 AIRbox Q900（`--chipset 9075`）上实测：AI Hub 下发的包常为下列布局： - +| 输出下标 | 形状 | 含义 | +| -------- | -------------- | --------- | +| `0` | `[1, 8400]` | scores | +| `1` | `[1, 8400]` | class_idx | +| `2` | `[1, 8400, 4]` | boxes | - +而上游 `yolov8_det.py` 默认按 `[boxes, scores, class_idx]` 解析，会导致 NMS 阶段： -- 准备输入图片，这里以以下图片为输入示例 +```text +IndexError: index ... is out of bounds for dimension 0 with size ... +``` - {" "} +在运行前，对 `ComputerVision/Object_Detection/yolov8_det/yolov8_det.py` 中 `Inference()` 内解析输出的代码做如下替换（建议先备份）： -
- - input image -
+将： -- 执行推理 +```python + # Run the inference. + model_output = yolov8.Inference(image) - + pred_boxes = torch.tensor(model_output[0].reshape(1, -1, 4)) + pred_scores = torch.tensor(model_output[1].reshape(1, -1)) + pred_class_idx = torch.tensor(model_output[2].reshape(1, -1)) +``` - ```bash - python3 yolov8_det/yolov8_det.py - ``` +替换为： + +```python + # Run the inference. + model_output = yolov8.Inference(image) + + # Output layout differs by model package: + # - some packages: [boxes, scores, class_idx] + # - QCS6490 / QCS9075 hub packages often: [scores, class_idx, boxes] + import numpy as _np + a0, a1, a2 = _np.array(model_output[0]), _np.array(model_output[1]), _np.array(model_output[2]) + if a2.ndim == 3 and a2.shape[-1] == 4: + pred_scores = torch.tensor(a0.reshape(1, -1)) + pred_class_idx = torch.tensor(a1.reshape(1, -1)) + pred_boxes = torch.tensor(a2.reshape(1, -1, 4)) + else: + pred_boxes = torch.tensor(a0.reshape(1, -1, 4)) + pred_scores = torch.tensor(a1.reshape(1, -1)) + pred_class_idx = torch.tensor(a2.reshape(1, -1)) +``` - +该写法会按最后一个输出是否为 `[..., 4]` 自动选择布局，兼容 6490 / 9075 与其它输出顺序。 - ```bash - $ python3 yolov8_det/yolov8_det.py - 0.0ms [WARNING] Initializing HtpProvider +### 执行推理 - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support + + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Object_Detection/yolov8_det/yolov8_det.py --chipset 6490 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 121.1ms [WARNING] Time: Read model file to memory. 6.22 + + - 0.0ms [WARNING] This META does not have Alloc2 Support +:::tip +SC8280XP 与 QCS6490 同为 Hexagon V68，下载模型时使用 `--chipset 6490`，并同样建议应用上方输出顺序修正。 +::: - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Object_Detection/yolov8_det/yolov8_det.py --chipset 6490 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Object_Detection/yolov8_det/yolov8_det.py --chipset 9075 +``` - 151.3ms [WARNING] Time: contextCreateFromBinary. 30.09 + - 151.4ms [WARNING] Time: UnmapViewOfFile. 0.00 + + - 152.3ms [WARNING] Time: model_initialize yolov8 152.21 +### 预期结果 - 230.4ms [WARNING] Time: model_inference yolov8 10.35 +完成输出顺序修正后，在 QCS6490 / QCS9075 上推理成功均会生成： - 0.0ms [WARNING] This META does not have Alloc2 Support +```text +ComputerVision/Object_Detection/yolov8_det/output.png +``` - 0.0ms [WARNING] This META does not have Alloc2 Support +无图形界面时可能打印 `Error: no DISPLAY environment variable specified`；只要 `output.png` 已生成且退出码为 0，可忽略。 - 0.0ms [WARNING] This META does not have Alloc2 Support +- 运行结果示意（示意图片，实际结果取决于输入与模型版本） - 0.0ms [WARNING] This META does not have Alloc2 Support +
+ + output image +
- /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 650.0ms [WARNING] Time: model_destroy yolov8 61.01 +:::tip +也可使用启动器（仍需先完成脚本中的输出顺序修正）： - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmpqk93qmdb.PNG' - ``` +```bash +python3 run_inference.py --model yolov8_det --args "--chipset 6490" +# 或 QCS9075： +# python3 run_inference.py --model yolov8_det --args "--chipset 9075" +``` -
- - output image -
+::: diff --git a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/README.md b/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/README.md index b965c98ce..23b594722 100644 --- a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/README.md +++ b/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/README.md @@ -2,8 +2,10 @@ sidebar_position: 99 --- -# CV 机器视觉示例 +# QAI AppBuilder 示例 -使用 Qai-APPBuilder Python API 利用 NPU 推理模型示例集合。 +基于上游 [QAI AppBuilder](https://github.com/qualcomm/qai-appbuilder) `samples` 中、且在 QCS6490 上可通过 `--chipset 6490` 下载模型的示例。完整菜单可用 `python3 run_inference.py --list` 查看。 + +请先完成 [**QAI AppBuilder 安装**](../qai-appbuilder)。 diff --git a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/convnext-base.md b/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/convnext-base.md deleted file mode 100644 index 51b2ea452..000000000 --- a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/convnext-base.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_convnext_base_qai-app-builder.mdx ---- - -# ConvNext-Base 示例 - -import CONVNEXTBASEQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_convnext_base_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/convnext-tiny.md b/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/convnext-tiny.md deleted file mode 100644 index bae91f430..000000000 --- a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/convnext-tiny.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_convnext_tiny_qai-app-builder.mdx ---- - -# ConvNext-Tiny 示例 - -import CONVNEXTTINYQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_convnext_tiny_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b0.md b/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b0.md deleted file mode 100644 index 6a72c90e2..000000000 --- a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b0.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_efficientnet_b0_qai-app-builder.mdx ---- - -# EfficientNet-B0 示例 - -import EFFICIENTNETB0QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_efficientnet_b0_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b4.md b/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b4.md deleted file mode 100644 index 376dba2ac..000000000 --- a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b4.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_efficientnet_b4_qai-app-builder.mdx ---- - -# EfficientNet-B4 示例 - -import EFFICIENTNETB4QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_efficientnet_b4_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-v2s.md b/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-v2s.md deleted file mode 100644 index b01eb3ce1..000000000 --- a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-v2s.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_efficientnet_v2s_qai-app-builder.mdx ---- - -# EfficientNet-V2-s 示例 - -import EFFICIENTNETV2SQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_efficientnet_v2s_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/fcn-resnet50.md b/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/fcn-resnet50.md deleted file mode 100644 index fc7ad3034..000000000 --- a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/fcn-resnet50.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_fcn-resnet50_qai-app-builder.mdx ---- - -# FCN-ResNet50 示例 - -import FCNRESNRT50QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_fcn-resnet50_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/levit.md b/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/levit.md deleted file mode 100644 index b47c4824d..000000000 --- a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/levit.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_levit_qai-app-builder.mdx ---- - -# LeViT 示例 - -import LEVITQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_levit_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/regnet.md b/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/regnet.md deleted file mode 100644 index e592cfa3f..000000000 --- a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/regnet.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_regnet_qai-app-builder.mdx ---- - -# RegNet 示例 - -import REGNETQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_regnet_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/sesr-m5.md b/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/sesr-m5.md deleted file mode 100644 index 5a6bbe267..000000000 --- a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/sesr-m5.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_sesr_m5_qai-app-builder.mdx ---- - -# SESR-M5 示例 - -import SESRM5QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_sesr_m5_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/shufflenet-v2.md b/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/shufflenet-v2.md deleted file mode 100644 index f5c4cec1a..000000000 --- a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/shufflenet-v2.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_shufflenet_v2_qai-app-builder.mdx ---- - -# Shufflenet-v2 示例 - -import SHUFFLENETV2QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_shufflenet_v2_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/squeezenet1-1.md b/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/squeezenet1-1.md deleted file mode 100644 index 90b454d02..000000000 --- a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/squeezenet1-1.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_squeezenet1_1_qai-app-builder.mdx ---- - -# SqueezeNet-1.1 示例 - -import SQUEEZENET11QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_squeezenet1_1_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/vit.md b/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/vit.md deleted file mode 100644 index da8ac71d8..000000000 --- a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/vit.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_vit_qai-app-builder.mdx ---- - -# VIT 示例 - -import VITQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_vit_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/wideresnet50.md b/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/wideresnet50.md deleted file mode 100644 index 5e221d59e..000000000 --- a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/wideresnet50.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_wideresnet50_qai-app-builder.mdx ---- - -# WideResNet50 示例 - -import WIDERESNET50QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_wideresnet50_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/xlsr.md b/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/xlsr.md deleted file mode 100644 index 9e1506d78..000000000 --- a/docs/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/xlsr.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_xlsr_qai-app-builder.mdx ---- - -# XLSR 示例 - -import XLSRQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_xlsr_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/README.md b/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/README.md index b965c98ce..fb7c6d6b6 100644 --- a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/README.md +++ b/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/README.md @@ -2,8 +2,10 @@ sidebar_position: 99 --- -# CV 机器视觉示例 +# QAI AppBuilder 示例 -使用 Qai-APPBuilder Python API 利用 NPU 推理模型示例集合。 +基于上游 [QAI AppBuilder](https://github.com/qualcomm/qai-appbuilder) `samples` 中、且在 SC8280XP 上可使用 `--chipset 6490`（Hexagon V68）下载模型的示例。完整菜单可用 `python3 run_inference.py --list` 查看。 + +请先完成 [**QAI AppBuilder 安装**](../qai-appbuilder)。 diff --git a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/convnext-base.md b/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/convnext-base.md deleted file mode 100644 index fdc07d22b..000000000 --- a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/convnext-base.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_convnext_base_qai-app-builder.mdx ---- - -# ConvNext-Base 示例 - -import CONVNEXTBASEQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_convnext_base_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/convnext-tiny.md b/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/convnext-tiny.md deleted file mode 100644 index 3534111c6..000000000 --- a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/convnext-tiny.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 1 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_convnext_tiny_qai-app-builder.mdx ---- - -# ConvNext-Tiny 示例 - -import CONVNEXTTINYQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_convnext_tiny_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b0.md b/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b0.md deleted file mode 100644 index ebed67bfa..000000000 --- a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b0.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 2 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_efficientnet_b0_qai-app-builder.mdx ---- - -# EfficientNet-B0 示例 - -import EFFICIENTNETB0QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_efficientnet_b0_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b4.md b/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b4.md deleted file mode 100644 index 1d568da1a..000000000 --- a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b4.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 3 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_efficientnet_b4_qai-app-builder.mdx ---- - -# EfficientNet-B4 示例 - -import EFFICIENTNETB4QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_efficientnet_b4_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-v2s.md b/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-v2s.md deleted file mode 100644 index c71e12a00..000000000 --- a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-v2s.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 4 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_efficientnet_v2s_qai-app-builder.mdx ---- - -# EfficientNet-V2-s 示例 - -import EFFICIENTNETV2SQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_efficientnet_v2s_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/fcn-resnet50.md b/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/fcn-resnet50.md deleted file mode 100644 index eafed3173..000000000 --- a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/fcn-resnet50.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 13 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_fcn-resnet50_qai-app-builder.mdx ---- - -# FCN-ResNet50 示例 - -import FCNRESNRT50QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_fcn-resnet50_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/levit.md b/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/levit.md deleted file mode 100644 index 4764b5ca0..000000000 --- a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/levit.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 7 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_levit_qai-app-builder.mdx ---- - -# LeViT 示例 - -import LEVITQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_levit_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/regnet.md b/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/regnet.md deleted file mode 100644 index e8d33003d..000000000 --- a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/regnet.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 8 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_regnet_qai-app-builder.mdx ---- - -# RegNet 示例 - -import REGNETQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_regnet_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/sesr-m5.md b/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/sesr-m5.md deleted file mode 100644 index 3d5408b73..000000000 --- a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/sesr-m5.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 18 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_sesr_m5_qai-app-builder.mdx ---- - -# SESR-M5 示例 - -import SESRM5QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_sesr_m5_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/shufflenet-v2.md b/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/shufflenet-v2.md deleted file mode 100644 index a3a2584d3..000000000 --- a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/shufflenet-v2.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 9 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_shufflenet_v2_qai-app-builder.mdx ---- - -# Shufflenet-v2 示例 - -import SHUFFLENETV2QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_shufflenet_v2_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/squeezenet1-1.md b/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/squeezenet1-1.md deleted file mode 100644 index f92fb838d..000000000 --- a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/squeezenet1-1.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 10 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_squeezenet1_1_qai-app-builder.mdx ---- - -# SqueezeNet-1.1 示例 - -import SQUEEZENET11QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_squeezenet1_1_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/vit.md b/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/vit.md deleted file mode 100644 index 76121de5d..000000000 --- a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/vit.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 11 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_vit_qai-app-builder.mdx ---- - -# VIT 示例 - -import VITQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_vit_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/wideresnet50.md b/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/wideresnet50.md deleted file mode 100644 index 59258d98a..000000000 --- a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/wideresnet50.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 12 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_wideresnet50_qai-app-builder.mdx ---- - -# WideResNet50 示例 - -import WIDERESNET50QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_wideresnet50_qai-app-builder.mdx'; - - diff --git a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/xlsr.md b/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/xlsr.md deleted file mode 100644 index ffb516e43..000000000 --- a/docs/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/xlsr.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 19 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - docs/common/ai/qualcomm/_xlsr_qai-app-builder.mdx ---- - -# XLSR 示例 - -import XLSRQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_xlsr_qai-app-builder.mdx'; - - diff --git a/docs/fogwise/airbox-q900/ai-dev/qai-appbuilder-demo/README.md b/docs/fogwise/airbox-q900/ai-dev/qai-appbuilder-demo/README.md index 1da42f2d1..7921da84c 100644 --- a/docs/fogwise/airbox-q900/ai-dev/qai-appbuilder-demo/README.md +++ b/docs/fogwise/airbox-q900/ai-dev/qai-appbuilder-demo/README.md @@ -2,8 +2,10 @@ sidebar_position: 99 --- -# CV 机器视觉示例 +# QAI AppBuilder 示例 -使用 Qai-APPBuilder Python API 利用 NPU 推理模型示例集合。 +对应上游 [QAI AppBuilder](https://github.com/qualcomm/qai-appbuilder) 中在 QCS9075 上可文档化的示例（Linux 运行需 `--chipset 9075`）。上游 `run_inference.py --list` 中部分条目仅支持 Snapdragon X Elite / X2 Elite（Windows on Snapdragon），未列入本页。 + +请先完成 [**QAI AppBuilder 安装**](../qai-appbuilder)。 diff --git a/docs/fogwise/airbox-q900/ai-dev/qai-appbuilder-demo/qwen-vl.md b/docs/fogwise/airbox-q900/ai-dev/qai-appbuilder-demo/qwen-vl.md new file mode 100644 index 000000000..be01a14e1 --- /dev/null +++ b/docs/fogwise/airbox-q900/ai-dev/qai-appbuilder-demo/qwen-vl.md @@ -0,0 +1,14 @@ +--- +sidebar_position: 0 + +doc_kind: wrapper +source_of_truth: common +imports_resolve_to: + - docs/common/ai/qualcomm/_qwen_vl_qai-app-builder.mdx +--- + +# Qwen VL 示例 + +import QWENVLQAIAPPBUILDER from '../../../../common/ai/qualcomm/\_qwen_vl_qai-app-builder.mdx'; + + diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_aotgan_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_aotgan_qai-app-builder.mdx index c409c6d7f..4384c875f 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_aotgan_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_aotgan_qai-app-builder.mdx @@ -1,152 +1,89 @@ This document describes how to use the [QAI AppBuilder](../qai-appbuilder) Python API to run inference with the [AOT-GAN](https://aihub.qualcomm.com/models/aotgan) image inpainting model on Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | +| Device | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run the Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` -### Run the Script - -- Navigate to the example directory - - - - - - +### Run the script - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- Enter the upstream samples directory - + - - - - -- Prepare input images. The following images are used as examples: + ```bash + cd qai-appbuilder/samples + ``` - {" "} + -
- - input image -
+- Prepare input data (use the sample input if provided, or pass script arguments) -
- - mask image -
+
+ + input image +
- Run inference ```bash - python3 aotgan/aotgan.py + python3 ComputerVision/Image_Editing/aotgan/aotgan.py --chipset 9075 ``` - ```bash - $ python3 aotgan/aotgan.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 136.6ms [WARNING] Time: Read model file to memory. 21.49 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +- Example result + +
+ + mask image +
+
+ + output image +
- 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 205.5ms [WARNING] Time: contextCreateFromBinary. 68.83 - - 205.6ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 207.5ms [WARNING] Time: model_initialize aotgan 207.45 - - 415.4ms [WARNING] Time: model_inference aotgan 154.66 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmpjvwp3wzi.PNG' - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 801.6ms [WARNING] Time: model_destroy aotgan 67.93 - - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmp0z61xxdt.PNG' - ``` +:::tip +The first run downloads the model via Qualcomm® AI Hub (network-dependent). You can also use the launcher from `samples`: - {" "} +```bash +python3 run_inference.py --list +python3 run_inference.py --model aotgan --args "--chipset 9075" +``` -
- - output image -
+::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_beit_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_beit_qai-app-builder.mdx index 3bc1cba91..ada71e626 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_beit_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_beit_qai-app-builder.mdx @@ -1,125 +1,69 @@ This document describes how to use the [QAI AppBuilder](../qai-appbuilder) Python API to run inference with the [Beit](https://aihub.qualcomm.com/models/beit) object recognition model on Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | +| Device | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run the Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` -### Run the Script - -- Navigate to the example directory - - - - - - +### Run the script - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- Enter the upstream samples directory - + - - - + ```bash + cd qai-appbuilder/samples + ``` -- Prepare input image. The following image is used as an example: + - {" "} +- Prepare input data (use the sample input if provided, or pass script arguments) -
- - input image -
+
+ + input image +
- Run inference ```bash - python3 beit/beit.py + python3 ComputerVision/Image_Classification/beit/beit.py --chipset 9075 ``` - ```bash - $ python3 beit/beit.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 179.0ms [WARNING] Time: Read model file to memory. 60.05 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 284.7ms [WARNING] Time: contextCreateFromBinary. 105.62 - - 284.8ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 288.1ms [WARNING] Time: model_initialize beit 288.01 - - 365.3ms [WARNING] Time: model_inference beit 17.72 - - Top 5 predictions for image: - - "Samoyed", 0.6322221756 - "Pomeranian", 0.1098636836 - "Keeshond", 0.0410530604 - "Japanese Chin", 0.0093044275 - "Chow Chow", 0.0058453484 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +:::tip +The first run downloads the model via Qualcomm® AI Hub (network-dependent). You can also use the launcher from `samples`: - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 483.8ms [WARNING] Time: model_destroy beit 113.14 - ``` +```bash +python3 run_inference.py --list +python3 run_inference.py --model beit --args "--chipset 9075" +``` - The output shows that `Samoyed` has the highest confidence score, which matches the content of the input image. +::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_convnext_base_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_convnext_base_qai-app-builder.mdx deleted file mode 100644 index 2fff4ef9f..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_convnext_base_qai-app-builder.mdx +++ /dev/null @@ -1,140 +0,0 @@ -This document describes using the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [ConvNext-Base](https://aihub.qualcomm.com/models/convnext_base) -target recognition model using Qualcomm® Hexagon™ Processor (NPU). - -**Supported Devices** - -| Device | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## Install QAI AppBuilder - -:::tip - -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). - -2. Please configure QAIRT environment variables according to [**Configure QAIRT Environment Variables**](../qai-appbuilder#configure-qairt-environment-variables). - ::: - -## Run Example - -### Install Dependencies - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### Run Script - -- Navigate to the example directory - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- Prepare input image, using the following image as input example - - {" "} - -
- - input image -
- -- Execute inference - - - - ```bash - python3 convnext_base/convnext_base.py - ``` - - - - ```bash - $ python3 convnext_base/convnext_base.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/convnext_base - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 193.9ms [WARNING] Time: Read model file to memory. 71.69 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 269.5ms [WARNING] Time: contextCreateFromBinary. 75.47 - - 269.5ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 270.9ms [WARNING] Time: model_initialize convnext_base 270.85 - - 348.3ms [WARNING] Time: model_inference convnext_base 30.17 - - Top 5 predictions for image: - - Samoyed 0.8726790547 - Pomeranian 0.0192712061 - keeshond 0.0150661934 - Japanese spaniel 0.0032834315 - Eskimo dog 0.0020474677 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 365.5ms [WARNING] Time: model_destroy convnext_base 14.44 - ``` - - The printed results show that `Samoyed` has the highest confidence, which matches the input image content. diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_convnext_tiny_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_convnext_tiny_qai-app-builder.mdx deleted file mode 100644 index 9dbddef11..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_convnext_tiny_qai-app-builder.mdx +++ /dev/null @@ -1,142 +0,0 @@ -This document describes using the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [ConvNext-Tiny](https://aihub.qualcomm.com/models/convnext_tiny) -target recognition model using Qualcomm® Hexagon™ Processor (NPU). - -**Supported Devices** - -| Device | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## Install QAI AppBuilder - -:::tip - -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder). - -2. Please configure QAIRT environment variables according to [**Configure QAIRT Environment Variables**](../qai-appbuilder). - ::: - -## Run Example - -### Install Dependencies - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### Run Script - -- Navigate to the example directory - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- Prepare input image, using the following image as input example - - {" "} - -
- - input image -
- -- Execute inference - - - - ```bash - python3 convnext_tiny/convnext_tiny.py - ``` - - - - ```bash - $ python3 convnext_tiny/convnext_tiny.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/convnext_tiny - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 132.1ms [WARNING] Time: Read model file to memory. 15.88 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 176.1ms [WARNING] Time: contextCreateFromBinary. 44.01 - - 176.2ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 176.6ms [WARNING] Time: model_initialize convnext_tiny 176.53 - - 245.2ms [WARNING] Time: model_inference convnext_tiny 11.95 - - Top 5 predictions for image: - - Samoyed 0.637629509 - Pomeranian 0.1382825226 - keeshond 0.0217001885 - Arctic fox 0.0138142202 - white wolf 0.0115938596 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 263.1ms [WARNING] Time: model_destroy convnext_tiny 15.29 - ``` - - The printed results show that `Samoyed` has the highest confidence, which matches the input image content. - ---- diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_depth_anything_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_depth_anything_qai-app-builder.mdx index c923c3ccb..0b4f0cdba 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_depth_anything_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_depth_anything_qai-app-builder.mdx @@ -1,132 +1,79 @@ This document describes how to use the [QAI AppBuilder](../qai-appbuilder) Python API to run inference with the [Depth-Anything](https://aihub.qualcomm.com/models/depth_anything) depth estimation model on Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | +| Device | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run the Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless matplotlib ``` -### Run the Script - -- Navigate to the example directory - - - - +### Run the script - +- Enter the upstream samples directory - ```bash - cd ai-engine-direct-helper/samples/python - ``` + - - - - - + ```bash + cd qai-appbuilder/samples + ``` -- Prepare input image. The following image is used as an example: + - {" "} +- Prepare input data (use the sample input if provided, or pass script arguments) -
- - input image -
+
+ + input image +
- Run inference ```bash - python3 depth_anything/depth_anything.py + python3 ComputerVision/Depth_Estimation/depth_anything/depth_anything.py --chipset 9075 ``` - ```bash - $ python3 depth_anything/depth_anything.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 167.0ms [WARNING] Time: Read model file to memory. 37.39 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +- Example result - 0.0ms [WARNING] This META does not have Alloc2 Support +
+ + output image +
- 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 329.2ms [WARNING] Time: contextCreateFromBinary. 162.14 - - 329.3ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 332.1ms [WARNING] Time: model_initialize depth_anything 332.05 - - 576.7ms [WARNING] Time: model_inference depth_anything 145.90 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 877.6ms [WARNING] Time: model_destroy depth_anything 85.31 +:::tip +The first run downloads the model via Qualcomm® AI Hub (network-dependent). You can also use the launcher from `samples`: - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmp_3i4qa54.PNG' - ``` +```bash +python3 run_inference.py --list +python3 run_inference.py --model depth_anything --args "--chipset 9075" +``` -
- - output image -
+::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_easy_ocr_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_easy_ocr_qai-app-builder.mdx index 643836330..7c4b90715 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_easy_ocr_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_easy_ocr_qai-app-builder.mdx @@ -1,111 +1,96 @@ This document describes how to use the [QAI AppBuilder](../qai-appbuilder) Python API to run inference with the [EasyOCR](https://aihub.qualcomm.com/models/easyocr) text recognition model on Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | +| Device | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to the [**QAI AppBuilder Installation Guide**](../qai-appbuilder). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Configure ADSP environment variables as described in [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run the Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless easyocr ``` -### Run the Script - -- Navigate to the example directory +### Run the script - +- Enter the upstream samples directory - + - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` + ```bash + cd qai-appbuilder/samples + ``` - + - +- Prepare input data (bundled `ch_en.png` / `english.png`, or pass `--Image_Path`) - +
+ + input image +
-- Prepare input image. The following image is used as an example: +:::warning +Verified on AIRbox Q900 (QCS9075): upstream `easy_ocr.py` uses `DataLoader(..., num_workers=1)`, which **forks worker processes** and releases the QNN HTP context. Recognition then fails with: - {" "} +```text +RuntimeError: Cannot perform 'Inference': context 'EasyOCRRecognizer' has been released. +``` -
- - input image -
+Before running, change every `num_workers=int(workers)` in `Multimodal/Image_To_Text/easy_ocr/easy_ocr.py` to **`num_workers=0`** (or set the `workers` default in `recognizer_get_text` to `0`). +::: - Run inference ```bash - python3 easy_ocr/easy_ocr.py + python3 Multimodal/Image_To_Text/easy_ocr/easy_ocr.py --chipset 9075 ``` - ```bash - $ python3 easy_ocr/easy_ocr.py - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - Calling EasyOCR_Detector::Inference on NPU - Calling EasyOCR_Recognizer::Inference on NPU - Calling EasyOCR_Recognizer::Inference on NPU - Calling EasyOCR_Recognizer::Inference on NPU - Calling EasyOCR_Recognizer::Inference on NPU - Calling EasyOCR_Recognizer::Inference on NPU - Calling EasyOCR_Recognizer::Inference on NPU - Calling EasyOCR_Recognizer::Inference on NPU - Calling EasyOCR_Recognizer::Inference on NPU - 白日依山尽 - 黄河入海流 - 欲穷千里目 - 更上一层楼 - The sun beyond the mountain glows, - The Yellow River seawards flows_ - You can enjoy a grander sight, - By climbing to a greater height。 - - Demo image display is disabled by default for remote servers. To override, set `QAIHM_ALWAYS_DISPLAY_OUTPUT=1` in your environment. - - Saving image to /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/samples/python/build/image.png - Logs will be sent to the system's default channel - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - ``` +On success the terminal prints recognized Chinese and English lines, for example (default bilingual sample image): + +```text +白日依山尽 +黄河入海流 +欲穷千里目 +更上一层楼 +The sun beyond the mountain glows, +The Yellow River seawards flows_ +You can enjoy a grander sight, +By climbing to a greater height。 +Saving image to .../samples/build/image.png +``` - Recognition results +:::tip +The first run downloads Detector / Recognizer models (including Chinese-English) via Qualcomm® AI Hub. Apply the `num_workers=0` change first, then you can use the launcher: - ```bash - 白日依山尽 - 黄河入海流 - 欲穷千里目 - 更上一层楼 - The sun beyond the mountain glows, - The Yellow River seawards flows_ - You can enjoy a grander sight, - By climbing to a greater height。 - ``` +```bash +python3 run_inference.py --list +python3 run_inference.py --model easy_ocr --args "--chipset 9075" +``` + +::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_efficientnet_b0_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_efficientnet_b0_qai-app-builder.mdx deleted file mode 100644 index cbcd28286..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_efficientnet_b0_qai-app-builder.mdx +++ /dev/null @@ -1,140 +0,0 @@ -This document describes using the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [EfficientNet-B0](https://aihub.qualcomm.com/models/efficientnet_b0) -target recognition model using Qualcomm® Hexagon™ Processor (NPU). - -**Supported Devices** - -| Device | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## Install QAI AppBuilder - -:::tip - -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). - -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). - ::: - -## Run Example - -### Install Dependencies - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### Run Script - -- Navigate to the example directory - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- Prepare input image, using the following image as input example - - {" "} - -
- - input image -
- -- Execute inference - - - - ```bash - python3 efficientnet_b0/efficientnet_b0.py - ``` - - - - ```bash - $ python3 efficientnet_b0/efficientnet_b0.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/efficientnet_b0 - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 122.3ms [WARNING] Time: Read model file to memory. 5.33 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 143.3ms [WARNING] Time: contextCreateFromBinary. 20.87 - - 143.3ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 143.7ms [WARNING] Time: model_initialize efficientnet_b0 143.64 - - 214.6ms [WARNING] Time: model_inference efficientnet_b0 6.08 - - Top 5 predictions for image: - - Samoyed 0.6965298057 - Pomeranian 0.1052966565 - keeshond 0.0131986532 - Eskimo dog 0.0075131184 - Arctic fox 0.0056676473 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 235.9ms [WARNING] Time: model_destroy efficientnet_b0 18.66 - ``` - - The printed results show that `Samoyed` has the highest confidence, which matches the input image content. diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_efficientnet_b4_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_efficientnet_b4_qai-app-builder.mdx deleted file mode 100644 index 8b15f9574..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_efficientnet_b4_qai-app-builder.mdx +++ /dev/null @@ -1,141 +0,0 @@ -This document describes using the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [EfficientNet-B4](https://aihub.qualcomm.com/models/efficientnet_b4) -target recognition model using Qualcomm® Hexagon™ Processor (NPU). - -**Supported Devices** - -| Device | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## Install QAI AppBuilder - -:::tip - -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). - -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). - ::: - -## Run Example - -### Install Dependencies - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### Run Script - -- Navigate to the example directory - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- Prepare input image, using the following image as input example - - {" "} - -
- - input image -
- -- Execute inference - - - - ```bash - python3 efficientnet_b4/efficientnet_b4.py - ``` - - - - ```bash - $ python3 efficientnet_b4/efficientnet_b4.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/efficientnet_b4 - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 134.2ms [WARNING] Time: Read model file to memory. 15.52 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 197.4ms [WARNING] Time: contextCreateFromBinary. 63.07 - - 197.4ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 197.8ms [WARNING] Time: model_initialize efficientnet_b4 197.73 - - 271.4ms [WARNING] Time: model_inference efficientnet_b4 12.60 - - Top 5 predictions for image: - - Samoyed 0.9568610787 - Pomeranian 0.0066686491 - keeshond 0.0054634647 - chow 0.0010403089 - Eskimo dog 0.0007608762 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 287.8ms [WARNING] Time: model_destroy efficientnet_b4 13.65 - - ``` - - The printed results show that `Samoyed` has the highest confidence, which matches the input image content. diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_efficientnet_v2s_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_efficientnet_v2s_qai-app-builder.mdx deleted file mode 100644 index 4641119b1..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_efficientnet_v2s_qai-app-builder.mdx +++ /dev/null @@ -1,140 +0,0 @@ -This document describes using the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [EfficientNet-V2-s](https://aihub.qualcomm.com/models/efficientnet_v2_s) -target recognition model using Qualcomm® Hexagon™ Processor (NPU). - -**Supported Devices** - -| Device | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## Install QAI AppBuilder - -:::tip - -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). - -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). - ::: - -## Run Example - -### Install Dependencies - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### Run Script - -- Navigate to the example directory - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- Prepare input image, using the following image as input example - - {" "} - -
- - input image -
- -- Execute inference - - - - ```bash - python3 efficientnet_v2_s/efficientnet_v2_s.py - ``` - - - - ```bash - $ python3 efficientnet_v2_s/efficientnet_v2_s.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/efficientnet_v2_s - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 129.0ms [WARNING] Time: Read model file to memory. 11.18 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 158.9ms [WARNING] Time: contextCreateFromBinary. 29.89 - - 159.0ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 159.4ms [WARNING] Time: model_initialize efficientnet_v2_s 159.34 - - 227.4ms [WARNING] Time: model_inference efficientnet_v2_s 10.06 - - Top 5 predictions for image: - - Samoyed 0.7607400417 - keeshond 0.0230328161 - Pomeranian 0.0204458293 - white wolf 0.0193288065 - Arctic fox 0.0062915389 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 239.8ms [WARNING] Time: model_destroy efficientnet_v2_s 9.40 - ``` - - The printed results show that `Samoyed` has the highest confidence, which matches the input image content. diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_face_attrib_net_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_face_attrib_net_qai-app-builder.mdx index 7d5230b1b..27ae13e7d 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_face_attrib_net_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_face_attrib_net_qai-app-builder.mdx @@ -1,271 +1,72 @@ This document describes how to use the [QAI AppBuilder](../qai-appbuilder) Python API to run inference with the [Facial-Attribute-Detection](https://aihub.qualcomm.com/models/face_attrib_net) face attribute recognition model on Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | +| Device | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run the Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` -### Run the Script - -- Navigate to the example directory - - - - - - +### Run the script - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- Enter the upstream samples directory - + - - - + ```bash + cd qai-appbuilder/samples + ``` -- Prepare input image. The following image is used as an example: + - {" "} +- Prepare input data (use the sample input if provided, or pass script arguments) -
- - input image -
+
+ + input image +
- Run inference ```bash - python3 face_attrib_net/face_attrib_net.py --image ./face_attrib_net/test.jpeg + python3 ComputerVision/Face_Recognition/face_attrib_net/face_attrib_net.py --chipset 9075 ``` - ```bash - $ python3 face_attrib_net/face_attrib_net.py --image ./face_attrib_net/test.jpeg - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 138.8ms [WARNING] Time: Read model file to memory. 17.12 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 171.9ms [WARNING] Time: contextCreateFromBinary. 33.00 - - 172.0ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 173.7ms [WARNING] Time: model_initialize face_attrib_net 173.70 - - 206.7ms [WARNING] Time: model_inference face_attrib_net 2.68 - - Model outputs are saved at: /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/samples/python/face_attrib_net/build/output.json - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 304.2ms [WARNING] Time: model_destroy face_attrib_net 91.15 - ``` +:::tip +The first run downloads the model via Qualcomm® AI Hub (network-dependent). You can also use the launcher from `samples`: -- Recognition results +```bash +python3 run_inference.py --list +python3 run_inference.py --model face_attrib_net --args "--chipset 9075" +``` - ```json - { - "id_feature": [ - -131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -110208.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, -65472.00390625, - -131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, -131008.0078125, 85824.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -9096.0009765625, -131008.0078125, 131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, 73472.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - -87232.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, -109696.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -31408.001953125, -131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -50720.00390625, - 131008.0078125, 131008.0078125, 95104.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - -131008.0078125, -68864.0078125, -131008.0078125, -28160.001953125, - 131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, -629.5000610351562, 131008.0078125, - -131008.0078125, 131008.0078125, 116608.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, -102656.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - -28240.001953125, 131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, -39840.00390625, -131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -40992.00390625, -131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, 111232.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, 73920.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, -131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, 131008.0078125, - 131008.0078125, 65184.00390625, -131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -129472.0078125, -131008.0078125, -131008.0078125, -131008.0078125, - 131008.0078125, -131008.0078125, 131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125, - 131008.0078125, 131008.0078125, -131008.0078125, -131008.0078125, - -131008.0078125, 131008.0078125, 131008.0078125, 131008.0078125 - ], - "liveness_feature": [ - 10344.0009765625, 22816.001953125, -4112.00048828125, -13632.0009765625, - 8156.00048828125, -15176.0009765625, 7100.00048828125, 19952.001953125, - 7816.00048828125, 17936.001953125, -2642.000244140625, -3474.000244140625, - -1823.0001220703125, 4340.00048828125, 8304.0009765625, 6308.00048828125, - 25936.001953125, -5192.00048828125, 7112.00048828125, 3862.000244140625, - 12288.0009765625, 2606.000244140625, -11192.0009765625, -8352.0009765625, - 8132.00048828125, 19664.001953125, -2890.000244140625, 1288.0001220703125, - -7612.00048828125, -19664.001953125, 2094.000244140625, -10056.0009765625 - ], - "eye_closeness": [131008.0078125, -131008.0078125], - "glasses": [76608.0078125, -79488.0078125], - "mask": [-86208.0078125, 67264.0078125], - "sunglasses": [0.9985352158546448, 0.0] - } - ``` +::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_facemap_3dmm_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_facemap_3dmm_qai-app-builder.mdx index 28ad2ae31..1e7f62478 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_facemap_3dmm_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_facemap_3dmm_qai-app-builder.mdx @@ -1,140 +1,84 @@ This document describes how to use the [QAI AppBuilder](../qai-appbuilder) Python API to run inference with the [Facial-Landmark-Detection](https://aihub.qualcomm.com/models/facemap_3dmm) facial landmark detection model on Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | +| Device | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run the Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless scikit-image onnxruntime ``` -### Run the Script - -- Navigate to the example directory - - - - +### Run the script - +- Enter the upstream samples directory - ```bash - cd ai-engine-direct-helper/samples/python - ``` + - - - - - + ```bash + cd qai-appbuilder/samples + ``` -- Prepare input image. The following image is used as an example: + - {" "} +- Prepare input data (use the sample input if provided, or pass script arguments) -
- - input image -
+
+ + input image +
- Run inference ```bash - python3 facemap_3dmm/facemap_3dmm.py + python3 ComputerVision/Face_Recognition/facemap_3dmm/facemap_3dmm.py --chipset 9075 ``` - ```bash - $ python3 facemap_3dmm/facemap_3dmm.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 120.2ms [WARNING] Time: Read model file to memory. 9.63 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +On AIRbox Q900 the sample may prefer the ONNX float model (requires `onnxruntime`). A successful run writes `output.jpg` and may print `ONNX model inference time: ... ms`. - 0.0ms [WARNING] This META does not have Alloc2 Support +- Example result - 0.0ms [WARNING] This META does not have Alloc2 Support +
+ + output image +
- 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 138.7ms [WARNING] Time: contextCreateFromBinary. 18.41 - - 138.8ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 140.3ms [WARNING] Time: model_initialize facemap_3dmm 140.23 - - 182.0ms [WARNING] Time: model_inference facemap_3dmm 1.43 - - /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/samples/python/facemap_3dmm/facemap_3dmm.py:145: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at /pytorch/torch/csrc/utils/tensor_new.cpp:253.) - _output = torch.tensor(output) - Saving image to /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/samples/python/facemap_3dmm/output.jpg - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 706.9ms [WARNING] Time: model_destroy facemap_3dmm 72.10 - - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmpkpcumu3y.PNG' - ``` +:::tip +The first run downloads the model via Qualcomm® AI Hub (network-dependent). You can also use the launcher from `samples`: - {" "} +```bash +python3 run_inference.py --list +python3 run_inference.py --model facemap_3dmm --args "--chipset 9075" +``` -
- - output image -
+::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_fcn-resnet50_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_fcn-resnet50_qai-app-builder.mdx deleted file mode 100644 index 0b7b36eb1..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_fcn-resnet50_qai-app-builder.mdx +++ /dev/null @@ -1,142 +0,0 @@ -This document describes using the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [FCN-ResNet50](https://aihub.qualcomm.com/compute/models/fcn_resnet50) -image segmentation model using Qualcomm® Hexagon™ Processor (NPU). - -**Supported Devices** - -| Device | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## Install QAI AppBuilder - -:::tip - -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). - -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). - ::: - -## Run Example - -### Install Dependencies - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### Run Script - -- Navigate to the example directory - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- Prepare input image, using the following image as input example - -
- - input image -
- -- Execute inference - - - - ```bash - python3 fcn_resnet50/fcn_resnet50.py --image ./input.jpg - ``` - - - - ```bash - $ python3 fcn_resnet50/fcn_resnet50.py --image ./input.jpg - Current file directory: /mnt/ssd/git_clone/ai-engine-direct-helper/samples/linux/python/fcn_resnet50 - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 127.2ms [WARNING] Time: Read model file to memory. 12.72 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 185.9ms [WARNING] Time: contextCreateFromBinary. 58.58 - - 186.0ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 187.1ms [WARNING] Time: model_initialize fcn_resnet50 187.09 - - 385.4ms [WARNING] Time: model_inference fcn_resnet50 112.79 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 463.2ms [WARNING] Time: model_destroy fcn_resnet50 10.20 - ``` - -- Output results - -
- - output image -
diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_googlenet_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_googlenet_qai-app-builder.mdx index e7be5b91e..574226d48 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_googlenet_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_googlenet_qai-app-builder.mdx @@ -1,152 +1,115 @@ -This document describes how to use the [QAI AppBuilder](../qai-appbuilder) Python API to run inference with the [GoogLeNet](https://aihub.qualcomm.com/models/googlenet?domain=Computer+Vision&useCase=Image+Classification&chipsets=qualcomm-qcs6490-proxy) object recognition model on Qualcomm® Hexagon™ Processor (NPU). +This document describes using the [QAI AppBuilder](../qai-appbuilder) Python API to run inference with the [GoogLeNet](https://aihub.qualcomm.com/models/googlenet?domain=Computer+Vision&useCase=Image+Classification&chipsets=qualcomm-qcs6490-proxy) image classification model on the Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | -| Dragon Q6A | QCS6490 | +| Device | SoC | +| -------------------- | -------- | +| Dragon Q6A | QCS6490 | | Dragon Q8B | SC8280XP | -| Fogwise® AIRbox Q900 | QCS9075 | +| Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run the Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` -### Run the Script - -- Navigate to the example directory - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` - - - - - - - -- Prepare input image. The following image is used as an example: - - {" "} - -
- - input image -
+### Run the script -- Run inference +- Enter the upstream samples directory - + ```bash - python3 googlenet/googlenet.py + cd qai-appbuilder/samples ``` - + - ```bash - $ python3 googlenet/googlenet.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support +Run inference (Linux requires `--chipset`; the first run downloads `dog.jpg`, labels, and the model) - 0.0ms [WARNING] This META does not have Alloc2 Support +:::tip +If the auto-downloaded `dog.jpg` is empty or corrupt, replace it with a valid local image or pass `--image /path/to/image.jpg`. +::: - 0.0ms [WARNING] This META does not have Alloc2 Support + + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 157.6ms [WARNING] Time: Read model file to memory. 8.91 +```bash +python3 ComputerVision/Image_Classification/googlenet/googlenet.py --chipset 6490 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + + - 0.0ms [WARNING] This META does not have Alloc2 Support +:::tip +SC8280XP and QCS6490 both use Hexagon V68. Use `--chipset 6490` when downloading models. +::: - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Image_Classification/googlenet/googlenet.py --chipset 6490 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + + - 180.6ms [WARNING] Time: contextCreateFromBinary. 22.97 + - 180.7ms [WARNING] Time: UnmapViewOfFile. 0.00 +```bash +python3 ComputerVision/Image_Classification/googlenet/googlenet.py --chipset 9075 +``` - 181.6ms [WARNING] Time: model_initialize googlenet 181.55 + - 255.7ms [WARNING] Time: model_inference googlenet 2.67 + + - Top 5 predictions for image: +### Expected result - Samoyed 0.948613584 - West Highland White Terrier 0.0060515446 - Alaskan tundra wolf 0.0052576731 - Pomeranian 0.0047871661 - Chow Chow 0.0035575195 - 0.0ms [WARNING] This META does not have Alloc2 Support +On Dragon Q6A (`--chipset 6490`), the default test image produces output similar to: - 0.0ms [WARNING] This META does not have Alloc2 Support +```text +Top 5 predictions for image: - 0.0ms [WARNING] This META does not have Alloc2 Support +Samoyed 0.9282982349 +West Highland White Terrier 0.0102122389 +Pomeranian 0.0064557223 +Pyrenean Mountain Dog 0.0059806537 +Chow Chow 0.0051328284 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support +HTP / FastRPC WARNING lines may appear and can be ignored when inference succeeds. - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 353.7ms [WARNING] Time: model_destroy googlenet 93.43 +:::tip +Launcher alternative: - ``` +```bash +python3 run_inference.py --model googlenet --args "--chipset 6490" +``` - The output shows that `Samoyed` has the highest confidence score, which matches the content of the input image. +::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_inception_v3_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_inception_v3_qai-app-builder.mdx index e2bba97f1..d37122b76 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_inception_v3_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_inception_v3_qai-app-builder.mdx @@ -1,151 +1,121 @@ -This document describes how to use the [QAI AppBuilder](../qai-appbuilder) Python API to run inference with the [Inception-v3](https://aihub.qualcomm.com/models/inception_v3?domain=Computer+Vision&useCase=Image+Classification&chipsets=qualcomm-qcs6490-proxy) object recognition model on Qualcomm® Hexagon™ Processor (NPU). +This document describes using the [QAI AppBuilder](../qai-appbuilder) Python API to run inference with the [Inception-v3](https://aihub.qualcomm.com/models/inception_v3?domain=Computer+Vision&useCase=Image+Classification&chipsets=qualcomm-qcs6490-proxy) image classification model on the Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | -| Dragon Q6A | QCS6490 | +| Device | SoC | +| -------------------- | -------- | +| Dragon Q6A | QCS6490 | | Dragon Q8B | SC8280XP | -| Fogwise® AIRbox Q900 | QCS9075 | +| Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run the Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` -### Run the Script - -- Navigate to the example directory - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - +### Run the script - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` +- Enter the upstream samples directory - + - - - - - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` - - - - - - - -- Prepare input image. The following image is used as an example: + ```bash + cd qai-appbuilder/samples + ``` - {" "} + -
- - input image -
+Run inference (Linux requires `--chipset`; the sample ships with `input.jpg`) -- Run inference + - + - ```bash - python3 inception_v3/inception_v3.py --image ./inception_v3/input.jpg - ``` + - +```bash +python3 ComputerVision/Image_Classification/inception_v3/inception_v3.py --chipset 6490 +``` - ```bash - $ python3 inception_v3/inception_v3.py --image ./inception_v3/input.jpg - 0.0ms [WARNING] Initializing HtpProvider + - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +:::tip +SC8280XP and QCS6490 both use Hexagon V68. Use `--chipset 6490` when downloading models. +::: - 0.0ms [WARNING] This META does not have Alloc2 Support + - 221.6ms [WARNING] Time: Read model file to memory. 17.28 +```bash +python3 ComputerVision/Image_Classification/inception_v3/inception_v3.py --chipset 6490 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Image_Classification/inception_v3/inception_v3.py --chipset 9075 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 258.5ms [WARNING] Time: contextCreateFromBinary. 36.82 + - 258.5ms [WARNING] Time: UnmapViewOfFile. 0.00 +Optional image path: - 260.2ms [WARNING] Time: model_initialize inceptionV3 260.14 +```bash +python3 ComputerVision/Image_Classification/inception_v3/inception_v3.py --chipset 6490 --image ComputerVision/Image_Classification/inception_v3/input.jpg +``` - 332.1ms [WARNING] Time: model_inference inceptionV3 3.50 +### Expected result - Top 5 predictions for image: +On Dragon Q6A (`--chipset 6490`), the default input produces output similar to: - Samoyed 0.9998983145 - Arctic fox 0.000054331 - white wolf 0.0000345349 - Eskimo dog 0.0000035142 - Pomeranian 0.0000030651 - 0.0ms [WARNING] This META does not have Alloc2 Support +```text +Top 5 predictions for image: - 0.0ms [WARNING] This META does not have Alloc2 Support +Samoyed 0.999714911 +white wolf 0.0001396882 +Arctic fox 0.0001165446 +Pomeranian 0.000006424 +Great Pyrenees 0.000006424 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support +HTP / FastRPC WARNING lines may appear and can be ignored when inference succeeds. - 0.0ms [WARNING] This META does not have Alloc2 Support +:::tip +Launcher alternative: - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 443.3ms [WARNING] Time: model_destroy inceptionV3 105.74 - ``` +```bash +python3 run_inference.py --model inception_v3 --args "--chipset 6490" +``` - The output shows that `Samoyed` has the highest confidence score, which matches the content of the input image. +::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_lama_dilated_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_lama_dilated_qai-app-builder.mdx index 3af583f17..91468b11b 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_lama_dilated_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_lama_dilated_qai-app-builder.mdx @@ -1,151 +1,89 @@ This document describes how to use the [QAI AppBuilder](../qai-appbuilder) Python API to run inference with the [LaMa-Dilated](https://aihub.qualcomm.com/models/lama_dilated) image inpainting model on Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | +| Device | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run the Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` -### Run the Script - -- Navigate to the example directory - - - - - - +### Run the script - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- Enter the upstream samples directory - + - - - - -- Prepare input images. The following images are used as examples: + ```bash + cd qai-appbuilder/samples + ``` - {" "} + -
- - input image -
+- Prepare input data (use the sample input if provided, or pass script arguments) -
- - mask image -
+
+ + input image +
- Run inference ```bash - python3 lama_dilated/lama_dilated.py + python3 ComputerVision/Image_Editing/lama_dilated/lama_dilated.py --chipset 9075 ``` - ```bash - $ python3 lama_dilated/lama_dilated.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 167.7ms [WARNING] Time: Read model file to memory. 39.91 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +- Example result + +
+ + mask image +
+
+ + output image +
- 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 255.5ms [WARNING] Time: contextCreateFromBinary. 87.63 - - 255.5ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 257.3ms [WARNING] Time: model_initialize lamadilated 257.28 - - 406.4ms [WARNING] Time: model_inference lamadilated 103.83 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmp0t30nvk9.PNG' - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmpd98hfol8.PNG' - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 978.3ms [WARNING] Time: model_destroy lamadilated 103.66 - ``` +:::tip +The first run downloads the model via Qualcomm® AI Hub (network-dependent). You can also use the launcher from `samples`: - {" "} +```bash +python3 run_inference.py --list +python3 run_inference.py --model lama_dilated --args "--chipset 9075" +``` -
- - output image -
+::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_levit_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_levit_qai-app-builder.mdx deleted file mode 100644 index a4c3b39c6..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_levit_qai-app-builder.mdx +++ /dev/null @@ -1,140 +0,0 @@ -This document describes using the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [LeViT](https://aihub.qualcomm.com/models/levit) -target recognition model using Qualcomm® Hexagon™ Processor (NPU). - -**Supported Devices** - -| Device | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## Install QAI AppBuilder - -:::tip - -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). - -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). - ::: - -## Run Example - -### Install Dependencies - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### Run Script - -- Navigate to the example directory - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- Prepare input image, using the following image as input example - - {" "} - -
- - input image -
- -- Execute inference - - - - ```bash - python3 levit/levit.py - ``` - - - - ```bash - $ python3 levit/levit.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/levit - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 120.8ms [WARNING] Time: Read model file to memory. 4.10 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 147.6ms [WARNING] Time: contextCreateFromBinary. 26.69 - - 147.6ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 148.5ms [WARNING] Time: model_initialize levit 148.48 - - 226.3ms [WARNING] Time: model_inference levit 5.23 - - Top 5 predictions for image: - - Samoyed 0.6532489061 - Pomeranian 0.0412057079 - acorn 0.013004588 - wallaby 0.007310994 - keeshond 0.0034880731 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 239.0ms [WARNING] Time: model_destroy levit 10.01 - ``` - - The printed results show that `Samoyed` has the highest confidence, which matches the input image content. diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_mediapipe_hand_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_mediapipe_hand_qai-app-builder.mdx index cf0e471cd..0862c2a4b 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_mediapipe_hand_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_mediapipe_hand_qai-app-builder.mdx @@ -1,171 +1,97 @@ This document describes how to use the [QAI AppBuilder](../qai-appbuilder) Python API to run inference with the [MediaPipe-Hand-Detection](https://aihub.qualcomm.com/models/mediapipe_hand) hand pose recognition model on Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | +| Device | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run the Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models pygame +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless pygame ``` -### Run the Script - -- Navigate to the example directory +The script also looks under `~/.qaihm` for [MediaPipePyTorch](https://github.com/zmurez/MediaPipePyTorch) (provides `blazepalm` and related modules). If missing: - + - +```bash +mkdir -p ~/.qaihm +git clone --depth 1 https://github.com/zmurez/MediaPipePyTorch.git ~/.qaihm/MediaPipePyTorch +``` - + - ```bash - cd ai-engine-direct-helper/samples/python - ``` +### Run the script - +- Enter the upstream samples directory - + - + ```bash + cd qai-appbuilder/samples + ``` -- Prepare input image. The following image is used as an example: + - {" "} +- Prepare input data (camera UI needs a GUI; use a static image in headless environments) -
- - input image -
+
+ + input image +
- Run inference ```bash - python3 mediapipe_hand/mediapipe_hand.py --imagefile ./mediapipe_hand/hand.jpg + python3 ComputerVision/Pose_Estimation/mediapipe_hand/mediapipe_hand.py \ + --chipset 9075 \ + --imagefile /path/to/hand_image.png \ + --displayPredict False \ + --DemoMode False ``` - ```bash - $ python3 mediapipe_hand/mediapipe_hand.py --imagefile ./mediapipe_hand/hand.jpg - /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/.venv/lib/python3.12/site-packages/pygame/pkgdata.py:25: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. - from pkg_resources import resource_stream, resource_exists - pygame 2.6.1 (SDL 2.28.4, Python 3.12.3) - Hello from the pygame community. https://www.pygame.org/contribute.html - Press ESC key to exit if you are opening camera app. - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 110.2ms [WARNING] Time: Read model file to memory. 2.70 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 133.4ms [WARNING] Time: contextCreateFromBinary. 23.04 - - 133.4ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 133.9ms [WARNING] Time: model_initialize hand_detector 133.82 - - 191.3ms [WARNING] Time: Read model file to memory. 3.24 +- Example result: `output.jpg` is written in the sample directory - 0.0ms [WARNING] This META does not have Alloc2 Support +
+ + output image +
- 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 204.0ms [WARNING] Time: contextCreateFromBinary. 12.63 - - 204.1ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 205.1ms [WARNING] Time: model_initialize landmark_detector 17.40 - - 242.5ms [WARNING] Time: model_inference hand_detector 2.86 - - 255.1ms [WARNING] Time: model_inference landmark_detector 2.10 - - /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/samples/python/mediapipe_hand/mediapipe_hand.py:906: DeprecationWarning: 'mode' parameter is deprecated and will be removed in Pillow 13 (2026-10-15) - out_image = Image.fromarray(pred_image[0], "RGB") - - Demo image display is disabled by default for remote servers. To override, set `QAIHM_ALWAYS_DISPLAY_OUTPUT=1` in your environment. - - Saving image to /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/samples/python/build/image.png - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 314.1ms [WARNING] Time: model_destroy hand_detector 11.84 - - Logs will be sent to the system's default channel - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 389.4ms [WARNING] Time: model_destroy landmark_detector 75.05 - ``` +:::tip +The first run downloads `handdetector` and `landmarkdetector` via Qualcomm® AI Hub. Prefer `--imagefile` when no display is available, and use `opencv-python-headless` to avoid Qt plugin initialization failures. - {" "} +```bash +python3 run_inference.py --list +python3 run_inference.py --model mediapipe_hand --args "--chipset 9075 --imagefile /path/to/hand_image.png --displayPredict False --DemoMode False" +``` -
- - output image -
+::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_nomic_embed_text_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_nomic_embed_text_qai-app-builder.mdx index 8377a0f59..d05267261 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_nomic_embed_text_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_nomic_embed_text_qai-app-builder.mdx @@ -1,237 +1,64 @@ This document describes how to use the [QAI AppBuilder](../qai-appbuilder) Python API to run inference with the [Nomic-Embed-Text](https://aihub.qualcomm.com/models/nomic_embed_text) text encoding model on Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | +| Device | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run the Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless transformers ``` -### Run the Script - -- Navigate to the example directory - - - - +### Run the script - +- Enter the upstream samples directory - ```bash - cd ai-engine-direct-helper/samples/python - ``` + - - - + ```bash + cd qai-appbuilder/samples + ``` - + - Run inference ```bash - python3 nomic_embed_text/nomic_embed_text.py + python3 Multimodal/Text_Generation/nomic_embed_text/nomic_embed_text.py --chipset 9075 ``` - ```bash - $ python3 nomic_embed_text/nomic_embed_text.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 226.8ms [WARNING] Time: Read model file to memory. 107.40 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 324.1ms [WARNING] Time: contextCreateFromBinary. 97.18 - - 324.1ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 327.1ms [WARNING] Time: model_initialize nomic_embed_text 327.00 - - 343.6ms [WARNING] Time: model_inference nomic_embed_text 10.11 - - Embeddings: - [ 2.41241474e-02 -4.50897263e-03 -2.15209976e-01 -4.53796424e-02 - 1.46770489e-03 4.48608436e-02 -2.05802941e-03 1.30081186e-02 - -2.21405048e-02 -6.17675819e-02 1.77621914e-05 7.71484450e-02 - 5.78918494e-02 5.52978553e-02 2.69470233e-02 -4.53491248e-02 - 3.90319861e-02 -8.25195387e-02 -5.13305701e-02 1.16806040e-02 - -2.47650165e-02 -5.70983924e-02 6.58798264e-03 -6.29806565e-03 - 2.16308609e-01 9.33837984e-03 -3.27453651e-02 1.02417000e-01 - 2.17590351e-02 3.63349938e-03 1.28631601e-02 -1.32064829e-02 - 1.50604257e-02 2.56042499e-02 6.51855543e-02 -3.80249061e-02 - 1.70040142e-03 3.19480919e-03 1.28250131e-02 1.43127451e-02 - 2.92968773e-03 -4.25720261e-03 8.76617525e-03 -5.45043983e-02 - 7.24487379e-02 -1.21612558e-02 -1.37939462e-02 -1.17721567e-02 - 6.06994666e-02 -1.56402607e-02 -3.29895057e-02 -1.77764904e-03 - -3.09295673e-02 3.49731483e-02 3.03497333e-02 5.01403846e-02 - 6.08215369e-02 2.45971698e-02 1.79595966e-02 1.87683124e-02 - 6.84204176e-02 7.29980543e-02 2.01873798e-02 6.91528395e-02 - 1.96990985e-02 -3.41796912e-02 -7.67231046e-04 8.52050856e-02 - 5.60913123e-02 -1.63269062e-02 7.55615309e-02 -2.16979999e-02 - 4.34875526e-02 1.93786640e-02 6.58798264e-03 1.62200946e-02 - 2.09045429e-02 -3.43017615e-02 -4.82482947e-02 -7.37915114e-02 - 5.18493690e-02 -9.25903395e-02 1.05407722e-01 -4.20837440e-02 - 1.24588022e-02 -4.14123572e-02 -1.32904062e-02 7.17163133e-03 - -4.33959998e-02 3.15551795e-02 6.09130897e-02 2.07519550e-02 - -2.68402118e-02 4.03747596e-02 -4.18090858e-02 1.57775898e-02 - -4.78744553e-03 -6.70623826e-03 -4.66308631e-02 -3.60717811e-02 - -9.71222017e-03 4.33349656e-03 -2.19726581e-02 2.30560321e-02 - 7.20214918e-02 4.43115272e-02 -4.73022498e-02 -7.61718825e-02 - -4.11987342e-02 -4.75769080e-02 -3.17077674e-02 3.72924842e-02 - -7.02667283e-03 2.29034442e-02 -1.76544208e-02 5.59234666e-03 - 3.74450721e-02 -8.04443434e-02 3.09295673e-02 7.23877028e-02 - -8.33129957e-02 2.06451435e-02 8.11576901e-04 4.22363319e-02 - 1.62200946e-02 4.13513221e-02 -8.30688551e-02 -3.15246619e-02 - 2.56729149e-03 -3.37829627e-02 -2.52990741e-02 -3.06549091e-02 - -1.01547251e-02 -4.36706580e-02 4.28466834e-02 9.04083345e-03 - -2.00805683e-02 -3.41491736e-02 4.34875526e-02 1.84631366e-02 - 3.17993201e-02 2.14386005e-02 1.50070200e-02 -5.69458045e-02 - -4.06494178e-02 -2.87017841e-02 6.92749098e-02 -3.66516151e-02 - 8.38756619e-04 -4.78744553e-03 -1.58081073e-02 2.99072284e-02 - 2.27508564e-02 3.61633338e-02 1.66473407e-02 -3.07617206e-02 - -5.39855994e-02 -9.80377290e-03 5.27343787e-02 -1.64947528e-02 - 2.93540978e-03 -4.39453162e-02 -8.68530348e-02 1.49841318e-02 - 7.54928635e-03 -9.79003981e-02 4.23812913e-03 3.79333533e-02 - 4.14428748e-02 2.45056171e-02 8.48388765e-03 -6.26831129e-02 - -3.14636268e-02 9.82284662e-04 -2.68707294e-02 -4.19921912e-02 - -1.14517221e-02 -6.64062575e-02 3.82385291e-02 7.20977830e-03 - 7.65991285e-02 -5.41992225e-02 2.29949970e-02 3.59191932e-02 - -2.84881610e-02 -3.38439979e-02 1.17034921e-02 9.36126802e-03 - -4.04663123e-02 -3.29284705e-02 -6.25610426e-02 1.85699482e-02 - -8.16650465e-02 -2.84881610e-02 -3.53088416e-02 -3.37829627e-02 - 1.52816782e-02 6.61850022e-03 5.45043983e-02 -5.33142127e-02 - -3.93066444e-02 -1.22985849e-02 -1.91650409e-02 -7.98034761e-03 - 7.43389188e-04 9.42993239e-02 4.79126014e-02 2.72216816e-02 - 5.44128455e-02 3.80859412e-02 1.03332527e-01 -6.76269606e-02 - -3.86047401e-02 -4.93774451e-02 5.10788010e-03 5.16414701e-04 - 3.11431903e-02 -4.71801795e-02 2.41699237e-02 2.67791767e-02 - 1.48315439e-02 -3.46374549e-02 5.94177283e-02 -1.06201181e-02 - 6.31713942e-02 -2.00958271e-02 -3.12347431e-02 1.44042978e-02 - -1.09252937e-01 2.44712853e-03 -5.78308143e-02 -2.55584735e-02 - 3.06243915e-02 2.87017841e-02 1.12235553e-04 6.24465989e-03 - 2.92663593e-02 4.35180701e-02 -2.05383319e-02 -6.28051832e-02 - -5.83190955e-02 7.96508864e-02 5.04760779e-02 6.17218064e-03 - -8.22753981e-02 3.06701679e-02 -4.98352088e-02 1.56021127e-02 - 1.25732431e-02 -1.12838754e-02 -3.18603553e-02 -2.19268817e-02 - -1.43661508e-02 3.89404334e-02 4.02221717e-02 -3.65600623e-02 - -5.61523475e-02 1.40838632e-02 -3.44238319e-02 -3.15856971e-02 - 4.85534705e-02 -1.01867683e-01 4.33654822e-02 -5.83190955e-02 - -4.94995154e-02 -1.06124887e-02 -6.00891151e-02 1.51214609e-02 - 6.64672926e-02 -6.98852614e-02 -3.58772301e-03 2.17895526e-02 - 3.07617206e-02 3.03955097e-02 -1.13906870e-02 -7.06481980e-03 - 2.38037133e-03 9.26208589e-03 -4.83398475e-02 9.26971529e-03 - -3.26232947e-02 -1.14822397e-02 -8.05664137e-02 -4.54406776e-02 - -1.70135517e-02 4.29992713e-02 5.15136756e-02 3.87268104e-02 - -2.23083515e-02 -2.96630878e-02 3.32946815e-02 1.15509043e-02 - 6.46972703e-03 1.03027351e-01 -5.15136756e-02 1.44805918e-02 - 6.10961951e-02 1.43203745e-02 -7.97119215e-02 -1.64337177e-02 - -3.32641639e-02 2.47497577e-02 3.29284705e-02 -4.26025428e-02 - -2.39105243e-02 1.45950327e-02 1.55792246e-02 -4.04357947e-02 - 7.18383864e-02 2.07824726e-02 -8.57544020e-02 6.94580153e-02 - -9.28344801e-02 4.58908128e-03 -5.43823279e-02 2.66723651e-02 - -1.87835712e-02 5.48095740e-02 9.75341871e-02 2.11181659e-02 - -4.49218787e-02 -5.65490760e-02 -5.21545447e-02 -6.97631910e-02 - 3.87878455e-02 1.12686167e-02 3.94897498e-02 5.50537147e-02 - 2.33612079e-02 -4.33959998e-02 2.44903583e-02 1.21383676e-02 - -4.91638221e-02 -2.73742694e-02 -4.21752967e-02 3.28674354e-02 - -1.45111093e-02 -3.08837909e-02 -1.08566293e-02 8.66088942e-02 - 2.14996357e-02 -4.25720261e-03 5.08308469e-04 -4.74548377e-02 - -2.05230732e-02 -5.14221229e-02 -8.78906343e-03 -5.35965012e-03 - -1.34201059e-02 1.50451669e-02 -1.49536142e-02 4.24499549e-02 - -2.27355976e-02 3.18298377e-02 -5.84125577e-04 -3.09143085e-02 - 7.52258347e-03 6.65283278e-02 2.72979755e-02 -2.58026142e-02 - 2.94303917e-03 3.52859526e-04 3.31420936e-02 9.76562593e-03 - -4.97131385e-02 2.02178978e-03 3.74450721e-02 2.48146080e-03 - 3.79333533e-02 8.08715913e-03 -3.27148475e-02 -2.69317646e-02 - -2.42157001e-02 4.19006385e-02 2.75268573e-02 4.73632850e-02 - -1.00097664e-01 -6.15234412e-02 1.02996835e-02 -2.12249774e-02 - -4.45251502e-02 -6.92138746e-02 4.54406776e-02 -2.92358417e-02 - 2.97927880e-03 6.73828200e-02 4.59899940e-02 2.00805683e-02 - 1.33285532e-02 1.31225598e-03 -7.61032151e-03 -8.88061617e-03 - 2.07061786e-02 1.19972241e-03 -1.21154794e-02 1.90429706e-02 - 5.27343787e-02 2.87017841e-02 -3.32031287e-02 1.87835712e-02 - -2.16217060e-02 -2.12478661e-03 -8.14056490e-03 -1.14501961e-01 - 1.14440927e-02 3.71933007e-03 -2.95410175e-02 -2.03399677e-02 - 3.63159217e-02 -1.47781381e-02 4.47082557e-02 -1.11846933e-02 - 1.22222910e-02 -3.87878455e-02 -7.10296677e-03 2.98004169e-02 - 3.20129432e-02 -6.11572303e-02 -3.82995643e-02 1.98974628e-02 - 8.17871168e-02 6.12487830e-02 3.95202674e-02 3.47900428e-02 - -3.25927772e-02 9.42993239e-02 5.86547889e-02 5.72814979e-02 - 1.37329111e-02 -9.53979567e-02 -1.60675067e-02 4.14123572e-02 - 2.33764667e-02 3.53698768e-02 -4.33044471e-02 -3.56750526e-02 - -5.64098416e-04 -2.61383075e-02 1.24282846e-02 -3.62586998e-03 - 4.86450233e-02 -3.13415565e-02 -5.13610877e-02 -1.89971942e-02 - 1.85394306e-02 1.38549820e-01 1.28051773e-01 -5.66711463e-02 - -4.99572791e-02 -5.78689622e-03 -1.12075815e-02 5.44433631e-02 - 5.10864295e-02 1.31454477e-02 1.12670906e-01 -8.23974684e-02 - 5.42907752e-02 -1.43814096e-02 -6.16455125e-03 2.03704853e-02 - 4.37622108e-02 4.82177772e-02 -4.18701209e-02 -1.38397226e-02 - 1.66168232e-02 -8.55712965e-02 1.35803232e-02 -2.05383319e-02 - 2.05688495e-02 2.71606464e-02 -3.46984901e-02 -3.33557166e-02 - 3.99475135e-02 -5.79528846e-02 -6.28051832e-02 2.44750995e-02 - -3.88717675e-03 2.40898156e-03 -6.31332444e-03 2.38952655e-02 - 1.12915048e-02 2.56347680e-03 -5.83190955e-02 -4.41894569e-02 - 1.88140888e-02 5.84411658e-02 2.09198017e-02 4.80041541e-02 - -9.94873140e-03 5.40771522e-02 9.94110201e-03 -6.61850022e-03 - 5.13305701e-02 2.78015155e-02 -3.58276404e-02 8.48388765e-03 - -3.97338904e-02 2.51159687e-02 -2.02331562e-02 -2.45513935e-02 - 1.50146494e-02 1.08795175e-02 -1.56173715e-02 3.21655311e-02 - 7.26699876e-03 -1.08947763e-02 5.22460975e-02 -1.05224617e-01 - -2.36206073e-02 2.08435077e-02 -5.32531776e-02 4.02221717e-02] - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 468.5ms [WARNING] Time: model_destroy nomic_embed_text 120.04 +On success the terminal prints an embedding vector (float array). The first run downloads the model and may also fetch Hugging Face tokenizer files. - ``` +:::tip +Launcher alternative from `samples`: + +```bash +python3 run_inference.py --list +python3 run_inference.py --model nomic_embed_text --args "--chipset 9075" +``` + +::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_openai_clip_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_openai_clip_qai-app-builder.mdx index cbfa1393b..009c94155 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_openai_clip_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_openai_clip_qai-app-builder.mdx @@ -1,149 +1,92 @@ This document describes how to use the [QAI AppBuilder](../qai-appbuilder) Python API to run inference with the [OpenAI-CLIP](https://aihub.qualcomm.com/models/openai_clip) model on Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | +| Device | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run the Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers openai-clip +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless "openai-clip==1.0.1" ftfy "setuptools<81" ``` -### Run the Script - -- Navigate to the example directory - - - - - - +### Run the script - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- Enter the upstream samples directory - + - - - + ```bash + cd qai-appbuilder/samples + ``` -- Prepare input images. The following images are used as examples: + - {" "} +- Prepare input data (use the sample input if provided, or pass script arguments) -
- - image1 image2 image3 -
+
+ + image1 image2 image3 +
- Run inference ```bash - python3 openai_clip/openai_clip.py --text camping under the stars + python3 Multimodal/Image_Classification/openai_clip/openai_clip.py --chipset 9075 --text "mountain" ``` - ```bash - $ python3 openai_clip/openai_clip.py - /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/.venv/lib/python3.12/site-packages/clip/clip.py:6: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. - from pkg_resources import packaging - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 232.1ms [WARNING] Time: Read model file to memory. 121.58 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +On success the terminal prints a similarity score per image, for example: - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 361.8ms [WARNING] Time: contextCreateFromBinary. 129.68 - - 361.9ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 365.4ms [WARNING] Time: model_initialize openai_clip 365.30 - - 1453.9ms [WARNING] Time: model_inference openai_clip 25.61 - - 1478.6ms [WARNING] Time: model_inference openai_clip 24.52 - - 1504.1ms [WARNING] Time: model_inference openai_clip 25.45 - - Searching images by prompt: camping under the stars - Image with name: image2.jpg has a similarity score=[19.906252] - Image with name: image1.jpg has a similarity score=[16.578127] - Image with name: image3.jpg has a similarity score=[26.828127] - Displaying the most relevant image - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +```text +Image with name: image2.jpg has a similarity score=[[18.687502]] +Image with name: image1.jpg has a similarity score=[[16.109377]] +Image with name: image3.jpg has a similarity score=[[34.031254]] +``` - /usr/bin/xdg-open: 882: www-browser: not found - 0.0ms [WARNING] This META does not have Alloc2 Support +(Scores vary with the images and query text; the highest score best matches the text.) - /usr/bin/xdg-open: 882: links2: not found - 0.0ms [WARNING] This META does not have Alloc2 Support +- Example result - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmpiq2mgemq.PNG' - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 7951.5ms [WARNING] Time: model_destroy openai_clip 122.86 - ``` +
+ + image3 +
- The results show that the image most similar to `camping under the stars` is `image3.jpg` +:::tip +`openai-clip` relies on the older `pkg_resources` API; pin `setuptools` below 81. Prefer running the script directly. The first run downloads the model via Qualcomm® AI Hub: - {" "} +```bash +python3 run_inference.py --list +python3 Multimodal/Image_Classification/openai_clip/openai_clip.py --chipset 9075 --text "mountain" +``` -
- - image3 -
+::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_openpose_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_openpose_qai-app-builder.mdx index c614257ec..d5b51cbd5 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_openpose_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_openpose_qai-app-builder.mdx @@ -1,137 +1,82 @@ This document describes how to use the [QAI AppBuilder](../qai-appbuilder) Python API to run inference with the [OpenPose](https://github.com/CMU-Perceptual-Computing-Lab/openpose) pose estimation model on Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | +| Device | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run the Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers openai-clip +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless scipy ``` -### Run the Script - -- Navigate to the example directory - - - - - - +### Run the script - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- Enter the upstream samples directory - + - - - + ```bash + cd qai-appbuilder/samples + ``` -- Prepare input image. The following image is used as an example: + - {" "} +- Prepare input data (use the sample input if provided, or pass script arguments) -
- - input image -
+
+ + input image +
- Run inference ```bash - python3 openpose/openpose.py + python3 ComputerVision/Pose_Estimation/openpose/openpose.py --chipset 9075 ``` - ```bash - $ python3 openpose/openpose.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 161.4ms [WARNING] Time: Read model file to memory. 49.87 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +- Example result - 0.0ms [WARNING] This META does not have Alloc2 Support +
+ + output image +
- 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 214.6ms [WARNING] Time: contextCreateFromBinary. 53.04 - - 214.6ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 216.5ms [WARNING] Time: model_initialize openpose 216.47 - - 273.6ms [WARNING] Time: model_inference openpose 24.91 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 929.9ms [WARNING] Time: model_destroy openpose 57.19 - - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmpfvxac2nz.PNG' - ``` +:::tip +The first run downloads the model via Qualcomm® AI Hub (network-dependent). You can also use the launcher from `samples`: - {" "} +```bash +python3 run_inference.py --list +python3 run_inference.py --model openpose --args "--chipset 9075" +``` -
- - output image -
+::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_qai-appbuilder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_qai-appbuilder.mdx index bf07c3725..dc9ce9bfa 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_qai-appbuilder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_qai-appbuilder.mdx @@ -1,47 +1,44 @@ -[**Quick AI Application Builder**](https://github.com/quic/ai-engine-direct-helper) (QAI AppBuilder) helps developers easily use the [Qualcomm® AI Runtime SDK](qairt-sdk#qairt) -to deploy AI models and design AI applications on Qualcomm® SoC platforms equipped with Qualcomm® Hexagon™ Processor (NPU). +[**Quick AI Application Builder**](https://github.com/qualcomm/qai-appbuilder) (QAI AppBuilder) helps developers easily use the [Qualcomm® AI Runtime SDK](qairt-sdk#qairt) +to deploy AI models and design AI applications on Qualcomm® SoC platforms equipped with the Qualcomm® Hexagon™ Processor (NPU). It encapsulates model deployment APIs into a set of simplified interfaces for loading models to the NPU and performing inference. QAI AppBuilder significantly reduces the complexity of model deployment for developers and provides multiple demos as references for designing their own AI applications. Supports Python API / C++ API
- - QAI-APPBUILDER Architecture + + QAI-APPBUILDER architecture
-**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | -| Dragon Q6A | QCS6490 | +| Device | SoC | +| -------------------- | -------- | +| Dragon Q6A | QCS6490 | | Dragon Q8B | SC8280XP | -| Fogwise® AIRbox Q900 | QCS9075 | +| Fogwise® AIRbox Q900 | QCS9075 | -## Installation Methods +## Installation methods :::tip -You can [**install via wheel package**](#install-via-wheel-package) or [**build from source**](#build-from-source) +[**Install with a wheel package**](#install-with-a-wheel-package) is recommended. To build from source, see [**Build from source**](#build-from-source). ::: -### Clone the Repository +### Clone the repository ```bash -git clone -b radxa-dev https://github.com/ZIFENG278/ai-engine-direct-helper.git && cd ai-engine-direct-helper +git clone https://github.com/qualcomm/qai-appbuilder.git && cd qai-appbuilder ``` -### Install via Wheel Package +### Install with a wheel package -##### Create a Virtual Environment +##### Create a virtual environment :::tip -Create a virtual environment to isolate the Python libraries +Create a virtual environment before installing Python packages for isolation. ::: @@ -54,481 +51,307 @@ source .venv/bin/activate - +##### Install qai_appbuilder -```bash -pip3 install https://github.com/ZIFENG278/ai-engine-direct-helper/releases/download/radxa-dev-2.38.0/qai_appbuilder-2.38.0-cp312-cp312-linux_aarch64.whl -``` - - - -### Build from Source +Download the Linux aarch64 wheel that matches your Python version from [**GitHub Releases**](https://github.com/qualcomm/qai-appbuilder/releases). For Python 3.12, for example: ```bash -git submodule update --init --recursive +pip3 install https://github.com/qualcomm/qai-appbuilder/releases/download/v2.48.40/qai_appbuilder-2.48.40-cp312-cp312-manylinux_2_39_aarch64.whl ``` -#### Download QAIRT SDK - :::tip -Please download and extract the QAIRT SDK according to [**Download QAIRT SDK Package**](./qairt-install#download-qairt-sdk-package) +Starting from v2.0.0, the Python wheel bundles the required Qualcomm® AI Runtime libraries (such as `libQnnHtp.so` and Hexagon skels). You usually do not need a full QAIRT SDK install for Python inference. ::: -##### Configure QAIRT Environment Variables - - - - - - ```bash - export PRODUCT_SOC=6490 DSP_ARCH=68 - ``` - - - - - - - - - ```bash - export PRODUCT_SOC=9075 DSP_ARCH=73 - ``` - - - - - - - - ```bash - export PRODUCT_SOC=8280 DSP_ARCH=68 - ``` - - - - - - +Verify the installation: ```bash -cd qairt/2.42.0.251225 -source bin/envsetup.sh -export ADSP_LIBRARY_PATH=$QNN_SDK_ROOT/lib/hexagon-v${DSP_ARCH}/unsigned -export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$QNN_SDK_ROOT/lib/aarch64-oe-linux-gcc11.2 +python3 -c "import qai_appbuilder; print(qai_appbuilder.__file__)" ``` -#### Compile qai_appbuilder +### Build from source -##### Create a Virtual Environment +To build from source, install build dependencies and prepare the [**QAIRT SDK**](./qairt-install#download-qairt-sdk-package). -:::tip -Create a virtual environment to isolate the Python libraries -::: +#### Install build dependencies ```bash -sudo apt install python3-venv python3-dev cmake gcc g++ +sudo apt update +sudo apt install -y cmake build-essential python3-dev python3-venv +git clone https://github.com/qualcomm/qai-appbuilder.git --recursive +cd qai-appbuilder python3 -m venv .venv source .venv/bin/activate +pip3 install wheel==0.45.1 setuptools==80.9.0 pybind11==2.13.6 build==1.4.0 ``` -##### Compile qai_appbuilder Wheel +#### Configure QAIRT environment variables + +Extract the QAIRT SDK first. Replace `` with the actual SDK path: ```bash -pip3 install setuptools -cd ai-engine-direct-helper -python3 setup.py bdist_wheel +export QNN_SDK_ROOT= +export QAI_TOOLCHAINS=aarch64-oe-linux-gcc11.2 ``` -After compilation, a `qai_appbuilder-2.38.0-cp312-cp312-linux_aarch64.whl` file will be generated in the `dist` directory - -#### Install qai_appbuilder - -##### Install qai_appbuilder Wheel +#### Build and install the qai_appbuilder wheel ```bash -cd dist -pip3 install ./qai_appbuilder-2.38.0-cp312-cp312-linux_aarch64.whl +python3 -m build -w +python3 -m pip install dist/qai_appbuilder-*.whl ``` -### Create QAIRT SDK LIB Symlink - -Create a `qai_libs` symlink in `ai-engine-direct-helper/samples/python` to link to the lib directory of the QAIRT SDK - - - - - :::tip -If you haven't downloaded the complete QAIRT_SDK, you can install `qcom-qairt` and then create the symlink. - -qcom-qairt is a slimmed-down version of QAIRT-SDK, installing only the required files +The wheel version follows the linked QAIRT SDK version. See the official [**BUILD.md**](https://github.com/qualcomm/qai-appbuilder/blob/main/BUILD.md) for details. ::: - - +### Create ADSP environment variables - - ```bash - sudo apt install qcom-qnn-sdk-v68 qcom-genie-sdk-v68 - ``` - - - - - - - - - ```bash - sudo apt install qcom-qnn-sdk-v73 qcom-genie-sdk-v73 - ``` - - - +Before running NPU inference on Linux, set `ADSP_LIBRARY_PATH` to a directory that contains the Hexagon skel libraries (for example `libQnnHtpV68Skel.so` / `libQnnHtpV73Skel.so`). - +#### Use the bundled wheel libraries (recommended) - - ```bash - sudo apt install qcom-qnn-sdk-v68 qcom-genie-sdk-v68 - ``` - - - - - - +After installing the wheel, point to the package `libs` directory: ```bash -ln -s /usr/lib/aarch64-linux-gnu samples/python/qai_libs +export ADSP_LIBRARY_PATH=$(python3 -c "import os, qai_appbuilder; print(os.path.join(os.path.dirname(qai_appbuilder.__file__), 'libs'))") ``` - - - +#### Use system QAIRT packages -:::tip -If you have already downloaded the complete QAIRT-SDK, you can link to the `lib/aarch64-oe-linux-gcc11.2/` directory under QAIRT-SDK -::: +System packages such as `qairt-libs` and `qairt-dsp-binaries` come from the Qualcomm apt repository (`ppa:ubuntu-qcom-iot/qcom-ppa`). **This repository is not configured by default** — add it manually before installing the packages: ```bash -ln -s qairt/2.42.0.251225/lib/aarch64-oe-linux-gcc11.2/ samples/python/qai_libs +sudo apt update && sudo apt install -y software-properties-common +sudo add-apt-repository -y ppa:ubuntu-qcom-iot/qcom-ppa +sudo apt update +sudo apt install -y qairt-libs qairt-dsp-binaries ``` - - - - -### Create ADSP Environment Variables - - - - - -:::tip -If you installed the `qcom-qairt` package, use the following command to create the `ADSP_LIBRARY_PATH` environment variable -::: +If packages such as `qairt-dsp-binaries` are already installed, you can also point `ADSP_LIBRARY_PATH` to the platform DSP directory. For QCS6490, for example: ```bash -export ADSP_LIBRARY_PATH=/usr/lib/aarch64-linux-gnu +export ADSP_LIBRARY_PATH=/usr/share/qcom/qcm6490/Thundercomm/RB3gen2/dsp/cdsp ``` - - - - :::tip -If you installed the complete QAIRT-SDK, use the following command to create the `ADSP_LIBRARY_PATH` environment variable +You can add the `export` line to `~/.bashrc` and run `source ~/.bashrc` to make it persistent. ::: - - - - - ```bash - export PRODUCT_SOC=6490 DSP_ARCH=68 - ``` - +## Application examples +:::tip +Upstream samples are reorganized by task type. On Linux, pass `--chipset` so the script can download the matching model package: - +- QCS6490 / SC8280XP: `--chipset 6490` +- QCS9075: `--chipset 9075` - +For more samples, see the upstream [**samples/README.md**](https://github.com/qualcomm/qai-appbuilder/blob/main/samples/README.md). You can also refer to local [**Demo examples**](./qai-appbuilder-demo). +::: - - ```bash - export PRODUCT_SOC=9075 DSP_ARCH=73 - ``` - +### Install sample dependencies - +Install common sample dependencies in the activated virtual environment: - - - - ```bash - export PRODUCT_SOC=8280 DSP_ARCH=68 - ``` - + +```bash +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless +``` - + - +### Enter the samples directory ```bash -cd qairt/2.42.0.251225 -export ADSP_LIBRARY_PATH=$(pwd)/lib/hexagon-v${DSP_ARCH}/unsigned +cd qai-appbuilder/samples ``` - +### Run samples - +You can run a sample script directly, or use the interactive launcher: + + + +```bash +python3 run_inference.py --list +python3 run_inference.py --model inception_v3 --args "--chipset 6490" +``` -## Application Examples + -:::tip -For QCS6490 Demos, please refer to [**Demo Examples**](./qai-appbuilder-demo) -::: +- Image classification - + | Model | How to run | + | ------------ | ----------------------------------------------------------------------------------------- | + | googlenet | `python3 ComputerVision/Image_Classification/googlenet/googlenet.py --chipset 6490` | + | inception_v3 | `python3 ComputerVision/Image_Classification/inception_v3/inception_v3.py --chipset 6490` | -```bash -cd ai-engine-direct-helper/samples/linux/python -``` +- Object detection - + | Model | How to run | + | ---------- | --------------------------------------------------------------------------------- | + | yolov8_det | `python3 ComputerVision/Object_Detection/yolov8_det/yolov8_det.py --chipset 6490` | + +- Super resolution -- Object Recognition - - | Model | Run Command | - | ----------------- | ------------------------------------------------ | - | convnext_base | `python3 convnext_base/convnext_base.py` | - | convnext_tiny | `python3 convnext_tiny/convnext_tiny.py` | - | efficientnet_b0 | `python3 efficientnet_b0/efficientnet_b0.py` | - | efficientnet_b4 | `python3 efficientnet_b4/efficientnet_b4.py` | - | efficientnet_v2_s | `python3 efficientnet_v2_s/efficientnet_v2_s.py` | - | googlenet | `python3 googlenet/googlenet.py` | - | inception_v3 | `python3 inception_v3/inception_v3.py` | - | levit | `python3 levit/levit.py` | - | regnet | `python3 regnet/regnet.py` | - | shufflenet_v2 | `python3 shufflenet_v2/shufflenet_v2.py` | - | squeezenet1_1 | `python3 squeezenet1_1/squeezenet1_1.py` | - | vit | `python3 vit/vit.py` | - | wideresnet50 | `python3 wideresnet50/wideresnet50.py` | - -- Image Segmentation - - | Model | Run Command | - | ------------ | -------------------------------------- | - | fcn_resnet50 | `python3 fcn_resnet50/fcn_resnet50.py` | - -- Object Detection - - | Model | Run Command | - | ---------- | ---------------------------------- | - | yolov8_det | `python3 yolov8_det/yolov8_det.py` | - -- Image Super-Resolution - - | Model | Run Command | - | ------------------------ | -------------------------------------------------------------- | - | quicksrnetmedium | `python3 quicksrnetmedium/quicksrnetmedium.py` | - | real_esrgan_general_x4v3 | `python3 real_esrgan_general_x4v3/real_esrgan_general_x4v3.py` | - | real_esrgan_x4plus | `python3 real_esrgan_x4plus/real_esrgan_x4plus.py` | - | sesr_m5 | `python3 sesr_m5/sesr_m5.py` | - | xlsr | `python3 xlsr/xlsr.py` | + | Model | How to run | + | ------------------------ | ----------------------------------------------------------------------------------------------------------------- | + | quicksrnetmedium | `python3 ComputerVision/Super_Resolution/quicksrnetmedium/quicksrnetmedium.py --chipset 6490` | + | real_esrgan_general_x4v3 | `python3 ComputerVision/Super_Resolution/real_esrgan_general_x4v3/real_esrgan_general_x4v3.py --chipset 6490` | + | real_esrgan_x4plus | `python3 ComputerVision/Super_Resolution/real_esrgan_x4plus/real_esrgan_x4plus.py --chipset 6490 --bin --no_show` | :::tip -For SC8280XP Demos, please refer to [**Demo Examples**](./qai-appbuilder-demo) +Upstream Linux sample scripts currently select model packages with `--chipset 6490` / `--chipset 9075`. SC8280XP and QCS6490 both use Hexagon V68, so use `--chipset 6490` when downloading models. ::: - +- Image classification -```bash -cd ai-engine-direct-helper/samples/linux/python -``` + | Model | How to run | + | ------------ | ----------------------------------------------------------------------------------------- | + | googlenet | `python3 ComputerVision/Image_Classification/googlenet/googlenet.py --chipset 6490` | + | inception_v3 | `python3 ComputerVision/Image_Classification/inception_v3/inception_v3.py --chipset 6490` | - +- Object detection -- Object Recognition - - | Model | Run Command | - | ----------------- | ------------------------------------------------ | - | convnext_base | `python3 convnext_base/convnext_base.py` | - | convnext_tiny | `python3 convnext_tiny/convnext_tiny.py` | - | efficientnet_b0 | `python3 efficientnet_b0/efficientnet_b0.py` | - | efficientnet_b4 | `python3 efficientnet_b4/efficientnet_b4.py` | - | efficientnet_v2_s | `python3 efficientnet_v2_s/efficientnet_v2_s.py` | - | googlenet | `python3 googlenet/googlenet.py` | - | inception_v3 | `python3 inception_v3/inception_v3.py` | - | levit | `python3 levit/levit.py` | - | regnet | `python3 regnet/regnet.py` | - | shufflenet_v2 | `python3 shufflenet_v2/shufflenet_v2.py` | - | squeezenet1_1 | `python3 squeezenet1_1/squeezenet1_1.py` | - | vit | `python3 vit/vit.py` | - | wideresnet50 | `python3 wideresnet50/wideresnet50.py` | - -- Image Segmentation - - | Model | Run Command | - | ------------ | -------------------------------------- | - | fcn_resnet50 | `python3 fcn_resnet50/fcn_resnet50.py` | - -- Object Detection - - | Model | Run Command | - | ---------- | ---------------------------------- | - | yolov8_det | `python3 yolov8_det/yolov8_det.py` | - -- Image Super-Resolution - - | Model | Run Command | - | ------------------------ | -------------------------------------------------------------- | - | quicksrnetmedium | `python3 quicksrnetmedium/quicksrnetmedium.py` | - | real_esrgan_general_x4v3 | `python3 real_esrgan_general_x4v3/real_esrgan_general_x4v3.py` | - | real_esrgan_x4plus | `python3 real_esrgan_x4plus/real_esrgan_x4plus.py` | - | sesr_m5 | `python3 sesr_m5/sesr_m5.py` | - | xlsr | `python3 xlsr/xlsr.py` | + | Model | How to run | + | ---------- | --------------------------------------------------------------------------------- | + | yolov8_det | `python3 ComputerVision/Object_Detection/yolov8_det/yolov8_det.py --chipset 6490` | - - - - -:::tip -For QCS9075 Demos, please refer to [**Demo Examples**](./qai-appbuilder-demo) -::: +- Super resolution - + | Model | How to run | + | ------------------------ | ----------------------------------------------------------------------------------------------------------------- | + | quicksrnetmedium | `python3 ComputerVision/Super_Resolution/quicksrnetmedium/quicksrnetmedium.py --chipset 6490` | + | real_esrgan_general_x4v3 | `python3 ComputerVision/Super_Resolution/real_esrgan_general_x4v3/real_esrgan_general_x4v3.py --chipset 6490` | + | real_esrgan_x4plus | `python3 ComputerVision/Super_Resolution/real_esrgan_x4plus/real_esrgan_x4plus.py --chipset 6490 --bin --no_show` | -```bash -cd ai-engine-direct-helper/samples/python -``` + - + -- Image Generation / Inpainting +- Image generation / image editing - | Model | Run Command | - | --------------------- | ----------------------------------------------------------------------------------------- | - | stable_diffusion_v2_1 | `python3 stable_diffusion_v2_1/stable_diffusion_v2_1.py --prompt "the prompt string ..."` | - | stable_diffusion_v1_5 | `python3 stable_diffusion_v1_5/stable_diffusion_v1_5.py --prompt "the prompt string ..."` | - | lama_dilated | `python3 lama_dilated/lama_dilated.py` | - | aotgan | `python3 aotgan/aotgan.py` | + | Model | How to run | + | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | + | stable_diffusion_v2_1 | `python3 GenerativeAI/Image_Generation/stable_diffusion_v2_1/stable_diffusion_v2_1.py --chipset 9075 --prompt "the prompt string ..."` | + | stable_diffusion_v1_5 | `python3 GenerativeAI/Image_Generation/stable_diffusion_v1_5/stable_diffusion_v1_5.py --chipset 9075 --prompt "the prompt string ..."` | + | lama_dilated | `python3 ComputerVision/Image_Editing/lama_dilated/lama_dilated.py --chipset 9075` | + | aotgan | `python3 ComputerVision/Image_Editing/aotgan/aotgan.py --chipset 9075` | -- Image Super-Resolution +- Super resolution - | Model | Run Command | - | ------------------------ | -------------------------------------------------------------- | - | real_esrgan_x4plus | `python3 real_esrgan_x4plus/real_esrgan_x4plus.py` | - | real_esrgan_general_x4v3 | `python3 real_esrgan_general_x4v3/real_esrgan_general_x4v3.py` | - | quicksrnetmedium | `python3 quicksrnetmedium/quicksrnetmedium.py` | + | Model | How to run | + | ------------------------ | ------------------------------------------------------------------------------------------------------------- | + | real_esrgan_x4plus | `python3 ComputerVision/Super_Resolution/real_esrgan_x4plus/real_esrgan_x4plus.py --chipset 9075` | + | real_esrgan_general_x4v3 | `python3 ComputerVision/Super_Resolution/real_esrgan_general_x4v3/real_esrgan_general_x4v3.py --chipset 9075` | + | quicksrnetmedium | `python3 ComputerVision/Super_Resolution/quicksrnetmedium/quicksrnetmedium.py --chipset 9075` | -- Image Recognition +- Image recognition - | Model | Run Command | - | ------------ | -------------------------------------- | - | inception_v3 | `python3 inception_v3/inception_v3.py` | - | beit | `python3 beit/beit.py` | - | googlenet | `python3 googlenet/googlenet.py` | + | Model | How to run | + | ------------ | ----------------------------------------------------------------------------------------- | + | inception_v3 | `python3 ComputerVision/Image_Classification/inception_v3/inception_v3.py --chipset 9075` | + | beit | `python3 ComputerVision/Image_Classification/beit/beit.py --chipset 9075` | + | googlenet | `python3 ComputerVision/Image_Classification/googlenet/googlenet.py --chipset 9075` | -- Pose Estimation +- Pose estimation - | Model | Run Command | - | -------------- | ------------------------------------------ | - | openpose | `python3 openpose/openpose.py` | - | mediapipe_hand | `python3 mediapipe_hand/mediapipe_hand.py` | + | Model | How to run | + | -------------- | ---------------------------------------------------------------------------------------- | + | openpose | `python3 ComputerVision/Pose_Estimation/openpose/openpose.py --chipset 9075` | + | mediapipe_hand | `python3 ComputerVision/Pose_Estimation/mediapipe_hand/mediapipe_hand.py --chipset 9075` | -- Image Segmentation / Depth Estimation +- Segmentation / depth estimation - | Model | Run Command | - | ----------------- | ------------------------------------------------ | - | unet_segmentation | `python3 unet_segmentation/unet_segmentation.py` | - | depth_anything | `python3 depth_anything/depth_anything.py` | + | Model | How to run | + | ----------------- | ---------------------------------------------------------------------------------------------------- | + | unet_segmentation | `python3 ComputerVision/Semantic_Segmentation/unet_segmentation/unet_segmentation.py --chipset 9075` | + | depth_anything | `python3 ComputerVision/Depth_Estimation/depth_anything/depth_anything.py --chipset 9075` | -- Object Detection +- Object detection - | Model | Run Command | - | ---------- | ---------------------------------- | - | yolov8_det | `python3 yolov8_det/yolov8_det.py` | + | Model | How to run | + | ---------- | --------------------------------------------------------------------------------- | + | yolov8_det | `python3 ComputerVision/Object_Detection/yolov8_det/yolov8_det.py --chipset 9075` | -- 3D Modeling +- 3D face modeling - | Model | Run Command | - | --------------- | -------------------------------------------- | - | facemap_3dmm | `python3 facemap_3dmm/facemap_3dmm.py` | - | face_attrib_net | `python3 face_attrib_net/face_attrib_net.py` | + | Model | How to run | + | --------------- | ------------------------------------------------------------------------------------------- | + | facemap_3dmm | `python3 ComputerVision/Face_Recognition/facemap_3dmm/facemap_3dmm.py --chipset 9075` | + | face_attrib_net | `python3 ComputerVision/Face_Recognition/face_attrib_net/face_attrib_net.py --chipset 9075` | -- Audio Models +- Audio models - | Model | Run Command | - | --------------- | -------------------------------------------- | - | yamnet | `python3 yamnet/yamnet.py` | - | whisper_base_en | `python3 whisper_base_en/whisper_base_en.py` | - | whisper_tiny_en | `python3 whisper_tiny_en/whisper_tiny_en.py` | + | Model | How to run | + | --------------- | ------------------------------------------------------------------------------------ | + | yamnet | `python3 audio/Audio_Classification/yamnet/yamnet.py --chipset 9075` | + | whisper_base_en | `python3 audio/Speech_Recognition/whisper_base_en/whisper_base_en.py --chipset 9075` | + | whisper_tiny_en | `python3 audio/Speech_Recognition/whisper_tiny_en/whisper_tiny_en.py --chipset 9075` | -- Text / Multimodal Embeddings +- Text / multimodal embedding - | Model | Run Command | - | ---------------- | ------------------------------------------------------ | - | nomic_embed_text | `python3 nomic_embed_text/nomic_embed_text.py` | - | openai_clip | `python3 openai_clip/openai_clip.py --text "mountain"` | + | Model | How to run | + | ---------------- | ------------------------------------------------------------------------------------------------------------------------ | + | nomic_embed_text | `python3 Multimodal/Text_Generation/nomic_embed_text/nomic_embed_text.py --chipset 9075` | + | openai_clip | `python3 Multimodal/Image_Classification/openai_clip/openai_clip.py --chipset 9075 --text "mountain"` | + | qwen_vl | Download model package first; `python3 Multimodal/qwen_vl/qwen_vl.py --model qwen2 --path ./qwen2vl2b` (see sample page) | -- 3D Action Recognition +- 3D action recognition - | Model | Run Command | - | --------- | -------------------------------- | - | resnet_3d | `python3 resnet_3d/resnet_3d.py` | + | Model | How to run | + | --------- | ----------------------------------------------------------------------------------- | + | resnet_3d | `python3 ComputerVision/Video_Classification/resnet_3d/resnet_3d.py --chipset 9075` | -- OCR (Text Recognition) +- OCR - | Model | Run Command | - | -------- | ------------------------------ | - | easy_ocr | `python3 easy_ocr/easy_ocr.py` | + | Model | How to run | + | -------- | ---------------------------------------------------------------------- | + | easy_ocr | `python3 Multimodal/Image_To_Text/easy_ocr/easy_ocr.py --chipset 9075` | diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_qairt-sdk.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_qairt-sdk.mdx index 9424e4774..a5d83f801 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_qairt-sdk.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_qairt-sdk.mdx @@ -49,11 +49,11 @@ For information on converting other model formats and inference methods for diff #### SoC Architecture Reference Table -| SoC | dsp_arch | soc_id | -| ----- | -------- | ------ | -| QCS6490 | v68 | 35 | +| SoC | dsp_arch | soc_id | +| -------- | -------- | ------ | +| QCS6490 | v68 | 35 | | SC8280XP | v68 | 37 | -| QCS9075 | v73 | 77 | +| QCS9075 | v73 | 77 | {/* Additional QAIRT references are maintained below. */} @@ -87,7 +87,7 @@ With AIMET, developers can quickly iterate to find the optimal quantization conf ### QAI-APPBUILDER -[**Quick AI Application Builder**](https://github.com/quic/ai-engine-direct-helper) (QAI AppBuilder) helps developers easily use the [Qualcomm® AI Runtime SDK](qairt-sdk#qairt) to deploy AI models and design AI applications on Qualcomm® SoC platforms equipped with the Qualcomm® Hexagon™ Processor (NPU). +[**Quick AI Application Builder**](https://github.com/qualcomm/qai-appbuilder) (QAI AppBuilder) helps developers easily use the [Qualcomm® AI Runtime SDK](qairt-sdk#qairt) to deploy AI models and design AI applications on Qualcomm® SoC platforms equipped with the Qualcomm® Hexagon™ Processor (NPU). It encapsulates the model deployment APIs into a set of simplified interfaces for loading models onto the NPU and performing inference. QAI AppBuilder significantly reduces the complexity of model deployment for developers and provides multiple demos as references for designing their own AI applications.
@@ -102,7 +102,7 @@ It encapsulates the model deployment APIs into a set of simplified interfaces fo - [**QAI AppBuilder**](./qai-appbuilder) -- [**QAI AppBuilder Repository**](https://github.com/quic/ai-engine-direct-helper) +- [**QAI AppBuilder Repository**](https://github.com/qualcomm/qai-appbuilder) ### QAI-Hub diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_quicksrnetmedium_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_quicksrnetmedium_qai-app-builder.mdx index 006d2787b..5a38457a4 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_quicksrnetmedium_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_quicksrnetmedium_qai-app-builder.mdx @@ -1,165 +1,109 @@ -This document describes how to use the [QAI AppBuilder](../qai-appbuilder) Python API to run inference with the [QuickSRNetMedium](https://aihub.qualcomm.com/models/quicksrnetmedium) image super-resolution model on Qualcomm® Hexagon™ Processor (NPU). +This document describes using the [QAI AppBuilder](../qai-appbuilder) Python API to run inference with the [QuickSRNetMedium](https://aihub.qualcomm.com/models/quicksrnetmedium) super-resolution model on the Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | -| Dragon Q6A | QCS6490 | +| Device | SoC | +| -------------------- | -------- | +| Dragon Q6A | QCS6490 | | Dragon Q8B | SC8280XP | -| Fogwise® AIRbox Q900 | QCS9075 | +| Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure QAIRT environment variables according to [**Configure QAIRT Environment Variables**](../qai-appbuilder#configure-qairt-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run the Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers openai-clip +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` -### Run the Script - -- Navigate to the example directory - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` - - - - - - - -- Prepare input image. The following image is used as an example: - - {" "} +### Run the script -
- - input image -
+- Enter the upstream samples directory -- Run inference - - + ```bash - python3 quicksrnetmedium/quicksrnetmedium.py + cd qai-appbuilder/samples ``` - + - ```bash - $ python3 quicksrnetmedium/quicksrnetmedium.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/quicksrnetmedium - Init - 0.0ms [WARNING] Initializing HtpProvider +Run inference (Linux requires `--chipset`; the first run downloads `input.png` and the model) - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Super_Resolution/quicksrnetmedium/quicksrnetmedium.py --chipset 6490 +``` - 112.4ms [WARNING] Time: Read model file to memory. 0.12 + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +:::tip +SC8280XP and QCS6490 both use Hexagon V68. Use `--chipset 6490` when downloading models. +::: - 0.0ms [WARNING] This META does not have Alloc2 Support + + +```bash +python3 ComputerVision/Super_Resolution/quicksrnetmedium/quicksrnetmedium.py --chipset 6490 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 118.0ms [WARNING] Time: contextCreateFromBinary. 5.51 + - 118.0ms [WARNING] Time: UnmapViewOfFile. 0.00 +```bash +python3 ComputerVision/Super_Resolution/quicksrnetmedium/quicksrnetmedium.py --chipset 9075 +``` - 118.3ms [WARNING] Time: model_initialize quicksrnetmedium 118.21 + - 174.0ms [WARNING] Time: model_inference quicksrnetmedium 25.10 + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +### Expected result - 0.0ms [WARNING] This META does not have Alloc2 Support +On Dragon Q6A, a successful run writes: - 0.0ms [WARNING] This META does not have Alloc2 Support +```text +ComputerVision/Super_Resolution/quicksrnetmedium/output.png +``` - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 353.0ms [WARNING] Time: model_destroy quicksrnetmedium 18.10 +Without a display (`DISPLAY` unset), you may see `Error: no DISPLAY environment variable specified`. If `output.png` exists and the process exits 0, ignore that message. - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmp751e_f48.PNG' - ``` +:::tip +Launcher alternative: - {" "} +```bash +python3 run_inference.py --model quicksrnetmedium --args "--chipset 6490" +``` -
- - output image -
+::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_qwen_vl_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_qwen_vl_qai-app-builder.mdx new file mode 100644 index 000000000..1c8bd9349 --- /dev/null +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_qwen_vl_qai-app-builder.mdx @@ -0,0 +1,90 @@ +This document describes running the upstream [Qwen VL](https://github.com/qualcomm/qai-appbuilder/tree/main/samples/Multimodal/qwen_vl) multimodal vision-language sample with [QAI AppBuilder](../qai-appbuilder) on Linux aarch64 (Gradio UI or in-script `Inference`). + +**Supported devices** + +| Device | SoC | +| -------------------- | ------- | +| Fogwise® AIRbox Q900 | QCS9075 | + +## Install QAI AppBuilder + +:::tip + +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). + +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). + ::: + +## Run the sample + +### Install dependencies + +Install sample dependencies in the activated virtual environment: + + + +```bash +pip3 install "transformers==4.57.0" qwen-vl-utils gradio Pillow torch torchvision opencv-python-headless +``` + + + +:::tip +`qwen_vl.py` imports Qwen3-related modules, so you need **transformers ≥ 4.57** (includes `transformers.models.qwen3_vl`). Also install `qwen-vl-utils`. +::: + +### Prepare the model + +This sample does **not** auto-download Qwen weights via `--chipset` / AI Hub. Follow the upstream README for a pre-quantized package, for example Qwen2-VL 2B: + + + +```bash +cd qai-appbuilder/samples/Multimodal/qwen_vl +wget -c https://www.aidevhome.com/data/adh2/models/suggested/qwen2vl2b.zip +unzip -o qwen2vl2b.zip +# Example extract layout: ./qwen2vl2b/ (veg.serialized.bin, weight_sharing_model_1_of_1.serialized.bin, tokenizer.json, ...) +``` + + + +The zip is about 1.8 GB; extracted size is about 2.6 GB. Reserve disk space accordingly. + +### Run the script + +- Enter the sample directory and set ADSP (same as the install page) + + + + ```bash + cd qai-appbuilder/samples/Multimodal/qwen_vl + export ADSP_LIBRARY_PATH=$(python3 -c "import os, qai_appbuilder; print(os.path.join(os.path.dirname(qai_appbuilder.__file__), 'libs'))") + ``` + + + +- Start the Gradio web UI (default `0.0.0.0:7861`) + + + + ```bash + python3 qwen_vl.py --model qwen2 --path ./qwen2vl2b --host 0.0.0.0 --port 7861 + ``` + + + +Open `http://:7861`, upload an image, and enter a question. + +- Optional headless check: after loading the model, call `Inference` once. On AIRbox Q900, with sample image `vlm_demo.png` and prompt _Describe this image in one sentence._, a response similar to the following was observed: + +```text +A webpage with a form for uploading files. +``` + +:::warning +`run_inference.py --model qwen_vl` does **not** replace model download. The entrypoint requires `--path` to the model directory and has **no** `--chipset 9075` argument. +::: + +:::tip +If TBB is installed, some setups set `LD_PRELOAD` to `libtbb.so` per the upstream README; on Q900 a one-shot inference also succeeded without it. +::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_real-esrgan-general_x4v3_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_real-esrgan-general_x4v3_qai-app-builder.mdx index dc265a577..1cf3128b5 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_real-esrgan-general_x4v3_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_real-esrgan-general_x4v3_qai-app-builder.mdx @@ -1,160 +1,105 @@ -This document describes how to use the [QAI AppBuilder](../qai-appbuilder) Python API to run inference with the [Real-ESRGAN-General-x4v3](https://aihub.qualcomm.com/models/real_esrgan_general_x4v3) image super-resolution model on Qualcomm® Hexagon™ Processor (NPU). +This document describes using the [QAI AppBuilder](../qai-appbuilder) Python API to run inference with the [Real-ESRGAN-General-x4v3](https://aihub.qualcomm.com/models/real_esrgan_general_x4v3) super-resolution model on the Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | -| Dragon Q6A | QCS6490 | +| Device | SoC | +| -------------------- | -------- | +| Dragon Q6A | QCS6490 | | Dragon Q8B | SC8280XP | -| Fogwise® AIRbox Q900 | QCS9075 | +| Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run the Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` -### Run the Script - -- Navigate to the example directory - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` - - - - - - - -- Prepare input image. The following image is used as an example: - - {" "} +### Run the script -
- - input image -
+- Enter the upstream samples directory -- Run inference - - + ```bash - python3 real_esrgan_general_x4v3/real_esrgan_general_x4v3.py + cd qai-appbuilder/samples ``` - - - ```bash - $ python3 real_esrgan_general_x4v3/real_esrgan_general_x4v3.py - 0.0ms [WARNING] Initializing HtpProvider + - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support +Run inference (Linux **requires** `--chipset`, or model download may fail) - 0.0ms [WARNING] This META does not have Alloc2 Support + + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Super_Resolution/real_esrgan_general_x4v3/real_esrgan_general_x4v3.py --chipset 6490 +``` - 248.6ms [WARNING] Time: Read model file to memory. 4.00 + - 0.0ms [WARNING] This META does not have Alloc2 Support + + - 0.0ms [WARNING] This META does not have Alloc2 Support +:::tip +SC8280XP and QCS6490 both use Hexagon V68. Use `--chipset 6490` when downloading models. +::: - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Super_Resolution/real_esrgan_general_x4v3/real_esrgan_general_x4v3.py --chipset 6490 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 324.8ms [WARNING] Time: contextCreateFromBinary. 76.10 +```bash +python3 ComputerVision/Super_Resolution/real_esrgan_general_x4v3/real_esrgan_general_x4v3.py --chipset 9075 +``` - 324.9ms [WARNING] Time: UnmapViewOfFile. 0.00 + - 326.0ms [WARNING] Time: model_initialize realesrgan 325.97 + + - 539.0ms [WARNING] Time: model_inference realesrgan 176.50 +### Expected result - 0.0ms [WARNING] This META does not have Alloc2 Support +On Dragon Q6A, a successful run writes: - 0.0ms [WARNING] This META does not have Alloc2 Support +```text +ComputerVision/Super_Resolution/real_esrgan_general_x4v3/output.jpg +``` - 0.0ms [WARNING] This META does not have Alloc2 Support +Without a display, `Error: no DISPLAY environment variable specified` may appear; ignore it if the output file exists and the exit code is 0. - 0.0ms [WARNING] This META does not have Alloc2 Support +:::tip +Launcher alternative: - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmp3vvli4p1.PNG' - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 3871.6ms [WARNING] Time: model_destroy realesrgan 101.28 - ``` +```bash +python3 run_inference.py --model real_esrgan_general_x4v3 --args "--chipset 6490" +``` -
- - output image -
+::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_real-esrgan-x4plus_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_real-esrgan-x4plus_qai-app-builder.mdx index e93bbded2..c5a7ee5b2 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_real-esrgan-x4plus_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_real-esrgan-x4plus_qai-app-builder.mdx @@ -1,160 +1,123 @@ -This document explains how to use the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [Real-ESRGAN-x4plus](https://aihub.qualcomm.com/models/real_esrgan_x4plus?domain=Computer+Vision&useCase=Super+Resolution&chipsets=qualcomm-qcs6490-proxy) image super-resolution model using the Qualcomm® Hexagon™ Processor (NPU). +This document describes using the [QAI AppBuilder](../qai-appbuilder) Python API to run inference with the [Real-ESRGAN-x4plus](https://aihub.qualcomm.com/models/real_esrgan_x4plus?domain=Computer+Vision&useCase=Super+Resolution&chipsets=qualcomm-qcs6490-proxy) super-resolution model on the Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | -| Dragon Q6A | QCS6490 | +| Device | SoC | +| -------------------- | -------- | +| Dragon Q6A | QCS6490 | | Dragon Q8B | SC8280XP | -| Fogwise® AIRbox Q900 | QCS9075 | +| Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to the [**QAI AppBuilder Installation Guide**](../qai-appbuilder). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure the ADSP environment variables according to [**Configuring ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` -### Run the Script - -- Navigate to the example directory - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` +### Run the script - +- Enter the upstream samples directory - + - - - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` - - - - - - - -- Prepare the input image. The following image is used as an example: - - {" "} + ```bash + cd qai-appbuilder/samples + ``` -
- - Input image -
+ -- Execute inference +- Prepare input data (the sample provides a default image) - +
+ + input image +
- ```bash - python3 real_esrgan_x4plus/real_esrgan_x4plus.py - ``` +Run inference - +:::warning +On QCS6490 / SC8280XP, **do not use the default float DLC**. The default `.dlc` path fails with `The SocModel doesn't support FP16`. Use **`--bin`** to download the precompiled HTP context binary. +::: - ```bash - $ python3 real_esrgan_x4plus/real_esrgan_x4plus.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support + + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Super_Resolution/real_esrgan_x4plus/real_esrgan_x4plus.py --chipset 6490 --bin --no_show +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 152.5ms [WARNING] Time: Read model file to memory. 44.96 + + - 0.0ms [WARNING] This META does not have Alloc2 Support +:::tip +SC8280XP and QCS6490 both use Hexagon V68. Use `--chipset 6490` with `--bin`. +::: - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Super_Resolution/real_esrgan_x4plus/real_esrgan_x4plus.py --chipset 6490 --bin --no_show +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Super_Resolution/real_esrgan_x4plus/real_esrgan_x4plus.py --chipset 9075 --no_show +``` - 631.4ms [WARNING] Time: contextCreateFromBinary. 478.81 + - 631.4ms [WARNING] Time: UnmapViewOfFile. 0.00 +:::tip +On QCS9075 the Hub default is a `.bin` HTP context that runs as-is. Use `--no_show` on headless sessions. +::: - 633.3ms [WARNING] Time: model_initialize realesrgan 633.22 + + - 2532.0ms [WARNING] Time: model_inference realesrgan 1851.89 +`--no_show` avoids opening an image viewer on headless SSH sessions. - 0.0ms [WARNING] This META does not have Alloc2 Support +### Expected result - 0.0ms [WARNING] This META does not have Alloc2 Support +A successful run writes: - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmpczsph22w.PNG' - 0.0ms [WARNING] This META does not have Alloc2 Support +```text +ComputerVision/Super_Resolution/real_esrgan_x4plus/output.png +``` - 0.0ms [WARNING] This META does not have Alloc2 Support +:::tip +Launcher alternative: - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 8196.8ms [WARNING] Time: model_destroy realesrgan 164.68 - ``` +```bash +python3 run_inference.py --model real_esrgan_x4plus --args "--chipset 6490 --bin --no_show" +``` -
- - Output image -
+::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_regnet_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_regnet_qai-app-builder.mdx deleted file mode 100644 index 226c5ae97..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_regnet_qai-app-builder.mdx +++ /dev/null @@ -1,140 +0,0 @@ -This document describes using the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [RegNet](https://aihub.qualcomm.com/models/regnet) -target recognition model using Qualcomm® Hexagon™ Processor (NPU). - -**Supported Devices** - -| Device | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## Install QAI AppBuilder - -:::tip - -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). - -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). - ::: - -## Run Example - -### Install Dependencies - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### Run Script - -- Navigate to the example directory - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- Prepare input image, using the following image as input example - - {" "} - -
- - input image -
- -- Execute inference - - - - ```bash - python3 regnet/regnet.py - ``` - - - - ```bash - $ python3 regnet/regnet.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/regnet - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 123.3ms [WARNING] Time: Read model file to memory. 9.22 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 143.9ms [WARNING] Time: contextCreateFromBinary. 20.52 - - 144.0ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 144.5ms [WARNING] Time: model_initialize regnet 144.44 - - 212.3ms [WARNING] Time: model_inference regnet 4.75 - - Top 5 predictions for image: - - Samoyed 0.7615189552 - Pomeranian 0.1356044859 - keeshond 0.0481537655 - white wolf 0.0241472572 - Eskimo dog 0.0072157839 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 223.8ms [WARNING] Time: model_destroy regnet 8.95 - ``` - - The printed results show that `Samoyed` has the highest confidence, which matches the input image content. diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_resnet3d_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_resnet3d_qai-app-builder.mdx index ffaa89516..9763d08d7 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_resnet3d_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_resnet3d_qai-app-builder.mdx @@ -1,139 +1,78 @@ This document explains how to use the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [ResNet-3D](https://aihub.qualcomm.com/models/resnet_3d) video classification model using the Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | +| Device | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to the [**QAI AppBuilder Installation Guide**](../qai-appbuilder). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure the ADSP environment variables according to [**Configuring ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers av +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` -### Run the Script - -- Navigate to the example directory - - - - - - +### Run the script - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- Enter the upstream samples directory - + - - - - -- Prepare the input video. The following video is used as an example: + ```bash + cd qai-appbuilder/samples + ``` - {" "} + - {" "} +- Prepare input data (use the sample input if provided, or pass script arguments) -
- -

- Input video -

-
+
+ +

+ Input video +

+
-- Execute inference +- Run inference ```bash - python3 resnet_3d/resnet_3d.py + python3 ComputerVision/Video_Classification/resnet_3d/resnet_3d.py --chipset 9075 ``` - ```bash - $ python3 resnet_3d/resnet_3d.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 157.9ms [WARNING] Time: Read model file to memory. 35.20 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 213.3ms [WARNING] Time: contextCreateFromBinary. 55.38 - - 213.4ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 215.6ms [WARNING] Time: model_initialize resnet_3d 215.54 - - /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/.venv/lib/python3.12/site-packages/torchvision/io/_video_deprecation_warning.py:5: UserWarning: The video decoding and encoding capabilities of torchvision are deprecated from version 0.22 and will be removed in version 0.24. We recommend that you migrate to TorchCodec, where we'll consolidate the future decoding/encoding capabilities of PyTorch: https://github.com/pytorch/torchcodec - warnings.warn( - 3691.5ms [WARNING] Time: model_inference resnet_3d 27.89 - - Top 5 predictions: - surfing water, kitesurfing, windsurfing, jetskiing, water sliding - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 3795.4ms [WARNING] Time: model_destroy resnet_3d 92.13 - ``` +:::tip +The first run downloads the model via Qualcomm® AI Hub (network-dependent). You can also use the launcher from `samples`: - Recognition Results +```bash +python3 run_inference.py --list +python3 run_inference.py --model resnet_3d --args "--chipset 9075" +``` - ```bash - Top 5 predictions: - surfing water, kitesurfing, windsurfing, jetskiing, water sliding - ``` +::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_sd1_5_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_sd1_5_qai-app-builder.mdx index 10b5effb8..df805468e 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_sd1_5_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_sd1_5_qai-app-builder.mdx @@ -1,107 +1,79 @@ This document explains how to use the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [Stable-Diffusion-v1.5](https://aihub.qualcomm.com/models/stable_diffusion_v1_5) text-to-image model using the Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | +| Device | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to the [**QAI AppBuilder Installation Guide**](../qai-appbuilder). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure the ADSP environment variables according to [**Configuring ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers diffusers +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless transformers diffusers ``` -### Run the Script - -- Navigate to the example directory +### Run the script - +- Enter the upstream samples directory - + - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` + ```bash + cd qai-appbuilder/samples + ``` - + - +- Prepare input data (use the sample input if provided, or pass script arguments) - +
+ + Output image +
-- Execute inference +- Run inference ```bash - python3 stable_diffusion_v1_5/stable_diffusion_v1_5.py --prompt spectacular view of northern lights from Alaska + python3 GenerativeAI/Image_Generation/stable_diffusion_v1_5/stable_diffusion_v1_5.py --chipset 9075 --prompt "a cat" ``` - ```bash - $ python3 stable_diffusion_v1_5/stable_diffusion_v1_5.py --prompt spectacular view of northern lights from Alaska - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - Step 0 Running... - Step 1 Running... - Step 2 Running... - Step 3 Running... - Step 4 Running... - Step 5 Running... - Step 6 Running... - Step 7 Running... - Step 8 Running... - Step 9 Running... - Step 10 Running... - Step 11 Running... - Step 12 Running... - Step 13 Running... - Step 14 Running... - Step 15 Running... - Step 16 Running... - Step 17 Running... - Step 18 Running... - Step 19 Running... - /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/samples/python/stable_diffusion_v1_5/stable_diffusion_v1_5.py:255: DeprecationWarning: 'mode' parameter is deprecated and will be removed in Pillow 13 (2026-10-15) - output_image = Image.fromarray(output_image, mode="RGB") - Image saved to '/mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/samples/python/stable_diffusion_v1_5/images/2025_10_30_03_55_02_6303777164_512.jpg' - time consumes for inference 5.059798002243042(s) - Logs will be sent to the system's default channel - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmpyor75jks.PNG' - Logs will be sent to the system's default channel - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - ``` +On success an image is written under the sample directory, for example: - {" "} +```text +GenerativeAI/Image_Generation/stable_diffusion_v1_5/images/_512.jpg +``` + +On AIRbox Q900 the first run downloads Text Encoder / UNet / VAE (on the order of ~1 GB total). Slow networks may take tens of minutes to hours. + +:::tip +Requires `transformers` and `diffusers`. Prefer running the script directly (avoids `run_inference.py --args` quoting issues with spaces in prompts): + +```bash +python3 GenerativeAI/Image_Generation/stable_diffusion_v1_5/stable_diffusion_v1_5.py --chipset 9075 --prompt "a cat" +``` -
- - Output image -
+::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_sd2_1_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_sd2_1_qai-app-builder.mdx index 152ffdca9..dd1eae45a 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_sd2_1_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_sd2_1_qai-app-builder.mdx @@ -1,107 +1,79 @@ This document explains how to use the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [Stable-Diffusion-v2.1](https://aihub.qualcomm.com/iot/models/stable_diffusion_v2_1) text-to-image model using the Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | +| Device | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to the [**QAI AppBuilder Installation Guide**](../qai-appbuilder). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure the ADSP environment variables according to [**Configuring ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers diffusers +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless transformers diffusers ``` -### Run the Script - -- Navigate to the example directory +### Run the script - +- Enter the upstream samples directory - + - - - ```bash - cd ai-engine-direct-helper/samples/python - ``` + ```bash + cd qai-appbuilder/samples + ``` - + - +- Prepare input data (use the sample input if provided, or pass script arguments) - +
+ + Output image +
-- Execute inference +- Run inference ```bash - python3 stable_diffusion_v2_1/stable_diffusion_v2_1.py --prompt spectacular view of northern lights from Alaska + python3 GenerativeAI/Image_Generation/stable_diffusion_v2_1/stable_diffusion_v2_1.py --chipset 9075 --prompt "a cat" ``` - ```bash - python3 stable_diffusion_v2_1/stable_diffusion_v2_1.py --prompt spectacular view of northern lights from Alaska - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - Step 0 Running... - Step 1 Running... - Step 2 Running... - Step 3 Running... - Step 4 Running... - Step 5 Running... - Step 6 Running... - Step 7 Running... - Step 8 Running... - Step 9 Running... - Step 10 Running... - Step 11 Running... - Step 12 Running... - Step 13 Running... - Step 14 Running... - Step 15 Running... - Step 16 Running... - Step 17 Running... - Step 18 Running... - Step 19 Running... - /mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/samples/python/stable_diffusion_v2_1/stable_diffusion_v2_1.py:275: DeprecationWarning: 'mode' parameter is deprecated and will be removed in Pillow 13 (2026-10-15) - output_image = Image.fromarray(output_image, mode="RGB") - Image saved to '/mnt/ssd/qualcomm/702/zzf_fork/test/ai-engine-direct-helper/samples/python/stable_diffusion_v2_1/images/2025_10_30_05_46_28_2081127356_512.jpg' - time consumes for inference 4.454082250595093(s) - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - Logs will be sent to the system's default channel - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmp0ci7lglx.PNG' - Logs will be sent to the system's default channel - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - ``` +On success an image is written under the sample directory, for example: - {" "} +```text +GenerativeAI/Image_Generation/stable_diffusion_v2_1/images/_512.jpg +``` + +On AIRbox Q900 the first run downloads Text Encoder / UNet / VAE (on the order of ~1.2 GB total). Slow networks may take tens of minutes to hours. + +:::tip +Requires `transformers` and `diffusers`. Prefer running the script directly (avoids `run_inference.py --args` quoting issues with spaces in prompts): + +```bash +python3 GenerativeAI/Image_Generation/stable_diffusion_v2_1/stable_diffusion_v2_1.py --chipset 9075 --prompt "a cat" +``` -
- - Output image -
+::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_sesr_m5_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_sesr_m5_qai-app-builder.mdx deleted file mode 100644 index 46d94ac50..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_sesr_m5_qai-app-builder.mdx +++ /dev/null @@ -1,150 +0,0 @@ -This document describes using the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [SESR-M5](https://aihub.qualcomm.com/models/sesr_m5) -image super-resolution model using Qualcomm® Hexagon™ Processor (NPU). - -**Supported Devices** - -| Device | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## Install QAI AppBuilder - -:::tip - -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). - -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). - ::: - -## Run Example - -### Install Dependencies - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### Run Script - -- Navigate to the example directory - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- Prepare input image, using the following image as input example - - {" "} - -
- - input image -
- -- Execute inference - - - - ```bash - python3 sesr_m5/sesr_m5.py - ``` - - - - ```bash - $ python3 sesr_m5/sesr_m5.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/sesr_m5 - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 113.8ms [WARNING] Time: Read model file to memory. 0.39 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 119.7ms [WARNING] Time: contextCreateFromBinary. 5.87 - - 119.8ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 119.9ms [WARNING] Time: model_initialize sesr_m5 119.89 - - 152.4ms [WARNING] Time: model_inference sesr_m5 11.00 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 330.6ms [WARNING] Time: model_destroy sesr_m5 18.65 - - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmpxodzub9w.PNG' - ``` - -
- - output image -
diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_shufflenet_v2_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_shufflenet_v2_qai-app-builder.mdx deleted file mode 100644 index 4f639be21..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_shufflenet_v2_qai-app-builder.mdx +++ /dev/null @@ -1,140 +0,0 @@ -This document describes using the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [ShuffleNet-v2](https://aihub.qualcomm.com/models/shufflenet_v2) -target recognition model using Qualcomm® Hexagon™ Processor (NPU). - -**Supported Devices** - -| Device | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## Install QAI AppBuilder - -:::tip - -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). - -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). - ::: - -## Run Example - -### Install Dependencies - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### Run Script - -- Navigate to the example directory - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- Prepare input image, using the following image as input example - - {" "} - -
- - input image -
- -- Execute inference - - - - ```bash - python3 shufflenet_v2/shufflenet_v2.py - ``` - - - - ```bash - $ python3 shufflenet_v2/shufflenet_v2.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/shufflenet_v2 - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 115.6ms [WARNING] Time: Read model file to memory. 0.91 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 125.6ms [WARNING] Time: contextCreateFromBinary. 10.02 - - 125.7ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 125.9ms [WARNING] Time: model_initialize shufflenet_v2 125.87 - - 199.9ms [WARNING] Time: model_inference shufflenet_v2 2.19 - - Top 5 predictions for image: - - Samoyed 0.8289067745 - wallaby 0.0915796161 - Pomeranian 0.0265247617 - Great Pyrenees 0.0101179369 - hare 0.0050830133 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 219.4ms [WARNING] Time: model_destroy shufflenet_v2 17.56 - ``` - - The printed results show that `Samoyed` has the highest confidence, which matches the input image content. diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_squeezenet1_1_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_squeezenet1_1_qai-app-builder.mdx deleted file mode 100644 index f0701d31b..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_squeezenet1_1_qai-app-builder.mdx +++ /dev/null @@ -1,140 +0,0 @@ -This document describes using the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [SqueezeNet-1.1](https://aihub.qualcomm.com/models/squeezenet1_1) -target recognition model using Qualcomm® Hexagon™ Processor (NPU). - -**Supported Devices** - -| Device | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## Install QAI AppBuilder - -:::tip - -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). - -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). - ::: - -## Run Example - -### Install Dependencies - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### Run Script - -- Navigate to the example directory - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- Prepare input image, using the following image as input example - - {" "} - -
- - input image -
- -- Execute inference - - - - ```bash - python3 squeezenet1_1/squeezenet1_1.py - ``` - - - - ```bash - $ python3 squeezenet1_1/squeezenet1_1.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/squeezenet1_1 - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 120.9ms [WARNING] Time: Read model file to memory. 2.75 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 132.8ms [WARNING] Time: contextCreateFromBinary. 11.81 - - 133.0ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 133.2ms [WARNING] Time: model_initialize squeezenet1_1 133.16 - - 225.5ms [WARNING] Time: model_inference squeezenet1_1 2.50 - - Top 5 predictions for image: - - Samoyed 0.8289067745 - wallaby 0.0915796161 - Pomeranian 0.0265247617 - Great Pyrenees 0.0101179369 - hare 0.0050830133 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 249.2ms [WARNING] Time: model_destroy squeezenet1_1 20.91 - ``` - - The printed results show that `Samoyed` has the highest confidence, which matches the input image content. diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_unet_seg_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_unet_seg_qai-app-builder.mdx index 9d1f599b3..71b352256 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_unet_seg_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_unet_seg_qai-app-builder.mdx @@ -1,144 +1,82 @@ This document explains how to use the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [Unet-Segmentation](https://aihub.qualcomm.com/models/unet_segmentation) image segmentation model using the Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | +| Device | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to the [**QAI AppBuilder Installation Guide**](../qai-appbuilder). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure the ADSP environment variables according to [**Configuring ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` -### Run the Script - -- Navigate to the example directory - - - - - - +### Run the script - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- Enter the upstream samples directory - + - - - + ```bash + cd qai-appbuilder/samples + ``` -- Prepare the input image. The following image is used as an example: + - {" "} +- Prepare input data (use the sample input if provided, or pass script arguments) -
- - Input image -
+
+ + Input image +
-- Execute inference +- Run inference ```bash - python3 unet_segmentation/unet_segmentation.py + python3 ComputerVision/Semantic_Segmentation/unet_segmentation/unet_segmentation.py --chipset 9075 ``` - ```bash - $ python3 unet_segmentation/unet_segmentation.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 153.4ms [WARNING] Time: Read model file to memory. 35.26 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +- Example result - 0.0ms [WARNING] This META does not have Alloc2 Support +
+ + Output image +
- 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 264.9ms [WARNING] Time: contextCreateFromBinary. 111.40 - - 265.0ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 267.2ms [WARNING] Time: model_initialize unet_segmentation 267.13 - - 627.1ms [WARNING] Time: model_inference unet_segmentation 246.67 - - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmp99xq_ell.PNG' - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /usr/bin/xdg-open: 882: www-browser: not found - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 2383.4ms [WARNING] Time: model_destroy unet_segmentation 67.34 - - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmp0hq03xtm.PNG' - ``` +:::tip +The first run downloads the model via Qualcomm® AI Hub (network-dependent). You can also use the launcher from `samples`: - {" "} +```bash +python3 run_inference.py --list +python3 run_inference.py --model unet_segmentation --args "--chipset 9075" +``` -
- - Output image -
+::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_vit_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_vit_qai-app-builder.mdx deleted file mode 100644 index da340e945..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_vit_qai-app-builder.mdx +++ /dev/null @@ -1,140 +0,0 @@ -This document describes using the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [ViT](https://aihub.qualcomm.com/models/vit) -target recognition model using Qualcomm® Hexagon™ Processor (NPU). - -**Supported Devices** - -| Device | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## Install QAI AppBuilder - -:::tip - -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). - -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). - ::: - -## Run Example - -### Install Dependencies - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### Run Script - -- Navigate to the example directory - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- Prepare input image, using the following image as input example - - {" "} - -
- - input image -
- -- Execute inference - - - - ```bash - python3 vit/vit.py - ``` - - - - ```bash - $ python3 vit/vit.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/vit - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 148.6ms [WARNING] Time: Read model file to memory. 32.50 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 351.2ms [WARNING] Time: contextCreateFromBinary. 202.52 - - 351.2ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 351.9ms [WARNING] Time: model_initialize vit 351.87 - - 560.6ms [WARNING] Time: model_inference vit 159.35 - - Top 5 predictions for image: - - Samoyed 0.7157644629 - Pomeranian 0.0566752702 - keeshond 0.0083217919 - Eskimo dog 0.0057556415 - Japanese spaniel 0.003955483 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 585.7ms [WARNING] Time: model_destroy vit 20.11 - ``` - - The printed results show that `Samoyed` has the highest confidence, which matches the input image content. diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_whisper_base_en_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_whisper_base_en_qai-app-builder.mdx index 23bdc4c48..02dfc078f 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_whisper_base_en_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_whisper_base_en_qai-app-builder.mdx @@ -1,250 +1,84 @@ This document explains how to use the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [Whisper-Base](https://aihub.qualcomm.com/models/whisper_base) speech recognition model using the Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | +| Device | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to the [**QAI AppBuilder Installation Guide**](../qai-appbuilder). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure the ADSP environment variables according to [**Configuring ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib openai-whisper audio2numpy samplerate transformers qai_hub_models==0.30.2 +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless audio2numpy ``` -### Run the Script - -- Navigate to the example directory - - - - - - +### Run the script - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- Enter the upstream samples directory - + - - - + ```bash + cd qai-appbuilder/samples + ``` -- Prepare the input audio. The following audio is used as an example: + - {" "} +- Prepare input data (use the sample input if provided, or pass script arguments) -
+

input audio

-- Execute inference +- Run inference ```bash - python3 whisper_base_en/whisper_base_en.py + python3 audio/Speech_Recognition/whisper_base_en/whisper_base_en.py --chipset 9075 ``` - ```bash - $ python3 whisper_base_en/whisper_base_en.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 184.8ms [WARNING] Time: Read model file to memory. 71.89 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 242.3ms [WARNING] Time: contextCreateFromBinary. 57.39 - - 242.3ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 244.9ms [WARNING] Time: model_initialize whisper_decoder 244.80 - - 282.6ms [WARNING] Time: Read model file to memory. 37.32 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 636.8ms [WARNING] Time: contextCreateFromBinary. 354.14 - - 636.9ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 638.6ms [WARNING] Time: model_initialize whisper_encoder 393.56 - - 1135.2ms [WARNING] Time: model_inference whisper_encoder 440.09 - - time consumes for encoder 0.4419412612915039(s) - Decoder Inference k_cache_cross type shape (6, 8, 64, 1500) type float32 - Decoder Inference v_cache_cross type shape (6, 8, 1500, 64) type float32 - start decode sample_len 224 - 1470.7ms [WARNING] Time: model_inference whisper_decoder 323.97 - - time consumes for decoder 0.3242673873901367(s) - 1796.0ms [WARNING] Time: model_inference whisper_decoder 322.57 - - time consumes for decoder 0.3229062557220459(s) - 2111.1ms [WARNING] Time: model_inference whisper_decoder 313.66 +The default input is the bundled `jfk.wav`. On success the terminal prints a transcription such as: - time consumes for decoder 0.31383657455444336(s) - 2425.6ms [WARNING] Time: model_inference whisper_decoder 313.01 - - time consumes for decoder 0.3132438659667969(s) - 2740.2ms [WARNING] Time: model_inference whisper_decoder 313.24 - - time consumes for decoder 0.3134174346923828(s) - 3055.0ms [WARNING] Time: model_inference whisper_decoder 313.45 - - time consumes for decoder 0.3136253356933594(s) - 3369.6ms [WARNING] Time: model_inference whisper_decoder 313.31 - - time consumes for decoder 0.31349658966064453(s) - 3684.4ms [WARNING] Time: model_inference whisper_decoder 313.44 - - time consumes for decoder 0.3136255741119385(s) - 3999.4ms [WARNING] Time: model_inference whisper_decoder 313.63 - - time consumes for decoder 0.3138108253479004(s) - 4314.0ms [WARNING] Time: model_inference whisper_decoder 313.20 - - time consumes for decoder 0.31337976455688477(s) - 4628.7ms [WARNING] Time: model_inference whisper_decoder 313.41 - - time consumes for decoder 0.313596248626709(s) - 4943.4ms [WARNING] Time: model_inference whisper_decoder 313.37 - - time consumes for decoder 0.3135509490966797(s) - 5258.1ms [WARNING] Time: model_inference whisper_decoder 313.29 - - time consumes for decoder 0.31348109245300293(s) - 5572.8ms [WARNING] Time: model_inference whisper_decoder 313.35 - - time consumes for decoder 0.31352949142456055(s) - 5887.5ms [WARNING] Time: model_inference whisper_decoder 313.35 - - time consumes for decoder 0.3135380744934082(s) - 6201.4ms [WARNING] Time: model_inference whisper_decoder 312.50 - - time consumes for decoder 0.31267857551574707(s) - 6515.8ms [WARNING] Time: model_inference whisper_decoder 313.02 - - time consumes for decoder 0.31319570541381836(s) - 6830.7ms [WARNING] Time: model_inference whisper_decoder 313.52 - - time consumes for decoder 0.31371450424194336(s) - 7145.6ms [WARNING] Time: model_inference whisper_decoder 313.57 - - time consumes for decoder 0.31376171112060547(s) - 7459.9ms [WARNING] Time: model_inference whisper_decoder 313.03 - - time consumes for decoder 0.3132154941558838(s) - 7774.9ms [WARNING] Time: model_inference whisper_decoder 313.60 - - time consumes for decoder 0.3137829303741455(s) - 8089.4ms [WARNING] Time: model_inference whisper_decoder 313.16 - - time consumes for decoder 0.31334543228149414(s) - 8404.7ms [WARNING] Time: model_inference whisper_decoder 313.92 - - time consumes for decoder 0.31411004066467285(s) - 8719.6ms [WARNING] Time: model_inference whisper_decoder 313.62 - - time consumes for decoder 0.31380605697631836(s) - 9034.5ms [WARNING] Time: model_inference whisper_decoder 313.51 - - time consumes for decoder 0.31369900703430176(s) - 9349.9ms [WARNING] Time: model_inference whisper_decoder 314.01 - - time consumes for decoder 0.31418848037719727(s) - 9664.7ms [WARNING] Time: model_inference whisper_decoder 313.53 - - time consumes for decoder 0.3137087821960449(s) - 9979.7ms [WARNING] Time: model_inference whisper_decoder 313.63 - - time consumes for decoder 0.3138093948364258(s) - Transcription: And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country. - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 10095.0ms [WARNING] Time: model_destroy whisper_decoder 13.25 - - Logs will be sent to the system's default channel - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 10198.2ms [WARNING] Time: model_destroy whisper_encoder 103.03 +```text +Transcription: And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country. +``` - ``` +:::tip +The first run downloads encoder/decoder models via Qualcomm® AI Hub. Decoding needs the `tiktoken` vocabulary; if the device cannot reach `openaipublic.blob.core.windows.net`, pre-warm `tiktoken.get_encoding("gpt2")` on a machine with network access. - Recognition Results +```bash +python3 run_inference.py --list +python3 run_inference.py --model whisper_base_en --args "--chipset 9075" +``` - ```bash - Transcription: And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country. - ``` +::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_whisper_tiny_en_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_whisper_tiny_en_qai-app-builder.mdx index 0a1488c9c..e9eb6d798 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_whisper_tiny_en_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_whisper_tiny_en_qai-app-builder.mdx @@ -1,220 +1,84 @@ This document explains how to use the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [Whisper-Tiny](https://aihub.qualcomm.com/models/whisper_tiny) speech recognition model using the Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | +| Device | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to the [**QAI AppBuilder Installation Guide**](../qai-appbuilder). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure the ADSP environment variables according to [**Configuring ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib openai-whisper audio2numpy samplerate transformers qai_hub_models==0.30.2 +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless audio2numpy ``` -### Run the Script - -- Navigate to the example directory - - - - - - +### Run the script - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- Enter the upstream samples directory - + - - - + ```bash + cd qai-appbuilder/samples + ``` -- Prepare the input audio. The following audio is used as an example: + - {" "} +- Prepare input data (use the sample input if provided, or pass script arguments) -
+

input audio

-- Execute inference +- Run inference ```bash - python3 whisper_tiny_en/whisper_tiny_en.py + python3 audio/Speech_Recognition/whisper_tiny_en/whisper_tiny_en.py --chipset 9075 ``` - ```bash - $ python3 whisper_tiny_en/whisper_tiny_en.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 175.0ms [WARNING] Time: Read model file to memory. 51.68 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 224.3ms [WARNING] Time: contextCreateFromBinary. 49.16 - - 224.3ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 226.7ms [WARNING] Time: model_initialize whisper_decoder 226.65 - - 245.7ms [WARNING] Time: Read model file to memory. 18.37 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 311.9ms [WARNING] Time: contextCreateFromBinary. 66.16 - - 312.0ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 313.2ms [WARNING] Time: model_initialize whisper_encoder 86.17 - - 567.2ms [WARNING] Time: model_inference whisper_encoder 199.78 - - Decoder Inference k_cache_cross type shape (4, 6, 64, 1500) type float32 - Decoder Inference v_cache_cross type shape (4, 6, 1500, 64) type float32 - start decode sample_len 224 - 728.7ms [WARNING] Time: model_inference whisper_decoder 159.73 - - 887.8ms [WARNING] Time: model_inference whisper_decoder 157.66 - - 1046.9ms [WARNING] Time: model_inference whisper_decoder 157.76 +The default input is the bundled `jfk.wav`. On success the terminal prints a transcription such as: - 1205.3ms [WARNING] Time: model_inference whisper_decoder 157.02 - - 1365.0ms [WARNING] Time: model_inference whisper_decoder 158.39 - - 1523.8ms [WARNING] Time: model_inference whisper_decoder 157.56 - - 1682.7ms [WARNING] Time: model_inference whisper_decoder 157.58 - - 1841.0ms [WARNING] Time: model_inference whisper_decoder 157.04 - - 2000.0ms [WARNING] Time: model_inference whisper_decoder 157.82 - - 2158.6ms [WARNING] Time: model_inference whisper_decoder 157.36 - - 2317.7ms [WARNING] Time: model_inference whisper_decoder 157.76 - - 2476.8ms [WARNING] Time: model_inference whisper_decoder 157.78 - - 2635.7ms [WARNING] Time: model_inference whisper_decoder 157.58 - - 2794.4ms [WARNING] Time: model_inference whisper_decoder 157.39 - - 2953.2ms [WARNING] Time: model_inference whisper_decoder 157.47 - - 3111.9ms [WARNING] Time: model_inference whisper_decoder 157.46 - - 3270.2ms [WARNING] Time: model_inference whisper_decoder 157.10 - - 3429.0ms [WARNING] Time: model_inference whisper_decoder 157.66 - - 3588.0ms [WARNING] Time: model_inference whisper_decoder 157.72 - - 3747.3ms [WARNING] Time: model_inference whisper_decoder 157.92 - - 3906.3ms [WARNING] Time: model_inference whisper_decoder 157.68 - - 4065.3ms [WARNING] Time: model_inference whisper_decoder 157.71 - - 4224.5ms [WARNING] Time: model_inference whisper_decoder 157.89 - - 4383.8ms [WARNING] Time: model_inference whisper_decoder 158.02 - - 4542.9ms [WARNING] Time: model_inference whisper_decoder 157.74 - - 4702.4ms [WARNING] Time: model_inference whisper_decoder 158.21 - - 4861.6ms [WARNING] Time: model_inference whisper_decoder 157.92 - - 5021.1ms [WARNING] Time: model_inference whisper_decoder 158.13 - - Transcription: And so my fellow Americans ask not what your country can do for you ask what you can do for your country. - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 5134.7ms [WARNING] Time: model_destroy whisper_decoder 14.94 - - Logs will be sent to the system's default channel - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +```text +Transcription: And so my fellow Americans ask not what your country can do for you, ask what you can do for your country. +``` - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 5208.4ms [WARNING] Time: model_destroy whisper_encoder 73.58 - ``` +:::tip +The first run downloads encoder/decoder models via Qualcomm® AI Hub. Decoding needs the `tiktoken` vocabulary; if the device cannot reach `openaipublic.blob.core.windows.net`, pre-warm `tiktoken.get_encoding("gpt2")` on a machine with network access. - Recognition Results +```bash +python3 run_inference.py --list +python3 run_inference.py --model whisper_tiny_en --args "--chipset 9075" +``` - ```bash - Transcription: And so my fellow Americans ask not what your country can do for you ask what you can do for your country. - ``` +::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_wideresnet50_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_wideresnet50_qai-app-builder.mdx deleted file mode 100644 index c071c2033..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_wideresnet50_qai-app-builder.mdx +++ /dev/null @@ -1,140 +0,0 @@ -This document describes using the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [WideResNet50](https://aihub.qualcomm.com/models/wideresnet50) -target recognition model using Qualcomm® Hexagon™ Processor (NPU). - -**Supported Devices** - -| Device | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## Install QAI AppBuilder - -:::tip - -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). - -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). - ::: - -## Run Example - -### Install Dependencies - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision -``` - - - -### Run Script - -- Navigate to the example directory - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- Prepare input image, using the following image as input example - - {" "} - -
- - input image -
- -- Execute inference - - - - ```bash - python3 wideresnet50/wideresnet50.py - ``` - - - - ```bash - $ python3 wideresnet50/wideresnet50.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/wideresnet50 - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 146.7ms [WARNING] Time: Read model file to memory. 30.54 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 179.0ms [WARNING] Time: contextCreateFromBinary. 32.18 - - 179.0ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 179.5ms [WARNING] Time: model_initialize wideresnet50 179.49 - - 243.2ms [WARNING] Time: model_inference wideresnet50 9.66 - - Top 5 predictions for image: - - Samoyed 0.8442133665 - keeshond 0.0415370725 - Pomeranian 0.0302517358 - white wolf 0.0258170888 - chow 0.0072637959 - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 258.1ms [WARNING] Time: model_destroy wideresnet50 11.75 - ``` - - The printed results show that `Samoyed` has the highest confidence, which matches the input image content. diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_xlsr_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_xlsr_qai-app-builder.mdx deleted file mode 100644 index 139867356..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_xlsr_qai-app-builder.mdx +++ /dev/null @@ -1,152 +0,0 @@ -This document describes using the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [XLSR](https://aihub.qualcomm.com/models/xlsr) -image super-resolution model using Qualcomm® Hexagon™ Processor (NPU). - -**Supported Devices** - -| Device | SoC | -| ---------- | ----- | -| Dragon Q6A | QCS6490 | -| Dragon Q8B | SC8280XP | - -## Install QAI AppBuilder - -:::tip - -1. Please install QAI AppBuilder according to [**QAI AppBuilder Installation Guide**](../qai-appbuilder#installation-methods). - -2. Please configure ADSP environment variables according to [**Create ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). - ::: - -## Run Example - -### Install Dependencies - - - -```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision matplotlib easyocr qai_hub_models transformers openai-clip -``` - - - -### Run Script - -- Navigate to the example directory - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - -- Prepare input image, using the following image as input example - - {" "} - -
- - input image -
- -- Execute inference - - - - ```bash - python3 xlsr/xlsr.py - ``` - - - - ```bash - $ python3 xlsr/xlsr.py - Current file directory: /mnt/ssd/qualcomm/701/zzf_fork/ai-engine-direct-helper/samples/linux/python/xlsr - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 113.4ms [WARNING] Time: Read model file to memory. 0.41 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 121.2ms [WARNING] Time: contextCreateFromBinary. 7.83 - - 121.3ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 121.5ms [WARNING] Time: model_initialize xlsr 121.46 - - 164.3ms [WARNING] Time: model_inference xlsr 7.48 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 300.2ms [WARNING] Time: model_destroy xlsr 6.65 - - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmp5i2_qhdq.PNG' - ``` - - {" "} - -
- - output image -
diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_yamnet_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_yamnet_qai-app-builder.mdx index c98a155df..fa09460e7 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_yamnet_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_yamnet_qai-app-builder.mdx @@ -1,135 +1,85 @@ This document explains how to use the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [YamNet](https://aihub.qualcomm.com/compute/models/yamnet) audio classification model using the Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | +| Device | SoC | +| -------------------- | ------- | | Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to the [**QAI AppBuilder Installation Guide**](../qai-appbuilder). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure the ADSP environment variables according to [**Configuring ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision resampy soundfile +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless soxr soundfile resampy ``` -### Run the Script - -- Navigate to the example directory - - - - - - +### Run the script - ```bash - cd ai-engine-direct-helper/samples/python - ``` +- Enter the upstream samples directory - + - - - + ```bash + cd qai-appbuilder/samples + ``` -- Prepare the input audio. The following audio is used as an example: + - {" "} +- Prepare input data (use the sample input if provided, or pass script arguments) -
+

input audio

-- Execute inference +- Run inference ```bash - python3 yamnet/yamnet.py + python3 audio/Audio_Classification/yamnet/yamnet.py --chipset 9075 ``` - ```bash - $ python3 yamnet/yamnet.py - 0.0ms [WARNING] Initializing HtpProvider - - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 113.7ms [WARNING] Time: Read model file to memory. 4.55 - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +The default input is the bundled `input.wav`. On success the terminal prints top-5 labels, for example: - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 127.6ms [WARNING] Time: contextCreateFromBinary. 13.42 - - 128.0ms [WARNING] Time: UnmapViewOfFile. 0.00 - - 129.0ms [WARNING] Time: model_initialize yamnet 128.97 - - 145.8ms [WARNING] Time: model_inference yamnet 0.80 - - Top 5 predictions: - Whistling | Alarm | Whistle | Field recording | Rattle (instrument) - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support - - 0.0ms [WARNING] This META does not have Alloc2 Support +```text +Top 5 predictions: +Whistling | Alarm | Whistle | Field recording | Rattle (instrument) +``` - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 229.1ms [WARNING] Time: model_destroy yamnet 79.87 - ``` +:::tip +The first run downloads the model via Qualcomm® AI Hub and needs audio libraries such as `soxr`. You can also use the launcher from `samples`: - Recognition Results +```bash +python3 run_inference.py --list +python3 run_inference.py --model yamnet --args "--chipset 9075" +``` - ```bash - Top 5 predictions: - Whistling | Alarm | Whistle | Field recording | Rattle (instrument) - ``` +::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_yolov8_det_qai-app-builder.mdx b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_yolov8_det_qai-app-builder.mdx index 98c42e373..7a097be4d 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_yolov8_det_qai-app-builder.mdx +++ b/i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_yolov8_det_qai-app-builder.mdx @@ -1,161 +1,179 @@ -This document explains how to use the [QAI AppBuilder](../qai-appbuilder) Python API to perform inference with the [YOLOv8-det](https://aihub.qualcomm.com/models/yolov8_det?domain=Computer+Vision&useCase=Object+Detection) object detection model using the Qualcomm® Hexagon™ Processor (NPU). +This document describes using the [QAI AppBuilder](../qai-appbuilder) Python API to run inference with the [YOLOv8-det](https://aihub.qualcomm.com/models/yolov8_det?domain=Computer+Vision&useCase=Object+Detection) object detection model on the Qualcomm® Hexagon™ Processor (NPU). -**Supported Devices** +**Supported devices** -| Device | SoC | -| -------------------- | ----- | -| Dragon Q6A | QCS6490 | +| Device | SoC | +| -------------------- | -------- | +| Dragon Q6A | QCS6490 | | Dragon Q8B | SC8280XP | -| Fogwise® AIRbox Q900 | QCS9075 | +| Fogwise® AIRbox Q900 | QCS9075 | ## Install QAI AppBuilder :::tip -1. Please install QAI AppBuilder according to the [**QAI AppBuilder Installation Guide**](../qai-appbuilder). +1. Install QAI AppBuilder by following the [**QAI AppBuilder installation guide**](../qai-appbuilder). -2. Please configure the ADSP environment variables according to [**Configuring ADSP Environment Variables**](../qai-appbuilder#create-adsp-environment-variables). +2. Configure ADSP environment variables as described in [**Create ADSP environment variables**](../qai-appbuilder#create-adsp-environment-variables). ::: -## Run Example +## Run the sample -### Install Dependencies +### Install dependencies + +Install sample dependencies in the activated virtual environment: ```bash -pip3 install requests tqdm qai-hub py3_wget opencv-python torch torchvision +pip3 install requests tqdm qai-hub py3-wget Pillow torch torchvision opencv-python-headless ``` -### Run the Script - -- Navigate to the example directory - - - - - - - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` - - - - - - - +### Run the script - +- Enter the upstream samples directory - ```bash - cd ai-engine-direct-helper/samples/linux/python - ``` + - + ```bash + cd qai-appbuilder/samples + ``` - + - +- Prepare input data (use the bundled sample input when available) - +
+ + input image +
- ```bash - cd ai-engine-direct-helper/samples/python - ``` +### Fix output order (required on QCS6490 / SC8280XP / QCS9075) - +Verified on Dragon Q6A (`--chipset 6490`) and AIRbox Q900 (`--chipset 9075`): AI Hub packages often use this layout: - +| Output index | Shape | Meaning | +| ------------ | -------------- | --------- | +| `0` | `[1, 8400]` | scores | +| `1` | `[1, 8400]` | class_idx | +| `2` | `[1, 8400, 4]` | boxes | - +Upstream `yolov8_det.py` assumes `[boxes, scores, class_idx]`, which leads to an NMS failure: -- Prepare the input image. The following image is used as an example: +```text +IndexError: index ... is out of bounds for dimension 0 with size ... +``` - {" "} +Before running, update the output parsing in `Inference()` inside `ComputerVision/Object_Detection/yolov8_det/yolov8_det.py` (back up the file first). -
- - input image -
+Replace: -- Execute inference +```python + # Run the inference. + model_output = yolov8.Inference(image) - + pred_boxes = torch.tensor(model_output[0].reshape(1, -1, 4)) + pred_scores = torch.tensor(model_output[1].reshape(1, -1)) + pred_class_idx = torch.tensor(model_output[2].reshape(1, -1)) +``` - ```bash - python3 yolov8_det/yolov8_det.py - ``` +with: + +```python + # Run the inference. + model_output = yolov8.Inference(image) + + # Output layout differs by model package: + # - some packages: [boxes, scores, class_idx] + # - QCS6490 / QCS9075 hub packages often: [scores, class_idx, boxes] + import numpy as _np + a0, a1, a2 = _np.array(model_output[0]), _np.array(model_output[1]), _np.array(model_output[2]) + if a2.ndim == 3 and a2.shape[-1] == 4: + pred_scores = torch.tensor(a0.reshape(1, -1)) + pred_class_idx = torch.tensor(a1.reshape(1, -1)) + pred_boxes = torch.tensor(a2.reshape(1, -1, 4)) + else: + pred_boxes = torch.tensor(a0.reshape(1, -1, 4)) + pred_scores = torch.tensor(a1.reshape(1, -1)) + pred_class_idx = torch.tensor(a2.reshape(1, -1)) +``` - +This auto-selects the layout when the last tensor is `[..., 4]`, and keeps compatibility with 6490 / 9075 and other packages. - ```bash - $ python3 yolov8_det/yolov8_det.py - 0.0ms [WARNING] Initializing HtpProvider +### Run inference - /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc - 0.0ms [WARNING] This META does not have Alloc2 Support + + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Object_Detection/yolov8_det/yolov8_det.py --chipset 6490 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 121.1ms [WARNING] Time: Read model file to memory. 6.22 + + - 0.0ms [WARNING] This META does not have Alloc2 Support +:::tip +SC8280XP and QCS6490 both use Hexagon V68. Use `--chipset 6490`, and apply the same output-order fix above. +::: - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Object_Detection/yolov8_det/yolov8_det.py --chipset 6490 +``` - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support + + - 0.0ms [WARNING] This META does not have Alloc2 Support + - 0.0ms [WARNING] This META does not have Alloc2 Support +```bash +python3 ComputerVision/Object_Detection/yolov8_det/yolov8_det.py --chipset 9075 +``` - 151.3ms [WARNING] Time: contextCreateFromBinary. 30.09 + - 151.4ms [WARNING] Time: UnmapViewOfFile. 0.00 + + - 152.3ms [WARNING] Time: model_initialize yolov8 152.21 +### Expected result - 230.4ms [WARNING] Time: model_inference yolov8 10.35 +After the output-order fix, a successful run on QCS6490 or QCS9075 writes: - 0.0ms [WARNING] This META does not have Alloc2 Support +```text +ComputerVision/Object_Detection/yolov8_det/output.png +``` - 0.0ms [WARNING] This META does not have Alloc2 Support +Without a display, `Error: no DISPLAY environment variable specified` may appear; ignore it if `output.png` exists and the exit code is 0. - 0.0ms [WARNING] This META does not have Alloc2 Support +- Example result (illustrative; actual result depends on the input and model package) - 0.0ms [WARNING] This META does not have Alloc2 Support +
+ + output image +
- /prj/qct/webtech_scratch20/mlg_user_admin/qaisw_source_repo/rel/qairt-2.37.1/point_release/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc - 650.0ms [WARNING] Time: model_destroy yolov8 61.01 +:::tip +Launcher alternative (still apply the output-order fix first): - /usr/bin/xdg-open: 882: www-browser: not found - /usr/bin/xdg-open: 882: links2: not found - /usr/bin/xdg-open: 882: elinks: not found - /usr/bin/xdg-open: 882: links: not found - /usr/bin/xdg-open: 882: lynx: not found - /usr/bin/xdg-open: 882: w3m: not found - xdg-open: no method available for opening '/tmp/tmpqk93qmdb.PNG' - ``` +```bash +python3 run_inference.py --model yolov8_det --args "--chipset 6490" +# or on QCS9075: +# python3 run_inference.py --model yolov8_det --args "--chipset 9075" +``` -
- - output image -
+::: diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/README.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/README.md index 9ae22a989..9cceafd83 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/README.md +++ b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/README.md @@ -2,8 +2,10 @@ sidebar_position: 99 --- -# CV Demos +# QAI AppBuilder samples -A collection of example models using NPU inference with the Qai-APPBuilder Python API. +Samples from upstream [QAI AppBuilder](https://github.com/qualcomm/qai-appbuilder) `samples` that can download models on QCS6490 with `--chipset 6490`. For the full menu, run `python3 run_inference.py --list`. + +Complete [**QAI AppBuilder installation**](../qai-appbuilder) first. diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/convnext-base.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/convnext-base.md deleted file mode 100644 index 511724b59..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/convnext-base.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_convnext_base_qai-app-builder.mdx ---- - -# ConvNext-Base Example - -import CONVNEXTBASEQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_convnext_base_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/convnext-tiny.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/convnext-tiny.md deleted file mode 100644 index 888c84284..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/convnext-tiny.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_convnext_tiny_qai-app-builder.mdx ---- - -# ConvNext-Tiny Example - -import CONVNEXTTINYQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_convnext_tiny_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b0.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b0.md deleted file mode 100644 index ec4fd4e9c..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b0.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_efficientnet_b0_qai-app-builder.mdx ---- - -# EfficientNet-B0 Example - -import EFFICIENTNETB0QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_efficientnet_b0_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b4.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b4.md deleted file mode 100644 index 738e00aaf..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b4.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_efficientnet_b4_qai-app-builder.mdx ---- - -# EfficientNet-B4 Example - -import EFFICIENTNETB4QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_efficientnet_b4_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-v2s.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-v2s.md deleted file mode 100644 index 4997eaff4..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-v2s.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_efficientnet_v2s_qai-app-builder.mdx ---- - -# EfficientNet-V2-s Example - -import EFFICIENTNETV2SQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_efficientnet_v2s_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/fcn-resnet50.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/fcn-resnet50.md deleted file mode 100644 index 0a2c19254..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/fcn-resnet50.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_fcn-resnet50_qai-app-builder.mdx ---- - -# FCN-ResNet50 Example - -import FCNRESNRT50QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_fcn-resnet50_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/levit.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/levit.md deleted file mode 100644 index a497fca7a..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/levit.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_levit_qai-app-builder.mdx ---- - -# LeViT Example - -import LEVITQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_levit_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/regnet.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/regnet.md deleted file mode 100644 index 0e66caf80..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/regnet.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_regnet_qai-app-builder.mdx ---- - -# RegNet Example - -import REGNETQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_regnet_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/sesr-m5.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/sesr-m5.md deleted file mode 100644 index eaf3a9350..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/sesr-m5.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_sesr_m5_qai-app-builder.mdx ---- - -# SESR-M5 Example - -import SESRM5QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_sesr_m5_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/shufflenet-v2.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/shufflenet-v2.md deleted file mode 100644 index be047eb4b..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/shufflenet-v2.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_shufflenet_v2_qai-app-builder.mdx ---- - -# Shufflenet-v2 Example - -import SHUFFLENETV2QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_shufflenet_v2_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/squeezenet1-1.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/squeezenet1-1.md deleted file mode 100644 index 65fbddf32..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/squeezenet1-1.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_squeezenet1_1_qai-app-builder.mdx ---- - -# SqueezeNet-1.1 Example - -import SQUEEZENET11QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_squeezenet1_1_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/vit.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/vit.md deleted file mode 100644 index 664272bc7..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/vit.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_vit_qai-app-builder.mdx ---- - -# VIT Example - -import VITQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_vit_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/wideresnet50.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/wideresnet50.md deleted file mode 100644 index 111c66cf4..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/wideresnet50.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_wideresnet50_qai-app-builder.mdx ---- - -# WideResNet50 Example - -import WIDERESNET50QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_wideresnet50_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/xlsr.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/xlsr.md deleted file mode 100644 index 8e1dd668c..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q6a/app-dev/npu-dev/qai-appbuilder-demo/xlsr.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_xlsr_qai-app-builder.mdx ---- - -# XLSR Example - -import XLSRQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_xlsr_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/README.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/README.md index 9ae22a989..518b6d382 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/README.md +++ b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/README.md @@ -2,8 +2,10 @@ sidebar_position: 99 --- -# CV Demos +# QAI AppBuilder samples -A collection of example models using NPU inference with the Qai-APPBuilder Python API. +Samples from upstream [QAI AppBuilder](https://github.com/qualcomm/qai-appbuilder) `samples` that can use `--chipset 6490` (Hexagon V68) on SC8280XP. For the full menu, run `python3 run_inference.py --list`. + +Complete [**QAI AppBuilder installation**](../qai-appbuilder) first. diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/convnext-base.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/convnext-base.md deleted file mode 100644 index 9c15be042..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/convnext-base.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 0 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_convnext_base_qai-app-builder.mdx ---- - -# ConvNext-Base Example - -import CONVNEXTBASEQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_convnext_base_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/convnext-tiny.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/convnext-tiny.md deleted file mode 100644 index 960d89be2..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/convnext-tiny.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 1 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_convnext_tiny_qai-app-builder.mdx ---- - -# ConvNext-Tiny Example - -import CONVNEXTTINYQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_convnext_tiny_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b0.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b0.md deleted file mode 100644 index 1f4553ec5..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b0.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 2 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_efficientnet_b0_qai-app-builder.mdx ---- - -# EfficientNet-B0 Example - -import EFFICIENTNETB0QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_efficientnet_b0_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b4.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b4.md deleted file mode 100644 index 6553415dc..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-b4.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 3 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_efficientnet_b4_qai-app-builder.mdx ---- - -# EfficientNet-B4 Example - -import EFFICIENTNETB4QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_efficientnet_b4_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-v2s.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-v2s.md deleted file mode 100644 index d1d7310da..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/efficientnet-v2s.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 4 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_efficientnet_v2s_qai-app-builder.mdx ---- - -# EfficientNet-V2-s Example - -import EFFICIENTNETV2SQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_efficientnet_v2s_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/fcn-resnet50.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/fcn-resnet50.md deleted file mode 100644 index 1f770d49a..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/fcn-resnet50.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 13 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_fcn-resnet50_qai-app-builder.mdx ---- - -# FCN-ResNet50 Example - -import FCNRESNRT50QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_fcn-resnet50_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/levit.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/levit.md deleted file mode 100644 index bb84b3f3c..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/levit.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 7 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_levit_qai-app-builder.mdx ---- - -# LeViT Example - -import LEVITQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_levit_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/regnet.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/regnet.md deleted file mode 100644 index e7c7c16f7..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/regnet.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 8 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_regnet_qai-app-builder.mdx ---- - -# RegNet Example - -import REGNETQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_regnet_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/sesr-m5.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/sesr-m5.md deleted file mode 100644 index afd916012..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/sesr-m5.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 18 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_sesr_m5_qai-app-builder.mdx ---- - -# SESR-M5 Example - -import SESRM5QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_sesr_m5_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/shufflenet-v2.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/shufflenet-v2.md deleted file mode 100644 index 1f672408b..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/shufflenet-v2.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 9 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_shufflenet_v2_qai-app-builder.mdx ---- - -# Shufflenet-v2 Example - -import SHUFFLENETV2QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_shufflenet_v2_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/squeezenet1-1.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/squeezenet1-1.md deleted file mode 100644 index 6cf7168ec..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/squeezenet1-1.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 10 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_squeezenet1_1_qai-app-builder.mdx ---- - -# SqueezeNet-1.1 Example - -import SQUEEZENET11QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_squeezenet1_1_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/vit.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/vit.md deleted file mode 100644 index d593b0aca..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/vit.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 11 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_vit_qai-app-builder.mdx ---- - -# VIT Example - -import VITQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_vit_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/wideresnet50.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/wideresnet50.md deleted file mode 100644 index 0396fc3b2..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/wideresnet50.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 12 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_wideresnet50_qai-app-builder.mdx ---- - -# WideResNet50 Example - -import WIDERESNET50QAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_wideresnet50_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/xlsr.md b/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/xlsr.md deleted file mode 100644 index 6c6889b8f..000000000 --- a/i18n/en/docusaurus-plugin-content-docs/current/dragon/q8b/app-dev/npu-dev/qai-appbuilder-demo/xlsr.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -sidebar_position: 19 - -doc_kind: wrapper -source_of_truth: common -imports_resolve_to: - - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_xlsr_qai-app-builder.mdx ---- - -# XLSR Example - -import XLSRQAIAPPBUILDER from '../../../../../common/ai/qualcomm/\_xlsr_qai-app-builder.mdx'; - - diff --git a/i18n/en/docusaurus-plugin-content-docs/current/fogwise/airbox-q900/ai-dev/qai-appbuilder-demo/README.md b/i18n/en/docusaurus-plugin-content-docs/current/fogwise/airbox-q900/ai-dev/qai-appbuilder-demo/README.md index 8a5a6ce2b..1356a4853 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/fogwise/airbox-q900/ai-dev/qai-appbuilder-demo/README.md +++ b/i18n/en/docusaurus-plugin-content-docs/current/fogwise/airbox-q900/ai-dev/qai-appbuilder-demo/README.md @@ -2,8 +2,10 @@ sidebar_position: 99 --- -# CV Demos +# QAI AppBuilder samples -Example collection of using Qai-APPBuilder Python API to utilize NPU inference models. +Covers [QAI AppBuilder](https://github.com/qualcomm/qai-appbuilder) samples that are documentable on QCS9075 (use `--chipset 9075` on Linux). Some entries in upstream `run_inference.py --list` only support Snapdragon X Elite / X2 Elite (Windows on Snapdragon) and are omitted here. + +Complete [**QAI AppBuilder installation**](../qai-appbuilder) first. diff --git a/i18n/en/docusaurus-plugin-content-docs/current/fogwise/airbox-q900/ai-dev/qai-appbuilder-demo/qwen-vl.md b/i18n/en/docusaurus-plugin-content-docs/current/fogwise/airbox-q900/ai-dev/qai-appbuilder-demo/qwen-vl.md new file mode 100644 index 000000000..0c7653e35 --- /dev/null +++ b/i18n/en/docusaurus-plugin-content-docs/current/fogwise/airbox-q900/ai-dev/qai-appbuilder-demo/qwen-vl.md @@ -0,0 +1,14 @@ +--- +sidebar_position: 0 + +doc_kind: wrapper +source_of_truth: common +imports_resolve_to: + - i18n/en/docusaurus-plugin-content-docs/current/common/ai/qualcomm/_qwen_vl_qai-app-builder.mdx +--- + +# Qwen VL Sample + +import QWENVLQAIAPPBUILDER from '../../../../common/ai/qualcomm/\_qwen_vl_qai-app-builder.mdx'; + +