Snapdragon 8 elite support

[Acs176]

Hi, we've successfully run whisper with QNN delegate on snapdragon 8 gen2, gen3 and snapdragon 7.
However, when running on 8 elite (SM8750) we get the next issue:

2025-02-26 07:44:51.491 11521-11552 QnnDsp                  com.whispertflite                    W   <W> Initializing HtpProvider
2025-02-26 07:44:51.491 11521-11552 QnnDsp                  com.whispertflite                    E   <E> Dsp startup: SoC model (SnapdragonModel) is unknown
2025-02-26 07:44:51.491 11521-11552 QnnDsp                  com.whispertflite                    E   <E> Htp startup on load failed
2025-02-26 07:44:51.491 11521-11552 QnnDsp                  com.whispertflite                    E   <E> Unable to load Remote symbols 4000
2025-02-26 07:44:51.491 11521-11552 QnnDsp                  com.whispertflite                    E   <E> Failed to init router
2025-02-26 07:44:51.491 11521-11552 QnnDsp                  com.whispertflite                    E   <E> Provider initialization failed, unable to detect platform info
2025-02-26 07:44:51.491 11521-11552 tflite                  com.whispertflite                    E  Failed to get HTP platform info. Error 1100
2025-02-26 07:44:51.491 11521-11552 tflite                  com.whispertflite                    E  Restored original execution plan after delegate application failure.

Any ideas?
Thanks in advance!

[mikel-brostrom]

What would be required to enable this @bpkeene, @keith4ever?

[keith4ever]

Hi @Acs176 , you must be doing a very interesting project if you're trying to run it on Snapdragon 8 elite.
First off, have you added the 'SM8750' to the list of qnn supported SoC? I'm guessing you did, and that is why you're getting those.
If that's the case can you post more detailed log after turning on the verbose option at CLI? Another option is to try that again with more recent qnn sdk & runtime version (2.31) which can be updated in scripts/Dockerfile and dev_env.sh.

[Acs176]

Hi @keith4ever, thanks for your answer! We have tried all that and still same output sadly.
I attach the logcat output.

samsung-SM-S938B-Android-15_2025-02-27_173330.logcat.zip

[bpkeene]

Can you confirm that running getprop ro.soc.model via adbshell yields the SM8750? That would match the runtime check with your modification. Separately, i'm seeing some compatibility warnings with the QNN delegate + (OS? libraries?) being used that look correlated with the backend initialization. Perhaps version mismatch is at play?

Making a note in our project tracking to improve how we support NPU enabled SoCs... there should be a better configuration for this that doesn't require manually adding SoC numbers to a list somewhere.

[mjnong]

We finally got it loading using QNN but while trying to transcribe and loading the model we get the errors

[LOADING WHISPER INIT]
Failed to open for writing: mel_spectrogram_tiny/2.24.043272174977434281231741162509

Failed to open for writing: whisper_encoder_tiny/2.24.027514202768396780201741162510

Failed to open for writing: whisper_decoder_tiny/2.24.0166175867607270514751741162550

[TRYING TO TRANSCRIBE]
Failed to open for writing: audio_input_tiny/2.24.0137216240413586050951741162565
QnnDsp <E> Graph qnn_delegate_graph_0 neither registered on skel nor in low memory mode to activate graph reload
QnnDsp <E> Graph qnn_delegate_graph_0 failed in execution with err 6001
qnn_graph_execute failed. Error 6001
Node number 271 (TfLiteQnnDelegate) failed to invoke.

The transcription result end-up being ""

[mjnong]

We finally got it loading using QNN but while trying to transcribe and loading the model we get the errors

[LOADING WHISPER INIT]
Failed to open for writing: mel_spectrogram_tiny/2.24.043272174977434281231741162509

Failed to open for writing: whisper_encoder_tiny/2.24.027514202768396780201741162510

Failed to open for writing: whisper_decoder_tiny/2.24.0166175867607270514751741162550

[TRYING TO TRANSCRIBE]
Failed to open for writing: audio_input_tiny/2.24.0137216240413586050951741162565
QnnDsp <E> Graph qnn_delegate_graph_0 neither registered on skel nor in low memory mode to activate graph reload
QnnDsp <E> Graph qnn_delegate_graph_0 failed in execution with err 6001
qnn_graph_execute failed. Error 6001
Node number 271 (TfLiteQnnDelegate) failed to invoke.

The transcription result end-up being ""

These errors seems to origin from tflite

[keith4ever]

Hi @marcusnagy ,

Thanks for the extra info.
One thing to clear: Is SM8750 supposed to be with Gen 4 HTP? If so, here is a strong speculation (that we can't confirm but very likely).
The reason might be that we're currently using QNN SDK & runtime version 2.27: https://github.com/argmaxinc/WhisperKitAndroid/blob/main/scripts/dev_env.sh#L48-L49

And I suspect SM8750 needs new recent library such as libQnnHtpV79Skel.so & libQnnHtpV79Stub.so, which are obviously missing in 2.27. If this is the case, you can use the latest version (2.31?) and see if it resolves this issue.
Replace those two lines in scripts/dev_env.sh:

aria2c $ARIA_OPTIONS -d $BUILD_DIR https://repo1.maven.org/maven2/com/qualcomm/qti/qnn-runtime/2.31.0/qnn-runtime-2.31.0.aar
aria2c $ARIA_OPTIONS -d $BUILD_DIR https://repo1.maven.org/maven2/com/qualcomm/qti/qnn-litert-delegate/2.31.0/qnn-litert-delegate-2.31.0.aar

Additionally replace the corresponding lines in Dockerfile: https://github.com/argmaxinc/WhisperKitAndroid/blob/main/scripts/Dockerfile#L41-L42
with this:

ARG QNN_RUNTIME=qnn-runtime-2.31.0.aar
ARG QNN_TFLITE_DELEGATE=qnn-litert-delegate-2.31.0.aar

And then rebuild the docker image. Also don't forget to run "make adb-push" to push all new .so files are available in the phone.

I hope this is the root cause, but if it still fails, your log would help us to determine the next step.

[mjnong]

This solved the issue but now we end up with the occasional issue of getting a transcriptions such as "!!!!!!!!!!..." a bit un-clear where this comes from.
This is the error we get when running

E Node number 557 (TfLiteQnnDelegate) failed to invoke.
E  QnnDsp <E> Graph qnn_delegate_graph_0 neither registered on skel nor in low memory mode to activate graph reload
E  QnnDsp <E> Graph qnn_delegate_graph_0 failed in execution with err 6001

This error msg is repeated multiple times.

If you want to try this out for yourself we've built a repo to complement his project.
https://github.com/mjnong/chatapp-v2

[Aakash356A]

sun:/ $ cd sdcard/argmax/tflite
sun:/sdcard/argmax/tflite $ export PATH=/data/local/tmp/bin:$PATH
sun:/sdcard/argmax/tflite $ export LD_LIBRARY_PATH=/data/local/tmp/lib
/argmax/tflite/models/openai_whisper-small --audio-path /sdcard/argmax/tflite/inputs/jfk_441khz.m4a <
SoC: SM8750P -> QNN HTP
INFO: Initialized TensorFlow Lite runtime.
VERBOSE: Replacing 21 out of 21 node(s) with delegate (TfLiteQnnDelegate) node, yielding 1 partitions for the whole graph.
VERBOSE: Replacing 911 out of 911 node(s) with delegate (TfLiteQnnDelegate) node, yielding 1 partitions for the whole graph.
Aborted

i am using the qnn-runtime-2.33.0.aar and qnn-litert-delegate-2.33.0.aar versions. and the model doesnt run. just the Tiny version runs. I tried the 2.32 and 2.31 version but still the small and base model dont work. I added the SM8750P at bool Runtime::check_qcom_soc() {
vector supported_socs{
"SM8650", "SM8550", "SM8450","SM8350","SM8750P","SM8750"
};

[Aakash356A]

ulting to ncvt=1
[2] WARNING: Could not find stride for conv cost (op_id=0x85570000080E, op_name: q::ConvLayer_fp16_S1_int_bias), defaulting to stride=1
[2] WARNING: Could not find stride for conv cost (op_id=0x85570000080E, op_name: q::ConvLayer_fp16_S1_int_bias), defaulting to stride=1
[2] WARNING: Could not find ncvt for conv cost (op_id=0x85570000080E, op_name: q::ConvLayer_fp16_S1_int_bias), defaulting to ncvt=1
[2] WARNING: Could not find stride for conv cost (op_id=0x855600000809, op_name: q::ConvLayer_fp16_S1_int_bias), defaulting to stride=1
[2] WARNING: Could not find stride for conv cost (op_id=0x855600000809, op_name: q::ConvLayer_fp16_S1_int_bias), defaulting to stride=1
[2] WARNING: Could not find ncvt for conv cost (op_id=0x855600000809, op_name: q::ConvLayer_fp16_S1_int_bias), defaulting to ncvt=1
[2] WARNING: Could not find stride for conv cost (op_id=0x855500000804, op_name: q::ConvLayer_fp16_S1_int_bias), defaulting to stride=1
[2] WARNING: Could not find stride for conv cost (op_id=0x855500000804, op_name: q::ConvLayer_fp16_S1_int_bias), defaulting to stride=1
[2] WARNING: Could not find ncvt for conv cost (op_id=0x855500000804, op_name: q::ConvLayer_fp16_S1_int_bias), defaulting to ncvt=1
[2] WARNING: Could not find stride for conv cost (op_id=0x8554000007FF, op_name: q::ConvLayer_fp16_S1_int_bias), defaulting to stride=1
[2] WARNING: Could not find stride for conv cost (op_id=0x8554000007FF, op_name: q::ConvLayer_fp16_S1_int_bias), defaulting to stride=1
[2] WARNING: Could not find ncvt for conv cost (op_id=0x8554000007FF, op_name: q::ConvLayer_fp16_S1_int_bias), defaulting to ncvt=1
[2] WARNING: unable to tile 0x1d210000009d q::Concat sufficiently, min size = 505344000 [1,8,4,32]
[2] WARNING: unable to tile 0x1d6b00000107 q::Concat sufficiently, min size = 505344000 [1,8,4,32]
[2] WARNING: unable to tile 0x1db500000171 q::Concat sufficiently, min size = 505344000 [1,8,4,32]
[2] WARNING: unable to tile 0x1dff000001e6 q::Concat sufficiently, min size = 505344000 [1,8,4,32]
[2] WARNING: unable to tile 0x1e4900000245 q::Concat sufficiently, min size = 505344000 [1,8,4,32]
[2] WARNING: unable to tile 0x1e93000002af q::Concat sufficiently, min size = 505344000 [1,8,4,32]
[2] WARNING: unable to tile 0x1edd00000319 q::Concat sufficiently, min size = 505344000 [1,8,4,32]
[2] WARNING: unable to tile 0x1f2700000383 q::Concat sufficiently, min size = 505344000 [1,8,4,32]
[2] WARNING: unable to tile 0x1f71000003ed q::Concat sufficiently, min size = 505344000 [1,8,4,32]
[2] WARNING: unable to tile 0x1fbb00000457 q::Concat sufficiently, min size = 505344000 [1,8,4,32]
[2] WARNING: unable to tile 0x2005000004c1 q::Concat sufficiently, min size = 505344000 [1,8,4,32]
[2] WARNING: unable to tile 0x204f0000052b q::Concat sufficiently, min size = 505344000 [1,8,4,32]
[2] WARNING: unable to tile 0x122a700000afd q::Concat sufficiently, min size = 9216000 [1,8,2,32]
[2] WARNING: unable to tile 0x1228e00000afd q::Concat sufficiently, min size = 9216000 [1,8,2,32]
[2] WARNING: unable to tile 0x1227500000afd q::Concat sufficiently, min size = 9216000 [1,8,2,32]
[2] WARNING: unable to tile 0x1225c00000afd q::Concat sufficiently, min size = 9216000 [1,8,2,32]
[2] WARNING: unable to tile 0x1224300000afd q::Concat sufficiently, min size = 9216000 [1,8,2,32]
[2] WARNING: unable to tile 0x1222a00000afd q::Concat sufficiently, min size = 9216000 [1,8,2,32]
[2] WARNING: unable to tile 0x1221100000afd q::Concat sufficiently, min size = 9216000 [1,8,2,32]
[2] WARNING: unable to tile 0x121f800000afd q::Concat sufficiently, min size = 9216000 [1,8,2,32]
[2] WARNING: unable to tile 0x121df00000afd q::Concat sufficiently, min size = 9216000 [1,8,2,32]
[2] WARNING: unable to tile 0x121c600000afd q::Concat sufficiently, min size = 9216000 [1,8,2,32]
[2] WARNING: unable to tile 0x121ad00000afd q::Concat sufficiently, min size = 9216000 [1,8,2,32]
[2] WARNING: unable to tile 0x1219400000afd q::Concat sufficiently, min size = 9216000 [1,8,2,32]

I tried the latest qnn-runtime-2.34.0.aar and qnn-litert-delegate-2.34.0.aar versions and the base model works but the small model has a lot of warnings and it gets aborted