Android OpenCL Crash on Adreno 740 GPU during Context Initialization

[ArindamRayMukherjee]

Summary

App crashes with SIGABRT during LlamaContext initialization when using OpenCL backend on Android device with Qualcomm Adreno 740 GPU. The crash occurs consistently after successful model loading and KV cache creation, during the context initialization phase.

Environment

Device:

• Model: Samsung Galaxy S23 Ultra (SM-S918B)
• SoC: Qualcomm Snapdragon 8 Gen 2
• GPU: Adreno 740
• OS: Android 15 (Build: AP3A.240905.015.A2/S918BXXS8DYI3)
• RAM: 12GB

Software:

• llama.rn version: 0.7.2
• React Native: 0.81.4
• Platform: Scala.js (ES2018, CommonJS)

Model:

• Name: Gemma 3 270M Instruction Tuned (QAT)
• Quantization: Q4_0
• File: gemma-3-270m-it-qat-q4_0.gguf
• Size: 224 MB
• Source: Google/Hugging Face official model

Configuration Used

const contextParams = {
  model: modelPath,
  n_ctx: 4096,
  n_gpu_layers: 99,
  use_mmap: true,
  use_mlock: false,
  flash_attn_type: "auto",
  cache_type_k: "q8_0",
  cache_type_v: "q8_0"
};

const context = await initLlama(contextParams);

AndroidManifest.xml:

<uses-native-library android:name="libOpenCL.so" android:required="false" />

Steps to Reproduce

1. Install llama.rn 0.7.2 on Android device with Adreno 740 GPU
2. Add OpenCL library declaration to AndroidManifest.xml
3. Download Gemma 3 270M Q4_0 GGUF model
4. Initialize context with OpenCL enabled (n_gpu_layers > 0)
5. App crashes during LlamaContext initialization

Attachments

1. Android adb log
2. Crash tombstone

tombstone_07.txt

applogs.txt

[jhen0409]

Try to not set cache_type_k and cache_type_v, it may not be supported yet for the backend (or the model).

[ArindamRayMukherjee]

@jhen0409 Thanks! Confirmed that fixes the situation.

[asif166037]

@jhen0409 , still not support the cache type K and V or it has fixed? if we add f16 then it will take too much time to get the response.