User:<image>Describe the image.
Answer: This is a surreal, humorous, and slightly absurd depiction of an astronaut relaxing on the Moon — likely in a fictional or satirical context.
Key elements:
-
Astronaut: A person dressed in a bulky white spacesuit with a gold-tinted visor helmet is reclining casually on what appears to be a makeshift chair or cushion made from a green cooler box (possibly repurposed as seating). Their posture suggests complete relaxation — legs stretched out, one hand holding a green glass bottle.
-
The Bottle: The astronaut is holding a clear green glass bottle with a label that reads “BrewDog” — a well-known Scottish craft beer brand. This adds to the comedic contrast: enjoying a beer in space? It’s impossible under current physics and regulations, but it fits the whimsical tone.
-
Setting: The background shows the lunar surface — gray, cratered terrain with no atmosphere or clouds. In the distance, Earth hangs prominently in the blackness of space, dotted with stars. A ladder leans against a structure (perhaps part of a base or rover) on the right side.
-
Tone & Style: The image blends realism (detailed spacesuit, realistic Moon surface and Earth) with fantasy and humor. It plays on pop culture tropes — astronauts drinking beer in space is often used for comedic effect in movies, cartoons, or memes.
Interpretation: This isn’t a real photograph — it’s digitally manipulated or conceptually imagined art meant to evoke amusement. It juxtaposes the solemnity of space exploration with the trivial pleasure of enjoying a drink, creating irony and levity.
In short: An astronaut lounging on the Moon, sipping BrewDog beer, with Earth watching over them. 🌕🍻🌍
Paper: Qwen3 Technical Report
LLMs (Large Language Models) are neural networks trained on large text datasets to understand and generate language.
VLMs (Vision-Language Models) add a visual encoder so the model can process images and text together.
A combined VLM+LLM system is often referred to as a multimodal model.
These models can be large—hundreds of millions to billions of parameters—which impacts accuracy, memory use, and runtime speed.
On edge devices like the RK3588, available RAM and compute are limited, and even the NPU has strict constraints on supported operations.
Because of this, models typically need to be quantised or simplified to fit.
Performance is usually expressed in tokens (words) per second.
Once converted to RKNN, parts of the model can run on the NPU, improving speed.
Despite these limits, models like Qwen3-2B run well on the RK3588 because the NPU efficiently accelerates the heavy math, and the vision encoder can be optimised. This makes advanced multimodal AI feasible on small, power-efficient devices.
All models, with C++ examples, can be found on the Q-engineering GitHub.
All LLM models are quantized to w8a8, while the VLM vision encoders use fp16.
| model | RAM (GB)1 | llm cold sec2 | llm warm sec3 | vlm cold sec2 | vlm warm sec3 | Resolution | Tokens/s |
|---|---|---|---|---|---|---|---|
| Qwen3.5-9B | 9.2 | 97.1 | 97.1 | 11.5 | 11.5 | 448 x 448 | 3.2 |
| Qwen3.5-4B | 5.4 | 52.8 | 6.2 | 8.5 | 0.9 | 448 x 448 | 5.2 |
| Qwen3.5-2B | 2.9 | 23.9 | 3.2 | 8.5 | 0.8 | 448 x 448 | 11.0 |
| Qwen3.5-0.8B | 1.3 | 10.6 | 1.9 | 2.7 | 0.2 | 448 x 448 | 21.6 |
| Qwen3-2B | 3.1 | 21.9 | 2.6 | 10.0 | 0.9 | 448 x 448 | 11.5 |
| Qwen3-4B | 8.7 | 49.6 | 5.6 | 10.6 | 1.1 | 448 x 448 | 5.7 |
| InternVL3.5-1B | 1.9 | 8.3 | 8.0 | 1.5 | 0.8 | 448 x 448 | 24 |
| InternVL3.5-2B | 3.0 | 22 | 8.0 | 2.7 | 0.8 | 448 x 448 | 11.2 |
| InternVL3.5-4B | 5.4 | 50 | 8.0 | 5.9 | 0.8 | 448 x 448 | 5 |
| InternVL3.5-8B | 8.8 | 92 | 8.0 | 50.5 | 5.8 | 448 x 448 | 3.5 |
| Qwen2.5-3B | 4.8 | 48.3 | 4.0 | 17.9 | 1.8 | 392 x 392 | 7.0 |
| Qwen2-7B | 8.7 | 86.6 | 34.5 | 37.1 | 20.7 | 392 x 392 | 3.7 |
| Qwen2-2.2B | 3.3 | 29.1 | 2.5 | 17.1 | 1.7 | 392 x 392 | 12.5 |
| InternVL3-1B | 1.3 | 6.8 | 1.1 | 7.8 | 0.75 | 448 x 448 | 30 |
| SmolVLM2-2.2B | 3.4 | 21.2 | 2.6 | 10.5 | 0.9 | 384 x 384 | 11 |
| SmolVLM2-500M | 0.8 | 4.8 | 0.7 | 2.5 | 0.25 | 384 x 384 | 31 |
| SmolVLM2-256M | 0.5 | 1.1 | 0.4 | 2.5 | 0.25 | 384 x 384 | 54 |
1 The total used memory; LLM plus the VLM.
2 When an llm/vlm model is loaded for the first time from your disk to RAM or NPU, it is called a cold start.
The duration depends on your OS, I/O transfer rate, and memory mapping.
3 Subsequent loading (warm start) takes advantage of the already mapped data in RAM. Mostly, only a few pointers need to be restored.
To run the application, you have to:
- OpenCV 64-bit installed.
- rkllm library.
- rknn library.
- Optional: Code::Blocks. (
$ sudo apt-get install codeblocks)
Start with the usual
$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get install cmake wget curl
To install OpenCV on your SBC, follow the Raspberry Pi 4 guide.
Or, when you have no intentions to program code:
$ sudo apt-get install libopencv-dev
$ git clone https://github.com/Qengineering/Qwen3-VL-4B-NPU
To run InternVL3, you need to have the rkllm-runtime library version 1.3.0 installed, as well as the rknpu driver version 0.9.8.
If you don't have these on your machine, or if you have a lower version, you need to install them.
We have provided the correct versions in the repo.
$ cd ./Qwen3-VL-4B-NPU/aarch64/library
$ sudo cp ./*.so /usr/local/lib
$ cd ../include
$ sudo cp ./*.h /usr/local/includeYour rkllm model must match the library.
If you use a model synthesized with the previous 1.2.3 rkllm library and run it with the latest 1.3.0, you will get a malfunction. The internal Byte-Pair Encoding (BPE) dictionary parsing gets misaligned.
The next step is downloading the models.
Both can be downloaded from our Hugging Face page.
- qwen3.5-4b_w8a8_rk3588.rkllm
- qwen3.5-4b-vision_rk3588.rknn
Copy both into this folder.
Once you have the two models, it is time to build your application.
You can use Code::Blocks.
- Load the project file *.cbp in Code::Blocks.
- Select Release, not Debug.
- Compile and run with F9.
- You can alter command line arguments with Project -> Set programs arguments...
Or use Cmake.
$ mkdir build
$ cd build
$ cmake ..
$ make -j4
The application switches dynamically between Single Image Mode and Video Sequence Mode based on how many image files you pass into the arguments.
./VLM_NPU RKNN_model RKLLM_model file1.jpg [file2.jpg file3.jpg ...]| Argument | Comment |
|---|---|
| RKNN_model | The visual encoder model (VLM) compiled for the NPU. |
| RKLLM_model | The large language model (LLM) compiled for the NPU. |
| file1.jpg ... | The images you want to process. Passing 1 file triggers Image Mode. Passing multiple files triggers Video Sequence Mode. |
In the context of the Rockchip RK3588 LLM (Large Language Model) library, NewTokens and ContextLength control the boundaries for text generation and memory allocation.
In main.cpp you will find the line:
RKLLM.LoadModel(vlm_model, llm_model, NewTokens, ContextLength);Here you set your context based on available memory.
NewTokens
This sets the maximum number of tokens (pieces of text, typically sub-word units) that the model is allowed to generate in response to a prompt during a single inference round. For example, if set to 300, the model will not return more than 300 tokens as output, regardless of the prompt length. It is important for controlling generation length to avoid run-on responses and manage resource use.
ContextLength (Dynamic KV Cache)
This specifies the maximum total number of tokens the model can hold in its memory at once, which includes the system prompt, the massive image/video embeddings, your text questions, and all previous generated answers.
We have synthesized the models with a larger KV Cache than normal. Ours can hold up to 16384 tokens!
-
For 32GB Boards (e.g., Rock 5C 32GB): You can safely push the KV Cache to
8192or16384to support processing long video sequences and maintaining deep, multi-turn conversations without the model forgetting the image.RKLLM.LoadModel(vlm_model, llm_model, 2048, 16384); -
For 8GB/16GB Boards: The KV cache is highly memory-intensive. You should keep this at
2048or4096. If you set this higher than your physical RAM can handle, the Linux Out-Of-Memory (OOM) killer will crash the application.RKLLM.LoadModel(vlm_model, llm_model, 2048, 4096);
Typical Command Line Examples:
Single Image Mode:
./VLM_NPU ./models/qwen3.5-4b-vision_rk3588.rknn ./models/qwen3.5-4b_w8a8_rk3588.rkllm ./frame1.jpg
Video Sequence Mode (Passing multiple frames):
./VLM_NPU ./models/qwen3.5-4b-vision_rk3588.rknn ./models/qwen3.5-4b_w8a8_rk3588.rkllm ./frame1.jpg ./frame2.jpg ./frame3.jpg
To process video input, individual frames are first extracted. The VLM converts each frame into embeddings, which are then transformed into vision tokens.
Even on a desktop PC, this process places a heavy load on memory and CUDA resources. It’s therefore no surprise that the Rock 5C, with its more limited hardware, struggles even more.
Each vision token corresponds to about 200 tokens, strongly limiting the number of frames processed per video. Therefore, the video is subsampled, and evenly spaced frames are extracted for processing by Qwen3.5.
For reference, each vision token occupies around 20 MB of RAM — a detail worth keeping in mind when working on systems with limited memory.
Keep also in mind that after loading the models, each frame has to be tokenised, which takes about 0.9 seconds per frame.
Using the application is simple. Once you provide the model and the media files, you can ask anything you want.
Remember, we are on a bare Rock 5C, so don't expect the same speed or quality as massive server-grade models like ChatGPT. On the other hand, as you will see in the examples below, the app performs amazingly well on the edge!
Interacting with Media:
- If you passed a single image and want to talk about it, you must include the
<image>tag in your prompt. (e.g.,"Describe this <image> in detail.") - If you passed multiple images (video sequence), you must use the
<video>tag instead. (e.g.,"What action is happening in this <video>?")
Chat Controls:
- The app remembers the dialogue context continuously. To wipe the model's memory and start a fresh conversation about the loaded media, type
clear. - To leave the application, type
exit.
Below, you find the surprisingly little code of main.cpp.
#include "RK35llm.h"
#include <vector>
int main(int argc, char** argv)
{
// Usage: ./VLM_VIDEO_NPU vlm_model llm_model frame1.jpg [frame2.jpg frame3.jpg ...]
if (argc < 4) {
std::cerr << "Usage: " << argv[0] << " vlm_model llm_model file1.jpg [file2.jpg file3.jpg ...]\n";
return -1;
}
std::string vlm_model = argv[1];
std::string llm_model = argv[2];
RK35llm RKLLM;
RKLLM.SetInfo(true);
RKLLM.SetSilence(false);
RKLLM.LoadModel(vlm_model, llm_model, 2048, 16384);
// Collect all image frames from arguments
std::vector<cv::Mat> frames;
for (int i = 3; i < argc; ++i) {
cv::Mat frame = cv::imread(argv[i]);
if (!frame.empty()) {
frames.push_back(frame);
} else {
std::cerr << "Warning: Could not load image file: " << argv[i] << "\n";
}
}
if (frames.empty()) {
std::cerr << "Error: No valid images loaded. Exiting.\n";
return -1;
}
// Dynamic routing: single image vs. video sequence
if (frames.size() == 1) {
std::cout << "\n[Info] Loading single image mode...\n";
RKLLM.LoadImage(frames[0]);
} else {
std::cout << "\n[Info] Loading video sequence mode (" << frames.size() << " frames)...\n";
RKLLM.LoadVideoFrames(frames);
}
std::string input_str;
std::string output_str;
while (true) {
printf("\nUser: ");
std::getline(std::cin, input_str);
if (input_str == "exit") break;
// Reminder: the user must input the correct tag (<image> or <video>) based on the mode!
output_str = RKLLM.Ask(input_str);
// std::cout << "\nLLM Reply: " << output_str << std::endl; }
return 0;
}Most code speaks for itself. One remark.
The LLM generates the answer to your request in little pieces of text. You see them as if the LLM is typing on your terminal.
RKLLM.SetSilence controls this behaviour. When set to true, it will suppress this output. RKLLM.Ask() also returns the complete answer.
In main.cpp uncomment this line:
std::cout << "\nLLM Answer: " << output_str << std::endl;| Image | Dialog |
|---|---|
 |
User: <image>Which city is this? Answer: This is Singapore. The image shows the iconic skyline of Singapore at dusk or twilight, featuring: - The Marina Bay Sands hotel on the left — recognizable by its three towers connected by a skybridge and distinctive roof structure. - The Singapore Flyer, one of the world’s largest observation wheels, located in Marina Bay. - Other modern skyscrapers along the waterfront, including the Esplanade – Theatres on the Bay (the two white shell-shaped buildings) near the center.The calm water in the foreground reflects the city lights and sky, capturing the serene yet vibrant atmosphere of this global hub. ✅ So, the answer is: Singapore. |
 |
User: <image>Tell me something about the history of this place. Answer: This image shows a section of the Great Wall of China, one of the most iconic and historically significant structures in human history — stretching over 21,000 kilometers (13,000 miles) across northern China’s rugged terrain. Historical Background: The Great Wall was not built all at once but evolved over centuries under different dynasties. Its construction began as early as the Warring States Period (475–221 BCE), when individual states like Qin, Yan, and Zhao built defensive walls to protect against nomadic invasions from the north — particularly the Xiongnu confederation. The most famous phase of its construction occurred under the Qin Dynasty (221–206 BCE). After unifying China, Emperor Qin Shi Huang ordered the connection and expansion of existing state walls into a continuous defensive system. He also added watchtowers, barracks, and fortified passes — laying the foundation for what we now recognize as the Great Wall. Later dynasties — especially the Ming Dynasty (1368–1644) — significantly expanded and reinforced the wall with brick-and-stone construction, adding more towers, battlements, and strategic fortifications. The Ming-era sections are often the ones tourists see today because they’re better preserved and more visually dramatic. Purpose: - Defense: To repel invasions by nomadic tribes from the steppes. - Control: To regulate trade and movement along the northern frontier. - Symbolism: It represented imperial power, unity, and resilience. Legacy: Today, parts of the Great Wall are UNESCO World Heritage Sites (e.g., Badaling, Mutianyu, Jinshanling). They stand as monumental testaments to ancient engineering prowess and enduring cultural identity. Despite centuries of war, natural erosion, and neglect, many sections remain intact — some even accessible for hiking or photography like in your image. In short: This wall isn’t just a barrier — it’s a story of survival, strategy, and human perseverance carved into the landscape over two millennia. 🏯🧱⛰️ |
 |
User: <image>What are the ingredients I need to make this? Answer:To recreate this pizza — which appears to be a classic Neapolitan-style Margherita with fresh basil and charred crust — you’ll need: 🍕 Ingredients: Dough (for 1 small Neapolitan pizza): - 250g (about 9 oz) “00” flour (Italian bread flour or high-protein all-purpose works too) - 375ml (1¼ cups) warm water (adjust slightly based on hydration — aim for ~68% hydration) - 7–8g active dry yeast (or instant yeast) - 2 tsp sea salt Optional: A pinch of sugar or honey to help yeast activate. Sauce: - 1 cup San Marzano tomato sauce (canned whole peeled tomatoes crushed by hand, strained if needed) - Optional: 1 clove garlic, minced (optional — traditional Margherita often omits it) - Freshly cracked black pepper or red pepper flakes (optional) Cheese: - 200g fresh mozzarella di bufala (buffalo mozzarella), sliced thin OR low-moisture whole milk mozzarella balls (bocconcini), torn into pieces Tip: Use low-moisture for easier handling; buffalo gives richer flavor. Toppings: - Fresh basil leaves, torn by hand (not pre-chopped — you want big, vibrant leaves) - Optional: A drizzle of extra virgin olive oil before baking 🔥 Equipment Needed: - Pizza stone or steel (preheat in oven to highest setting + broil on high for last 5–10 mins) - Baking sheet lined with parchment paper (if no stone/steel) - Peeler or knife to score dough before baking - Tongs or peel to transfer pizza 🧑🍳 Quick Instructions: 1. Make Dough: Mix flour, yeast, salt, and water until shaggy. Let rise 2–4 hours (or overnight in fridge). 2. Preheat Oven + Stone/Steel: Set oven to max heat (~500°F / 260°C or higher if possible), broil on high for last 10 mins. 3. Prepare Sauce & Cheese: Mix sauce; set aside cheese and basil. 4. Shape Dough: Punch down dough, stretch gently into round (don’t overwork!). Transfer to parchment-lined sheet. 5. Add Toppings: Spread sauce lightly, add mozzarella, top with fresh basil leaves. 6. Bake: Slide onto hot stone/steel or baking sheet. Bake 8–12 minutes until crust is puffed and charred in spots, cheese bubbly. 7. Finish: Drizzle with EVOO if desired. Serve immediately! 💡 Pro Tips: - Let dough rest after shaping to relax gluten before baking. - Don’t overcrowd toppings — let them breathe for even cooking. - For authentic char, use a very hot oven and broil at the end. Enjoy your homemade Neapolitan Margherita! 🍕✨ User:What is the first step? Answer:The first step to making this pizza is: Prepare the dough. This means mixing together: - 250g “00” flour (or high-protein all-purpose) - 375ml warm water - 7–8g active dry yeast - 2 tsp sea salt Mix until a shaggy dough forms, then let it rise for at least 2 hours (or overnight in the fridge). This gives you the foundation for that charred, airy Neapolitan crust. Once the dough is ready and risen, you can move on to making the sauce, cheese, and assembling the pizza. ✅ So — Step 1: Make and let rise your pizza dough. |
 |
User: <image>Which movie is this? Answer:This iconic scene is from the 1997 romantic drama film Titanic. The image shows Leonardo DiCaprio (as Jack Dawson) and Kate Winslet (as Rose DeWitt Bukater) on the bow of the RMS Titanic, arms outstretched toward a sunset — one of the most famous and emotionally resonant moments in cinema history. It symbolizes their brief but transformative love story aboard the ill-fated ship. The film was directed by James Cameron and became a global phenomenon, earning multiple Academy Awards including Best Picture. |
User: <video> Describe the video.
Answer: The video shows a man practicing martial arts in what appears to be a dojo or training hall with tatami mats on the floor and calligraphy scrolls hanging on light blue walls. The sequence captures him performing a dynamic move, possibly a kick or flip, which results in him falling onto his back as he completes the motion. His attire consists of dark clothing suitable for martial arts practice.
Notice the nice Markdown makeup of the answer!
User: Can you make me a C++ call that embeds the Pythagorean theorem?
Answer: Sure! Here's a simple and clear C++ program that demonstrates the Pythagorean theorem:
( a^2 + b^2 = c^2 )
wherecis the hypotenuse of a right triangle, anda,bare the other two sides.
#include <iostream>
#include <cmath> // for std::sqrt()
using namespace std;
int main() {
double a, b, c;
cout << "Enter side 'a': ";
cin >> a;
cout << "Enter side 'b': ";
cin >> b;
// Calculate hypotenuse using Pythagorean theorem: c = sqrt(a² + b²)
c = sqrt(pow(a, 2) + pow(b, 2));
cout << "\nHypotenuse (c) = " << c << endl;
return 0;
}- Uses
pow(base, exponent)to square the sides. - Uses
sqrt()from<cmath>to compute the square root of the sum. - Prompts user for two legs (
aandb) and outputs the hypotenuse (c).
Enter side 'a': 3
Enter side 'b': 4
Hypotenuse (c) = 5
Let me know if you’d like to add validation, handle non-right triangles, or make it a function-based design! 📐✨
To get a taste, try our professional Qwen3 AI-chatbot running on a Rock 5C: https://rock5gpt.qengineering.eu
