Run MiniCPM4 with CPU only

calendar Jun 8, 2025 · 1 min read · ai  ·
Share on: linkedin copy

This is a guide for serving the MiniCPM4 0.5b model with CPU only(in my laptop WSL Ubuntu 24.04).

The model is served by llama.cpp, and I break down it for several steps.

  • Download the model from hf
1huggingface-cli download openbmb/MiniCPM4-0.5B
  • Install(and compile) the llama.cpp
 1# build llama.cpp
 2git clone https://github.com/ggml-org/llama.cpp.git
 3sudo apt install python3-dev build-essential cmake libcurl4-openssl-dev
 4cmake -B build  
 5cmake --build build --config Release
 6
 7# prepare llama tools
 8uv venv --python=3.12 .venv
 9source .venv/bin/activate
10uv pip install -r requirements/requirements-convert_hf_to_gguf.txt --index-strategy unsafe-best-match
  • Convert the downloaded safetensors model to gguf format
1# the default model location of huggingface-cli
2python convert_hf_to_gguf.py \
3~/.cache/huggingface/hub/models--openbmb--MiniCPM4-0.5B/snapshots/ebf6ddf19764646a49d94e857fb4eb439f35ecfb/ \
4--outfile /path/to/minicpm4-0.5b.gguf
  • Serve the gguf file with llama.cpp
1build/bin/llama-server --model /path/to/minicpm4-0.5b.gguf

Here you can call the model with OpenAI API protocol at http//127.0.0.1:8080/v1 .

The .gguf file can also be served with ollama:

1echo 'FROM /path/to/minicpm4-0.5b.gguf' > Modelfile
2ollama create MiniCPM4:0.5b -f Modelfile