1. Pengenalan
llama.cpp ialah runtime inference LLM berasaskan C/C++ yang popular kerana ringan, pantas, dan sesuai untuk menjalankan model GGUF secara local. Ia boleh digunakan pada:
Server x86_64
Workstation Linux
Mini PC
Raspberry Pi
Orange Pi
SBC ARM64
Container Linux
Dalam deployment sebenar, terdapat dua pendekatan utama:
Native build
Compile terus pada mesin yang akan menjalankan llama.cpp.
Cross compile
Compile pada mesin lebih laju (contohnya PC x86_64), tetapi menghasilkan binary untuk platform lain (contohnya ARM64 Orange Pi).
Bahagian 1 — Persediaan Debian 12/13
1.1 Install dependency asas
sudo apt update
sudo apt install -y \
git \
build-essential \
cmake \
ninja-build \
pkg-config
Enter fullscreen mode Exit fullscreen mode
Komponen utama:
Package Fungsi
git Ambil source code
build-essential GCC, G++, make
cmake Build configuration
ninja-build Build engine lebih pantas
pkg-config Cari library dependency
Bahagian 2 — Clone llama.cpp
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
Enter fullscreen mode Exit fullscreen mode
Semak versi:
git log -1 --oneline
Enter fullscreen mode Exit fullscreen mode
Bahagian 3 — Compile Native (Mesin Sama)
Contoh:
Debian 12/13 x86_64
Debian ARM64
Orange Pi
Raspberry Pi
3.1 Configure CMake
Build menggunakan Ninja:
cmake -B build \
-G Ninja \
-DCMAKE_BUILD_TYPE=Release
3.2 Compile
ninja -C build -j$(nproc)
Enter fullscreen mode Exit fullscreen mode
atau:
cmake --build build
Enter fullscreen mode Exit fullscreen mode
plaintext
3.3 Hasil build
Semak:
ls build/bin
Enter fullscreen mode Exit fullscreen mode
plaintext
Contoh:
llama-cli
llama-server
llama-bench
llama-perplexity
Enter fullscreen mode Exit fullscreen mode
shell
Bahagian 4 — Enable OpenBLAS (Pilihan)
OpenBLAS boleh membantu operasi matrix CPU.
Install:
sudo apt install libopenblas-dev
Enter fullscreen mode Exit fullscreen mode
cmake
Build:
cmake -B build \
-G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DGGML_BLAS=ON \
-DGGML_BLAS_VENDOR=OpenBLAS
Enter fullscreen mode Exit fullscreen mode
shell
Kemudian:
ninja -C build
Enter fullscreen mode Exit fullscreen mode
Nota Penting: CMake Cache
Jika pernah configure dengan:
-DGGML_BLAS=ON
Enter fullscreen mode Exit fullscreen mode
kemudian buang option tersebut, CMake masih menyimpan konfigurasi lama.
Contoh masalah:
BLAS not found
missing: BLAS_LIBRARIES
Penyelesaian:
rm -rf build
Enter fullscreen mode Exit fullscreen mode
Kemudian configure semula.
Sentiasa ingat:
CMakeCache.txt menyimpan konfigurasi lama.
Bahagian 5 — Cross Compile x86_64 → ARM64
Contoh:
PC Debian 12 x86_64
|
|
v
Orange Pi ARM64
Kelebihan:
Compile lebih cepat
Tidak membebankan SBC
Sesuai untuk production image
5.1 Install ARM64 cross compiler
sudo apt install -y \
gcc-12-aarch64-linux--gnu\
g++-12-aarch64-linux-gnu
Enter fullscreen mode Exit fullscreen mode
sudo apt install -y \
gcc-13-aarch64-linux--gnu\
g++-13-aarch64-linux-gnu
Enter fullscreen mode Exit fullscreen mode
Semak:
aarch64-linux-gnu-gcc --version
Enter fullscreen mode Exit fullscreen mode
5.2 Configure cross build
Bersihkan dahulu:
rm -rf build-arm
Enter fullscreen mode Exit fullscreen mode
Kemudian:
cmake -B build-arm \
-G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_SYSTEM_NAME=Linux \
-DCMAKE_SYSTEM_PROCESSOR=aarch64 \
-DCMAKE_C_COMPILER=aarch64-linux-gnu-gcc \
-DCMAKE_CXX_COMPILER=aarch64-linux-gnu-g++
5.3 Compile
ninja -C build-arm -j$(nproc)
Enter fullscreen mode Exit fullscreen mode
Hasil:
ls build-arm/bin
Enter fullscreen mode Exit fullscreen mode
Bahagian 6 — Semak Architecture Binary
Gunakan:
file build-arm/bin/llama-server
Enter fullscreen mode Exit fullscreen mode
Contoh output berjaya:
ELF 64-bit LSB pie executable,
ARM aarch64,
dynamically linked
Enter fullscreen mode Exit fullscreen mode
Maksud:
Output Maksud
ELF 64-bit Binary 64-bit
ARM aarch64 Untuk ARM64
dynamically linked Perlukan shared library
PIE executable Linux security hardening
Bahagian 7 — Semak Dependency .so
Jangan guna ldd untuk cross binary
Jika compile ARM64 tetapi check pada PC x86:
ldd llama-server
Enter fullscreen mode Exit fullscreen mode
boleh gagal:
not a dynamic executable
Enter fullscreen mode Exit fullscreen mode
Sebab:
PC:
x86_64 loader
Binary:
ARM64 loader
Gunakan readelf
aarch64-linux-gnu-readelf \
-d build-arm/bin/llama-server | grep NEEDED
Contoh:
Shared library: [libllama.so]
Shared library: [libggml.so]
Shared library: [libstdc++.so.6]
Enter fullscreen mode Exit fullscreen mode
Cari semua .so
find build-arm -name "*.so"
Enter fullscreen mode Exit fullscreen mode
Contoh:
libllama.so
libggml.so
libggml-base.so
libggml-cpu.so
Enter fullscreen mode Exit fullscreen mode
Semak architecture:
file build-arm/bin/*.so
Enter fullscreen mode Exit fullscreen mode
Output:
ARM aarch64
Bahagian 8 — Dynamic vs Static Binary
Enter fullscreen mode Exit fullscreen mode
Semak:
file llama-server
Enter fullscreen mode Exit fullscreen mode
Contoh dynamic:
dynamically linked
Enter fullscreen mode Exit fullscreen mode
Perlu:
lib*.so
Contoh static:
statically linked
Tidak perlu .so.
Bahagian 9 — Installation ke Linux
Pilihan standard
Binary:
/usr/local/bin
Enter fullscreen mode Exit fullscreen mode
Library:
/usr/local/lib
Enter fullscreen mode Exit fullscreen mode
Contoh:
sudo cp llama-server /usr/local/bin/
sudo cp llama-cli /usr/local/bin/
sudo cp *.so /usr/local/lib/
sudo ldconfig
Enter fullscreen mode Exit fullscreen mode
Pilihan appliance / embedded
Untuk SBC:
/opt/llama.cpp/
llama-server
llama-cli
libllama.so
libggml.so
Enter fullscreen mode Exit fullscreen mode
Kemudian:
export LD_LIBRARY_PATH=/opt/llama.cpp
Enter fullscreen mode Exit fullscreen mode
Sesuai untuk:
Orange Pi
kiosk AI
edge inference node
Bahagian 10 — Deploy ke Orange Pi
Copy:
scp build-arm/bin/llama-server \
orangepi:/usr/local/bin/
Enter fullscreen mode Exit fullscreen mode
scp build-arm/bin/llama-cli \
orangepi:/usr/local/bin/
Enter fullscreen mode Exit fullscreen mode
Jika perlu:
scp build-arm/bin/*.so \
orangepi:/usr/local/lib/
Enter fullscreen mode Exit fullscreen mode
Pada Orange Pi:
sudo ldconfig
Enter fullscreen mode Exit fullscreen mode
Semak:
uname -m
Enter fullscreen mode Exit fullscreen mode
Expected:
aarch64
Enter fullscreen mode Exit fullscreen mode
Bahagian 11 — Cadangan Production Architecture
Untuk sistem AI agent:
+—————-+
| Go Agent |
| Tool Router |
+——-+——–+
|
|
HTTP API
|
v
+—————-+
| llama-server |
| llama.cpp |
+—————-+
|
|
GGUF
Model
Kelebihan:
Go agent tidak perlu embed model
Model boleh tukar tanpa rebuild
llama.cpp boleh upgrade sendiri
Mudah scale ke banyak node
Kesimpulan
Workflow yang stabil:
Native
cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
ninja -C build
Cross Compile ARM64
sudo apt install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu
rm -rf build-arm
cmake -B build-arm \
-G Ninja \
-DCMAKE_SYSTEM_NAME=Linux \
-DCMAKE_SYSTEM_PROCESSOR=aarch64 \
-DCMAKE_C_COMPILER=aarch64-linux-gnu-gcc \
-DCMAKE_CXX_COMPILER=aarch64-linux-gnu-g++
ninja -C build-arm
Verification
file llama-server
aarch64-linux-gnu-readelf -d llama-server | grep NEEDED
find . -name “*.so”
Dengan proses ini, satu mesin Debian 12/13 boleh menjadi build server untuk menghasilkan node AI ARM64 seperti Orange Pi, Raspberry Pi, atau edge inference appliance.
답글 남기기