Running Kimi K3 on CPU via kimi-k3-in-c engine

A portable inference engine, kimi-k3-in-c, has been introduced for running the Kimi K3 model with 2.78 trillion parameters on a single CPU.

The program is written in C99 and does not require BLAS, frameworks, or GPUs, ensuring independence from external libraries and hardware. It requires a fast NVMe SSD with at least 1.56 TB of free space to store the model checkpoint.

Key Project Features

• Programming Language: C99
• No GPU or third-party library dependencies
• Storage Requirement: NVMe SSD with at least 1.56 TB capacity
• Generation Speed: Approximately 32.7 seconds per token using 8 GB of RAM

The project is available at https://github.com/FareedKhan-dev/kimi-k3-in-c