KTransformers Runs LLMs Across CPU and GPU Hardware

KTransformers is a Python framework for experimenting with efficient large language model inference and fine-tuning across CPU and GPU resources. It focuses on heterogeneous computing, letting researchers and engineers test optimization strategies for serving and adapting LLMs with different hardware layouts. The project targets practical inference, supervised fine-tuning, and future performance work for large models.

AI / ML llm inference fine-tuning python gpu

Stars: 19K
Forks: 1.5K
Issues: 453
PRs: 16

Open on GitHub Archive stats

KTransformers Runs LLMs Across CPU and GPU Hardware

KTransformers is a research-focused Python framework for efficient LLM inference and fine-tuning. It explores CPU-GPU heterogeneous computing, which can help teams test how large models behave across mixed hardware setups.

Why it matters

The project gives researchers and engineers a place to try inference optimizations, supervised fine-tuning workflows, and hardware-aware model execution strategies.

Good fit

Use it if you are experimenting with LLM serving, fine-tuning, or performance tuning on machines where CPU and GPU resources both matter.