llama.cpp vs ik_llama.cpp: The Fork, Drama and Performance Race

If you spend enough time in LocalLLaMA, sooner or later somebody says:

Try llama.cpp.

Then somebody else appears five comments later and says:

No, try ik_llama.cpp.

Then a third person says:

Mainline has that now.

Then somebody posts a benchmark from a dual-3090 machine with 256 GB of RAM, six flags you have never seen before, and a quantization name that looks like a Wi-Fi password.

Welcome to one of the more entertaining family stories in local AI.

This is the story of llama.cpp and ik_llama.cpp: parent project and fork, mainstream and experimental branch, shared history and growing divergence.

There is real technical history here. There is also community mythology.

So let us separate the two without removing all the fun.

The short version

llama.cpp is the giant mainstream local-LLM runtime.

It has a large community, broad hardware support, active model support, a server, a web UI, OpenAI-compatible APIs, many backends, and a huge amount of tooling around GGUF.

ik_llama.cpp is a fork created by longtime llama.cpp contributor ikawrakow.

According to the current ik_llama.cpp README, the fork began in June 2024 and was last synced with upstream llama.cpp in August 2024.

Since then, the projects have moved farther apart.

The ik fork focuses heavily on things local-model power users love:

new quantization types
fast CPU inference
CPU + GPU hybrid inference
MoE optimization
fine-grained tensor placement
multi-GPU work
experimental inference features

And this is where the story gets interesting.

Because several ideas that appeared early in ik_llama.cpp later appeared in some form in mainline llama.cpp too.

That has created a recurring LocalLLaMA plot line:

Is ik_llama.cpp the weird laboratory where tomorrow’s llama.cpp features appear first?

Sometimes, yes.

But the full story is less dramatic — and more interesting — than saying one project simply “copies” the other.

Before the fork, ikawrakow was already part of llama.cpp history

The fork did not come from a random person cloning llama.cpp one weekend.

Years before ik_llama.cpp became its own thing, ikawrakow was already deeply involved with quantization work in llama.cpp.

In April 2023, for example, he opened the QX_4 quantization work in the llama.cpp project.

In late 2023 and early 2024, discussions around low-bit quantization and what became the iQuant family were already happening in the main llama.cpp community.

In January 2024, he opened a llama.cpp discussion called:

Even more quantization types?

The question was basically a classic open-source problem:

More quant types
→ finer control over size vs quality

but

More quant types
→ more code, more maintenance, more user confusion

That tension never really went away.

It is important context because when people discover ik_llama.cpp today and see a forest of names such as:

IQ2_K
IQ3_K
IQ4_K
IQ4_KS
IQ4_KSS
Trellis quants
R4 variants

it can look like a completely separate quantization universe.

It is not.

The two projects share a long technical family tree.

Then came the fork

The current ik_llama.cpp README gives a very clean timeline:

Forked from llama.cpp:
June 2024

Last synced with upstream:
August 2024

After that, the fork stopped trying to remain a near-copy of mainline.

This matters.

A fork that regularly merges upstream changes is basically a modified version of the parent.

A fork that stops syncing and keeps building its own architecture becomes something else.

By 2025 and 2026, calling ik_llama.cpp “llama.cpp plus a few patches” had become misleading.

The codebases had diverged.

The command-line flags diverged.

The performance work diverged.

Some model support arrived at different times.

And the two communities developed different priorities.

That is when the love story became a soap opera.

So why did ikawrakow leave mainline?

Here comes the disappointing part for anyone hoping for a Netflix documentary.

There is no clean public statement saying:

Person A did X,
therefore Person B forked the project.

People have looked.

In a March 2025 ik_llama.cpp discussion literally titled Diverging from llama.cpp, a user asked almost exactly this question.

The user had researched earlier discussions and still could not find one simple reason for the fork.

Ikawrakow’s own answer was much less dramatic than the community mythology.

His explanation was essentially:

I am hacking here to keep my brain active.
I am having fun.
I am not chasing fame or mass adoption.
A smaller number of technical users is fine.

He even noted that a very popular project would create support work for non-technical users that he was not sure he wanted.

That tells you a lot about ik_llama.cpp.

It is not trying to win a market-share contest.

It is closer to:

What happens if a very strong low-level inference engineer
gets a playground and does not need to please everybody?

The answer is ik_llama.cpp.

The “mad scientist fork” description is not completely unfair

Mainline llama.cpp has to think about many things at once:

many CPUs
many GPUs
many operating systems
many model families
APIs
servers
web UI
backward compatibility
packaging
new users
old users
CI
regressions

That is the cost of becoming infrastructure.

Ik_llama.cpp can be more opinionated.

It can spend a lot of energy on a weird quant, a specific MoE bottleneck, CPU prompt processing, or a new split strategy because somebody finds the problem interesting.

The README itself says there is no single document describing every new feature and that pull requests are often the best source of information.

That is both charming and slightly terrifying.

Mainline says:

Here is the supported interface.

Ik sometimes feels more like:

Read PR #1373.
Then PR #1501.
Then this discussion.
Do not use -rtr unless you understand why.
Good luck.

For a certain type of LocalLLaMA user, this is not a warning.

It is advertising.

Where ik_llama.cpp became famous: quants, CPUs and giant MoE models

The clearest identity of ik_llama.cpp is performance under awkward hardware limits.

Think about the local-AI user who says:

I have one or two gaming GPUs,
128 GB of system RAM,
and absolutely no intention of paying for an H100.

This is exactly the kind of person who starts reading ik_llama.cpp discussions at 2 a.m.

The fork has focused heavily on:

CPU inference
hybrid CPU/GPU inference
quantized matrix kernels
MoE expert handling
fine tensor offload
low-bit quantization
multi-GPU layouts

That became especially interesting when enormous Mixture-of-Experts models such as DeepSeek arrived.

A model might have hundreds of billions of total parameters while activating a much smaller subset for each token.

Suddenly the question was not simply:

Can the whole model fit in VRAM?

It became:

Can I keep a lot of it in RAM,
put the right parts on GPU,
and still get useful speed?

That is the sort of problem ik_llama.cpp likes very much.

The famous ik quants

If normal GGUF quant names already looked complicated, ik_llama.cpp decided the situation needed more letters.

You may run into families such as:

IQ2_K
IQ3_K
IQ4_K
IQ5_K
IQ4_KS
IQ4_KSS
IQ*_R4
IQ*_KT

Some are aimed at better quality per bit.

Some are designed around different packing and hardware kernels.

Some make more sense on CPU.

Some have CUDA paths.

Some are useful in hybrid inference.

And some will make a beginner close the browser tab and install Ollama.

That is okay.

The important point is that ik_llama.cpp treats quantization as an active performance engineering space, not a solved list of Q4/Q5/Q6 presets.

If you want the calmer version first, read GGUF Quantization Levels Explained: Q4, Q5, Q6 and Q8.

Then come back when you are emotionally ready for IQ4_KSS_R4.

Meanwhile, llama.cpp kept becoming the boring successful adult

While ik_llama.cpp was exploring unusual performance paths, llama.cpp kept expanding as the mainstream runtime.

Today mainline offers an enormous list of hardware backends and tools.

It supports CPU execution, CUDA, Metal, HIP, Vulkan, SYCL and more.

Its server supports OpenAI-compatible endpoints, multimodal inference, tool calling, parallel decoding, continuous batching and a built-in web UI.

Its new unified llama tooling also makes basic use much easier than the old days.

This is the mainline advantage:

You are more likely to find:

a guide
an issue
an existing build
an app integration
a recent model implementation
someone who has seen your error before

For most beginners, that matters more than squeezing another chunk of performance out of a strange hardware configuration.

If you are starting from zero, llama.cpp for Beginners: Build, Run and Tune Your First Local LLM is the better first stop.

Then the community noticed features moving in both directions

Here is where the gossip starts.

The current ik_llama.cpp README explicitly says that several inference features appeared in the fork before later becoming available in llama.cpp.

It names examples including:

MLA
quant repacking
fused delta-net / Gated Delta Net
Tensor Parallel
MTP
DFlash

There is even an ik_llama.cpp discussion whose title is basically:

Mainline is now copying stuff from ik_llama.cpp

That title is very Reddit-compatible.

It is also not the same thing as proving misconduct.

Open-source projects routinely solve the same problems, study each other’s work, reimplement ideas, port techniques, or arrive at similar designs independently.

And remember: ik_llama.cpp itself began as a llama.cpp fork.

So the family tree already has arrows in both directions.

A better way to describe it is:

ik_llama.cpp has often acted as an aggressive experimental branch of the local-inference ecosystem, and some ideas it explored early later became relevant to mainline too.

Less spicy, but more accurate.

Reddit mostly cares about performance, not family therapy

The LocalLLaMA attitude is wonderfully practical.

People will argue about project history for twenty comments.

Then somebody posts:

Step 3.5
16K prompt
X tokens/sec
2 GPUs
this quant
these flags

and suddenly everyone becomes an accountant.

Recent LocalLLaMA threads show users choosing ik_llama.cpp because it performs well for specific CPU+GPU or multi-GPU workloads, while others stay with mainline because it is easier, more compatible, or fast enough.

One user testing Step 3.5 and MiniMax on ik_llama.cpp described the combination of strong open models and local inference as finally creating a feeling of freedom.

That is probably the healthiest way to see the fork.

Not:

Team llama.cpp vs Team ik_llama.cpp

but:

We have two serious engines pushing each other.
Nice.

Is ik_llama.cpp always faster?

No.

Please do not turn this article into that conclusion.

Even the ik_llama.cpp issue tracker contains cases where users report a model or workload being slower than mainline llama.cpp.

Performance depends on:

model architecture
quant type
CPU
GPU
memory bandwidth
context
prompt processing vs generation
how much is offloaded
backend
build flags
number of GPUs

A fork optimized for one workload can lose on another.

This is why “ik is faster” and “mainline is faster” can both be true on the same day.

You need the rest of the sentence.

For example:

faster for what model?
on what hardware?
with what quant?
at what context?
for prompt processing or generation?

Without that, a tokens-per-second number is mostly decoration.

The biggest downside of ik_llama.cpp is the same thing that makes it fun

It moves fast and goes deep.

That creates friction.

The fork’s own README says there is no single reference covering every feature.

Users may need to read pull requests and discussions.

Some flags differ from mainline.

The projects have diverged enough that instructions for one do not always apply cleanly to the other.

A GitHub discussion in 2026 even asks why ik_llama arguments are not consistent with llama.cpp and related tools.

That is the price of a real fork.

At some point you cannot have both:

complete freedom to redesign things

and:

perfect compatibility with upstream forever

The biggest downside of mainline is also the same thing that makes it useful

Mainline has to carry everybody.

That means architecture decisions are more conservative.

A change that looks brilliant on one CPU family or one exotic MoE setup may create maintenance or compatibility costs across many backends.

So mainline can feel slower to adopt specialized ideas.

But the payoff is ecosystem stability.

When a new GGUF lands on Hugging Face, most people asking:

Does llama.cpp support this?

mean mainline llama.cpp.

That network effect is enormous.

Which one should a normal person use?

My non-dramatic answer is:

Start with llama.cpp.

Especially if you are:

new to local AI
using one normal GPU
running common dense models
using apps that expect mainline behavior
following online tutorials
building a simple local server

Mainline gives you the smoothest path.

Then try ik_llama.cpp when you have a reason.

Good reasons include:

you are CPU-heavy
you run CPU + GPU hybrid
you are squeezing giant MoE models into local hardware
you want ik-specific quants
you need a feature that landed there first
you enjoy benchmarking more than sleeping

That last group already knows who they are.

Can the same GGUF run in both?

Often, standard GGUF models can work across both projects because ik_llama.cpp remains part of the broader GGUF ecosystem.

But do not assume every ik-specific quant or feature is supported by mainline.

That is where people get confused.

A model file named something familiar such as:

Q4_K_M

is one thing.

A quant or packing method designed specifically around ik_llama.cpp may require the fork.

Always check the model publisher’s instructions.

If the model page says:

requires ik_llama.cpp

believe it.