Ollama and llama.cpp: It's Complicated

There is a polite version of the Ollama story: llama.cpp built an extraordinary local-inference engine, Ollama wrapped a friendlier product around it, and both projects helped local AI grow.

Then there is the version you hear after spending enough time with people who actually benchmark GGUFs.

In that version, llama.cpp does years of ugly upstream work — kernels, quantization, model loaders, GPU backends, memory management, compatibility — while Ollama puts a cleaner interface in front of it, becomes the brand many users remember, diverges from upstream when convenient, and periodically rediscovers why upstream existed in the first place.

That second version is obviously partisan.

It is also much harder to dismiss after reading the primary sources.

This is not a claim that Ollama contributes nothing. Model management, scheduling, API stability, compatibility and one-command UX are real engineering. But if we are telling the history of this relationship, symmetry for its own sake obscures the central fact: llama.cpp is the upstream technical project whose work made much of the modern GGUF local-inference ecosystem possible, while Ollama has repeatedly had to negotiate the consequences of building a product abstraction on top of — and sometimes away from — that upstream.

And by 2026, a significant part of the enthusiast community had stopped being subtle about which side it preferred.

Act I: llama.cpp builds the machinery

llama.cpp describes its mission simply: LLM inference in C/C++ with minimal setup and strong performance across a huge hardware surface.

The simplicity of that sentence hides the work.

CPU instruction sets, Metal, CUDA, HIP, Vulkan and SYCL; low-bit quantization; CPU/GPU hybrid inference; GGUF loading; KV-cache behavior; sampling; architecture after architecture; constant kernel work as new hardware and new models arrive.

This is the layer where local AI either works or does not.

Ollama solved a different problem. It made the machinery easier to consume: ollama run, a model library, an API, model lifecycle management, scheduling and defaults that save users from learning every inference knob.

That was valuable. It was also a recipe for attribution asymmetry.

The better an infrastructure layer works, the easier a product above it can make that infrastructure disappear.

For many newcomers, “runs in Ollama” eventually became almost synonymous with “runs locally.” That is a remarkable product victory for Ollama. It is also easy to understand why people working closer to llama.cpp found the cultural inversion irritating: the engine was increasingly invisible behind the dashboard.

Act II: the attribution dispute becomes public

In March 2024, Justine Tunney (jart) opened Ollama issue #3185 and alleged that Ollama’s distributed artifacts were not carrying copyright notices required for statically linked dependencies including llama.cpp.

The issue included commands intended to demonstrate that the llama.cpp copyright notice could not be found in installed Ollama files.

That allegation should not be inflated into a legal judgment. It was a public licensing-compliance accusation, not a court ruling.

But focusing only on the legal question misses why the thread mattered.

Two arguments became entangled.

The first was mechanical: MIT-licensed code comes with notice obligations when copied or distributed in substantial portions.

The second was cultural: was Ollama adequately acknowledging how much of its core inference capability came from llama.cpp and the ggml ecosystem?

The second question proved much more durable.

As Ollama grew into the recognizable consumer-facing brand, some llama.cpp supporters increasingly saw a familiar open-source pattern: upstream engineers perform difficult foundational work; downstream productization captures most of the mindshare.

You do not need to accept every accusation in issue #3185 to understand the resentment. The existence and persistence of the thread establish that attribution was not an invented Reddit grievance. It became a real fault line.

Act III: calling Ollama “just a wrapper” is wrong — but not in the way Ollama critics think

It is tempting to dismiss Ollama as a shell script around llama.cpp. Its source tree makes that description inaccurate.

Ollama maintains scheduling, model distribution, APIs, lifecycle management, backend integration and substantial compatibility machinery. Its llama integration pins a LLAMA_CPP_VERSION, has an upstream-update workflow and carries compatibility patches.

That is real engineering.

But there is another way to read the same architecture.

Every downstream compatibility layer is also a ledger of divergence.

Ollama’s current compatibility code can rewrite GGUF metadata, rename tensors, hide embedded projector or MTP tensors, merge and split tensors, promote data types and perform architecture-specific transformations so older Ollama artifacts remain loadable by newer llama.cpp code.

The project explicitly describes the desired end state as moving back toward llama.cpp-compatible metadata and tensor layouts on disk, and its maintenance instructions tell developers to check whether upstream has gained support for functionality previously carried as a local patch.

That is not evidence that Ollama is trivial.

It is evidence of something more interesting: once you fork the behavior of a fast-moving inference upstream, you inherit the cost of eventually reconciling with it.

That cost would become painfully visible with GPT-OSS.

Act IV: GPT-OSS, the fork, and Georgi enters the chat

This is the episode that makes the entire relationship difficult to describe as a merely imagined community feud.

When GPT-OSS arrived in August 2025, Ollama wanted day-one support. Ollama PR #11672 implemented support using a forked ggml inference path.

Then users tried ordinary GPT-OSS GGUFs from the broader ecosystem.

Ollama issue #11714 filled with reports of failures such as:

gguf_init_from_file_impl: tensor 'blk.0.ffn_down_exps.weight' has invalid ggml type 39 (NONE)
gguf_init_from_file_impl: failed to read tensor info

Users reported GGUFs that failed under Ollama while upstream llama.cpp builds could load them.

And then Georgi Gerganov — creator of llama.cpp and ggml — appeared in the Ollama issue himself.

His explanation was not diplomatic.

Gerganov said Ollama’s developers had forked the ggml inference engine to implement GPT-OSS support without coordinating those changes with upstream. He characterized the resulting implementation as incompatible with most GPT-OSS GGUFs used elsewhere and as significantly slower and unoptimized.

Then came the line that turned a technical diagnosis into open-source theater.

After, as he put it, the release blogs and marketing had circulated and “the dust has settled,” Gerganov said it was time for Ollama to throw out its ggml fork and copy the upstream implementation through Ollama PR #11823. He finished with the prediction that once upstream ggml returned, “ollama will once again be fast and compatible.”

That is extraordinary context for the Ollama/llama.cpp relationship.

It was not a random Reddit commenter accusing Ollama of riding upstream work. The upstream project’s creator was publicly pointing at a concrete fork, a concrete compatibility failure and a concrete return to upstream.

The episode also illustrates why day-one model support can be misleading as an engineering metric. Shipping first creates enormous marketing value. But if the implementation fragments the surrounding model format, performs worse and is soon replaced by the upstream path, the announcement date tells only part of the story.

The subsequent discussion widened beyond GPT-OSS. Other developers argued that Ollama had repeatedly implemented local solutions rather than upstreaming them, only to later consume fixes from ggml/llama.cpp. Whether one accepts that characterization in every historical case or not, GPT-OSS gave the criticism a unusually clean public example.

Act V: llama.cpp quietly removes the reason many power users needed Ollama

The original argument for Ollama was compelling partly because raw local inference was awkward.

That gap has narrowed dramatically.

Modern llama.cpp has a CLI, direct Hugging Face workflows, llama-server, an OpenAI-compatible HTTP API and a web UI. It can expose the knobs enthusiasts actually care about rather than deciding that those knobs are implementation details.

For a power user, this changes the calculation.

If you are choosing exact GGUF quants, measuring prompt processing and token generation separately, tuning GPU offload, experimenting with KV-cache types, controlling context, tensor splits, samplers or speculative decoding, an abstraction layer can stop being convenience and start becoming distance from the system you are trying to understand.

This is the strongest technical version of the anti-Ollama argument.

It does not say Ollama has no use.

It says the more you know about local inference, the smaller Ollama’s convenience advantage becomes and the larger the cost of hidden defaults can become.

For RAMGPT’s own kind of work — reproducible benchmarks, quantization analysis, long-context experiments, backend comparisons and memory forensics — llama.cpp is usually the more natural instrument precisely because the machinery remains visible.

Act VI: “Friends don’t let friends use Ollama”

By 2026, the technical grievance had acquired a slogan.

On April 15, Zetaphor published “Friends Don’t Let Friends Use Ollama.” It was unapologetically a prosecution brief. The essay consolidated complaints that had been scattered across GitHub and local-AI discussions: attribution, downstream divergence, model packaging and naming, hidden inference controls, quantization choices, cloud direction and venture-backed commercialization on top of open-source infrastructure.

This was not an official llama.cpp campaign. There is no basis for claiming the llama.cpp maintainers organized it.

But the slogan escaped the essay.

By June, a r/LocalLLaMA post titled “Stop using Ollama” had accumulated more than 1,600 upvotes. In technical discussions, “use llama.cpp” or “stop using Ollama” increasingly became a canned answer to questions about performance, unsupported quants, GPU behavior and model compatibility.

The meme had a technical thesis:

If you care enough to tune local inference, stop hiding local inference from yourself.

That thesis landed because llama.cpp had become easier at exactly the same time that Ollama’s abstraction was becoming more controversial among enthusiasts.

The movement certainly overreached at times. Performance comparisons depend on versions, backends, defaults and models. Not every Ollama user needs access to every llama.cpp switch. A stable product API and automatic model management are legitimate reasons to choose Ollama.

But reducing the backlash to elitism would miss the historical context.

The attribution dispute had already primed the community. Downstream/upstream divergence was visible in code. The GPT-OSS incident had produced an unusually public rebuke from Gerganov himself. And llama.cpp was no longer an inaccessible low-level library that required heroic effort to use.

The slogan worked because a large group of users already had receipts for the sentiment behind it.

Act VII: the irony — Ollama keeps coming back to llama.cpp

The funniest part of the story is that Ollama’s later engineering makes llama.cpp’s importance more explicit, not less.

Ollama 0.30 publicly announced improved GGUF compatibility through llama.cpp and credited performance work from NVIDIA and the llama.cpp teams. Its own maintenance documentation treats llama.cpp updates as consequential integration events capable of affecting GPU discovery, scheduling, conversion, speculative paths, sampler behavior and runtime compatibility.

Meanwhile the compatibility layer explicitly seeks to retire downstream transformations when upstream llama.cpp gains native support.

None of this means Ollama is surrendering its identity. The opposite is true: Ollama increasingly wants to be a stable product surface above multiple inference technologies, including MLX on Apple hardware.

But that actually sharpens the distinction.

Ollama’s durable asset is the product layer. llama.cpp’s durable asset is the inference engineering.

When evaluating who contributed what to local AI, those should not be confused.

The money makes attribution matter more

In July 2026 Ollama announced an $88 million financing round and said it served 8.9 million developers.

Success changes the emotional temperature of open source.

Nobody spends much time fighting over attribution to an obscure downstream utility. Once that utility becomes a venture-backed company and a household name among developers, the causal chain suddenly matters.

Who built the kernels?

Who defined and maintained the formats?

Who did the model bring-up?

Who made it work on weird hardware?

Who packaged that work into the thing users actually installed?

The last question matters. Ollama deserves credit for productization.

But productization should not retroactively blur the answers to the first four.

This is why I find the llama.cpp side of the dispute more persuasive. Ollama created real value by hiding complexity. The problem begins when hiding the complexity also hides too much of the provenance of the engineering underneath it — or when downstream divergence creates problems that upstream had already solved differently.

So should you stop using Ollama?

Not automatically.

If you want a straightforward local API, easy model lifecycle management and sensible defaults, Ollama can still be the better product for the job. Convenience is not a moral failure.

But for technically demanding local-AI work, my answer has shifted.

If you benchmark models, care about exact quantizations, debug memory behavior, tune context and KV cache, compare GPU backends, need new model support quickly, or simply want to know exactly what the inference engine is doing, start with llama.cpp unless you have a specific reason not to.

That is not because Ollama is “just a wrapper.” It plainly is not.

It is because llama.cpp puts you closer to the upstream implementation where much of the important work is actually happening. Fewer layers mean fewer questions about whether a surprising result belongs to the model, llama.cpp, Ollama’s pinned version, an Ollama patch, a compatibility translation or an Ollama default.

For reproducible systems work, that distinction is enormous.

The relationship status

The original conclusion of this article was deliberately balanced:

llama.cpp won the right to be infrastructure; Ollama won the right to make that infrastructure disappear.

After following the history further, I think that sentence lets Ollama off too easily.

A better version is:

llama.cpp built infrastructure good enough for Ollama to hide. The controversy began when the hiding became more visible than the infrastructure.

Ollama absolutely made local models accessible to millions of people. That achievement should survive criticism.

But llama.cpp and ggml have repeatedly supplied the lower-level work that determines whether those models load, fit, accelerate and run correctly across hardware. The GPT-OSS episode is almost a parable: diverge from upstream for the launch, encounter incompatibility, then return to upstream after the dust settles — with the upstream creator narrating the process in public.

Add the attribution dispute, the compatibility debt, llama.cpp’s steady climb toward easier direct use, and the 2026 “Friends Don’t Let Friends Use Ollama” backlash, and the supposed feud stops looking like random tribalism.

There is a substantive argument underneath it.

Ollama is a successful product built above local-inference infrastructure.

llama.cpp is a large part of why that infrastructure exists in the form people now take for granted.

If the history feels asymmetrical, that is because the engineering relationship itself is asymmetrical.

And if you are the kind of person who wants to see the machinery rather than have it hidden from you, the community’s punch line is increasingly difficult to improve upon:

use llama.cpp.