Tensor, Data, Pipeline and Expert Parallelism for LLM Inference
AI Foundations #32 explains the four major ways inference workloads are split across GPUs, what each one replicates or shards, and why communication changes the trade-off.
AI Fundamentals Contributor
William
William writes about AI fundamentals from the perspective of a technically curious learner without a traditional computer science background. His articles focus on building accurate mental models of neural networks, inference, probability, optimization, and local AI systems without assuming advanced prior knowledge.
Published work
Articles
20 published
Prefix Caching and KV Reuse: Why Repeated Prompts Can Be Cheaper
AI Foundations #29 explains prefix caching, KV reuse, exact token matching, cache hits, eviction, prompt-template effects, and why reuse reduces prefill work without caching the answer.
By William ·
Latency and Throughput: Why LLM Speed Has More Than One Number
AI Foundations #27 explains time to first token, time per output token, prefill, decode, batching, throughput, context length, and why one tokens-per-second number can hide the real user experience.
By William ·
llama.cpp 'blk.32.attn_norm.weight' Not Found: Fix MTP Metadata
Fix the Qwen3.5-family blk.32.attn_norm.weight missing-tensor error when config metadata declares an MTP/NextN layer that the checkpoint does not contain.
By William ·
Alignment and Guardrails: Why Capable AI Still Needs Boundaries
AI Foundations #25 explains alignment, preference training, guardrails, instruction hierarchy, runtime policy, and why capability and safe system behavior are different goals.
By William ·
Retrieval-Augmented Generation: How an AI Model Uses External Knowledge
AI Foundations #23 explains retrieval-augmented generation, embeddings, chunking, retrieval, context injection, citations, failure modes, and why RAG is different from training.
By William ·
Reasoning and Reinforcement Learning: Beyond Pretraining
AI Foundations #21 explains reasoning tokens, inference-time compute, reward signals, RLHF, PPO, and how reinforcement-learning post-training changes model behavior.
By William ·
Mixture of Experts: How MoE Models Activate Only Part of the Network
AI Foundations #19 follows tokens through an MoE router and explains expert selection, active versus total parameters, load balance, memory, and inference cost.
By William ·
Context and KV Cache: How an LLM Reuses Earlier Tokens
AI Foundations #17 follows KV state through prefill, decoding, prefix reuse, edited prompts, and request completion, explaining when cached work remains valid.
By William ·
Inference: What Happens After You Press Send to a Trained Language Model
AI Foundations #15 follows one language-model request through tokenization, prefill, decoding, sampling, and the memory used along the way.
By William ·
Pretraining: How a Random Transformer Learns From Next-Token Prediction
AI Foundations #13 connects the Transformer architecture to the training objective that turns random parameters into a useful language model.
By William ·
Attention: How Each Token Decides Which Other Tokens Matter
AI Foundations #11 follows tokenization and embeddings into attention: queries, keys, values, similarity scores, softmax weights, and context-dependent representations.
By William ·
Embeddings: How AI Turns Meaning Into Coordinates
AI Foundations #9 explains embeddings as learned vectors that give neural networks a useful numerical space for tokens and other objects.
By William ·
Gradient Descent: How a Neural Network Takes Its First Step Toward a Lower Loss
AI Foundations #7 explains gradient descent as the rule that turns loss into a direction for changing weights, using one small numerical example before backpropagation.
By William ·
What Is a Forward Pass? How Data Moves Through a Neural Network
AI Foundations #5 calculates a complete two-layer forward pass and distinguishes parameters, activations, nonlinearities, inference, and training.
By William ·
Parameters, Weights, and Biases: What Are They in a Neural Network?
AI Foundations #3 counts the weights and biases in a small layer, shows how they change its output, and connects parameter count to model memory.
By William ·
AI, Machine Learning, Neural Networks, and LLMs: A Beginner's Map
AI Foundations #1 separates artificial intelligence, machine learning, neural networks, deep learning, generative AI, LLMs, models, runtimes, and applications.
By William ·
How to Clone a Voice Locally with Audio8 TTS 0.1B
A beginner-friendly guide to local voice cloning with Audio8 TTS Preview 0.1B, from reference audio and transcripts to generation and FFmpeg speed control.
By William ·
Running Qwen3.8-27B on an RTX 4090 with vLLM + OpenWebUI
A hands-on guide to running Qwen3.8-27B on one RTX 4090 with patched vLLM, 64K context, MTP speculative decoding, tool calling, and OpenWebUI.
By William ·
llama.cpp for Beginners: Build, Run and Tune Your First Local LLM
A simple step-by-step llama.cpp guide for beginners: compile from source, run a GGUF model, use GPU acceleration, and tune the main inference settings.
By William ·