Stories from the gateway
Fundamentals
LLM Latency: Why Time to First Token Is the Number to Watch
LLM latency is two different problems wearing one name. Understanding prefill and decode tells you which knob to turn when a request feels slow.
· 9 min read
Fundamentals
Mixture of Experts: Why Trillion-Parameter Models Are Fast
MoE models decouple parameter count from compute per token, which is why a trillion-parameter model can be cheap to serve. Here is the mechanism and what it costs you.
· 8 min readFundamentals
Model Distillation Explained
· 8 min read
Fundamentals
Multimodal Models Explained: How Images Become Tokens
· 8 min read
Fundamentals
Open Weights vs Open Source: The Distinction That Matters
· 9 min readFundamentals
Prompt Caching: Stop Paying for the Same Tokens Twice
· 9 min readFundamentals
Quantization Explained: Big Models on Smaller Hardware
· 9 min readFundamentals
RAG vs Long Context: Retrieval Did Not Become Obsolete
· 8 min read
Fundamentals
Reasoning Models Explained: When Thinking Longer Helps
· 9 min read
Fundamentals
Speculative Decoding: Faster Output, Identical Tokens
· 8 min read
Fundamentals
System Prompts: What They Actually Do and How to Write One
· 9 min readFundamentals
Temperature, Top-p and Sampling: What the Knobs Really Do
· 8 min readFundamentals
Tokenization Explained: Why Your Bill Is Not About Words
· 8 min readFundamentals
What Is a Parameter Count, and Does It Matter?
· 8 min readReady to start?
Every model, one key. Start free and upgrade when you are ready.