Topics
Browse posts by category and tag — every topic we cover, with the latest pieces under each.
Tags
- #observability 7
- #production-llm 7
- #llmops 6
- #vllm 4
- #cost-optimization 3
- #inference 3
- #llm-serving 3
- #deployment 2
- #gpu 2
- #gpu-optimization 2
- #guardrails 2
- #infrastructure 2
- #latency 2
- #llm-eval 2
- #llm-inference 2
- #llm-security 2
- #production 2
- #prompt-injection 2
- #prompt-management 2
- #rag 2
- #self-hosted-llm 2
- #tooling 2
- #arize 1
- #benchmark 1
- #best-practices 1
- #ci-cd 1
- #cost-monitoring 1
- #cost-reduction 1
- #debugging 1
- #deepeval 1
- #drift 1
- #evidently 1
- #feature-engineering 1
- #governance 1
- #langfuse 1
- #mlops 1
- #model-registry 1
- #monitoring 1
- #nemo-guardrails 1
- #open-source 1
- #prompt-versioning 1
- #promptfoo 1
- #quantization 1
- #ragas 1
- #ray-serve 1
- #retrieval 1
- #retrieval-augmented-generation 1
- #review 1
- #semantic-caching 1
- #sglang 1
- #tensorrt-llm 1
- #testing 1
- #tgi 1
- #token-tracking 1
- #training-serving-skew 1
- #vector-database 1
Categories
Platform 5 posts
- LLMOps Tools on GitHub: The Open-Source StackA layer-by-layer map of the open-source LLMOps stack on GitHub, from serving and gateways to tracing, evaluation and guardrails, plus how to vet a repo.
- Best Vector Database for RAG: A Practical Comparison (2026)Pinecone, Weaviate, Qdrant, pgvector, Chroma and Milvus compared on the published recall@k, p99 latency, filtered-search and cost figures for each.
- Prompt Versioning and Deployment: The Operational WorkflowVersioning prompts is the easy part. The hard parts are decoupling prompt releases from code deploys and keeping staging and production labels honest.
- LLMOps Best Practices: From Prototype to ProductionLLMOps best practices that separate fragile demos from reliable production systems: prompt versioning, observability, evaluation and cost control.
- Model Registry Patterns That Actually WorkWhat the hype skips about model registries, what mature teams actually do, and how to avoid the metadata graveyard most registries become.
Cost 4 posts
- Serverless vs Dedicated GPU for LLM HostingServerless GPU containers scale to zero but pay for it in cold starts and lost batching. Here is the duty-cycle math that decides which one is cheaper.
- LLM GPU Cost Optimization Techniques That Move the NeedleFour LLM GPU cost optimization techniques that pay off: quantization, KV cache management, continuous batching, and parallelism, and when each fits.
- Self Hosting LLM vs API Cost: A TCO Breakdown for 2026Self hosting LLM vs API cost, broken down: hardware, cloud GPU rental, engineering overhead, and the utilization trap that breaks most breakeven math.
- Token-Cost Observability: What You Measure vs What You ShouldMost LLM apps track total spend and call it done. Per-feature cost, per-user attribution and anomaly bands all require deliberate instrumentation.
Evaluation 3 posts
- LLM Eval Pipelines in CI/CD: Gates That Actually Catch ThingsRunning LLM evals in CI is easy to set up and easy to get wrong. How to build quality and red-team gates that block bad prompts before they ever ship.
- Guardrails in the Serving Path: Defense in Depth for LLMsGuardrails are not a single check you bolt on — they're layers in the request path, each catching what the others miss.
- Training/Serving Skew: The Silent KillerHow training/serving skew happens, why it's so hard to see, and the specific places to look when your model works in eval and breaks in prod.
Observability 3 posts
- RAG Observability: Monitoring the Retrieval Layer in ProductionWhen a RAG system gives a bad answer, the retrieval layer is usually to blame — and your LLM monitoring can't see it.
- MLOps Tool Review: Arize vs EvidentlyAn honest comparison of two ML observability tools—where each fits, where each frustrates, and what neither one solves.
- Concept Drift Detection in Production: Practical ThresholdsHow to actually detect concept drift in live systems, what thresholds matter, and why your monitoring dashboard is probably lying to you.
Serving 3 posts
- vLLM vs TGI Serving Comparison: Throughput, Latency, and EOL RiskA vLLM vs TGI serving comparison covering PagedAttention, continuous batching, and why TGI's maintenance-mode status now outweighs raw benchmark numbers.
- Best LLM Serving Frameworks 2026: vLLM, SGLang, and TensorRT-LLMHow vLLM, SGLang, TensorRT-LLM, and Ray Serve stack up on throughput, TTFT, and operational complexity — and which one fits your workload in 2026.
- Semantic Caching for LLM Serving: Hit Rates and Failure ModesExact-match caching misses most LLM cache hits — paraphrases tank hit rate. Semantic caching, threshold tuning, and the production failure modes that bite.