#production-llm
-
Prompt Injection Detection in Production: What to Alert On
Where injection detection sits in the serving path, which metric to tune (recall at a fixed false-positive rate), and how to shadow-test it.
-
LLMOps Tools on GitHub: The Open-Source Stack
A layer-by-layer map of the open-source LLMOps stack on GitHub, from serving and gateways to tracing, evaluation and guardrails, plus how to vet a repo.
-
Semantic Caching for LLM Serving: Hit Rates and Failure Modes
Exact-match caching misses most LLM cache hits — paraphrases tank hit rate. Semantic caching, threshold tuning, and the production failure modes that bite.
-
RAG Observability: Monitoring the Retrieval Layer in Production
When a RAG system gives a bad answer, the retrieval layer is usually to blame — and your LLM monitoring can't see it.
-
Guardrails in the Serving Path: Defense in Depth for LLMs
Guardrails are not a single check you bolt on — they're layers in the request path, each catching what the others miss.
-
LLMOps Best Practices: From Prototype to Production
LLMOps best practices that separate fragile demos from reliable production systems: prompt versioning, observability, evaluation and cost control.
-
Token-Cost Observability: What You Measure vs What You Should
Most LLM apps track total spend and call it done. Per-feature cost, per-user attribution and anomaly bands all require deliberate instrumentation.