An LLM model registry needs to identify the complete release that produced an answer: prompt version, model revision, serving configuration, and retrieval dependencies. A prompt registry stores part of that record. The operational work is making its version history agree with what the application actually resolves at runtime.
This guide focuses on LLM and prompt registry patterns. For the general platform pattern, use model registry patterns in production. For the surrounding stack, see LLMOps tools on GitHub.
Separate immutable versions from deployment pointers
MLflow’s Prompt Registry documents versioned prompts, aliases, and metadata. Langfuse’s prompt version control similarly distinguishes versions from labels used to resolve a prompt.
A version identifies content; an environment pointer selects content for use. Moving a production alias does not mean the previous version disappeared. The release system should retain the old version and record the transition.
Treat this as a recommended operating contract: create a new version for a behavior change, attach its evaluation evidence, then update the environment pointer through the release process. Record the resolved immutable version on each generation so an incident can be reproduced after the pointer moves.
A registry entry describes desired state. Confirm actual state from the application or serving deployment. Cached prompts, partially rolled-out services, or a changed gateway route can leave more than one release serving traffic even when an alias has a single value.
The LLM release bundle
Store references to the artifacts below in a release record. This is a suggested manifest, not a claim that every registry has identical built-in fields.
| Artifact | Record | Why it belongs with the prompt |
|---|---|---|
| Prompt | Immutable version and template variables | Reconstruct the instructions used |
| Model | Provider/model revision or weights reference | Identify the generation dependency |
| Serving | Engine version, tokenizer, chat template, precision | Preserve the execution configuration |
| Retrieval | Corpus/index revision, embedding model, filters | Explain which context could be retrieved |
| Application | Code release, routing, generation settings | Reconstruct orchestration and defaults |
| Evaluation | Dataset revision, scorer configuration, result | Identify the evidence permitting promotion |
| Deployment | Environment pointer, resolved versions, rollout status | Compare desired and observed state |
Do not include credentials or unrestricted prompt payloads in a broadly accessible manifest. Reference the relevant configuration without copying secret values. Use access controls appropriate to the underlying content.
The vLLM vs TGI guide explains why serving compatibility belongs in this bundle, and choosing the serving framework covers the wider field that the pinned engine version is drawn from. Changing the engine can change client behavior or capacity even if the weights and prompt stay fixed.
Promotion needs a gate and an observation
Keep candidate creation, evaluation, promotion, and runtime observation as distinct steps. The sequence below is an editorial workflow built around the version-and-pointer capabilities described in the source documentation.
First, resolve all dependencies to fixed versions for evaluation. Save the dataset and scorer configuration alongside the result. A report labeled only with a mutable model or prompt alias cannot establish what was evaluated later.
Next, promote the release bundle through a limited rollout. Record which deployment cohorts should use it and which remain on the previous version. A single “production” tag is too coarse to explain a partial rollout by itself.
Finally, compare the runtime’s reported versions with the intended manifest. Alert on missing or unexpected versions as a release-consistency problem. Keep quality alerts separate: correct artifact deployment does not establish that output meets the task’s requirements.
LLM eval pipelines in CI/CD covers the gate’s placement. Prompt versioning and deployment workflows covers the environment transition and caching concerns.
Roll back the dependency set
Reverting prompt text alone may leave a new model route or retrieval index active. Define the rollback unit before promotion and retain the artifacts required to restore it.
For example, a proposed release might change both the system prompt and the number of retrieved chunks. Restoring the old prompt while keeping the larger context budget is a different configuration from the earlier release. Its token cost and latency may differ even when the displayed prompt version looks familiar.
Keep the previous retrieval index available for the required rollback window if rebuilding it would exceed the recovery objective. The vector database comparison discusses retrieval configuration and rebuild capacity.
Use a benign verification request to confirm the resolved release after rollback. Record its version metadata, application outcome, and the deployment cohort that handled it. This is a suggested release check, not a claimed execution by this site.
Metadata should be captured automatically
Fields that can be derived from a build or deployment should be attached by that process. Use manual notes for intent and exceptions, rather than asking an operator to retype immutable identifiers.
Reject incomplete release records at the boundary where they become deployable. At minimum, require the prompt reference, model reference, code/configuration revision, and evaluation reference. Allow an explicit “not applicable” for retrieval in an application that has none.
Store the deployment history as events: candidate created, evaluation completed, pointer moved, rollout observed, and rollback if required. A mutable summary field alone loses the sequence needed to investigate a regression.
Registry decisions also affect cost
A prompt change can increase input length or add another generation step. Attach release identifiers to the cost ledger so spend changes can be grouped by the configuration responsible.
Evaluate a release’s cost at the same request mix and quality target as its baseline. A cheaper average across a different mix of users does not establish an improvement. The self-hosting versus API cost guide provides the model for comparing equivalent completed work.
Common registry mistakes
A prompt registry is not evidence that every application fetches the intended version. A model alias is not an immutable revision. An evaluation score without a dataset and scorer version is not a reproducible release decision.
The practical goal is a traceable dependency bundle with observable deployment state. Use MLflow or Langfuse’s documented versioning features where they fit, then define the application-specific release contract around them. Keep general model-platform selection in the platform registry guide so this workflow stays focused on LLM releases.
