---
title: You Upgraded Your Embedding Model. Your RAG Pipeline Just Got Worse.
description: Embedding models get deprecated and retrained, and when marketing teams never re-embed, retrieval quality rots silently. Here is the migration playbook that catches it before it costs you a deal.
author: LETSGROW Dev Team
date: 2026-08-07
category: AI Tools
tags: ["AI Tools", "RAG", "Vector Databases", "AI Agents", "MarTech Engineering"]
url: "https://letsgrow.dev/blog/embedding-model-migration-rag-retrieval-debt"
---
# You Upgraded Your Embedding Model. Your RAG Pipeline Just Got Worse.

You upgraded your embedding model. Your RAG pipeline just got worse, and nobody on your team can tell you why.

Marketing teams treat retrieval-augmented generation as a setup task. Pick an embedding model, embed the knowledge base, wire it to a vector database, ship it. Nobody schedules what comes next, because nothing in that checklist prepares you for the day the embedding model itself changes. Providers retrain their models, deprecate old endpoints, and ship new versions with better benchmarks. When that happens to a system you already trust, retrieval quality degrades quietly, and the first sign you get is a marketing AI agent citing the wrong case study in front of a prospect.

This is not a hypothetical. Every major embedding provider has deprecated at least one model generation in the last two years. If your RAG pipeline has been running since early 2025, some fraction of your vectors were almost certainly produced by a model that no longer exists in its original form. The question is not whether this happens to you. It is whether you notice before or after it costs you something.

## Why Your Vectors Have a Shelf Life

An embedding is a coordinate in a specific model's geometry. Two documents that sit close together in an OpenAI text-embedding-3 space are not guaranteed to sit close together in a Voyage or Cohere space, and they are not guaranteed to sit close together in the next major version of the same provider's model either. Embedding spaces are not portable and they are not stable across versions. That single fact breaks three assumptions most marketing teams make without realizing it.

The first assumption is that mixing vectors from different model versions in one index is safe. It is not. Cosine similarity between a v1 vector and a v2 vector is meaningless even if both vectors describe the same document, because the two models never agreed on what "close" means. The second assumption is that a deprecated model keeps working indefinitely. Providers give you a sunset window, not a permanent guarantee, and once the endpoint goes away you cannot re-derive the same vectors from the same text. The third assumption is that nobody needs to know which model produced which vector. That metadata gap is what makes every migration harder than it should be, because you end up re-embedding the entire corpus instead of just the stale slice.

## The Failure Nobody Notices Until It's Too Late

Retrieval degradation does not throw an error. It shows up as a slow erosion of relevance that looks, at first, like a prompting problem. Teams spend weeks tuning system prompts and reranking logic before anyone checks whether the underlying vectors are still the right ones. By the time someone traces it back to embedding drift, the AI agent has been quietly serving mediocre context for months.

::checklist{title="Signs your RAG retrieval has already started to rot"}
- Similarity scores for known-good query and document pairs have spread out compared to six months ago
- Your content or support team has flagged "the AI cited the wrong doc" more than once in the last month
- Your vector index still contains embeddings from a model your provider has marked deprecated or legacy
- Nobody on the team can tell you which embedding model version produced any given vector in the index
- You have never run a recall@k evaluation against a fixed set of test queries with known correct answers
::

If two or more of those are true, you are not maintaining a RAG pipeline anymore. You are running one on borrowed time.

## The Migration Playbook

Re-embedding an entire knowledge base is not free, and the strategy you pick should match how much downtime and budget you can absorb. There are three real options, and most teams default to the worst one because it is the first one they think of.

::compare-table{title="Re-embedding strategies compared"}
| Strategy | Downtime | Cost profile | Best for |
| --- | --- | --- | --- |
| Full re-embed and swap | Hours to days, index is stale mid-migration | High, one-time compute spike | Small corpora or a hard provider cutover deadline |
| Shadow index, dual-write | None, old index serves until cutover | Medium, double storage during overlap | Production systems with an SLA on retrieval quality |
| Incremental rolling re-embed | None, oldest vectors refresh first | Low, spread over weeks | Large corpora with no urgent deprecation deadline |
::

Whichever strategy you choose, the mechanics stay the same. Tag every vector with the model name and version that produced it, so you can query for stale vectors instead of guessing. Build a fixed evaluation set of twenty to fifty real queries with known correct source documents before you touch anything, because without a baseline you cannot tell if the new model actually improved retrieval or just moved the errors around. Dual-write to both the old and new index during the transition, run the eval set against both, and only retire the old index once the new one matches or beats it on recall. Skipping the eval step is how teams migrate to a "better" model and end up with worse retrieval, because a higher MTEB benchmark score does not guarantee better performance on your specific corpus.

## What to Build Before You Need It

The teams that handle this well did not get lucky. They built three things ahead of time: version metadata on every vector so a migration is a query, not an archaeology project; a standing recall evaluation harness that runs on a schedule, not just when someone is suspicious; and a line item in the AI infrastructure budget for re-embedding, so it is not a surprise cost fight when a provider sends a deprecation notice.

RAG was never a one-time integration. It is a system with a maintenance schedule, the same way a database needs reindexing and a cache needs invalidation. Treat the embedding layer like infrastructure that ages, because it does, whether or not you are watching it.