---
id: "prereq-rag-pipelines"
type: "prereq"
source_timestamps: ["00:06:03", "00:11:32"]
tags: ["technical-knowledge"]
related: ["concept-context-engineering", "concept-shadow-agents", "entity-langchain"]
reason: "Required to understand the limitations of current Context Engineering and why organizations are building fragmented 'Shadow Agents'."
sources: ["s24-prompt-engineering-dead"]
sourceVaultSlug: "s24-prompt-engineering-dead"
originDay: 24
---
# Understanding of RAG Pipelines

## What Is Required

The speaker assumes the audience understands what a **RAG (Retrieval-Augmented Generation) pipeline** is. RAG is referenced as:

- The baseline implementation of [[concept-context-engineering]].
- The mechanism behind every team's bespoke [[concept-shadow-agents|shadow agent]].
- The technical predecessor to standardized protocols like [[entity-mcp]].

## Quick Refresher

A RAG pipeline:

1. **Ingests** organizational documents (PDFs, Slack messages, wiki pages, CRM records).
2. **Chunks** them into manageable segments.
3. **Embeds** each chunk into a vector representation.
4. **Stores** embeddings in a vector database.
5. At query time, **retrieves** the most relevant chunks.
6. **Injects** them into the LLM's prompt as additional context before generation.

Frameworks like [[entity-langchain]] and LlamaIndex are the de facto standards for building these pipelines.

## Why This Matters for the Argument

Understanding RAG is necessary to grasp:

- Why "context engineering" became a discrete discipline.
- Why every team can build their own context stack — and why that produces shadow agents.
- Why protocol-level standardization ([[entity-mcp]]) is needed to retire shadow agents and reach [[concept-unified-context-infrastructure|unified context infrastructure]].

Without this baseline, the rest of the architectural argument is hard to follow.