---
id: "question-metadata-extraction-reliability"
type: "open-question"
source_timestamps: ["00:21:17", "00:21:28"]
tags: ["technical-limitations", "machine-learning"]
related: ["framework-open-brain-architecture"]
resolutionPath: "Advancements in structured output enforcement (like OpenAI's Structured Outputs) and specialized, smaller models fine-tuned specifically for metadata extraction."
sources: ["s22-saas-replacement"]
sourceVaultSlug: "s22-saas-replacement"
originDay: 22
---
# How can LLM metadata extraction be made perfectly reliable?

## Question

The speaker concedes that the **Process** step of [[framework-open-brain-architecture]] — using an LLM to extract people, topics, and action items from raw text — is *not always perfect*. Misclassifications happen. How can this pipeline be made fully reliable without human review?

## Why It Matters

If metadata is wrong, downstream queries return wrong results. Trust in the [[concept-open-brain-d22]] depends on the underlying structured data being accurate enough that semantic search complements (rather than fights) keyword/metadata filters.

## Resolution Path

- Adoption of structured-output enforcement (OpenAI Structured Outputs, JSON-mode-with-schema, constrained decoding).
- Smaller specialized models fine-tuned just for metadata extraction.
- Self-correcting pipelines where a second pass validates the first.
- Human-in-the-loop review for high-stakes captures only.