Don't classify. Hallucinate!
Let LLMs freely hallucinate tags, then map them to your real vocabulary via vector similarity, solving traditional classification's generalization limits.
- Traditional tag classification struggles with large vocabularies, context limits, and poor coverage of new tags
- Core approach: Let the model freely generate descriptive tags without a preset vocabulary, then map them to the existing system via vector search
- Prompts only need structural examples (e.g., multi-level category paths), not an exhaustive tag list
- This method combines LLM creativity with vector search precision, drastically reducing system maintenance overhead
The Scale Curse of Tag Management
If you have ever managed a content platform or knowledge base, you know the pain: tags multiply from dozens to thousands. Every new piece of content needs to be slotted into an existing label. The traditional approach is to dump the entire tag list into a prompt and ask the LLM to pick. Reality bites though. A large list consumes your context window, causes the model to miss or mislabel items, and completely fails when new concepts emerge that are not in your vocabulary. Forcing classification in this scenario only makes the system rigid and brittle.
Replacing Choice with Controlled Hallucination
Doug Turnbull offers a counterintuitive yet brilliant solution: do not give the model the tag list. Let it invent. More precisely, allow the model to freely generate descriptive tags based on the content. These tags do not need to exist in your system; they just need to be semantically close to the actual content. Once generated, you use these hallucinated tags to perform a vector similarity search against your real vocabulary, finding the closest existing matches.
Consider an example. Your vocabulary contains Living Room Furniture / Coffee Tables and Decor / Throw Pillows. When the query is brown coffee table, your prompt only needs to instruct the model to generate tags with a structure like Furniture / Living Room / Coffee Tables. The model might output Wooden Living Room Low Table or Casual Area Side Table. Even if these exact phrases are not in your library, their vector representations will align closely with the correct category. By calculating vector distance, you map them back precisely.
From Closed Classification to Open Generation and Vector Alignment
This reveals a deeper shift in AI-era knowledge organization: we are moving from preset rules to dynamic generation paired with semantic alignment. Previously, we tried to cover every possibility through exhaustive lists. Now, we leverage the generalization power of LLMs to diverge, then use vector technology to converge. This generate-then-retrieve paradigm is spreading rapidly beyond tagging into search, recommendation engines, and data cleaning.
You might think vector search is only for semantic retrieval, but it is actually becoming the universal glue that bridges free model output with fixed business systems. The LLM handles intent understanding and creative expression, while the vector store handles grounding and standardization. Together, they preserve flexibility while maintaining business boundaries.
Low-Cost Upgrade for Your Tagging System
If you are tagging content, organizing a knowledge base, or building a vertical search engine, this approach is ready to deploy. You do not need to train a custom classifier or maintain massive prompt templates. The workflow requires just three steps: write a prompt that provides structural examples of your tags, let the model generate candidates, and use an embedding model to compute similarity and map the top results. This runs in a few lines of code. As your vocabulary grows, the system gets smarter because the vector space automatically absorbs new semantics.
Hallucination Is a Feature, Not a Bug
Most developers are still trying to suppress LLM hallucination, but here it is intentionally amplified as a core capability. The model does not need to be correct; it only needs to be relevant. This mindset shift is crucial. In engineering practice, we do not need the model to act as an omniscient judge. We need it to be a creative brainstorming partner, leaving the deterministic algorithms to handle the final grounding. This might be a sign of AI application maturity: stop chasing perfect end-to-end solutions and learn to let different technologies play to their strengths.
Analysis by BitByAI · Read original