RAG (Retrieval-Augmented Generation) is a technique in which an AI model (LLM) retrieves relevant information from an external/additional data source and uses that information to generate an answer to a user's query.
There are three main conceptual steps in RAG:
- Retrieval
- Augmentation
- Generation
Two main pipelines in RAG
RAG generally involves two pipelines:
- Data Ingestion Pipeline
- Data Retrieval Pipeline
1. Data Ingestion Pipeline
In this pipeline, the data is prepared and stored so that it can be retrieved later.
Step 1: Get the data
Data can come from sources such as:
- Excel
- CSV
- Database
- Website
- Documents
- Text, video, audio, images, etc.
Note: For audio, video, and images, the RAG system needs to process or transform these types of data into a form that can be indexed and retrieved. For example, audio may need to be transcribed, while images may require vision-based processing.
Step 2: Chunking
The data is split into smaller segments called chunks.
Step 3: Embedding
The chunks are converted into vector embeddings.
Embeddings are numerical representations of data or information that capture its semantic meaning. An embedding model or API is used to generate these vector embeddings from the chunks.
Step 4: Store in a Vector Database
The vector embeddings are stored in a vector database.
Traditional databases generally retrieve data using structured queries and fields, while vector databases can retrieve information based on semantic similarity.
For example:
"Having a problem with vision"
and
"Poor eyesight"
have different words but similar meanings. A vector search can identify this semantic similarity.
2. Data Retrieval Pipeline
Step 5: Get the user's query
The user's query is received by the RAG system.
Step 6: Convert the query into an embedding
The query is converted into a vector embedding using an embedding model.
Step 7: Search the vector database
The query embedding is compared with the stored embeddings, and the most semantically similar information is retrieved.
The retrieved information is called the context.
Up to this point, we are mainly dealing with the Retrieval process.
Augmentation
The original user query is combined with the retrieved context to create a meaningful prompt for the LLM.
Generation
The augmented prompt is sent to the AI model (LLM), which uses the retrieved context to generate the answer for the user.
Chunking Types
Fixed-size chunking
Divides content into chunks based on a fixed number of characters, words, or tokens.
Hierarchical chunking
Creates chunks at different levels, such as document → section → paragraph, while preserving their relationships.
Semantic chunking
Divides content based on changes in meaning or topic rather than simply using a fixed size.
Popular RAG Architectures
1. Standard or Naive RAG
This is the basic RAG architecture explained above.
Flow:
User Query → Embedding → Retrieval → Context → Prompt → LLM → Response
2. Hybrid RAG
Combines vector/semantic search with keyword search to improve retrieval.
3. RAG with Memory
Uses previous conversation history along with retrieved information to provide more context-aware responses.
4. Graph RAG
Uses a graph containing entities (nodes) and relationships between them to retrieve and connect relevant information.
Benefits of RAG
1. Can reduce hallucinations
RAG provides the LLM with relevant information from the available sources, which can help reduce incorrect or unsupported answers. However, RAG does not completely eliminate hallucinations.
2. Can provide more up-to-date information
RAG can provide current information when the underlying data source is updated and the new information is available for retrieval.
Without RAG, an LLM's built-in knowledge may be limited by its training data and knowledge cutoff.
3. Can be more cost-effective
When we need to provide an LLM with additional or frequently changing information, RAG can be more cost-effective than retraining or fine-tuning the model.
4. Can help control data access
RAG can retrieve only the information relevant to a user's query rather than providing the entire data source to the LLM.
However, data privacy depends on how the overall RAG system, data sources, access controls, vector database, and LLM are configured.
In simple terms:
RAG = Retrieve relevant information → Add it to the prompt → Let the LLM generate the answer.
RAG is one of the important techniques for connecting LLMs with external/additional and domain-specific knowledge.
No comments:
Post a Comment