Vector Databases in RAG¶

Why do we need a Vector Database?¶

Large Language Models (LLMs) cannot directly understand or search through raw text documents efficiently.

Instead, the RAG pipeline converts every document chunk into an embedding vector. When a user asks a question, the query is also converted into an embedding vector. The Vector Database performs Nearest Neighbor Search to retrieve the most semantically similar chunks.

Pipeline:

Documents

↓

Chunking

↓

Embeddings

↓

Vector Database

↓

Similarity Search

↓

Relevant Chunks

↓

LLM

↓

Answer

Why not SQL?¶

Traditional databases perform exact matching.

Example

SELECT * FROM employees
WHERE name='John'

Vector databases perform semantic matching.

Example

User Query

↓

"What is Dependency Injection?"

↓

Find documents with similar meaning

↓

Return top-k results

This makes them ideal for semantic search, recommendations, and Retrieval-Augmented Generation (RAG).

Search Library vs Vector Database¶

This is one of the most common interview questions.

Search Library	Vector Database
Only performs similarity search	Stores vectors + metadata + text
No persistence by default	Persistent storage
No replication	Supports replication
No scaling	Scalable
Developer manages infrastructure	Database manages infrastructure
Example: FAISS	Example: Pinecone, Qdrant

FAISS¶

What is FAISS?¶

FAISS (Facebook AI Similarity Search) is an open-source vector similarity search library developed by Meta.

FAISS is NOT a Vector Database.

It is only responsible for performing fast similarity search.

Architecture¶

Documents

↓

Embeddings

↓

FAISS Index

↓

Nearest Neighbor Search

Advantages¶

Extremely fast
Lightweight
Multiple indexing algorithms
Excellent for experimentation

Disadvantages¶

No persistence by default
No metadata support
No user management
No scaling
No cloud management

Best Use Cases¶

Learning
Local RAG
Research
Offline experimentation

LangChain Example¶

from langchain_community.vectorstores import FAISS

vectorstore = FAISS.from_documents(
    docs,
    embeddings
)

So, what is vectorstore here?

It’s not a full vector database like Pinecone or Weaviate.
It’s an in-memory FAISS index managed by LangChain. Think of it as a lightweight, local vector store that lives in your Python process.
You can persist it to disk (vectorstore.save_local(...)) and reload later, but it doesn’t have distributed storage, metadata filtering, or cloud-native scaling.

ChromaDB¶

What is ChromaDB?¶

Chroma is an open-source Vector Database built specifically for AI applications.

Think of it as

SQLite

for

Vector Embeddings

Architecture¶

Documents

↓

Embeddings

↓

Collections

↓

Similarity Search

Advantages¶

Built-in persistence
Metadata filtering
Collections
Very easy to use
Great LangChain integration

Disadvantages¶

Not designed for very large production workloads
Scaling is limited compared to cloud databases

Best Use Cases¶

Local development
AI prototypes
Internal tools
Small-to-medium RAG systems

LangChain Example¶

from langchain_chroma import Chroma

vectorstore = Chroma.from_documents(
    docs,
    embeddings
)

Pinecone¶

What is Pinecone?¶

Pinecone is a fully managed cloud Vector Database. Instead of managing servers and indexes yourself, you simply upload vectors and query them.

Architecture¶

Application

↓

Pinecone Cloud

↓

Managed Indexes

↓

Similarity Search

Advantages¶

Fully managed
Automatic scaling
Metadata filtering
High availability
Production ready

Disadvantages¶

Paid service
Vendor lock-in
Less infrastructure control

Best Use Cases¶

SaaS products
Customer-facing AI
Production RAG
Millions of vectors

LangChain Example¶

from langchain_pinecone import PineconeVectorStore

vectorstore = PineconeVectorStore(
    index=index,
    embedding=embeddings
)

Qdrant¶

What is Qdrant?¶

Qdrant is an open-source production-grade Vector Database. It is designed for high-performance semantic search and supports advanced filtering and hybrid search.

Advantages¶

Open source
Metadata filtering
Hybrid Search
REST API
gRPC support
High performance

Disadvantages¶

Requires self-hosting (unless using Qdrant Cloud)
Slightly higher operational complexity

Best Use Cases¶

Production AI applications
Recommendation systems
Large-scale semantic search
Open-source deployments

LangChain Example¶

from langchain_qdrant import QdrantVectorStore

vectorstore = QdrantVectorStore.from_documents(
    docs,
    embeddings
)

AstraDB¶

What is AstraDB?¶

AstraDB is DataStax's managed cloud database built on Apache Cassandra with native vector search capabilities.

It combines

Traditional NoSQL storage
Vector Search
Metadata filtering

in a single managed database.

Advantages¶

Enterprise grade
Massive scalability
Cassandra ecosystem
Built-in vector search
Metadata filtering

Disadvantages¶

More suited for enterprise use cases
Slightly steeper learning curve

Best Use Cases¶

Enterprise AI
Large organizations
Cassandra users
Massive datasets

LangChain Example¶

from langchain_astradb import AstraDBVectorStore

vectorstore = AstraDBVectorStore(
    embedding=embeddings
)

Comparison¶

Feature	FAISS	Chroma	Pinecone	Qdrant	AstraDB
Type	Search Library	Vector DB	Managed Vector DB	Vector DB	Managed Vector DB
Open Source	✅	✅	❌	✅	❌
Local Development	✅	✅	❌	✅	❌
Managed Cloud	❌	Partial	✅	✅	✅
Metadata Filtering	❌	✅	✅	✅	✅
Persistence	Manual	Built-in	Built-in	Built-in	Built-in
Scaling	Poor	Moderate	Excellent	Excellent	Excellent
Best For	Learning	Local RAG	SaaS & Production	Open-source Production	Enterprise

Trade-offs¶

High-Level Trade-offs¶

Database	Biggest Strength	Biggest Weakness
FAISS	Extremely fast local similarity search	No persistence, metadata, or scalability
ChromaDB	Very easy local RAG development	Limited scalability for enterprise workloads
Pinecone	Fully managed production infrastructure	Paid service and vendor lock-in
Qdrant	Open-source production-grade vector database	Requires infrastructure management (unless using Qdrant Cloud)
AstraDB	Enterprise scalability with Cassandra ecosystem	More complex than needed for small projects

Performance Trade-offs¶

Feature	FAISS	Chroma	Pinecone	Qdrant	AstraDB
Search Speed	⭐⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐⭐
Setup Simplicity	⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐	⭐⭐
Local Development	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐	⭐⭐⭐⭐	⭐
Production Readiness	⭐⭐	⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐
Metadata Filtering	❌	✅	✅	✅	✅
Scalability	⭐	⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐
Infrastructure Control	⭐⭐⭐⭐⭐	⭐⭐⭐⭐	⭐	⭐⭐⭐⭐	⭐⭐
Operational Overhead	Low	Low	Very Low	Medium	Medium

Cost vs Control¶

Database	Cost	Control
FAISS	⭐ Free	⭐⭐⭐⭐⭐ Full Control
ChromaDB	⭐ Free	⭐⭐⭐⭐ High Control
Pinecone	💰 Paid	⭐ Minimal Control
Qdrant	⭐ Free (Self-hosted) / 💰 Cloud	⭐⭐⭐⭐ High Control
AstraDB	💰 Managed Pricing	⭐⭐⭐ Moderate Control

Decision Flow¶

Are you learning?

        │
        ▼
      FAISS

──────────────

Need local persistence?

        │
        ▼
     ChromaDB

──────────────

Need production infrastructure?

        │
        ▼

Want Managed?
      │
      ├── Yes → Pinecone
      │
      └── No
             │
             ▼
          Qdrant

──────────────

Already using Cassandra?

        │
        ▼
      AstraDB

Interview Insight ⭐

There is no universally "best" vector database.

Always justify your choice based on the application's requirements, such as:

📦 Dataset size
🚀 Expected traffic
🏗 Infrastructure ownership
💰 Budget
🔎 Metadata filtering requirements
☁️ Deployment model (local vs cloud)
👥 Team expertise

In interviews, avoid saying

"Pinecone is the best."

Instead say

"The choice depends on the application's scale, operational requirements, and infrastructure preferences. For example, I'd choose FAISS for local experimentation, Chroma for rapid prototyping, Qdrant for self-hosted production systems, Pinecone for managed SaaS deployments, and AstraDB for enterprise environments already leveraging Cassandra."

LangChain Comparison¶

Notice how only one line changes.

# FAISS
FAISS.from_documents(docs, embeddings)

# Chroma
Chroma.from_documents(docs, embeddings)

# Qdrant
QdrantVectorStore.from_documents(docs, embeddings)

#Pinecone
Pinecone.from_documents(docs, embeddings)

# AstraDB
AstraDBVectorStore.from_documents(docs, embeddings)

The RAG architecture remains exactly the same. Only the storage backend changes.

Interview Questions¶

What is a Vector Database?¶

Answer

A Vector Database stores embedding vectors and enables efficient nearest-neighbor search. It typically stores the embedding, original text, metadata, and document identifiers, allowing semantic retrieval for applications like RAG.

Why can't SQL replace a Vector Database?¶

Answer

SQL databases are optimized for exact matches and relational queries. Vector databases are optimized for similarity search in high-dimensional vector spaces using algorithms such as Approximate Nearest Neighbor (ANN) search.

Why is FAISS not considered a Vector Database?¶

Answer

FAISS is a similarity search library. It performs fast vector search but does not provide built-in persistence, metadata management, replication, user management, or cloud scalability.

Difference between FAISS and Pinecone?¶

Answer

FAISS is a local similarity search library where developers manage storage and infrastructure. Pinecone is a fully managed cloud Vector Database that provides persistence, scaling, metadata filtering, and production infrastructure.

What does a Vector Database actually store?¶

Answer

A Vector Database typically stores:

Embedding vectors
Original text
Metadata
Document IDs

Only the embeddings are used for similarity search, while the text and metadata are returned after retrieval.

Why do Vector Databases support Metadata Filtering?¶

Answer

Metadata filtering narrows the search space before similarity search.

Example:

Author = Aristotle

↓

Perform similarity search only on Aristotle's documents

This improves retrieval accuracy and reduces unnecessary search.

Can we change the embedding model without rebuilding the Vector Database?¶

Answer

No.

Changing the embedding model changes the vector space itself. All documents must be embedded again and the vector index rebuilt. This process is called Re-indexing.

Which Vector Database would you choose?¶

Scenario	Choice
Learning	FAISS
Local Prototype	Chroma
Startup MVP	Chroma / Qdrant
Production SaaS	Pinecone
Enterprise	AstraDB

Key Takeaways¶

A Vector Database enables semantic search using embeddings.
FAISS is a vector similarity search library, not a full database.
Chroma is ideal for local development and rapid RAG prototyping.
Pinecone is a fully managed cloud Vector Database designed for production.
Qdrant is a powerful open-source production Vector Database with advanced filtering and hybrid search.
AstraDB combines Cassandra with native vector search for enterprise-scale AI applications.
LangChain abstracts most implementation differences, allowing the same RAG architecture to work across multiple Vector Databases.
Choosing a Vector Database is primarily an infrastructure decision, while the RAG pipeline (Chunking → Embeddings → Retrieval → LLM) remains fundamentally unchanged.