Vector Databases Explained: The Storage Layer for AI Search
How vector databases store meaning and enable AI-powered search. What they are, how they work, and which options to consider for your RAG system.
How vector databases store meaning and enable AI-powered search. What they are, how they work, and which options to consider for your RAG system.
A vector database is a specialised database designed to store and search vectors: lists of numbers that represent the meaning of text, images, or other data. In the context of AI and RAG systems, it is where you store the embeddings of your document chunks.
Traditional databases search by exact match or keyword. Vector databases search by similarity. This means you can find content that is semantically similar to a query, even if it uses completely different words.
Here's a simplified version:
The "closeness" is measured using mathematical methods like cosine similarity or Euclidean distance. In practice, you do not need to worry about the maths. The database handles it.
Why this matters: A user searching for "annual leave policy" will find a document titled "Employee Holiday Entitlements" because the meanings are similar, even though the words are different.
In a RAG system, the vector database is the retrieval engine. When a user asks a question:
Without a vector database, you'd need to search by keyword, which misses synonyms, paraphrases, and conceptual matches. Keyword search finds "leave policy." Vector search also finds "time off entitlements," "holiday allowance," and "PTO guide."
| Database | Type | Best for |
|---|---|---|
| pgvector | PostgreSQL extension (free) | Teams already using PostgreSQL, smaller datasets, cost-sensitive |
| Pinecone | Managed cloud service | Fast setup, low ops overhead, scaling without management |
| Weaviate | Open-source / managed | Hybrid search (vector + keyword), complex filtering |
| OpenSearch | Open-source / AWS managed | Existing AWS stacks, combined text + vector search |
| Qdrant | Open-source / managed | High performance, advanced filtering, self-hosted |
| ChromaDB | Open-source | Prototyping, local development, small projects |
For most Australian business RAG deployments, we recommend:
The choice matters less than you think at the start. Most vector databases perform similarly for datasets under a million documents. Start with what is easy and migrate later if needed.
A vector database stores and searches vectors, which are lists of numbers that represent the meaning of text, images or other data. In a retrieval augmented generation (RAG) system, the vector database holds the embeddings of document chunks. Traditional databases search by exact match or keyword, while a vector database searches by similarity, finding content that means the same thing in different words.
A vector database search compares meaning rather than exact words. An embedding model turns each piece of text into a vector, perhaps 1,536 numbers long, stored with the original text and its metadata. A search query is turned into a vector the same way, the database measures how close it is to every stored vector, and the closest matches come back as the most similar results.
In a retrieval augmented generation (RAG) system, the vector database is the retrieval engine. A user's question is turned into a vector, the database finds the most relevant document chunks, and those chunks are passed to the language model so its answer is grounded in the retrieved content. Without a vector database, search falls back to keywords, which misses synonyms, paraphrases and conceptual matches.
A business already on PostgreSQL can start with pgvector, a free extension that handles moderate scale well. Pinecone or Weaviate Cloud suit teams that want a managed service with minimal operations, at a higher cost. On Amazon Web Services (AWS), Amazon OpenSearch with vector search keeps data inside the business's virtual private cloud, and Weaviate or OpenSearch both handle combined vector and keyword search.
The choice of vector database matters less than most people think at the start. Most vector databases perform similarly for datasets under a million documents, so start with what is easy and migrate later if needed. The vector database is a critical component but not the only one: how documents are chunked and the retrieval strategy matter just as much.
Ask the author
Ask it here and it comes straight to the founder. No sales call, no obligation, and a real answer even if the answer is that you do not need us.
Kasun Wijayamanna
Founder, replies within one business day
Tell us what you are working on. We will come back with a practical recommendation and clear next steps.
Built here. Your data stays here.
Thanks for reaching out. We will get back to you within one business day.
See what else we do