Vector Databases Explained: The Storage Layer for AI Search

How vector databases store meaning and enable AI-powered search. What they are, how they work, and which options to consider for your RAG system.

What is a vector database?

A vector database is a specialised database designed to store and search vectors: lists of numbers that represent the meaning of text, images, or other data. In the context of AI and RAG systems, it is where you store the embeddings of your document chunks.

Traditional databases search by exact match or keyword. Vector databases search by similarity. This means you can find content that is semantically similar to a query, even if it uses completely different words.

How they work

Here's a simplified version:

  1. Your text is converted into a vector (a list of perhaps 1,536 numbers) by an embedding model.
  2. The vector is stored in the database alongside the original text and any metadata.
  3. When you search, your query is also converted into a vector.
  4. The database calculates how "close" the query vector is to every stored vector.
  5. The closest matches are returned. Those are the most semantically similar results.

The "closeness" is measured using mathematical methods like cosine similarity or Euclidean distance. In practice, you do not need to worry about the maths. The database handles it.

Why this matters: A user searching for "annual leave policy" will find a document titled "Employee Holiday Entitlements" because the meanings are similar, even though the words are different.

Why RAG needs them

In a RAG system, the vector database is the retrieval engine. When a user asks a question:

  • The question is embedded into a vector
  • The vector database finds the most relevant document chunks
  • Those chunks are passed to the language model as context
  • The model generates an answer grounded in the retrieved content

Without a vector database, you'd need to search by keyword, which misses synonyms, paraphrases, and conceptual matches. Keyword search finds "leave policy." Vector search also finds "time off entitlements," "holiday allowance," and "PTO guide."

Database Type Best for
pgvector PostgreSQL extension (free) Teams already using PostgreSQL, smaller datasets, cost-sensitive
Pinecone Managed cloud service Fast setup, low ops overhead, scaling without management
Weaviate Open-source / managed Hybrid search (vector + keyword), complex filtering
OpenSearch Open-source / AWS managed Existing AWS stacks, combined text + vector search
Qdrant Open-source / managed High performance, advanced filtering, self-hosted
ChromaDB Open-source Prototyping, local development, small projects

How to choose

For most Australian business RAG deployments, we recommend:

  • If you are already on PostgreSQL: Start with pgvector. It's free, familiar, and handles moderate scale well.
  • If you want managed with minimal ops: Pinecone or Weaviate Cloud. More expensive but you do not manage infrastructure.
  • If you are on AWS: Amazon OpenSearch with vector search is a natural fit and keeps data in your VPC.
  • If you need hybrid search: Weaviate or OpenSearch both handle vector + keyword search natively.

The choice matters less than you think at the start. Most vector databases perform similarly for datasets under a million documents. Start with what is easy and migrate later if needed.

Key takeaways

  • Vector databases store numerical representations of text that capture meaning, not just keywords.
  • They enable semantic search, finding content that means the same thing even with different words.
  • For most business RAG systems, pgvector (free, runs on PostgreSQL) or a managed service like Pinecone work well.
  • The vector database is a critical but not the only component. Chunking and retrieval strategy matter just as much.
Kasun Wijayamanna
Kasun Wijayamanna Founder & Lead Developer

Postgraduate Researcher (AI & RAG), Curtin University - Western Australia

View profile →

Vector databases: common questions

What is a vector database?

A vector database stores and searches vectors, which are lists of numbers that represent the meaning of text, images or other data. In a retrieval augmented generation (RAG) system, the vector database holds the embeddings of document chunks. Traditional databases search by exact match or keyword, while a vector database searches by similarity, finding content that means the same thing in different words.

How does a vector database search work?

A vector database search compares meaning rather than exact words. An embedding model turns each piece of text into a vector, perhaps 1,536 numbers long, stored with the original text and its metadata. A search query is turned into a vector the same way, the database measures how close it is to every stored vector, and the closest matches come back as the most similar results.

Why does a RAG system need a vector database?

In a retrieval augmented generation (RAG) system, the vector database is the retrieval engine. A user's question is turned into a vector, the database finds the most relevant document chunks, and those chunks are passed to the language model so its answer is grounded in the retrieved content. Without a vector database, search falls back to keywords, which misses synonyms, paraphrases and conceptual matches.

Which vector database should a business choose?

A business already on PostgreSQL can start with pgvector, a free extension that handles moderate scale well. Pinecone or Weaviate Cloud suit teams that want a managed service with minimal operations, at a higher cost. On Amazon Web Services (AWS), Amazon OpenSearch with vector search keeps data inside the business's virtual private cloud, and Weaviate or OpenSearch both handle combined vector and keyword search.

Does the choice of vector database matter much at the start?

The choice of vector database matters less than most people think at the start. Most vector databases perform similarly for datasets under a million documents, so start with what is easy and migrate later if needed. The vector database is a critical component but not the only one: how documents are chunked and the retrieval strategy matter just as much.

Ask the author

Still have a question?

Ask it here and it comes straight to the founder. No sales call, no obligation, and a real answer even if the answer is that you do not need us.

Kasun Wijayamanna, Founder Kasun Wijayamanna
Founder, replies within one business day

Ready to discuss your project?

Tell us what you are working on. We will come back with a practical recommendation and clear next steps.

Australian owned and operated

Built here. Your data stays here.

  • No offshore development. Everything is written by our own team in Australia. Nothing is subcontracted overseas.
  • Your data stays onshore. Hosted in Australia, on infrastructure you own, under Australian law.
  • Every state, not just ours. Perth, Melbourne, Sydney, Brisbane, Adelaide and everywhere between.