Secure RAG on AWS: Architecture, Security & Data Residency

How to deploy a retrieval-augmented generation system on AWS with proper security, VPCs, encryption, and Australian data residency.

Why deploy RAG on AWS?

For Australian businesses, AWS is the most common choice for production RAG deployments. The reasons are practical:

  • Sydney region (ap-southeast-2): All data stays in Australia, which is important for Privacy Act compliance
  • Complete stack: Everything you need is available: compute, storage, vector search, LLM access
  • Security: VPCs, encryption, IAM, and CloudTrail logging. Enterprise-grade by default
  • AWS Bedrock: Access to Claude, Mistral, and other models via API, with data never leaving your account
  • Most organisations are already on AWS: Reduces the integration overhead

Reference architecture

A typical production RAG deployment on AWS includes:

Ingestion pipeline

  • S3: Document storage (PDFs, Word files, HTML). The source of truth.
  • Lambda / ECS: Document processing: text extraction, chunking, metadata extraction.
  • Amazon Bedrock: Embedding generation (Titan Embeddings or third-party models).
  • OpenSearch / Aurora pgvector: Vector storage. Chunks + embeddings + metadata.

Query pipeline

  • API Gateway + Lambda / ECS: API layer for user queries.
  • OpenSearch / pgvector: Vector search to retrieve relevant chunks.
  • Amazon Bedrock: LLM inference (Claude, Mistral) for answer generation.
  • CloudWatch: Logging, monitoring, cost tracking.

Front-end

  • CloudFront + S3: Static web app (React, Next.js) or integration into your existing app.
  • Cognito: User authentication and access control.

Security layers

Network isolation

Deploy all components within a VPC. Use VPC endpoints for AWS services (S3, Bedrock, OpenSearch) so no traffic leaves the private network.

Encryption

  • At rest: KMS-managed encryption for S3, OpenSearch, Aurora, and any EBS volumes.
  • In transit: TLS everywhere. API calls, database connections, inter-service communication.

Access control

  • IAM policies: Least-privilege access for all services and roles.
  • Document-level permissions: Tag chunks with access levels and filter at query time.
  • Cognito groups: Different users see different documents based on their role.

Audit trail

  • CloudTrail: Log every API call and service interaction.
  • Application logging: Log queries, retrievals, and generated answers for review.
  • Retention policies: Define how long logs are kept, align with compliance requirements.

Australian data residency

For Privacy Act and APRA compliance, ensure:

  • All S3 buckets are in ap-southeast-2
  • OpenSearch / Aurora instances are in ap-southeast-2
  • Bedrock model inference happens in ap-southeast-2 (Claude and Titan are available there)
  • No cross-region replication is enabled unless explicitly required
  • CloudFront caching policies do not store sensitive data at edge locations

AWS Bedrock advantage: When you use Bedrock, your data is processed within your AWS account. It's not used to train models and does not leave the region.

Cost considerations

RAG on AWS typically costs less than people expect. For a mid-size deployment:

  • Bedrock (LLM calls): Charged per token, at a rate that depends on the model. Budget for your expected query volume.
  • Embeddings: One-time cost per document. Titan Embeddings are very cheap.
  • OpenSearch: A monthly cost, starting with a small cluster. Scales with data volume.
  • Compute (Lambda/ECS): Pay per use for Lambda, or fixed for ECS. Depends on query volume.
  • Storage (S3): Negligible for document storage.

Total cost for a typical deployment serving ~1,000 queries/day is a monthly running cost, significantly less than the manual effort it replaces.

Getting started

  1. Define your use case: What knowledge domain? What users? What questions?
  2. Audit your data: What format? How much? How often does it change?
  3. Set up infrastructure: VPC, S3, Bedrock access, vector database.
  4. Build ingestion pipeline: Process documents, generate embeddings, store in vector DB.
  5. Build query pipeline: API → retrieval → generation → response.
  6. Test with real users: Measure answer quality, iterate on chunking and prompts.
  7. Harden for production: Add monitoring, logging, error handling, access control.

Key takeaways

  • AWS provides a complete stack for secure RAG: compute, storage, vector search, and LLM APIs.
  • Deploy in ap-southeast-2 (Sydney) to keep all data in Australia.
  • Use VPC endpoints, encryption at rest and in transit, and IAM policies to lock down the system.
  • AWS Bedrock lets you use frontier models without your data leaving your AWS account.
Kasun Wijayamanna
Kasun Wijayamanna Founder & Lead Developer

Postgraduate Researcher (AI & RAG), Curtin University - Western Australia

View profile →

Secure RAG on AWS: common questions

How do you keep a RAG system's data in Australia on AWS?

Keep a retrieval augmented generation (RAG) system's data in Australia by deploying it in the Amazon Web Services (AWS) Sydney region, ap-southeast-2. Put all S3 buckets and OpenSearch or Aurora databases there, run Bedrock model inference there, avoid cross-region replication unless explicitly required, and make sure CloudFront caching does not store sensitive data at edge locations.

Does Amazon Bedrock use business data to train AI models?

No, with Amazon Bedrock on Amazon Web Services (AWS), business data is processed within your own AWS account, is not used to train models and does not leave the region. Bedrock gives access to models such as Claude and Mistral, and inference can run in the Sydney region, where Claude and Titan models are available.

How do you secure a RAG system on AWS?

Secure a retrieval augmented generation (RAG) system on Amazon Web Services (AWS) in layers. Run every component inside a virtual private cloud (VPC) with VPC endpoints so traffic stays on the private network, encrypt data at rest and in transit, give every service least-privilege access, tag document chunks with access levels and filter at query time, and log every call with CloudTrail.

Which AWS services make up a typical RAG system?

A typical retrieval augmented generation (RAG) system on Amazon Web Services (AWS) stores documents in S3, processes them with Lambda or ECS, creates embeddings with Amazon Bedrock and stores vectors in OpenSearch or Aurora pgvector. Queries arrive through API Gateway, Bedrock generates answers with models such as Claude, CloudWatch handles logging and monitoring, and Cognito manages user sign-in.

How do you get started building a RAG system on AWS?

Start a retrieval augmented generation (RAG) project on Amazon Web Services (AWS) by defining the use case: the knowledge domain, the users and their questions. Audit the data for format, volume and how often it changes. Then set up the infrastructure, build the ingestion and query pipelines, test with real users to tune chunking and prompts, and harden the system for production.

Ask the author

Still have a question?

Ask it here and it comes straight to the founder. No sales call, no obligation, and a real answer even if the answer is that you do not need us.

Kasun Wijayamanna, Founder Kasun Wijayamanna
Founder, replies within one business day

Ready to discuss your project?

Tell us what you are working on. We will come back with a practical recommendation and clear next steps.

Australian owned and operated

Built here. Your data stays here.

  • No offshore development. Everything is written by our own team in Australia. Nothing is subcontracted overseas.
  • Your data stays onshore. Hosted in Australia, on infrastructure you own, under Australian law.
  • Every state, not just ours. Perth, Melbourne, Sydney, Brisbane, Adelaide and everywhere between.