RAG Powered Apps with Amazon Bedrock, Part 2: Automating the RAG Pipeline with Terraform

Before you start: This picks up where Part 1 left off. In Part 1, we walked through setting up a Bedrock Knowledge Base manually through the AWS Console.

Introduction & Motivation

I started this project with a simple goal: build a reusable Terraform setup that could deploy the infrastructure behind a “Chat with PDF” application end to end.

When Amazon Bedrock launched, I immediately started experimenting. Like many early GenAI workflows, my first few prototypes were built directly through the AWS Console. The console experience is great for quickly understanding how the pieces fit together, but once I started iterating, I wanted something more repeatable.

I wanted to be able to spin environments up, tear them down, and quickly test different architecture decisions. For example, I wanted to compare the cost differences between using S3 Vectors and OpenSearch as the vector store without manually rebuilding the infrastructure every time.

At the time, I couldn’t find a Terraform module that covered the complete RAG pipeline I was looking for: S3, Bedrock Knowledge Bases, OpenSearch Serverless, Lambda, IAM, and all the glue in between.

So I built my own. Partly because I needed it, and partly because understanding how these services connect together is the best way to learn.

What Are We Building?

A couple of terraform modules to automate everything we clicked through manually in Part 1. One terraform apply brings up the full stack:

• S3 Bucket: your document store. Encrypted at rest, versioning on, zero public access.
• OpenSearch Serverless: the vector database. Stores the embeddings Bedrock generates during ingestion.
• Bedrock Knowledge Base: orchestrates the chunking, embedding, and storage of documents, and retrieval at query time.
• Ingestion Lambda: triggered automatically when you upload a file to S3. Starts a Bedrock ingestion job so documents are chunked, embedded, and indexed without ClickOps.
• Query Lambda: accepts a natural language question, calls RetrieveAndGenerate, and returns an answer with source citations.

Full source code + ReadMe: Bedrock Project. If you run into issues or want to extend the module, feel free to open an issue.

Architecture

Project Structure

rag-bedrock-project/
├── main.tf
├── variables.tf
├── outputs.tf
├── backend.tf
├── terraform.tfvars.example
├── bootstrap/
└── modules/
    ├── storage/
    ├── opensearch/
    ├── bedrock/
    └── lambda/

Enter fullscreen mode Exit fullscreen mode

Each module owns one piece of the infrastructure and exposes only what other modules need through outputs. The root main.tf connects everything by passing outputs from one module as inputs to another.

Multiple modules might feel like overkill, but I learned early in my development journey that clear boundaries make systems easier to reason about. Each module owns one responsibility, which makes debugging much easier when something breaks.

The Lambda module does not need to know how OpenSearch is configured. It only receives the IDs and values it needs through variables.

Implementation

Step 1: Bootstrap Remote State First

Before running terraform apply on anything, you need somewhere to store your Terraform state.

If you’re new to Terraform, state is how Terraform keeps track of the infrastructure it manages. Every resource it creates is recorded in a terraform.tfstatefile so future runs know what already exists.

Keeping state locally ties it to your machine. This project uses S3 remote state so Terraform has a durable, shared source of truth. With Terraform 1.10, native S3 state locking removes the need for a separate DynamoDB table.

The bootstrap/ directory provisions the resources needed for Terraform’s remote backend and deployment permissions. Run the Terraform commands below once before deploying the main stack.

All commands use aws-vault, which stores AWS credentials in your OS keychain and injects temporary credentials at runtime. The --no-session flag skips STS session tokens, which some IAM operations reject. Ignore this if you’re not using aws-vault

aws-vault exec YOUR_PROFILE --no-session -- \
  terraform -chdir=bootstrap init

aws-vault exec YOUR_PROFILE --no-session -- \
  terraform -chdir=bootstrap apply \
  -var="project_name=my-rag" -var="environment=dev"

Enter fullscreen mode Exit fullscreen mode

Bootstrap creates two things: the S3 state bucket and a scoped deployer IAM policy.

The initial bootstrap requires AdministratorAccess because the deployer policy does not exist yet. Once bootstrap completes, attach the generated policy to your IAM user and remove AdministratorAccess. From that point forward, deployments run with least-privilege permissions.

The bootstrap step outputs the S3 backend configuration and deployer policy ARN. Copy the backend configuration into backend.tf, then initialize Terraform again to migrate state.

aws-vault exec YOUR_PROFILE --no-session -- terraform init

Enter fullscreen mode Exit fullscreen mode

Then swap to the scoped policy (this can also be done through the AWS Console):

# Attach the deployer policy
aws-vault exec YOUR_PROFILE --no-session -- aws iam attach-user-policy \
  --user-name YOUR_IAM_USER \
  --policy-arn YOUR_DEPLOYER_POLICY_ARN

# Drop AdministratorAccess
aws-vault exec YOUR_PROFILE --no-session -- aws iam detach-user-policy \
  --user-name YOUR_IAM_USER \
  --policy-arn arn:aws:iam::aws:policy/AdministratorAccess

Enter fullscreen mode Exit fullscreen mode

Step 2: Configure Your Variables

cp terraform.tfvars.example terraform.tfvars

Enter fullscreen mode Exit fullscreen mode

The module only requires a few inputs. The setting worth paying attention to is embedding_dimensions because it controls the size of the vectors stored in OpenSearch.

For this development environment, we use 512 dimensions. It reduces storage costs while keeping retrieval quality acceptable. We’ll revisit this choice when looking at the Bedrock Knowledge Base configuration.

embedding_dimensions = 512  # 256 or 512, half the storage cost vs 1024

Enter fullscreen mode Exit fullscreen mode

💡 First time using Bedrock?

Bedrock subscribes your account to a foundation model automatically the first time you invoke it, but only if the identity making that first call has the AWS Marketplace permissions (aws-marketplace:ViewSubscriptions and aws-marketplace:Subscribe) to finish the subscription. There is a setup window of up to 15 minutes where your calls may even succeed while the subscription is still being finalized in the background.

Here is the trap I fell into. My first invocation came from the scoped Lambda role, which does not carry those Marketplace permissions. Calls worked for a few minutes, then every request started failing with an access error once the subscription failed to complete. The fix is a one-time step: subscribe the model once using an identity that does have Marketplace permissions, such as your admin user, either by accepting the model agreement in the Bedrock console or by invoking the model once. The subscription is account-wide, so you do it once rather than per region. After that, the Lambda role needs only bedrock:InvokeModel.

One extra step for Anthropic models like Claude: before that first invocation you also have to submit a one-time First Time Use form describing your use case. You do it once per account, or once at the organization’s management account, from the model catalog in the Bedrock console or with the PutUseCaseForModelAccess API. Access is granted as soon as the form goes through.

Step 3: Lambda The Query Handler and the Trigger

Lambda Module

Lambda connects the application workflow together. There are two functions with both running Node.js 20 on ARM64. ARM64 (Graviton) is cheaper to run than x86 for the same memory allocation, and since these functions are mostly waiting on AWS services rather than doing heavy computation, there was no reason to pay the x86 premium.

The ingestion handler (ingest.mjs) handles the document ingestion flow. It is triggered by S3 uploads and starts a Bedrock ingestion job so new documents are processed and indexed:

// lambda/src/ingest.mjs

export async function handler(event) {
  const command = new StartIngestionJobCommand({
    knowledgeBaseId: process.env.KNOWLEDGE_BASE_ID,
    dataSourceId: process.env.DATA_SOURCE_ID,
  });

  try {
    const response = await client.send(command);
    return { statusCode: 202, body: JSON.stringify({ jobId: response.ingestionJob.ingestionJobId }) };
  } catch (error) {
    if (error.name === "ConflictException") {
      // An ingestion job is already running. That's fine, the new file will be picked up.
      return { statusCode: 202, body: JSON.stringify({ message: "Ingestion already in progress" }) };
    }
    throw error;
  }
}

Enter fullscreen mode Exit fullscreen mode

One subtle issue: Bedrock only allows one ingestion job at a time. If several files are uploaded together, multiple S3 events can trigger Lambda invocations, and subsequent requests receive a ConflictException.

We treat this as expected behavior and return 202, because the existing ingestion job will pick up the newly uploaded files.

The query handler (query.mjs) handles retrieval and generation. It accepts a natural language question and uses Bedrock Knowledge Bases to retrieve relevant context and generate an answer with citations:

// lambda/src/query.mjs

export async function handler(event, context) {
  // Parses event.body.query (API Gateway proxy format)
  const query = parseQuery(event);

  const command = new RetrieveAndGenerateCommand({
    input: { text: query.trim() },
    retrieveAndGenerateConfiguration: {
      type: "KNOWLEDGE_BASE",
      knowledgeBaseConfiguration: {
        knowledgeBaseId: KNOWLEDGE_BASE_ID,
        modelArn: MODEL_ARN,
      },
    },
  });

  // Retries once on throttle/5xx if there's > 15 seconds left in the timeout budget
  let response;
  try {
    response = await client.send(command);
  } catch (error) {
    if (isRetryable(error) && hasTimeForRetry(context)) {
      response = await client.send(command);
    } else {
      return buildResponse(503, { error: "Service temporarily unavailable" });
    }
  }

  return buildResponse(200, {
    answer: response.output?.text || "",
    citations,
  });
}

Enter fullscreen mode Exit fullscreen mode

For observability, logging follows a strict rule: structured JSON, no PII. We log request IDs, durations, and error types, but never store the actual user query or generated response.

function log(level, requestId, message) {
  console.log(JSON.stringify({
    level,
    timestamp: new Date().toISOString(),
    requestId: requestId || undefined,
    message,
  }));
}

Enter fullscreen mode Exit fullscreen mode

Security: Lambda dependencies are locked in package-lock.json. No floating version ranges means deployments are reproducible and less likely to unexpectedly pull in a compromised dependency.

Step 4: Deploy

The Bedrock and OpenSearch modules have a circular dependency. The Bedrock Knowledge Base requires the AOSS collection endpoint, while the OpenSearch data access policy requires the Bedrock KB role ARN. On a fresh deployment, Terraform cannot resolve both resources at the same time.

The solution is to break the cycle by hand: create the collection first, tell the rest of the stack where it lives, then apply everything else.

# Create just the AOSS collection
aws-vault exec YOUR_PROFILE --no-session -- terraform apply -target=module.opensearch

Enter fullscreen mode Exit fullscreen mode

Once it exists, grab its endpoint and paste it into terraform.tfvars so Bedrock knows where to point:

opensearch_collection_endpoint = "https://your-collection-id.us-east-1.aoss.amazonaws.com"

Enter fullscreen mode Exit fullscreen mode

Now deploy the rest. This first full run also pulls in the OpenSearch Terraform provider, so it needs an init in front of it:

aws-vault exec YOUR_PROFILE --no-session -- terraform init
aws-vault exec YOUR_PROFILE --no-session -- terraform apply

Enter fullscreen mode Exit fullscreen mode

The first AOSS deployment takes a while (~10 minutes). After that the endpoint is captured in your state and tfvars, so day-to-day changes are just terraform apply. The README has the exact commands if you want to follow along line by line.

One thing worth knowing if you’re using aws-vault: some IAM operations reject the session tokens it generates by default. If you encounter an InvalidClientTokenId error during deployment, retry with --no-session.

The README contains the complete deployment reference.

Step 5: Test It

Upload, check status, and invoke

Upload a document. The ingestion Lambda triggers automatically through the S3 event:

aws-vault exec YOUR_PROFILE --no-session -- \
  aws s3 cp ./my-document.pdf \
  s3://$(terraform output -raw document_bucket_name)/

Enter fullscreen mode Exit fullscreen mode

Before querying, verify that Bedrock has finished processing the document:

aws-vault exec YOUR_PROFILE --no-session -- \
  aws bedrock-agent list-ingestion-jobs \
  --knowledge-base-id $(terraform output -raw knowledge_base_id) \
  --data-source-id $(terraform output -raw data_source_id) \
  --region us-east-1

Enter fullscreen mode Exit fullscreen mode

Once the ingestion job is COMPLETE, invoke the query Lambda:

aws-vault exec YOUR_PROFILE --no-session -- \
  aws lambda invoke \
  --function-name $(terraform output -raw query_function_name) \
  --payload '{"body":"{\"query\":\"What is the document about?\"}","requestContext":{"requestId":"test-1"},"headers":{}}' \
  --cli-binary-format raw-in-base64-out \
  /tmp/response.json && cat /tmp/response.json

Enter fullscreen mode Exit fullscreen mode

The payload uses the API Gateway format expected by the Lambda handler, which is why the query is nested inside body.

A successful response looks like:

{
  "statusCode": 200,
  "headers": { "Content-Type": "application/json" },
  "body": "{\"answer\":\"The document covers...\",\"citations\":[...]}"
}

Enter fullscreen mode Exit fullscreen mode

If you get an empty answer, check the ingestion status first. The Knowledge Base cannot retrieve documents until the ingestion job completes.

Step 6: Cleanup

Before running terraform destroy, there is one thing to update. The Bedrock data source uses a data_deletion_policy that defaults to DELETE.

During teardown, Bedrock attempts to remove vectors from OpenSearch as part of deleting the data source. If the AOSS collection is destroyed in the same operation, Bedrock can no longer reach it and the deletion can get stuck.

Set the policy to RETAIN first, apply the change, then destroy:

# modules/bedrock/main.tf

resource "aws_bedrockagent_data_source" "s3" {
  name                 = "${var.config.environment}-${var.config.project_name}-s3-source"
  knowledge_base_id    = aws_bedrockagent_knowledge_base.main.id
  data_deletion_policy = "RETAIN"

  # ... rest of config
}

Enter fullscreen mode Exit fullscreen mode

Apply the change:

aws-vault exec YOUR_PROFILE --no-session -- terraform apply

Enter fullscreen mode Exit fullscreen mode

Full cleanup: tear down the stack

aws-vault exec YOUR_PROFILE --no-session -- terraform destroy

Enter fullscreen mode Exit fullscreen mode

To also remove the bootstrap resources, first empty the versioned Terraform state bucket, then destroy the bootstrap stack:

# Remove all object versions from the state bucket
aws-vault exec YOUR_PROFILE --no-session -- \
  aws s3api delete-objects \
  --bucket YOUR_STATE_BUCKET_NAME \
  --delete "$(aws-vault exec YOUR_PROFILE --no-session -- \
    aws s3api list-object-versions \
    --bucket YOUR_STATE_BUCKET_NAME \
    --query '{Objects: Versions[].{Key:Key,VersionId:VersionId}}' \
    --output json)"

# Destroy bootstrap
aws-vault exec YOUR_PROFILE --no-session -- \
  terraform -chdir=bootstrap destroy

Enter fullscreen mode Exit fullscreen mode

Gotchas and Cost

AOSS is the biggest cost driver in this stack. Even the minimum capacity configuration can add hundreds of dollars per month for a continuously running collection. If you are experimenting, destroy the stack when you are done.

Common issues:

AOSS collection won’t create

You likely hit the policies-first timing issue. Make sure encryption, network, and access policies are included in depends_on.

KB ingestion fails

Verify that the KB role has s3:GetObject access to the bucket and aoss:APIAccessAll access to the collection. Also confirm the AOSS collection is in an ACTIVE state before starting ingestion.

Lambda returns 400 (query is required)

The query Lambda expects event.body.query. Make sure the payload wraps the query inside a body field as shown in Step 5.

Lambda returns 503

The function collapses any Bedrock failure into a generic 503, so the real reason is in CloudWatch, not the response body. Two things cause it in practice.

The boring one is throttling. The function retries once automatically, and sustained traffic may need backoff or a limit increase.

The one that actually cost me time is IAM. This project generates with Claude Sonnet 4.5, which you reach through a cross-region inference profile rather than a plain model ID. An inference profile is really a router: it forwards your request to the underlying model in whichever US region has capacity. So IAM checks you twice, once on the profile and again on the foundation model it lands on. Grant bedrock:InvokeModel on only the profile and the call fails with an AccessDeniedException on GetInferenceProfile, which the Lambda dutifully turns into a 503. The role needs InvokeModel and GetInferenceProfile on the profile, plus InvokeModel on the foundation model in each region the profile can route to (us-east-1, us-east-2, us-west-2). The Terraform in the repo already wires this up, but it is worth knowing why all those grants are there.

Empty citations array

The document format may not be supported well by Bedrock chunking. Plain text, PDF, Markdown, and HTML generally work best.

terraform destroy gets stuck

The default data_deletion_policy = "DELETE" can cause teardown issues when the AOSS collection is destroyed at the same time. Set it to RETAIN, apply, and then destroy.

What’s Next?

The Knowledge Base is live and queryable from the CLI. In Part 3, we’re putting an API Gateway in front of the query Lambda and wiring up a React frontend. The query Lambda’s response shape already works with API Gateway’s proxy integration, and the only thing left is CORS headers and the API Gateway resource itself.

Disclaimer

This is strictly for educational purposes. You will be charged for the resources created when you follow along. Remember to clean up after use.

Links & Resources

• https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base.html
• https://docs.aws.amazon.com/opensearch-service/latest/developerguide/serverless-getting-started.html
• https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/bedrockagent_knowledge_base
• https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html
• https://developer.hashicorp.com/terraform/language/backend/s3
• https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html