Architecture Evolution

From static EC2 instances to a fully serverless, highly scalable MVP.

🏢 Current: EC2 VM Pool

💻

React UI

User inputs command

⚙️

Express API

Runs on a static EC2 instance

🧠

Bedrock LLM

Agent reasoning

📬

SQS Queue

Task buffer

🖥️

EC2 VM Pool

Virtual Machines running 24/7. Expensive, slow to boot, requires manual patching.

🚀 Envisioned: Serverless MVP

💻

React UI

User inputs command

Express API (App Runner)

Serverless, auto-scales with web traffic

🧠

Bedrock LLM

Agent reasoning

📬

SQS Queue

Task buffer

🐳

Fargate Containers

Ephemeral Docker containers. Boot instantly, die when finished, pay per second.

Data Lifecycle & Token Efficiency

How the system ensures state is preserved across ephemeral tasks without burning through expensive LLM context windows.

🗄️ DynamoDB (Durable State)

The executors are totally stateless. When a Fargate container finishes a task, it writes the result to DynamoDB and is destroyed. The API reads from DynamoDB, meaning your data lives safely in a managed database forever, unaffected by containers dying.

📦 S3 (Payload Offloading)

LLMs have strict token limits and charge per token. If an agent tries to read a 10MB log file, feeding that directly into Bedrock would crash it. The executor smartly pushes large files to S3, and only passes the S3 link and metadata (like byte size) back to the LLM.

🪙 Context Management

Because the Node.js API acts as the middleman between the queue and Bedrock, it can summarize and trim history before sending it back to the AI. Bedrock only sees the immediate context it needs ("Command succeeded, here is a 50-line summary") rather than ingesting huge raw files.

Execution Models: Polling vs. On-Demand

Understanding how Fargate provisions Firecracker microVMs and how to achieve true 1:1 user isolation.

🔄 The Polling Model (Current Setup)

A Fargate Service runs continuously, containing a script that constantly polls an SQS queue. When User A submits a task, the container runs it. When User B submits a task, the same container runs it next.

Good for: Simple API tasks. Cheaper and faster.

Drawback: Tasks execute sequentially on the same hard drive. Less secure for untrusted AI agents.

⚡ The On-Demand Model (Target State)

Instead of a queue, the Express API uses the AWS SDK to call ECS.RunTask(). Fargate instantly provisions a brand new, unpolluted Firecracker microVM specifically for that single task or user.

Good for: Executing untrusted bash scripts or downloading arbitrary code.

Advantage: Zero-trust security. Complete 1:1 isolation per user.