💬 1. Sending a Message & Chatting
User Action: Types "Hello" and hits send in the UI.
- React UI: Sends an HTTP POST to the Express backend.
- Express API: Verifies the Cognito JWT
userId to ensure the user owns this specific thread.
- DynamoDB: Saves the user's message to the
hermes_messages table.
- Amazon Bedrock: The backend streams the conversation history to the Claude 3.5 Sonnet model.
- SSE Stream: As Bedrock generates a response token-by-token, the Express API streams it directly back to the React UI via Server-Sent Events (SSE).
📄 2 & 3. Creating & Downloading a Text File
User Action: "Write a python script for me to download."
- Fargate Boot: Express API triggers
ECS.RunTask(). AWS instantly provisions a sterile Firecracker microVM specifically for this user's workspace.
- LLM Tool Call: Bedrock decides to use the
write_file tool. Node.js writes this pending event to DynamoDB.
- Dynamo Polling: Inside the microVM,
executor.py polls DynamoDB, sees the event, and executes it, writing the file to the local /tmp disk.
- Artifact Upload: Knowing the container is ephemeral, the LLM is instructed to call the
upload_artifact tool. The executor pushes the file to the S3 Artifact Bucket.
- Download: The UI receives the S3 pre-signed URL, and the user downloads the file natively.
sequenceDiagram
actor User
participant API as Express API
participant DB as DynamoDB
participant ECS as Fargate (RunTask)
participant S3 as Amazon S3
User->>API: "Write python script"
API->>ECS: Provision new MicroVM
API->>DB: Write 'write_file' event
ECS->>DB: Poll for events
ECS-->>ECS: Execute python locally
ECS->>S3: upload_artifact to bucket
S3-->>API: Return S3 URL
API->>User: Download File Link
⚡ 4. Rapid-Fire Queueing
User Action: Spams 5 messages in a row before the agent replies.
- Stateless API: The Express API simply appends all 5 messages to the DynamoDB
hermes_messages table instantly.
- Context Retrieval: When the Bedrock LLM is ready to think again, it queries DynamoDB, sees the entire context of all 5 messages, and responds accordingly. No messages are dropped, and the UI stays perfectly synced.
👁️ 5. Vision Inputs (Sending a Picture)
User Action: Uploads a screenshot and asks "What is wrong with this code?"
- S3 Upload: The React UI pushes the raw image bytes to the Express API, which uploads it directly to S3.
- Multimodal Payload: The Express API retrieves the image payload/URI and passes it to Claude 3.5 Sonnet using Bedrock's native multimodal message format. Bedrock analyzes the image directly.
🎨 6. Image Generation Extension
User Action: "Generate a picture of a server architecture."
- Tool Invocation: Bedrock uses a new
generate_image tool.
- Titan/DALL-E Integration: The Express API catches this tool call and pings Amazon Titan Image Generator.
- S3 Storage: Titan returns the image bytes. The Node.js API saves the image to S3, returns the S3 URL to Bedrock, and the React UI renders the image tag natively in the chat!
📊 7. Autonomous Data Science
User Action: Uploads a 50MB .zip file and says "Analyze this CSV dataset."
- S3 Offloading: The zip is pushed to S3. The LLM is never given the raw zip file to protect token limits.
- Fargate Execution: A new Fargate microVM boots.
executor.py downloads the zip from S3 and unzips it onto the local disk.
- Autonomous Loop: Bedrock enters an "Agent Loop". It writes a Python script, executes it via bash, reads the output, generates charts, and uploads the charts back to S3.
- Self-Destruction: After the complex task completes, the Fargate container cleanly self-destructs.
graph TD
A[S3 Bucket: Dataset.zip] -->|Download| B(Fargate MicroVM)
B -->|1. write_script.py| C{Agent Loop}
C -->|2. bash run_script| B
B -->|3. terminal output| C
C -->|4. Fix bugs| C
C -->|5. Generate chart.png| D[S3 Bucket: Artifacts]
D --> E((Container Self-Destructs))
⏱️ 8. Recurring Live Agent Extension
User Action: "Check the stock market API every 90 minutes."
- EventBridge Scheduler: Because Fargate tasks self-destruct to save money, the Express API creates an Amazon EventBridge Rule set to trigger every 90 minutes.
- Scheduled Trigger: Every 90 minutes, EventBridge sends a webhook to the Express API.
- Fresh MicroVM: The API calls
ECS.RunTask(). A completely fresh microVM boots up, runs the requested analysis script, sends the user a summary message via email or UI, and instantly terminates.
sequenceDiagram
participant EB as Amazon EventBridge
participant API as Express API
participant ECS as Fargate MicroVM
participant LLM as Amazon Bedrock
Note over EB,LLM: Automatically triggers every 90 mins
EB->>API: Webhook: Run Stock Script
API->>ECS: Boot fresh isolated container
ECS->>LLM: Analyze live stock data
LLM-->>ECS: Return analysis
ECS->>API: Send summary alert to User
Note over ECS: Container is instantly destroyed