Agent DataSphere

AI agents are only as good as the context they can reason over, and an organization’s context is scattered across data lakes, warehouses, lakehouses, databases, and streams, and in knowledge no one has written down. Until agents can reach that context safely, their decisions can’t be trusted. Senior AWS solutions architects and engineers were shaping the idea for Agent DataSphere: a knowledge-graph agent platform that acts as the semantic and context layer of an agentic data lake, so AI agents can find, understand, and trust an organization’s data. The closest public example today is AWS Context, which AWS announced in June 2026.

I was given the task of building the MVP: a working prototype of Agent DataSphere. I built the platform from scratch, with four core deliverables: an unstructured-data pipeline, a searchable knowledge base, a question-answering agent, and a human-in-the-loop interface.

• Built document ingestion that parses text, Markdown, JSON, and CSV files, extracts entities and relationships, and merges them into an Amazon Neptune knowledge graph, with uploads through S3 presigned URLs.
• Built the embedding pipeline and vector index on Amazon S3 Vectors (chosen over OpenSearch to reduce cost), behind a semantic search API that also returns related graph entities.
• Wrote the orchestrator agent: a tool-use loop on Amazon Bedrock and Claude that decides, per question, whether to search documents, traverse the graph, or both in parallel, with runtime model selection and a deterministic fallback so it never hard-fails.
• Built a React interface for people to review and correct the graph: create, edit, and delete entities and relationships, with per-entity history, an audit log in DynamoDB, and one-click revert.
• Developed MCP and A2A integrations connecting the agents to enterprise applications, APIs, and data platforms.
• Deployed the full stack on AWS: an EC2 app server behind an Application Load Balancer with an Elastic IP and TLS, CloudFront and AWS WAF at the edge, Amazon Cognito sign-in with Lambda@Edge authorization, least-privilege IAM, and Neptune provisioned through CloudFormation, all with repeatable deploy scripts.
• Added an evaluation harness that scores gold questions on document hit rate, entity recall, phrase coverage, and groundedness, and measures p50/p95/p99 latency per pipeline stage, publishable to CloudWatch as a regression guard.

• Delivered all core deliverables (the unstructured-data pipeline, the semantic-search knowledge base, the question-answering agent, and the human-in-the-loop interface), then went past scope with a production deployment, enterprise security, cross-source BI reports with source citations, and full auditability.
• Shipped about 17,000 lines of Python across roughly 100 modules and a 3,400-line React front end, running live on AWS.
• Every answer shows its provenance (tool calls, per-stage timings, and the engine used), and CloudWatch dashboards and a live status page watch the deployment.
• Wrote the architecture diagrams and design docs, then presented and defended the design with a live demo in two internship review rounds.