aws-samples/sample-Agentic-Ai-Data-Operations
Agentic Data Onboarding Platform(ADOP) — an AI-driven system that automates Silver→Bronze→Gold data pipeline lifecycle (ETL scripts, quality checks, Airflow DAGs, semantic layer) from a natural language description, reducing weeks of
ARCHETYPE
MCP Hoarder
Six or more MCP servers wired in. Every tool, always in context.
CONTEXT TAX · EVERY TURN
up to ~36.5k tokens
Claude Code loads MCP tools on demand, so the real cost is usually lower.
Context hog · median rig: 2.3k · breakdown
Copy this rig
# review before running: this installs third-party code $ claude mcp add glue-athena -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -- uv run --no-project --with fastmcp --with boto3 --python 3.13 mcp-servers/glue-athena-server/server.py $ claude mcp add lakeformation -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -- uv run --no-project --with fastmcp --with boto3 --python 3.13 mcp-servers/lakeformation-server/server.py $ claude mcp add sagemaker-catalog -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -- uv run --no-project --with fastmcp --with boto3 --python 3.13 mcp-servers/sagemaker-catalog-server/server.py $ claude mcp add core -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -e FASTMCP_LOG_LEVEL=YOUR_FASTMCP_LOG_LEVEL -- uvx --from awslabs-core-mcp-server awslabs.core-mcp-server $ claude mcp add iam -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -e FASTMCP_LOG_LEVEL=YOUR_FASTMCP_LOG_LEVEL -- uvx --from awslabs-iam-mcp-server awslabs.iam-mcp-server $ claude mcp add lambda -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -e FASTMCP_LOG_LEVEL=YOUR_FASTMCP_LOG_LEVEL -- uvx --from awslabs-lambda-mcp-server awslabs.lambda-mcp-server $ claude mcp add s3-tables -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -e FASTMCP_LOG_LEVEL=YOUR_FASTMCP_LOG_LEVEL -- uvx --from awslabs-s3-tables-mcp-server awslabs.s3-tables-mcp-server $ claude mcp add cloudtrail -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -e FASTMCP_LOG_LEVEL=YOUR_FASTMCP_LOG_LEVEL -- uvx --from awslabs-cloudtrail-mcp-server awslabs.cloudtrail-mcp-server $ claude mcp add redshift -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -e FASTMCP_LOG_LEVEL=YOUR_FASTMCP_LOG_LEVEL -- uvx --from awslabs-redshift-mcp-server awslabs.redshift-mcp-server $ claude mcp add cloudwatch -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -e FASTMCP_LOG_LEVEL=YOUR_FASTMCP_LOG_LEVEL -- uvx --from awslabs-cloudwatch-mcp-server awslabs.cloudwatch-mcp-server $ claude mcp add cost-explorer -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -e FASTMCP_LOG_LEVEL=YOUR_FASTMCP_LOG_LEVEL -- uvx --from awslabs-cost-explorer-mcp-server awslabs.cost-explorer-mcp-server $ claude mcp add dynamodb -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -e FASTMCP_LOG_LEVEL=YOUR_FASTMCP_LOG_LEVEL -- uvx --from awslabs-dynamodb-mcp-server awslabs.dynamodb-mcp-server $ claude mcp add pii-detection -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -- uv run --no-project --with boto3 --with fastmcp --python 3.13 mcp-servers/pii-detection-server/server.py $ npx degit aws-samples/sample-Agentic-Ai-Data-Operations/.claude ./rig-sample-agentic-ai-data-operations # inspect, then merge into .claude/
MCP servers are added to Claude Code at local scope; env vars are shown as YOUR_… placeholders — we never store values. Files are fetched with degit into a separate folder so you can review before merging.
$ claude mcp add glue-athena -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -- uv run --no-project --with fastmcp --with boto3 --python 3.13 mcp-servers/glue-athena-server/server.py $ claude mcp add lakeformation -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -- uv run --no-project --with fastmcp --with boto3 --python 3.13 mcp-servers/lakeformation-server/server.py $ claude mcp add sagemaker-catalog -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -- uv run --no-project --with fastmcp --with boto3 --python 3.13 mcp-servers/sagemaker-catalog-server/server.py $ claude mcp add core -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -e FASTMCP_LOG_LEVEL=YOUR_FASTMCP_LOG_LEVEL -- uvx --from awslabs-core-mcp-server awslabs.core-mcp-server $ claude mcp add iam -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -e FASTMCP_LOG_LEVEL=YOUR_FASTMCP_LOG_LEVEL -- uvx --from awslabs-iam-mcp-server awslabs.iam-mcp-server $ claude mcp add lambda -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -e FASTMCP_LOG_LEVEL=YOUR_FASTMCP_LOG_LEVEL -- uvx --from awslabs-lambda-mcp-server awslabs.lambda-mcp-server $ claude mcp add s3-tables -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -e FASTMCP_LOG_LEVEL=YOUR_FASTMCP_LOG_LEVEL -- uvx --from awslabs-s3-tables-mcp-server awslabs.s3-tables-mcp-server $ claude mcp add cloudtrail -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -e FASTMCP_LOG_LEVEL=YOUR_FASTMCP_LOG_LEVEL -- uvx --from awslabs-cloudtrail-mcp-server awslabs.cloudtrail-mcp-server $ claude mcp add redshift -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -e FASTMCP_LOG_LEVEL=YOUR_FASTMCP_LOG_LEVEL -- uvx --from awslabs-redshift-mcp-server awslabs.redshift-mcp-server $ claude mcp add cloudwatch -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -e FASTMCP_LOG_LEVEL=YOUR_FASTMCP_LOG_LEVEL -- uvx --from awslabs-cloudwatch-mcp-server awslabs.cloudwatch-mcp-server $ claude mcp add cost-explorer -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -e FASTMCP_LOG_LEVEL=YOUR_FASTMCP_LOG_LEVEL -- uvx --from awslabs-cost-explorer-mcp-server awslabs.cost-explorer-mcp-server $ claude mcp add dynamodb -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -e FASTMCP_LOG_LEVEL=YOUR_FASTMCP_LOG_LEVEL -- uvx --from awslabs-dynamodb-mcp-server awslabs.dynamodb-mcp-server $ claude mcp add pii-detection -e AWS_PROFILE=YOUR_AWS_PROFILE -e AWS_REGION=YOUR_AWS_REGION -- uv run --no-project --with boto3 --with fastmcp --python 3.13 mcp-servers/pii-detection-server/server.py
$ npx degit aws-samples/sample-Agentic-Ai-Data-Operations/plugins/claude-code/skills/data-compliance .claude/skills/data-compliance $ npx degit aws-samples/sample-Agentic-Ai-Data-Operations/plugins/claude-code/skills/decision-engine .claude/skills/decision-engine $ npx degit aws-samples/sample-Agentic-Ai-Data-Operations/plugins/claude-code/skills/discovery-protocol .claude/skills/discovery-protocol $ npx degit aws-samples/sample-Agentic-Ai-Data-Operations/plugins/claude-code/skills/observability .claude/skills/observability $ npx degit aws-samples/sample-Agentic-Ai-Data-Operations/plugins/claude-code/skills/pipeline-conventions .claude/skills/pipeline-conventions
$ curl -fsSL --create-dirs -o .claude/agents/dag-agent.md https://raw.githubusercontent.com/aws-samples/sample-Agentic-Ai-Data-Operations/main/.claude/agents/dag-agent.md $ curl -fsSL --create-dirs -o .claude/agents/iac-agent.md https://raw.githubusercontent.com/aws-samples/sample-Agentic-Ai-Data-Operations/main/.claude/agents/iac-agent.md $ curl -fsSL --create-dirs -o .claude/agents/metadata-agent.md https://raw.githubusercontent.com/aws-samples/sample-Agentic-Ai-Data-Operations/main/.claude/agents/metadata-agent.md $ curl -fsSL --create-dirs -o .claude/agents/ontology-agent.md https://raw.githubusercontent.com/aws-samples/sample-Agentic-Ai-Data-Operations/main/.claude/agents/ontology-agent.md $ curl -fsSL --create-dirs -o .claude/agents/quality-agent.md https://raw.githubusercontent.com/aws-samples/sample-Agentic-Ai-Data-Operations/main/.claude/agents/quality-agent.md $ curl -fsSL --create-dirs -o .claude/agents/transformation-agent.md https://raw.githubusercontent.com/aws-samples/sample-Agentic-Ai-Data-Operations/main/.claude/agents/transformation-agent.md
Merge into .claude/settings.json (project) or ~/.claude/settings.json (user). Hook commands reference scripts in the source repo — copy those too.
{
"hooks": {
"PreToolUse": [
{
"matcher": "Edit|Write",
"hooks": [
{
"type": "command",
"command": ".claude/hooks/check-discovery-gate.sh"
},
{
"type": "command",
"command": ".claude/hooks/check-logging.sh"
}
]
},
{
"matcher": "Write|Edit|MultiEdit",
"hooks": [
{
"type": "command",
"command": "python3 .claude/hooks/enforce_template_codegen.py"
}
]
}
]
}
} MCP servers (13)
| server | source | est. tokens |
|---|---|---|
| fastmcp · "glue-athena" env: AWS_PROFILE, AWS_REGION | pypi | 2.5k |
| fastmcp · "lakeformation" env: AWS_PROFILE, AWS_REGION | pypi | 2.5k |
| fastmcp · "sagemaker-catalog" env: AWS_PROFILE, AWS_REGION | pypi | 2.5k |
| awslabs-core-mcp-server · "core" env: AWS_PROFILE, AWS_REGION, FASTMCP_LOG_LEVEL | pypi | 2.5k |
| awslabs-iam-mcp-server · "iam" env: AWS_PROFILE, AWS_REGION, FASTMCP_LOG_LEVEL | pypi | 2.5k |
| awslabs-lambda-mcp-server · "lambda" env: AWS_PROFILE, AWS_REGION, FASTMCP_LOG_LEVEL | pypi | 2.5k |
| awslabs-s3-tables-mcp-server · "s3-tables" env: AWS_PROFILE, AWS_REGION, FASTMCP_LOG_LEVEL | pypi | 2.5k |
| awslabs-cloudtrail-mcp-server · "cloudtrail" env: AWS_PROFILE, AWS_REGION, FASTMCP_LOG_LEVEL | pypi | 2.5k |
| awslabs-redshift-mcp-server · "redshift" env: AWS_PROFILE, AWS_REGION, FASTMCP_LOG_LEVEL | pypi | 2.5k |
| awslabs-cloudwatch-mcp-server · "cloudwatch" env: AWS_PROFILE, AWS_REGION, FASTMCP_LOG_LEVEL | pypi | 2.5k |
| awslabs-cost-explorer-mcp-server · "cost-explorer" env: AWS_PROFILE, AWS_REGION, FASTMCP_LOG_LEVEL | pypi | 2.5k |
| awslabs-dynamodb-mcp-server · "dynamodb" env: AWS_PROFILE, AWS_REGION, FASTMCP_LOG_LEVEL | pypi | 2.5k |
| boto3 · "pii-detection" env: AWS_PROFILE, AWS_REGION | pypi | 2.5k |
Skills (5)
Subagents (6)
| dag-agent model: sonnet | Builds the DAG spec and renders the workload's Airflow DAG via the deterministic codegen renderer. Spawned by the Data Onboarding Agent at Step 4.5. Produces dags/{workload}_dag.py plus unit and integ |
| iac-agent model: opus | Converts a built workload's artifacts (Glue PySpark, MWAA DAG, YAML configs, quality rules, Cedar policies) into deployable Terraform / CDK Python / CDK TypeScript / CloudFormation plus an APPLY_GUIDE |
| metadata-agent model: opus | Extracts, infers, classifies and catalogs metadata for a workload's data source. Spawned by the Data Onboarding Agent in Phase 3 (profiling) and Step 4.2 (formalize catalog). Produces config/source.ya |
| ontology-agent model: opus | Induces an OWL2 ontology and R2RML mappings from a workload's semantic.yaml plus its Glue Gold-zone schema, validates the Turtle, and stages three artifacts locally for AWS Semantic Layer handoff. Spa |
| quality-agent model: opus | Builds the quality spec (5 dimensions, rules, gates, anomaly detection) and renders quality check scripts via the deterministic codegen renderer. Spawned by the Data Onboarding Agent at Step 4.4. Prod |
| transformation-agent model: opus | Produces Bronze/Silver/Gold transform specs and renders AWS Glue ETL (PySpark + Iceberg) scripts via the deterministic codegen renderer. Spawned by the Data Onboarding Agent at Step 4.3. Produces conf |
Hooks (5)
| event | matcher | runs |
|---|---|---|
| PreToolUse | Edit|Write | .claude/hooks/check-discovery-gate.sh |
| PreToolUse | Edit|Write | .claude/hooks/check-logging.sh |
| PreToolUse | Write|Edit|MultiEdit | python3 .claude/hooks/enforce_template_codegen.py |
| SessionStart | * | python3 .claude/hooks/inject_run_context.py |
| PostToolUse | AskUserQuestion | python3 .claude/hooks/log_conversation.py |
Slash commands (3)
/devops-workflow/onboard-workflow/security-review
Similar rigs
surajrimal07/NepseAPI-Unofficial
Unlock Nepal Stock Exchange (NEPSE) data with this powerful unofficial API. Features REST, WebSocket, and a first-of-its-kind Model Context Protocol (MCP) server, enabling AI models like Claude to perform live market analysis with over 20 s
Pragmatist 2.5k tok · · Q0
Spazzy757/dotfiles
Dotfile collection for my local environment
Minimalist 2.3k tok · · Q5
newrelic-experimental/preflight
Observability for AI coding assistants — cost tracking, efficiency scoring, and anti-pattern detection for Claude Code, Cursor, Copilot, and more. Local-first; New Relic optional.
Pragmatist 10.2k tok · · Q10
trotor/metsainfo
Suomen metsävaratietojen karttasovellus ja MCP-palvelin tekoälytyökaluille. Forest data map viewer and MCP server for AI tools.
Pragmatist 7.2k tok · · Q0