# Data Engineering

> Equips your coding agent with a data engineer, pipeline commands and skills for Airflow, dbt, Spark and data quality.

Data Engineering is a plugin from Seth Hobson's agents collection (wshobson/agents), a community project. It adds two subagents, two commands and four skills for building data pipelines: a data engineer for pipelines, warehouses and streaming, a backend architect for the services around them, a pipeline-design command, a step-by-step data-driven feature workflow, and skills for Airflow, dbt, Spark and data quality.

What's inside
- data-engineer: designs batch and streaming pipelines, warehouses and lakehouses across Spark, dbt, Airflow, Kafka and the cloud data services, with data quality checks, monitoring and cost in mind.
- backend-architect: designs the APIs, services and event-driven architecture around your data.
- /data-pipeline: designs and implements a batch or streaming pipeline, from ingestion and orchestration to dbt transformations, data quality checks and monitoring.
- /data-driven-feature: takes a feature from exploratory analysis and an A/B test design through instrumentation, build and gradual rollout to statistical analysis, stopping for your approval between phases.
- airflow-dag-patterns (skill): DAG design, operators, sensors, testing and deployment for Apache Airflow.
- dbt-transformation-patterns (skill): model layers, naming, tests, documentation and incremental models in dbt.
- spark-optimization (skill): partitioning, caching, shuffles and memory tuning for slow Spark jobs.
- data-quality-frameworks (skill): validation with Great Expectations, dbt tests and data contracts.

How to install
In Claude Code, run /plugin marketplace add wshobson/agents, then /plugin install data-engineering@claude-code-workflows. Codex CLI uses codex plugin marketplace add wshobson/agents. The repository also has install routes for Cursor, OpenCode, Antigravity CLI, GitHub Copilot and Pi.

Good to know
- It's a community plugin, not a data vendor's.
- It doesn't connect to your warehouse or pipelines; it guides your agent's design and code.
- /data-driven-feature keeps its working files in a .data-driven-feature folder in your project.

- Type: Plugin
- Made by: wshobson
- Category: Data & analytics
- Formats: Claude plugins
- Version: 1.3.2
- Website: https://github.com/wshobson/agents
- Source: https://github.com/wshobson/agents/tree/HEAD/plugins/data-engineering
- Page: https://pluginboard.ai/plugins/data-engineering

## Works with

- Claude
- GitHub Copilot
- Claude Code
- OpenClaw

## Install

### Claude

In Claude (web or desktop): Customize → Plugins → Add → Add marketplace, enter wshobson/agents, then add data-engineering

### Claude Code

In Claude Code:

```text
/plugin marketplace add wshobson/agents
/plugin install data-engineering@claude-code-workflows
```

---
From Pluginboard (https://pluginboard.ai).