millitool
tools/

data-pipeline

Design and implement a scalable data pipeline for: $ARGUMENTS

datapipeline
Content Preview
---
model: claude-sonnet-4-0
---

# Data Pipeline Architecture

Design and implement a scalable data pipeline for: $ARGUMENTS

Create a production-ready data pipeline including:

1. **Data Ingestion**:
   - Multiple source connectors (APIs, databases, files, streams)
   - Schema evolution handling
   - Incremental/batch loading
   - Data quality checks at ingestion
   - Dead letter queue for failures

2. **Transformation Layer**:
   - ETL/ELT architecture decision
   - Apache Beam/Spark transfor
How to Use

Recommended: Install to project (local)

mkdir -p .claude/skills
curl -o .claude/skills/data-pipeline.md \
  https://raw.githubusercontent.com/wshobson/commands/main/tools/data-pipeline.md

Skill is scoped to this project only. Add .claude/skills/ to your .gitignoreif you don't want to commit it.

Alternative: Clone full repo

git clone https://github.com/wshobson/commands

Then reference at tools/data-pipeline.md

Related Skills

all-tools
Display All Available Development Tools
commandsalltools

by qdhenry · claude-command-suite

all_tools
List all available tools detailed in your system prompt. Display them in bullet points. Display them in typescript function signature format and suffix the purpose of the tool. Double line break between each tool for readability.
commandsalltools

by disler · disler-hooks-mastery

data-normalization-tool
This skill provides automated assistance for data normalization tool tasks within the ML Tr
data-normalization-tooldatanormalizationtool

by jeremylongshore · plugins-plus-skills

pinecone
Pinecone vector database integration. Streamline your Pinecone development with powerful tools for managing vector indexes, querying data, and rapid prototyping. Use slash commands like /quickstart to generate AGENTS.md files and initialize Python projects and /query to quickly explore indexes. Access the Pinecone MCP server for creating, describing, upserting and querying indexes with Claude. Perfect for developers building semantic search, RAG applications, recommendation systems, and other vector-based applications with Pinecone.
pluginpluginmarketplacepinecone

by Anthropic · anthropic-official-plugins