Agent Planning Data
Structured Planning Data
Structuring complex task planning into directed acyclic graphs (DAGs) through human annotation, generating high-quality training data for planner models' task decomposition and multi-agent coordination capabilities.
Demo Cases
Interactive DAG visualizations from our annotated dataset
Build a full-stack todo web application with authentication, database, and deployment.
Click any node to view detailed annotation
| From | To | Data Flow |
|---|---|---|
| root | arch | parsed_requirements |
| arch | db | tech_stack_config |
| arch | auth | auth_requirements |
| arch | api | api_spec |
| db | fe | schema_types |
| auth | fe | auth_context |
| db | integrate | db_client |
| auth | integrate | auth_middleware |
| api | integrate | route_handlers |
| fe | test | components |
| integrate | test | api_endpoints |
| test | deploy | test_results |
From Todo Lists to DAGs
Why linear planning tools are no longer enough
Linear Todo List
- Models need frequent reminders to stay on track (system reminder every 5 turns)
- Fixed checklists limit the model's ability to autonomously adjust
- Sub-agents cannot share coordination state
- Linear structure cannot express parallelism or dependencies
DAG Task Graph
- Dependencies between tasks are explicitly modeled (directed edges)
- Supports parallel execution of independent subtasks
- Agents can pass context and status updates through edges
- Models can dynamically add or remove nodes, adjusting plans flexibly
Reference: Anthropic's Claude Code team found that as model capabilities improved, the linear TodoWrite tool became a bottleneck. They introduced the Task Tool with dependency support and cross-agent communication (DAG structure), significantly improving multi-agent collaboration.
View original post
What We Annotate
DAG Topology
Complete graph structures with nodes (subtasks) and directed edges (dependencies)
Node Metadata
Agent roles, tool invocations, input/output descriptions for each node
Execution Semantics
Parallel groups, critical paths, data flow direction, and error handling strategies
Annotation Workflow
Task Collection & Design
- Collect real user queries
- Filter for complex tasks suitable for DAG decomposition (exclude single-step tasks)
- Design annotation templates: node fields + edge fields + metadata
DAG Structure Annotation
- Annotators decompose queries into subtask nodes (with agent roles, tools, descriptions)
- Annotate inter-node dependencies (directed edges), ensuring acyclicity
- Annotate parallelism: which nodes can execute concurrently
- Annotate data flow direction for each edge (input/output mapping)
Quality Review
- Cross-review: another annotator independently reviews DAG structural validity
- Automated validation: cycle detection, connectivity checks, orphan node detection
- Consistency assessment: compare two annotators' DAGs, compute structural similarity
- Expert arbitration: senior annotator resolves disagreements
Data Augmentation & Training
- Convert annotated DAGs to JSON format training data
- Data augmentation: generate multiple valid DAG variants for the same query
- Train Planner models: input query → output DAG structure
- Evaluation: compare with ground-truth DAGs for topological similarity & execution success rate
Annotation Guidelines
Node Granularity
Each node should correspond to an independently completable subtask, roughly equivalent to one agent tool call
Edge Semantics
Edges represent data/control dependencies, must annotate data flow direction, no redundant edges allowed
Parallel Annotation
Independent nodes should be marked as parallelizable and explicitly declared in parallel_groups
Agent Assignment
Each node specifies an appropriate agent role and available tool set, ensuring executability
Data Format Example
{
"query": "Build a full-stack todo app with auth and deploy",
"dag": {
"nodes": [
{ "id": "arch", "label": "Architecture Design", "agent": "Architect", "tools": ["SequentialThinking"] },
{ "id": "db", "label": "Database Design", "agent": "DB-Agent", "tools": ["Bash","Write"] },
{ "id": "auth", "label": "Auth Module", "agent": "Auth-Agent", "tools": ["Write","Read"] }
],
"edges": [
{ "from": "arch", "to": "db", "data_flow": "tech_stack_config" },
{ "from": "arch", "to": "auth", "data_flow": "auth_requirements" }
],
"parallel_groups": [["db", "auth", "api"]],
"critical_path": ["arch", "db", "fe", "test", "deploy"]
},
"metadata": { "annotator": "expert_03", "time_spent_min": 11, "difficulty": "medium" }
}