Project Automation Extract Eval

Evaluation of knowledge extraction for Project Automation.

---

Run — 2026-07-25

Fresh extraction review after Plans 11–12 (Agent Handoff edges, Forge Chat routing, dual-mode Project Runner, stale schedule references fixed).

Verdict

Approved

Extraction Summary

#	Extraction	Destination	Status
1	Agent Handoff edges + Forge Chat routing rules	Forge/Forge_Agent_Orchestration.md → ## Agent Handoffs section	✅
2	Interactive vs Autonomous Mode (arch)	Forge/Forge_Agent_Orchestration.md → ### Interactive vs Autonomous Mode	✅
3	Interactive vs Autonomous Mode (user-facing)	Forge/Forge_Project_User_Guide.md → ### Interactive vs Autonomous Mode	✅
4	SRP Agents table descriptions	Forge/Forge_Agent_Orchestration.md → SRP Agents table	✅
5	Stale LangFuse/model-selection references	Forge/Projects/Project_Automation/AGENTS.md → removed	✅
6	Empty WIP file	Trashed via uvilo-trash, sidebar entry removed	✅

Items

#	Type	Description	Status	Disposition
—	✅ Clean	All extraction complete and verified	🟢 Resolved	—

Verification Checks

1. Content at correct permanent locations — All 6 extractions verified on disk. Forge_Agent_Orchestration.md has ## Agent Handoffs with routing rules, ### Interactive vs Autonomous Mode, and updated SRP Agents table. Forge_Project_User_Guide.md has user-facing Interactive vs Autonomous section. agent-sync.ts has edges on Forge Chat family pointing to Project Runner. Forge_Chat_Prompt.md has ## PROJECT ROUTING section. ✅
2. Self-contained — No dangling references to the project folder in any permanent doc. ✅
3. No useful information trapped — Remaining project files are historical artifacts (Vision, Spec, Plans, Evals, State files, Research, Summary, Learnings, Audit Results, Runs). All generalizable knowledge extracted. ✅
4. No duplication — Orchestration doc is canonical architecture reference; User Guide section is complementary user-facing explanation. No redundant content. ✅
5. Skills listed in Forge_Chat_Prompt.md — All 47 skill names in prompt match directories on disk (one pre-existing discrepancy: Commands listed but no Commands/ skill directory — Create_Command is the actual skill. Not introduced by this extraction round.) ✅
6. Department-level knowledge linked from Forge/AGENTS.md — User Guide, Workflow, and Orchestration docs all linked. No stale references. ✅
7. Stale name references — No Project_Fix, Project_Evaluate, Evaluation_Template, Spec_Audit, Plan_Audit, Requirements_Eval, LangFuse, model-selection, or A/B test in permanent docs. ✅
8. Cron schedule — Forge_Agent_Orchestration.md Cron Scheduling table shows Project Runner as “Hourly” (not stale “Once daily”). ✅

Previous Extractions (already approved)

Extraction	Destination	Status
Project lifecycle & phase model	Forge_Project_Workflow.md, Forge_Chat_Prompt.md, Project_Snippet.md	✅
Eval skills (Vision_Eval, Spec_Eval, Plan_Eval, Execute_Eval, Extract_Eval)	Forge/Skills/Project_{phase}_Eval/	✅
Eval templates	Forge/Skills/Project_Create/templates/	✅
User Guide	Forge/Forge_Project_User_Guide.md	✅
Forge Optimizer project awareness	Forge/Skills/Forge_Optimizer/	✅
Agent Run Summary (generate-runs.ts)	Forge/Skills/Forge_Optimizer/scripts/	✅
Agent prompts	Forge/Configs/Agents/	✅
Project_Create name uniqueness	Forge/Skills/Project_Create/	✅
forge-discovery index.ts	Forge/Configs/MCP_Servers/forge-discovery/src/	✅
Lifecycle canonical string	Forge_Chat_Prompt.md, Project_Snippet.md	✅
AgentJob project and phase columns	Forge_Agent_Orchestration.md (AgentJob table + Spawn Tool Parameters table)	✅

---

Run — 2026-07-27

Fresh-session independent evaluation of Plan 12 re-extraction quality. Prior run (2026-07-25) covered Plans 1–12 and passed. This run verifies the re-extraction for Plan 12 specifically.

Verdict

Approved

Plan 12 Contributions Verified

#	Contribution	Permanent Location	Verified
1	Cron schedule: Project Runner = “Hourly” (not stale)	Forge_Agent_Orchestration.md → Cron Scheduling table	✅
2	Agent Handoffs architecture	Forge_Agent_Orchestration.md → ## Agent Handoffs (line 246)	✅
3	Interactive vs Autonomous Mode (architectural)	Forge_Agent_Orchestration.md → ### Interactive vs Autonomous Mode (line 272)	✅
4	Interactive vs Autonomous Mode (user-facing)	Forge_Project_User_Guide.md → ### Interactive vs Autonomous Mode (line 242)	✅
5	Agent Handoff edges config	agent-sync.ts → Forge Chat family edges → agent_yG6v4v0wfi5EEHQ8htTXH with Interactive Mode passthrough	✅
6	Forge Chat PROJECT ROUTING rules	Forge/Configs/Agents/Forge_Chat_Prompt.md → ## PROJECT ROUTING (line 212)	✅
7	Forge/AGENTS.md links	Links to Forge_Agent_Orchestration.md and Forge_Project_User_Guide.md	✅

Verification Checks

#	Check	Result
1	Completeness — All Plan 12 content at permanent locations	✅
2	Self-contained — No dangling references to Forge/Projects/Project_Automation/ in permanent docs	✅
3	No trapped knowledge — Project folder contains only historical artifacts (Vision, Spec, Plans, Evals, State, Research, Summary, Learnings, Audit, Runs)	✅
4	No duplication — Orchestration doc is technical architecture; User Guide is complementary user-facing explanation. Same concept, different audiences, different wording.	✅
5	Findability — system-wide — All 47 skills in Forge_Chat_Prompt.md match disk directories (pre-existing Commands vs Create_Command discrepancy; not from this extraction)	✅
6	Findability — department-level — Forge/AGENTS.md links to Orchestration doc, User Guide, and Workflow doc. All status: published.	✅
7	No stale names — No Project_Fix, Project_Evaluate, Evaluation_Template, Spec_Audit, Plan_Audit, Requirements_Eval, LangFuse, model-selection, or A/B test in permanent (non-archived, non-project) docs	✅
8	Cron schedule — Forge_Agent_Orchestration.md Cron Scheduling table shows `Project Runner	Hourly` (not stale “Once daily”)

Notes

• Project AGENTS.md lists Project_Eval which no longer exists as a standalone skill (eval skills are now split into Project_Vision_Eval, Project_Spec_Eval, etc.). This is a project-internal artifact, not a permanent doc. Non-blocking — stays with the project folder as historical record.
• Commands skill name in Forge_Chat_Prompt.md skills list has no matching Commands/ directory (Create_Command is the actual skill). Pre-existing discrepancy carried from prior eval. Non-blocking.
• Archived references: Project_Fix, Project_Evaluate etc. appear only in Forge/Archived/Forge_Project/ — not in active permanent docs. Acceptable.

Disposition

All Plan 12 extraction verified. No findings requiring auto-fix. Extraction passes.