Project Automation Vision
Vision
A fully reliable, well-documented agentic project automation workflow where projects move smoothly through all phases, every review artifact is persisted to files, human review happens at the correct gates, project names are globally conflict-free, and project execution data is captured well enough to support review, optimization, and future model-selection work.
Requirements
R1: Full Workflow Audit
Every component of the project automation workflow must be audited for correctness and completeness: the Project Flow lifecycle definition, all project skills, all project agent prompts, the shared Project Snippet and Env Snippet, all document templates, and related knowledge docs. Gaps, inconsistencies, and missing steps must be identified and fixed.
R2: Formal Evaluation Phase Lifecycle
The project lifecycle must use formal evaluation phases after the appropriate content-producing phases. The canonical lifecycle is:
Create → Vision → Vision_Eval → Research → Spec → Spec_Eval → Plan → Plan_Eval → Execute → Execute_Eval → Verify → Extract → Extract_Eval → Complete
The lifecycle definition, Phase naming convention, Project Snippet, Project Runner routing, Task Runner routing, and agent prompts must all reflect this as the formal project flow.
R3: Persisted Review and Evaluation Reports
Every review- or evaluation-producing phase must save its output to a file. This includes Vision_Eval, Spec_Eval, Plan_Eval, Eval, Verify, Extract_Eval, workflow audit results, and any other structured project review artifact. Re-runs may append to an existing file when that is the correct workflow, but the result must always be linkable and reviewable as a durable project artifact.
R4: Complete Work Products
Every phase must produce all required work products. Every step in every skill must be actionable and ordered. No required file creation, report persistence, sidebar update, build, commit, or push may be missing from any skill procedure.
R5: Smooth Agent Handoffs
Handoff between agent runs must be seamless. When Project Runner dispatches Project Thinker for Vision, Spec, or Plan work, when evaluators hand findings back to the previous phase, and when Project Worker starts Execute after Plan approval, each transition must leave the project in a state the next agent can pick up without confusion or missing context.
R6: Human-in-the-Loop at Correct Points
The workflow must invoke human review at precisely the right moments: after Vision drafting and Vision_Eval, after each Research decision, after Spec drafting and Spec_Eval, after Plan drafting and Plan_Eval, after each Verify run, and anywhere else the system requests a human decision. The system must not proceed past these gates without human approval, and must make it easy for the human to see what needs review.
R7: Easy Human Review and Feedback
The human must be able to view results and give feedback easily. This includes clear presentation of drafts for review, visible pending approval states, Erik_Todo entries that link directly to the persisted report or document needing review, and stable file targets for later re-review.
R8: User Guide
A clear, complete User Guide must be produced describing the entire project automation flow for human consumption. It must explain each phase, what the agents do, when the human is involved, what documents exist, what reports are produced, and how to monitor and steer a project through its lifecycle.
R9: Forge Optimizer Integration
The Forge Optimizer must be integrated into the project workflow to review project agent executions and produce actionable improvements to skills and prompts. A process must exist to run the Optimizer after significant agent runs, surface findings, and implement approved changes.
R10: Model Selection System [Deferred / Not required in this iteration]
The full model-selection system is deferred for a future iteration. This project does not need to implement LangFuse CLI installation, A/B test execution, or the scoring rubric. It does need to preserve the foundations those future systems require: persisted review files, project-level run tracking, and stable report links.
R11: Agent Run Summary
The user must be able to get a summary of all agent runs involved in a project’s lifecycle. For each conversation (agent run), the summary must include: which agent/model was used, the task/message it was given, how long the run took, token usage per turn and in total, and the cost of that conversation. The summary must also compute the total cost across all conversations for the project.
R12: Global Project Name Uniqueness
Project_Create must ensure that a new project name never conflicts with any existing project anywhere in the repository, including projects in other departments and projects already moved to Archived folders. The check must be done before creating the project, and a conflict must stop creation with a clear message to the user.
Success Criteria
| # | Criterion | Measured by |
|---|---|---|
| V1 | Every project skill, agent prompt, template, and lifecycle doc is audited and all gaps documented | Complete gap list with file references |
| V2 | The lifecycle is formalized as Create → Vision → Vision_Eval → Research → Spec → Spec_Eval → Plan → Plan_Eval → Execute → Execute_Eval → Verify → Extract → Extract_Eval → Complete | Lifecycle string matches across canonical files and routing follows it |
| V3 | Every review/evaluation phase writes to a durable file artifact | Erik_Todo links and project docs reference saved report files |
| V4 | Every skill procedure has all required steps with no missing build/commit/push/sidebar/report updates | Skill-by-skill verification checklist passes |
| V5 | A project can move through the full lifecycle via agent handoffs without manual intervention between phases except at explicit human gates | End-to-end dry run succeeds |
| V6 | Human gates exist at all required points and the system stops at each one | Gate checklist matches workflow |
| V7 | Pending human items are surfaced clearly and the human can review and approve/reject using linked project artifacts | Erik_Todo entries link to stable files |
| V8 | User Guide is complete, accurate, and usable by a human unfamiliar with the system | Independent readability review |
| V9 | Forge Optimizer can be invoked on project agent executions and findings lead to implemented improvements | At least one Optimizer run on a project agent produces actionable, implemented changes |
| V10 | Deferred model-selection work is explicitly documented while required data/reporting foundations are preserved | Vision, Research, and Spec all state the deferment and the preserved foundations |
| V11 | Agent Run Summary can be generated for any project, listing all conversations with model, message, duration, tokens per turn and total, cost per conversation, and total project cost | Summary output matches schema; totals are accurate |
| V12 | Project_Create blocks duplicate names across all departments, including Archived projects | Conflict test fails cleanly before any files are created |
Out Of Scope
- Building a UI/dashboard for project visualization
- Automated phase transitions without human approval at gates
- Integration with external project management tools (Jira, etc.)
- Rewriting agent infrastructure or MCP servers beyond the minimal changes required by this project
- Real-time monitoring dashboards for agent execution
- Full model-selection system implementation (LangFuse CLI installation, A/B runner, rubric scoring)
- Multi-repo project support