Skip to content
Uvilo OS
published Visibility internal Owner erik@uvilo.com Approver _ Created _ Updated _

Project Execute Eval

When to Use

A Plan’s execution is complete (all State items checked) and needs evaluation.

Inputs

  • Project path: {Dept}/Projects/{Project}/
  • Which Plan to evaluate (by Plan number N)

Context Boundary

Read only the current Plan, current Execute_State, Learnings, Changelog, and (when re-evaluating) the previous Execute_Eval report. Never read Vision, Research, Spec, or README. If context is missing from the Plan, that is a Plan authoring failure — note it in the evaluation but do not consult upstream documents.

Prerequisite

HARD GATE: If you are in the same session that executed this Plan, STOP. Tell the user: “Evaluation requires a fresh session — please start a new conversation and run Project Execute Eval there.” Do NOT proceed with evaluation.

Procedure

  1. Read: current Plan, current Execute_State, Learnings
  2. Update {Project}_Phase.md: set Phase: Execute_Eval_{N}_Started
  3. Build, commit, push
  4. Review the execution against the Plan:
    • Compare actual implementation to Plan intent — are all tasks truly complete, correct, and free of defects?
    • Use the Changes table in Changelog as a checklist — confirm every changed file aligns with Plan intent and no unexpected files were modified
  5. Write findings to the per-plan report file {Project}_Execute_Eval_{N}.md by spawning the Page Manager bot via spawnAgent with { botGroup: 'forge', botHandle: 'page-manager' } (create the page, add slug to sidebar, build, commit, push — atomically):
    • Use the Execute_Eval_Template.md from Forge/Forge_Project/Skills/Project_Create/templates/
    • If the file already exists from a previous run, append the new run as a new section with a ## Run — {date} header (edit the file directly; sidebar entry already exists)
    • No issues found: Use the ✅ Clean single-row format — one row with ✅ Clean type and 🟢 Resolved status. Do NOT list empty categories like “No defects”, “No gaps”, etc.
    • Issues found: Use the multi-row table — only include rows for types that have issues. Omit empty categories entirely.
    • Each issue item is one of:
      • Defect — execution doesn’t match the Plan’s intent
      • Gap — the Plan specified something that wasn’t implemented
      • Variance — execution differs from Plan for a reason
      • Learning — entry from Learnings that needs permanent storage
    • Issue items start as 🔴 Open
  6. Auto-fix loop: If findings exist, attempt to resolve them automatically before escalating to the user:
    • For straightforward fixes (clear defects, obvious gaps), apply the fix disposition directly
    • Re-evaluate after fixes; iterate up to 3 rounds
    • Only escalate to the user when the Thinker and Evaluator cannot converge (ambiguous issues, trade-off decisions, or after 3 rounds)
    • Log auto-dispositions in the Resolution Log with note “Auto-dispositioned”
  7. If auto-fix loop converged (no remaining open findings): set Verdict to Approved. Otherwise: Present remaining open findings to the user. For each item, the user decides: fix, update, reject, or defer
  8. Process user dispositions and update item statuses in the Execute_Eval report:
    • fix: Add a corrective task to {Project}_Execute_State_{N}.md (using the Execute_Eval item ID as the task label, e.g. E1 — Fix the Reject disposition description), marked [ ] ⬜. Item remains 🔴 Open with Disposition Fix.
    • update: Don’t fix the implementation; update the Spec to match instead. Add a task to {Project}_Execute_State_{N}.md to update the Spec, marked [ ] ⬜. Item 🟡 In progress → 🟢 Resolved when done.
    • reject: I disagree with the finding; do nothing. Item → 🟢 Resolved (no action).
    • defer: Add a task to the next Plan’s Execute_State (or note it in the next Plan if that State doesn’t yet exist), marked [ ] ⬜. Item → 🟢 Resolved (deferred).
  9. If any items have Disposition Fix or Update: set Verdict to Rejected. The project returns to Project_Execute for the same Plan to implement the tasks.
  10. If no items have Disposition Fix or Update: set Verdict to Approved (all remaining items resolved or deferred).
  11. Update the Execute_Eval report after each iteration (statuses and Resolution Log)
  12. If later Plans exist and are affected by changes, revise them (Plans are only revised before their execution begins)
  13. Build, commit, push
  14. If Verdict is Approved:
    • Set current Plan and Execute_State frontmatter status to approved
    • Set the Execute_Eval report’s frontmatter status to approved
    • Update {Project}_Phase.md: set Phase: Execute_Eval_{N}_Completed
    • Build, commit, push
  15. If Verdict is Rejected:
    • Incorporate any Learnings entries into their Destination documents, then clear the Learnings file
    • Update {Project}_Phase.md: set Phase: Execute_{N}_Started (returns to Execute for fixes)
    • Build, commit, push

Report File Conventions

  • One report file per Plan: {Project}_Execute_Eval_{N}.md in the project folder
  • First evaluation creates the file from the template
  • Subsequent evaluations of the same Plan (after re-execution) append as a new ## Run — {date} section
  • Each run section includes: Verdict, Implementation Summary table, Items table, Resolution Log, Notes
  • The report file is a durable artifact — it persists across the entire Execute/Execute_Eval cycle

Rules

  • Plans are frozen once the next Plan’s execution starts — during a Plan’s own Execute/Execute_Eval cycle, the Plan can be updated
  • Accepted deviations update the current Plan via the “update” disposition. Vision/Spec may also be updated if a Learning explicitly calls for it, but not because a partial execution doesn’t yet cover them.
  • Learnings incorporation is done during re-execution — Execute_Eval only notes which entries need incorporation
  • Evaluation must be a separate agent session from the execution
  • Failed Execute_Eval returns directly to Execute — there is no separate Fix phase
  • The report file is the single source of truth for evaluation findings — never duplicate it inline in State or chat
  • Every todo item created for the user that references a work-product must embed the work-product as an artifact block inside the todo body so the user can open it directly from the todo item. Do not include raw URLs, repo paths, or paste instructions. The user records their verdict in the work-product; the system picks it up from there.