Skip to content
Uvilo OS
archived Visibility internal Owner erik@uvilo.com Approver _ Created 2026-05-19 Updated 2026-05-25

Orchestration Plan 1

Scope: Empirical investigation of LibreChat’s Agents API and Agent Handoffs — determine what actually works, what parameters are supported, how long-running calls behave, and whether the API is viable as the delegation mechanism for the Supervisor pattern.

Research: Orchestration Research

Prior plan: None

Task 1 — Enable and Configure the Agents API

The Agents API must be enabled before any endpoint can be tested. The remoteAgents interface config must be set in librechat.yaml and an API key generated.

  1. Read the current Forge/Configs/LibreChat_Service/librechat.yaml
  2. Add the remoteAgents interface configuration: 
    interface:
  remoteAgents:
    use: true
    create: true
  3. Redeploy LibreChat (or restart the service) so the config takes effect
  4. Log in to LibreChat as admin, navigate to the API key management section, and generate an API key for the Agents API
  5. Store the API key in Railway environment variables as LIBRECHAT_AGENTS_API_KEY
  6. Verify the endpoint is reachable: 
    curl -X GET https://librechat.uvilo.ai/api/agents/v1/models \
  -H "Authorization: Bearer $LIBRECHAT_AGENTS_API_KEY"
  7. Record the full response — this lists all available agents as models and reveals the agent ID format (MongoDB ObjectId? slug? custom string?)

Deliverable: Agents API enabled, API key stored, agent ID format documented.

Task 2 — Map the Full Chat Completions API Surface

The docs show only model, messages, and stream. We need to know every parameter the endpoint actually accepts, what it returns, and whether it supports the operational features orchestration requires (conversation ID, title, model overrides, structured output).

  1. Send a minimal Chat Completions request to confirm basic functionality: 
    curl -X POST https://librechat.uvilo.ai/api/agents/v1/chat/completions \
  -H "Authorization: Bearer $LIBRECHAT_AGENTS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<agent_id_from_task_1>",
    "messages": [{"role": "user", "content": "Hello, respond with just the word OK"}],
    "stream": false
  }'
  2. Record the full response — note every field in the response object, especially:
    • Is there a conversation ID? What field? What format?
    • Is there usage/token data?
    • What does finish_reason say?
    • Is there any metadata beyond the standard OpenAI format?
  3. Test streaming mode with the same request but "stream": true. Capture the SSE event types and data shapes. Note whether intermediate tool-call events are streamed or only the final response.
  4. Test additional parameters that the OpenAI Chat Completions format supports, one at a time, recording which are accepted vs. ignored:
    • temperature — can you override the agent’s configured temperature?
    • max_tokens / max_completion_tokens
    • response_format — can you request JSON mode or structured output?
    • stop
    • top_p
    • Any custom/non-standard parameters
  5. Check LibreChat’s source code for the Chat Completions route handler. Clone or browse danny-avila/LibreChat and find the endpoint handler (likely under api/server/routes/ or similar). Document the actual parameter schema — what the code validates and accepts vs. what the docs claim.

Deliverable: Complete parameter map for Chat Completions endpoint — accepted params, ignored params, response shape, streaming event types.

Task 3 — Map the Full Responses API Surface

The Responses endpoint is LibreChat’s “future direction” per the docs. It follows the Open Responses specification. We need to understand its parameter surface, which may differ from Chat Completions.

  1. Read the Open Responses specification at https://www.openresponses.org/ — document the full parameter surface that the spec defines
  2. Send a minimal Responses request: 
    curl -X POST https://librechat.uvilo.ai/api/agents/v1/responses \
  -H "Authorization: Bearer $LIBRECHAT_AGENTS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<agent_id>",
    "input": "Hello, respond with just the word OK"
  }'
  3. Record the full response shape — compare it with Chat Completions. Does it return a conversation ID? Different metadata?
  4. Test streaming mode and capture SSE event types
  5. Test any additional parameters from the Open Responses spec — which ones does LibreChat actually support?
  6. Check LibreChat’s source code for the Responses route handler. Document the actual parameter schema.

Deliverable: Complete parameter map for Responses endpoint, comparison with Chat Completions, Open Responses spec alignment.

Task 4 — Investigate Conversation Lifecycle and Tracking

Orchestration requires knowing the conversation ID (for R7 transcript review, for Ralph Loop reconnection, for state tracking). The docs don’t mention how to get or use conversation IDs.

  1. After Task 2/3, check if the response includes a conversation ID. If yes, note the field name and format.
  2. If no conversation ID in the response, try these discovery methods:
    • Check LibreChat’s MongoDB directly — query the conversations collection after an API call to find the new conversation and its ID
    • Check the SSE stream for early events that might contain metadata (conversation ID, thread ID) before the main response
    • Check response headers for custom headers (e.g., X-Conversation-Id)
  3. Once a conversation ID is known, test:
    • Can you query conversation status/history via any API endpoint?
    • Can you send a follow-up message to the same conversation? (i.e., continue a chat)
    • Is the conversation visible in the LibreChat Chat History UI?
    • Can you set a conversation title? If not via the API, can you update it via the Agent Builder API or MongoDB directly?
  4. Test conversation continuity — send a second request referencing the same conversation (if the API supports it). Does the agent retain context from the first message?

Deliverable: Conversation lifecycle documentation — how to get the ID, how to set the title, whether conversations are continuable, whether they appear in Chat History.

Task 5 — Test Long-Running Agent Invocations

Orchestration agents may run for 5–30 minutes with dozens of tool calls. The Agents API’s behavior under long-running execution is unknown. We need to know: does the HTTP connection survive? Does the agent complete server-side even if the client disconnects? What are the timeout limits?

  1. Select or create a test agent that performs multiple tool calls (e.g., an agent with filesystem MCP tools). Give it a task that requires at least 5–10 tool calls.
  2. Run it via the Chat Completions API with stream: true and measure:
    • Total execution time
    • Whether the SSE stream stays alive throughout
    • What events are emitted during tool calls (progress indicators? heartbeats?)
    • Whether the final response includes tool call details or just the final message
  3. Run the same task with stream: false and measure:
    • How long before the HTTP response arrives
    • Whether Railway’s proxy times out (check Railway logs)
    • Whether the response is complete when it arrives
  4. Test disconnect resilience:
    • Start a long-running agent call with stream: true
    • After 30 seconds, kill the client (Ctrl+C the curl)
    • Check LibreChat’s Chat History — did the agent continue running and complete?
    • If it completed, can you retrieve the full transcript from Chat History?
  5. Test a deliberately long task (15+ minutes if possible) and observe behavior. Check Railway request timeout settings.
  6. Document all findings — timeout limits, streaming behavior, server-side persistence.

Deliverable: Long-running execution profile — max safe duration, streaming behavior, disconnect resilience, whether agents complete server-side independently of the client connection.

Task 6 — Investigate the LibreChat UI Chat API

The OpenAI-compatible Agents API (Tasks 2–5) is a stateless proxy — no persistence, no conversation ID, no transcript. The LibreChat UI chat endpoint (/api/agents/chat/agents) returns a conversationId immediately and likely triggers the full message persistence pipeline. This task investigates whether the UI API solves the deal breakers identified in Tasks 2–5.

  1. Reproduce the UI API call. Using the JWT auth token from the UI, send a chat request to /api/agents/chat/agents with a simple prompt. Confirm the response includes conversationId and streamId.
  2. Verify message persistence. After the agent completes, check MongoDB messages collection for the conversation. Confirm that BOTH user messages AND assistant messages (including tool call details) are persisted — not just the final text response.
  3. Verify transcript visibility in Chat History. Open LibreChat UI → Chat History. Confirm the conversation appears with the full transcript including tool calls, intermediate steps, etc.
  4. Test streaming behavior. Repeat the call with streaming and capture the SSE events. Determine whether events are streamed in real-time during tool execution (unlike the OpenAI-compatible API which is silent until completion) or replayed in a burst.
  5. Test conversation continuation. Send a second message to the same conversation (using the conversationId and parentMessageId from the first response). Confirm the agent has context from the first message.
  6. Test long-running execution. Give the agent a multi-step task requiring 5+ tool calls. Measure duration, streaming behavior, and whether the full transcript is preserved.
  7. Investigate auth mechanism. The UI API requires a JWT session token, not an API key. Determine:
    • How long do JWT tokens last?
    • Can we refresh tokens programmatically (the refreshToken cookie suggests yes)?
    • What’s the refresh endpoint and flow?
    • Is there a way to use API key auth with this endpoint instead?
  8. Check LibreChat source code for the /api/agents/chat/agents route handler. Understand how it differs from the OpenAI-compatible endpoint — why does it persist messages when the other doesn’t?

Deliverable: UI Chat API profile — persistence confirmation, streaming behavior, conversation lifecycle, auth flow. Verdict on whether this endpoint solves the deal breakers for orchestration.

Task 7 — Compile Findings and Update Research

All previous tasks produce raw findings. This task synthesizes them into an updated Research document with concrete, evidence-based recommendations that replace the speculative sections.

  1. Compile all findings from Tasks 1–6 into a structured appendix or updated sections of Orchestration_Research.md
  2. For each Research section that had a pending decision, update with the evidence-based verdict:
    • Section 1 (LibreChat Current State) — update with actual API capabilities discovered
    • Section 2 (Architecture Pattern) — confirm or revise the Supervisor + Agents API recommendation based on what actually works
    • Section 7 (Sub-Agent Transcript Review) — update with actual conversation ID and title mechanics
    • Section 8 (SRP Agent Design) — update with whether programmatic deployment is feasible
  3. Add a new section: “Agents API Operational Profile” — the long-running execution limits, streaming behavior, and disconnect resilience findings from Task 5
  4. Add a new section: “UI Chat API vs OpenAI-Compatible API” — findings from Task 6
  5. If any findings invalidate previous recommendations (e.g., the UI API solves the persistence problem), propose revised architecture alternatives
  6. Update all “Decision: Pending user approval” fields to “Decision: Supported by investigation” or “Decision: Needs revision — [reason]”

Deliverable: Updated Orchestration_Research.md with evidence-based findings replacing all speculation. All pending decisions resolved or flagged for revision.