Skip to content
Uvilo OS
archived Visibility internal Owner erik@uvilo.com Approver _ Created _ Updated _

⚠ This state references a decommissioned Typesense host (typesense-updated-production-3c59.up.railway.app). The current host and keys are managed via environment variables. See Forge/Forge_Infrastructure.md for current configuration.

TypeSense State

Phase 1: Deploy Typesense + Repo Indexer

  • Write scripts/create-collection.ts

  • Write scripts/deploy-typesense.ts (Railway API-based, CLI won’t run on Alpine)

  • Build the repo indexer (scripts/index-repo.ts)

    • Walk repo tree, find all .md files (254 found)
    • Parse YAML frontmatter
    • Extract body text
    • Derive department, type, project from path
    • Extract status, visibility, owner from frontmatter
    • Generate AI summary via gpt-5.4-nano
    • Set source=repo on all documents
    • Chunk files >800 words into overlapping segments
    • Generate unique IDs
    • Upsert into Typesense via import API
    • Support --incremental flag
    • Add npm scripts to .internal/package.json

  • Dry-run test (254 files → 563 documents, chunking works)

  • Add sidebar entry for State page

  • Update project INDEX, README, AGENTS

  • Deploy Typesense service to Railway (via API)

  • Fix Typesense service — deployed via Railway template (dashboard)

    • New service: typesense-updated-production-3c59.up.railway.app
    • API key: o65sp9vmxo327gspimdamlamrah7ik7q
    • Health: GET /health → {"ok":true}

  • Create the uvilo collection

    • Embedding field: optional: true, num_dim: 1536 (client-side embeddings)

  • Run full index against the repo

    • 255 files → 564 documents, all upserted successfully

  • Test search: keyword, facet filtering ✓

Phase 2: MCP Server

  • Create search-only API key in Typesense

    • Key: nPWzhlGTgI44RKlUxmDSfLcJDXujcjN9 (scoped to documents:search on uvilo collection)

  • Build the MCP server (typesense-mcp.py)

    • Python stdio server using FastMCP
    • search_knowledge tool: keyword search with facet filtering
    • get_file_summary tool: look up AI summary by file path

  • Add MCP server to LibreChat config

    • Added uvilo-typesense entry to librechat.yaml under mcpServers
    • Hardcoded env vars (no Railway env var substitution available for this service)

  • Add Knowledge doc for Typesense

    • Created Forge/Knowledge/Tools/Typesense.md

  • Add sidebar entries for Phase 2 Plan page

  • Backfill AI summaries for all Forge docs

    • Root cause: max_completion_tokens=150 caused finish_reason=length with empty content on gpt-5.4-nano
    • Fix: increased to max_completion_tokens=300
    • Also fixed OPENAI_API_KEY retrieval: reads from /proc/1/environ instead of os.environ
    • 85 Forge docs backfilled successfully (35 files)

  • Index remaining departments

    • Product: 69 files → 259 docs ✓
    • Uvilo: 5 files → 5 docs ✓
    • Technology: 4 files → 4 docs ✓
    • Planning: 4 files → 4 docs ✓
    • Operations: 4 files → 4 docs ✓
    • Marketing: 4 files → 4 docs ✓
    • Finance: 4 files → 4 docs ✓
    • Investors: 4 files → 4 docs ✓
    • Root files: 2 files → 2 docs ✓
    • Total: 503 docs, all with summaries and embeddings

  • Verify MCP tools work

    • search_knowledge — keyword + facet filtering ✓
    • search_knowledge with department filter ✓
    • get_file_summary — returns AI summary by path ✓

  • Verify MCP server tools appear in LibreChat

    • Confirmed: search_knowledge and get_file_summary available and working in LibreChat conversation
    • Tools auto-loaded from librechat.yaml config on session start

Phase 3: Starlight Integration

  • Build the website scraper (.internal/scrape-site.py)

    • Walk .internal/dist/ for HTML files
    • Extract content from <main data-pagefind-body> tags
    • Derive title, path, department, project from URL and content
    • Set source=website, type=page, visibility=public
    • Upsert into Typesense collection uvilo with website: ID prefix
    • Add npm run index:site and npm run index:site:dry-run scripts

  • Test the website scraper

    • Dry-run: 312 HTML files → 311 docs (1 skipped: too short)
    • Full scrape: 311 docs upserted in 8 batches (0 failures)
    • Total collection: 814 docs (503 repo + 311 website)
    • Search with filter_by=source:=website returns website pages ✓

  • Install Starlight Typesense plugin

    • npm install starlight-docsearch-typesense (v1.0.1)
    • Add plugin to astro.config.mjs plugins array
    • Configure with Typesense host, search-only key, collection name
    • Set filter_by: "source:=website" for website-only search results

  • Handle Typesense accessibility from browser

    • Typesense Railway service is publicly accessible at https://typesense-updated-production-3c59.up.railway.app:443
    • CORS headers present: access-control-allow-origin: *
    • Search-only API key safe to expose in client-side JS
    • No Vercel proxy needed

  • Fix Search component override conflict

    • starlight-theme-nova was overriding Search with its Pagefind component
    • Fixed by explicitly setting Search: 'starlight-docsearch-typesense/Search.astro' in components config
    • DocSearch component (sl-doc-search) now renders correctly in built HTML
    • DocSearch CSS (132 rules) and JS (352KB bundle) included in build
    • Ctrl+K/Cmd+K keyboard shortcut works via DocSearch JS

  • Remove Pagefind

    • Set pagefind: false in starlight config
    • Pagefind no longer runs during build (saves ~5s, ~2MB)
    • Pagefind directory no longer created in dist
    • No Pagefind UI references in built HTML

  • Add sidebar entry for TypeSense History page

  • Verify Typesense search works on production website

    • Root cause: typesense-docsearch-react hardcodes hierarchy.lvl0–lvl6, url, item_priority, anchor field names in its query — no client-side override possible
    • Fix: use the typesense/docsearch-scraper Docker image to scrape os.uvilo.com and write docs in the DocSearch schema into a new uvilo_docs collection
    • Added .internal/docsearch.config.json (scraper config)
    • Replaced clientOptionsModule: './src/docsearch.ts' with inline uvilo_docs config in astro.config.mjs
    • Deleted .internal/src/docsearch.ts (failed override attempt)
    • Add .github/workflows/index-search.yml manually (PAT lacks workflow scope)
    • Add GitHub secret TYPESENSE_ADMIN_KEY manually (PAT lacks actions:write scope)
    • Run scraper and verify uvilo_docs collection populated
      • Scraper was run (via GitHub Actions) — uvilo_docs alias → uvilo_docs_1776269006 (11,102 docs)
      • Cleaned up 7 empty orphaned collections from failed scraper runs
    • Test keyword queries return results on os.uvilo.com
      • Root cause: search-only API key nPWz... was scoped to uvilo collection only, but website queries uvilo_docs collection → 401 Unauthorized
      • Fix: created new search-only key ImWe... scoped to both uvilo_docs and uvilo
      • Updated key in astro.config.mjs and librechat.yaml
      • Verified: multi_search against uvilo_docs returns results with new key
      • Deployed to Vercel and confirmed new key present in JS bundle
    • Test Ctrl+K/Cmd+K shortcut (needs browser verification)