TypeSense Index Research Decisions
Extracted from TypeSense_Index_Research.md. This file preserves the exact decision content so the Spec can reference it without paraphrasing.
R1 Decision — Incremental Indexing via Diff
Trigger on **/*.md (no path narrowing on the workflow trigger), but only index the .md files that changed since the last push. The filtering logic moves from the workflow trigger to the indexer itself, which diffs against the last push to determine changed files and only processes those. This simplifies the trigger config and makes the indexer incremental by default.
R2 Decision — p-limit Concurrency
Use p-limit for concurrency. Concurrency of 5–8, configurable. Summary and embedding calls for the same file run in parallel (they’re independent). Multiple files process in parallel. Batch upsert to Typesense batches by BATCH_SIZE (40 docs).
R3 Decision — index-department.yml GitHub Action
New index-department.yml workflow with workflow_dispatch trigger. Supports inputs:
mode— incremental / all / departmentdepartment— (optional) department name for department modesince_ref— (optional) defaults to HEAD~1
Runs on ubuntu-latest runners, eliminating Railway timeout issues.
R4/R5 Decision — Separate Bundled TypeScript Projects
Two separate bundled TypeScript projects:
index-department— atForge/Typesense/Maintenance/index-department/typesense-mcp— atForge/Configs/MCP_Servers/typesense-mcp/
Collection setup script: Forge/Typesense/Maintenance/create-typesense-collections.ts (moved from Forge/Configs/).
Minimal shared code is acceptable to duplicate (HTTP helpers, env var reading).
R6 Decision — Update Skill Docs After Migration
Update Typesense MCP skill documentation after R4/R5 TypeScript migration is complete, as part of the same implementation cycle. Remove: “no curl in container” references, Python dependency references, /proc/1/environ-only env var pattern references.
R7 Decision — Default Status Filter
Default status to "published". "all" omits the filter entirely. Any explicit status value filters normally.
Original Python Indexer Constants
| Constant | Value |
|---|---|
| COLLECTION | uvilo |
| CHUNK_WORD_LIMIT | 800 |
| HEADING_CHUNK_MIN_WORDS | 50 |
| BATCH_SIZE | 40 |
| MAX_RETRIES | 3 |
| RETRY_DELAY | 2 |
| OPENAI models | gpt-5.4-nano (summary), text-embedding-3-small (embedding) |
Original OpenAI Summary Prompt
You generate concise summaries of documentation files. Summarize the purpose and key content of the file in 100 words or fewer. Be factual and specific. Do not use phrases like ‘this file contains’ or ‘this document describes’.