90 lines
5.8 KiB
Markdown
90 lines
5.8 KiB
Markdown
`llmorbiter` **LLM tool-calling orchestrator with provider
|
|
abstraction**.
|
|
|
|
## Layout
|
|
|
|
```
|
|
src/
|
|
index.js # public entry — re-exports everything
|
|
LLMOrbiter.js # core orchestrator (~364 lines)
|
|
Tool.js # defineTool() + ToolError + DirectReturnError
|
|
providers/
|
|
OpenRouterProvider.js # streaming OpenRouter SSE provider
|
|
|
|
examples/
|
|
weather.js # minimal tool-calling demo
|
|
tool-error-retry.js # tool-error + LLM self-correction demo (has PASS/FAIL asserts)
|
|
pdf-pages.js # PDF -> per-page LLM processing (500 lines, largest)
|
|
tools/weather.js # sample static-data weather tool
|
|
```
|
|
|
|
## Public API (`src/index.js`)
|
|
|
|
Exports: `LLMOrbiter`, `OpenRouterProvider`, `defineTool`, `ToolError`, `DirectReturnError`.
|
|
|
|
- `defineTool({ name, description, parameters, strict=false, invoke })` -> `{ spec, invoke, name }`.
|
|
- `spec`: OpenAI-compatible function schema (`{ type: 'function', function: { name, description, parameters, strict } }`).
|
|
- `parameters`: JSON Schema object (defaults to `{ type: 'object', properties: {}, additionalProperties: false }`).
|
|
- `invoke(args, { signal })`: local async implementation.
|
|
- `ToolError(message)` — throw from a tool to signal a recoverable error back to the LLM.
|
|
- `DirectReturnError(data)` — throw from a tool to stop the loop immediately; `run()` resolves
|
|
with `{ directReturn: true, data, ... }`. Any remaining tool calls in the same turn are closed
|
|
with synthetic skipped tool responses so the conversation transcript remains valid.
|
|
|
|
### `new LLMOrbiter(provider, opts)`
|
|
|
|
`opts` are lifecycle callbacks:
|
|
- `onLLMCall({ model, toolCount, tools, promptSize, messageCount, messages, iteration })` — called before each LLM call; `messages` is a comma-separated summary of roles (e.g. `'user,assistant,tool_response'`).
|
|
- `onUsage({ cached, input, output, cost })` — called after each LLM call with that call's usage; provider's raw usage object is attached on non-enumerable `.raw`.
|
|
- `onToolCall({ name, args })` — called before executing a tool.
|
|
- `onToolResult({ name, result })` — called on successful tool execution.
|
|
- `onToolError({ name, error, args, rawError })` — called when tool execution throws, returns `{ error }`, has invalid JSON args, or references an unknown tool.
|
|
- `onStatus(status)` — forwarded from provider: `'connected'` (HTTP headers received), `'firstevent'` (first SSE line/comment/event received), `'reasoning'` (first reasoning delta received), `'content'` (first content delta received), or `'toolcall'` (first tool call delta received).
|
|
- `onContent(chunk)` — forwarded from provider: incremental output content string (`choice.delta.content`).
|
|
- `onReasoning(text)` — forwarded from provider: **accumulated** reasoning string so far (aggregated from `delta.reasoning`, `delta.reasoning_content`, and `delta.reasoning_details`).
|
|
|
|
### `orbiter.run(opts)`
|
|
|
|
`{ prompt, messages, model, tools, maxIterations=100, extraBody, signal }`.
|
|
|
|
- `prompt`: user message string to append (omit or pass `null` to continue without adding one).
|
|
- `messages`: transcript array to seed from; a leading `{ role: 'system' }` is the system prompt.
|
|
- `tools`: array of tool objects created with `defineTool()`.
|
|
- `maxIterations`: safety limit on tool-calling loop iterations (default: `100`).
|
|
- `extraBody`: extra parameters merged into provider request payload.
|
|
- `signal`: external `AbortSignal`.
|
|
|
|
Loop: send messages + tool specs -> if `tool_calls` come back, invoke locals and feed results
|
|
back -> if an iteration returns neither tool calls nor content (no response), the loop continues -> repeat until a response is produced or `maxIterations`.
|
|
|
|
Returns:
|
|
- `content`: final assistant message text (or `null`).
|
|
- `toolResults`: `{ [toolName]: [result, ...] }` map of all tool results executed across iterations.
|
|
- `usage`: aggregated usage object `{ input, output, cached, tokenIn, tokenOut, tokenCached, costTotal, cost, raw }`.
|
|
- `rawUsage`: single provider usage object (if 1 iteration) or array of usage objects (if multiple iterations).
|
|
- `rawUsages`: array of all iteration usage objects from the provider.
|
|
- `iterations`: count of iterations performed.
|
|
- `messages`: conversation transcript array for continuity in subsequent runs.
|
|
- `directReturn`: `true` if terminated by `DirectReturnError`.
|
|
- `data`: data payload passed to `DirectReturnError`.
|
|
|
|
`orbiter.abort(reason)` cancels the active run; aborts surface as `DOMException('...', 'AbortError')`.
|
|
|
|
### `OpenRouterProvider`
|
|
|
|
`new OpenRouterProvider({ apiKey, apiUrl, siteUrl, siteName, sessionId })`.
|
|
- `apiUrl` defaults to `https://openrouter.ai/api/v1/chat/completions`.
|
|
- `siteUrl` sets `HTTP-Referer` header.
|
|
- `siteName` sets `X-Title` header.
|
|
- `sessionId` sets session ID / cache key (defaults to a process-lifetime UUID created when the app starts; sent via `x-session-id` HTTP header and `session_id` body parameter).
|
|
|
|
Method:
|
|
`chatCompletions({ model, messages, tools, signal, extraBody, onStatus, onContent, onReasoning, sessionId })`
|
|
— streams SSE, assembles `{ content, tool_calls, finish_reason, usage, reasoning_details, reasoning }`.
|
|
- If reasoning was returned, automatically appends `{ role: 'assistant', content: result.reasoning }` to `messages`.
|
|
- Delta handling:
|
|
- first SSE event/line (including comments) -> triggers `onStatus('firstevent')`.
|
|
- `delta.reasoning` / `delta.reasoning_content` / `delta.reasoning_details` -> triggers `onStatus('reasoning')` on first reasoning delta, and passes **accumulated** string to `onReasoning(text)`.
|
|
- `delta.content` -> triggers `onStatus('content')` on first content delta, and passes chunk to `onContent(chunk)`.
|
|
- `delta.tool_calls` -> triggers `onStatus('toolcall')` once when tool calls begin streaming (tool calls are assembled into `result.tool_calls` without per-chunk streaming callbacks).
|