# THE BPHEN/DIS MASTER COPY
## Deterministic Inference System — Code Documentation
### Version: BASE (v3.1.0 — pre-upgrade master)
### Created: August 31, 2026
### Author: Javelion Griffin (JU), DBA Javelion Analytics™
### Documented by: SuperNinja AI (integrity auditor)

---

## PURPOSE OF THIS DOCUMENT

This is the "functions" half of the DIS master copy. It maps every concept documented in the conceptual framework (`DIS_MASTER_COPY_CONCEPTUAL.md`) to the actual code that implements it. If the conceptual document answers "what makes the DIS the DIS," this document answers "what code turns those concepts into functions for Ida."

The entire DIS runs from a single Cloudflare Worker file: `ida-archive-worker.js`. This file is 4,304 lines of JavaScript. It contains the server-side logic (API routes, Supabase queries, Gemini AI integration, Ida's tool architecture), AND the embedded client-side JavaScript (the boot terminal UI, archive browser, workspace interface, PENS methodology viewer). Both server and client live in one file — the client code is stored in the `CLIENT_SCRIPT` variable (lines 90–2578) and injected into the HTML pages served from Cloudflare Pages.

This document is organized by functional layer, not by line number. Each section covers one subsystem, lists the functions that implement it, describes what each function does, and notes which Supabase tables or external APIs it touches.

---

## LAYER 1: AUTHENTICATION & SESSION MANAGEMENT

The DIS uses a two-step boot terminal for authentication. The user enters an ACCESS CODE, then a SECURITY CODE. If both match, an HMAC-signed session cookie is issued. All operational API endpoints require this session.

### Functions

| Line | Function | Purpose |
|------|----------|---------|
| 15 | `base64UrlEncode(bytes)` | Encodes bytes to URL-safe base64 for HMAC operations |
| 23 | `base64UrlDecode(value)` | Decodes URL-safe base64 back to bytes |
| 35 | `constantTimeEqual(left, right)` | Constant-time byte comparison to prevent timing attacks on credential checks |
| 45 | `hmacOwnerSession(env, value)` | Creates an HMAC-SHA256 signature for a session value using the worker's secret key |
| 60 | `ownerSessionCookie(value, maxAge)` | Formats a `Set-Cookie` header for the session cookie with the signed value and TTL |
| 65 | `createOwnerSession(env)` | Generates a random session token, signs it with HMAC, and returns the signed cookie value |
| 73 | `hasOwnerSession(request, env)` | Reads the session cookie from the request, verifies the HMAC signature, returns true/false |
| 2650 | `authorizeOperationsRead(request, env)` | Secondary auth check for operations/audit endpoints — accepts either an owner session or an `x-ida-ops-token` header |
| 4184 | `handleLogin(request, env)` | The login API handler. Receives POST with `{step, value}`. Step 1 checks ACCESS_CODE, step 2 checks SECURITY_CODE. On successful SECURITY_CODE, creates session and returns `Set-Cookie` |
| 4213 | `handleLogout(request)` | Clears the session cookie by setting it with maxAge=0 |

### Configuration Variables

| Line | Variable | Value |
|------|----------|-------|
| 3405 | `IDA_OWNER_KEY` | `"credential:IDA_SECURITY_CODE_READ"` — the owner key used to scope all workspace data in Supabase |
| — | `IDA_ACCESS_CODE` | Set as Cloudflare Worker environment variable (value: `20019128`) |
| — | `IDA_SECURITY_CODE` | Set as Cloudflare Worker environment variable (value: `89JX`) |

### Security Design

The authentication uses constant-time comparison (`constantTimeEqual`) to prevent timing attacks. The session cookie is HMAC-signed, meaning it cannot be forged without the worker's secret key. The two-step process (ACCESS_CODE → SECURITY_CODE) creates a layered defense — even if one code is compromised, the other still blocks entry.

---

## LAYER 2: THE CLIENT-SIDE UI (BOOT TERMINAL + APPLICATIONS)

The entire user interface is embedded as a JavaScript string in the `CLIENT_SCRIPT` variable (lines 90–2578). When the worker serves an HTML page from Cloudflare Pages, it injects this script. The script builds the boot terminal, the archive browser, the workspace (conversations, notes, project folders), and the PENS methodology viewer.

### 2A: Core UI Utilities (Lines 108–230)

| Line | Function | Purpose |
|------|----------|---------|
| 108 | `escapeHtml(value)` | Escapes HTML special characters to prevent XSS in dynamically rendered content |
| 114 | `formatDate(value)` | Formats an ISO date string to a readable date (e.g., "Aug 28, 2026") |
| 126 | `formatDateTime(value)` | Formats an ISO date string to a readable date+time |
| 140 | `auditTag(item)` | Generates an audit status tag element for an archive item |
| 164 | `auditSection(item)` | Generates a full audit detail section for an archive item |
| 212 | `closeButton(body, callback, label)` | Creates a close button for overlays/modals |
| 223 | `getOverlay()` | Gets or creates the global overlay container element |

### 2B: Overlay & Branding System (Lines 231–340)

| Line | Function | Purpose |
|------|----------|---------|
| 231 | `applyReaderFrame(overlay, title, body)` | Applies the standard reader frame styling to an overlay (header bar, scrollable body) |
| 277 | `styleUniversalActionButton(button)` | Styles action buttons with the DIS color tokens |
| 297 | `installUniversalOverlayActions(header, close, overlay)` | Installs minimize/close behavior on overlay headers |
| 324 | `removeWorkspaceMetadata()` | Removes metadata divs from workspace pages |
| 330 | `applyWorkspaceBranding()` | Applies DIS branding (colors, fonts) to workspace elements |

### 2C: The Ida Terminal (Lines 394–596)

The Ida Terminal is the chat interface where users talk to Ida. It renders tool steps as they execute, showing a console-like trace of what Ida is doing.

| Line | Function | Purpose |
|------|----------|---------|
| 394 | `terminalStepLabel(step)` | Maps tool names to human-readable labels (e.g., `search_archive` → "ARCHIVE SEARCH", `cycle_distribution` → "CYCLE DISTRIBUTION", `detect_transitions` → "TRANSITION DETECT") |
| 406 | `terminalStepSummary(step)` | Creates a one-line summary of what a tool step did (shows filter args, result count) |
| 441 | `terminalConsoleLine(kind, text)` | Creates a single console line element (kind: "user", "ida", "tool", "step") |
| 453 | `stopTerminalReveal()` | Stops the typewriter reveal animation |
| 460 | `renderTerminalConsole(body)` | Renders the full terminal console UI (message area, input, send button) |
| 490 | `beginTerminalTask()` | Starts a task indicator in the terminal |
| 498 | `completeTerminalTask(steps)` | Completes the task indicator and renders tool step traces |
| 517 | `openIdaTerminal()` | Opens the Ida Terminal overlay (the main chat interface) |
| 528 | `renderTerminalFooter(body)` | Renders the terminal footer with copy/save buttons |

### 2D: Quick Task & Thread Windows (Lines 597–760)

| Line | Function | Purpose |
|------|----------|---------|
| 597 | `openQuickTaskWindow()` | Opens a quick task window for sending a one-off message to Ida |
| 637 | `openNewThreadWindow()` | Opens a window to create a new conversation thread |
| 678 | `openUtilityTabWindow()` | Opens a utility tab for managing project folders and conversations |

### 2E: Project Folder Management (Lines 741–860)

| Line | Function | Purpose |
|------|----------|---------|
| 774 | `utilityRenameFolder(folderId, newName)` | Renames a project folder via the workspace API |
| 785 | `utilityDeleteFolder(folderId)` | Deletes a project folder |
| 792 | `utilityMoveFolder(folderId, direction)` | Reorders a project folder up or down |
| 808 | `utilityMoveThread(threadId, direction)` | Reorders a conversation thread |
| 823 | `projectFolderKey(name)` | Generates a unique key for a project folder by name |
| 827 | `createProjectFolder(projectFolder, key, isSystem)` | Creates a project folder DOM element |
| 853 | `renderProjectFolders()` | Renders all project folders in the sidebar |
| 864 | `setWorkspaceButtonState()` | Updates workspace button states based on current mode |

### 2F: Workspace Conversations (Lines 897–1428)

| Line | Function | Purpose |
|------|----------|---------|
| 897 | `workspaceRequest(path, init)` | Makes a fetch request to the workspace API with session credentials |
| 905 | `workspaceApi(resource, path, init)` | Makes a workspace API request to a specific resource endpoint |
| 925 | `loadConversations()` | Loads all conversations from the workspace API |
| 955 | `createConversation(title, folderId)` | Creates a new conversation |
| 966 | `renameConversation(id, title)` | Renames a conversation |
| 971 | `deleteConversation(id)` | Deletes a conversation |
| 979 | `selectConversation(id)` | Selects a conversation and loads its messages |
| 998 | `sendConversationMessage(text)` | Sends a message to Ida in the current conversation — calls `/api/workspace/conversations/{id}/messages` |
| 1061 | `renderWorkspaceConversations(element)` | Renders conversations in a container |
| 1067 | `renderWorkspaceConversationsUI()` | Renders the full workspace conversations UI (sidebar + message area + composer) |
| 1429 | `workspaceMessage(body, message, tone)` | Shows a status message in the workspace |

### 2G: Notes (Lines 1216–1428)

| Line | Function | Purpose |
|------|----------|---------|
| 1216 | `loadNotes()` | Loads all notes from the workspace API |
| 1225 | `createNote(title, folderId)` | Creates a new note |
| 1238 | `saveNote(id, title, content, folderId)` | Saves a note's content |
| 1247 | `deleteNote(id)` | Deletes a note |
| 1256 | `openNotesWorkspace(selectedNoteId)` | Opens the notes workspace UI |

### 2H: PENS Methodology Viewer (Lines 1618–1791)

This is the PENS access point. PENS data is NOT embedded in the DIS — it is accessed through this viewer, which calls the PENS API endpoint. This is the correct architecture: PENS lives in Supabase tables and is accessed on-demand through the Private Workspace, not embedded in the system.

| Line | Function | Purpose |
|------|----------|---------|
| 1618 | `pensMessage(body, text, isError)` | Shows a status message in the PENS viewer |
| 1632 | `pensButton(label)` | Creates a PENS viewer button |
| 1641 | `pensHeader(body, label, title, detail)` | Creates a PENS viewer header |
| 1659 | `promptPensStudy(body, analysis)` | Prompts for a PENS study access code |
| 1681 | `openPensStudy(body, analysis)` | Opens a specific PENS training analysis study |
| 1731 | `renderPensMethodology(body, payload)` | Renders the list of available PENS methodologies and training analyses |
| 1774 | `openPensMethodology()` | Opens the PENS methodology viewer (the main entry point) |

### 2I: Archive Browser (Lines 1799–2578)

| Line | Function | Purpose |
|------|----------|---------|
| 1792 | `setupProjectFolders()` | Initializes project folder system folders |
| 1799 | `openArchive()` | Opens the archive browser — loads and displays all archive records |
| 1848 | `makeSelect(label, key, options, body)` | Creates a dropdown selector for archive filtering (cycle, source type, phenomenon, era) |

---

## LAYER 3: HTML PROXYING & CONTENT REWRITING

The DIS serves its HTML from Cloudflare Pages (`https://bphen-dis.pages.dev`). The worker fetches HTML from this origin, rewrites it (strips retired PWA shell, rewrites retired language, injects the client script), and serves it to the user.

### Functions

| Line | Function | Purpose |
|------|----------|---------|
| 2594 | `stripWorkspaceMetadata(html)` | Removes workspace metadata divs from served HTML |
| 2599 | `rewriteRetiredWorkspaceLanguage(source)` | Rewrites retired terminology throughout the HTML: "BLOODHOUND" → "DIS TRACKER", "SCANNER" → "COLLECTION STATUS", "operating system" → "BPHEN/DIS", "Archive Connector" → "BPHEN Archive", and many more replacements. This function is critical — it transforms the old Replit-era HTML into the current DIS language |
| 2625 | `stripPwaShell(html)` | Removes PWA (Progressive Web App) shell elements: manifest links, apple-touch icons, service worker scripts, viewport meta tags |
| 4225 | `serveAsset(request, env)` | The main asset-serving function. Fetches HTML from `BPHEN_PAGES_ORIGIN`, applies all rewriting functions, injects `CLIENT_SCRIPT`, and returns the response. This is the fallback handler for all non-API routes |
| 4252 | `serveEnhancedMainScript(request)` | Serves the enhanced main navigation script |

### Configuration

| Line | Variable | Value |
|------|----------|-------|
| 11 | `BPHEN_PAGES_ORIGIN` | `"https://bphen-dis.pages.dev"` — the Cloudflare Pages origin where HTML is hosted |

---

## LAYER 4: SUPABASE INTEGRATION

All data operations go through Supabase. The worker uses the Supabase REST API (PostgREST) with a service key for authenticated access.

### Functions

| Line | Function | Purpose |
|------|----------|---------|
| 2630 | `selectHeaders(env, extras)` | Builds the headers for Supabase API calls: `apikey` and `authorization: Bearer` using `SUPABASE_SERVICE_KEY` |
| 2639 | `supabase(env, path, init)` | The core Supabase helper. Takes a path (e.g., `archive_items?review_status=eq.APPROVED&select=*`) and returns the fetch response. All Supabase queries in the system go through this function |

### Configuration (Environment Variables)

| Variable | Purpose |
|----------|---------|
| `SUPABASE_URL` | The Supabase project URL (e.g., `https://xxx.supabase.co`) |
| `SUPABASE_SERVICE_KEY` | The Supabase service role key for authenticated REST API access |

### Supabase Tables Used by the DIS

**Core Archive Tables:**
- `archive_items` — All 980 records. Fields: id, title, snippet, source_type (LIVE/HISTORICAL), primary_cycle (C1-C4), phenomena, era, geo_scope, published_at, collected_at, source_name, source_url, review_status
- `archive_folders` — System folders (the 11 eras). Fields: id, slug, name
- `archive_folder_items` — Junction table linking items to folders (maps records to eras)

**Collection Tracking Tables:**
- `collection_log` — Log of collection events
- `archive_audit_queue` — Items pending audit review
- `archive_sync_runs` — Sync/mirror run history
- `cycle_snapshots` — Cycle state snapshots (dominant_cycle, dominant_phase, dominant_confidence, cycle_scores, active_phenomena, items_in_window, window_start, window_end, snapshot_at)

**Workspace Tables:**
- `workspace_conversations` — Chat conversations (id, title, folder_id, created_at, updated_at, owner_key)
- `workspace_messages` — Individual messages within conversations (role: USER/IDA, content, created_at)
- `workspace_notes` — User-created notes (id, title, content, folder_id, owner_key)
- `workspace_project_folders` — Project folders for organizing conversations and notes

**PENS Tables (Private Workspace — accessed through applications, NOT embedded in DIS):**
- `ida_pens_methodologies` — PENS methodology definitions
- `ida_pens_training_analyses` — PENS training analysis studies

---

## LAYER 5: ARCHIVE API (`/api/archive`)

The archive API is the public-facing data layer. It serves archive records, stats, audit data, health metrics, activity feeds, and cycle state. Most endpoints are public (no auth required); the audit endpoint requires operations auth.

### Route Handler

**`handleArchive(request, env)`** (line 3360) — Routes based on the `type` query parameter:
- `type=stats` → `archiveStats(env)`
- `type=audit` → `archiveAudit(env)`
- `type=health` → `archiveHealth(env)`
- `type=activity` → `archiveActivity(env)`
- `type=cycle-state` → `archiveCycleState(env)`
- `type=folders` → `archiveFolders(env)`
- `type=folder-items` → `archiveFolderItems(env, url)`
- `type=items` (with filter params) → `archiveItems(env, url)`
- Default → `proxyOriginalArchive(request, env)` (proxies to the original archive backend)

### Data Functions

| Line | Function | Purpose |
|------|----------|---------|
| 2782 | `filterDerivedBrowsableItems(env, items)` | Filters out derived/synthetic items from browsable results |
| 2810 | `allBrowsableArchiveItems(env, order)` | Fetches ALL browsable archive items from Supabase: `archive_items?review_status=eq.APPROVED&select=*&order=collected_at.desc&limit=1000`. Paginated if needed. Returns complete records with all metadata fields. This is the foundation function for archive browsing and Ida's search tool |
| 2827 | `archiveStats(env)` | Computes archive statistics: counts per cycle, per source_type, per phenomenon, per era, totals. Returns a structured stats object |
| 2874 | `archiveAudit(env)` | Returns audit workload data — items pending review, recently reviewed, audit queue status. Requires operations auth |
| 2894 | `archiveAuditWorkload(env)` | Computes detailed audit workload metrics |
| 3014 | `archiveHealth(env)` | Returns system health metrics: collection freshness, mirror sync status, collector health, overall system status |
| 3104 | `archiveItems(env, url)` | The filtered items API. Supports filtering by: `public`, `source_type`, `phenomenon`, `cycle`, `geo`, `era`, `search`. Returns matching items with full metadata. This is the API that the archive browser UI calls |
| 3143 | `archiveFolders(env)` | Returns all system folders (the 11 eras) from `archive_folders` |
| 3187 | `archiveFolderItems(env, url)` | Returns items belonging to a specific folder (era) via the `archive_folder_items` junction table |
| 3260 | `archiveCycleState(env)` | Reads the current cycle state from `cycle_snapshots` table. Returns: dominant_cycle, dominant_phase, dominant_confidence, cycle_scores, active_phenomena, items_in_window, window_start, window_end, snapshot_at |
| 3276 | `archiveActivity(env)` | Returns recent collection activity by merging data from: collection_log, archive_audit_queue, archive_sync_runs, archive_items (LIVE), and cycle_snapshots |
| 3329 | `diversifyActivityEvents(events, options)` | Deduplicates and diversifies activity events so the feed shows varied activity types |

### Helper Functions

| Line | Function | Purpose |
|------|----------|---------|
| 2951 | `collectionSourceEvidence(details, sourceType)` | Generates evidence metadata for collection sources |
| 2958 | `deriveCollectionHealth({...})` | Derives overall collection health status from multiple signals |
| 2983 | `disTrackerCollectorHealth(newestLiveCollectedAt, lastVerifiedPollAt)` | Computes collector health based on freshness of live data |
| 3000 | `archiveMirrorHealth(latestRun)` | Computes mirror sync health from the latest sync run |
| 3096 | `clampLimit(value, fallback)` | Clamps a limit value to a safe range for pagination |
| 3172 | `normalizePhenomena(value)` | Parses the phenomena field, which can be an array, a JSON string, or a comma-separated string. Returns a clean array of phenomenon names. Used everywhere phenomena are processed |
| 3245 | `archiveActivityEvent(kind, row, ...)` | Creates a normalized activity event object from a database row |
| 3218 | `proxyOriginalArchive(request, env)` | Proxies requests to the original archive backend (fallback for unmatched archive routes) |

---

## LAYER 6: PENS API (`/api/pens/methodology`)

The PENS API serves PENS methodology data and training analyses. This is the server-side access point for the PENS data that lives in the Private Workspace. PENS data is stored in separate Supabase tables and accessed through this API — it is NOT embedded in the DIS core.

### Functions

| Line | Function | Purpose |
|------|----------|---------|
| 2662 | `pensMethodology(env)` | Fetches PENS methodologies from `ida_pens_methodologies` table and training analyses from `ida_pens_training_analyses` table. Returns a combined payload. Public endpoint (methodologies listed without auth) |
| 2697 | `validPensAnalysisId(value)` | Validates a PENS analysis ID format |
| 2702 | `pensTrainingAnalysis(request, env, analysisId)` | Fetches a specific PENS training analysis by ID. Requires authorization (either owner session or access code) |
| 2739 | `handlePens(request, env)` | The PENS API route handler. Routes `/api/pens/methodology` to `pensMethodology()` and `/api/pens/methodology/training/{id}` to `pensTrainingAnalysis()` |

---

## LAYER 7: IDA — THE AI ANALYST

Ida is the AI analyst that queries the classified archive. She uses Google's Gemini model (`gemini-flash-lite-latest`) through an agentic loop that can call tools to gather data before responding. This is the layer where the concepts (cycles, phenomena, eras) become queryable through Ida's tools.

### Configuration

| Line | Variable | Value |
|------|----------|-------|
| 3406 | `GEMINI_MODEL` | `"gemini-flash-lite-latest"` — the Gemini model Ida uses |
| 3407 | `IDA_SYSTEM_PROMPT` | The base system prompt defining Ida's identity. **THIS IS WHERE THE PENS EMBEDDING ISSUE LIVES** — the prompt names "PENS methodology studies" as a core function, causing Ida to default to PENS. See PENS Embedding Investigation document for the recommended fix |
| 3789 | `TERMINAL_SYSTEM_PROMPT` | The terminal-specific system prompt. Concatenates `IDA_SYSTEM_PROMPT` with detailed tool usage instructions, methodology guidance, and metadata field query patterns. This is the prompt that governs Ida's behavior in the agentic terminal |

### Ida's System Prompt (Current — with PENS embedding issue)

```javascript
var IDA_SYSTEM_PROMPT = "You are Ida, the AI system built by Javelion Analytics™
  (Jay Griffin / JU) for the BPHEN/DIS archive and the Dependency Loop™ reparations
  research project. You are precise, evidence-driven, and direct. You help review,
  verify, and analyze archive records, PENS methodology studies, and workspace notes.
  Keep responses concise and useful. Never fabricate sources or data.";
```

The phrase "PENS methodology studies" in this prompt is the root cause of Ida's PENS focus. The recommended clean version (pending JU approval) removes PENS from Ida's identity and adds cycles/phenomena/era as her core function:

```javascript
// RECOMMENDED CLEAN VERSION (not yet implemented):
var IDA_SYSTEM_PROMPT = "You are Ida, the AI system built by Javelion Analytics™
  (Jay Griffin / JU) for the BPHEN/DIS archive and the Dependency Loop™ reparations
  research project. You are precise, evidence-driven, and direct. You help review,
  verify, and analyze archive records classified by microcycle, phenomenon, and era.
  Keep responses concise and useful. Never fabricate sources or data.";
```

### Ida's Tool Architecture (TERMINAL_TOOLS, line 3416)

Ida has 6 tools available in her agentic loop. These tools are what allow her to access the DIS's classified data without that data being embedded in her prompt.

#### Tool 1: search_archive (line 3416)
**Implementation:** `archiveSearchTool(env, query, limit, filters)` (line 3496)

Searches the archive by text query AND/OR metadata field filters. This is Ida's primary research tool. It calls `allBrowsableArchiveItems()` to get all records, then filters in-memory by:
- `query` — text search against title and snippet
- `primary_cycle` — filter by cycle (FREEDOM_SUPPRESSION, ADVANCEMENT_RESISTANCE, INSTITUTIONAL_RESPONSE, CULTURAL_POWER_DYNAMICS)
- `source_type` — LIVE or HISTORICAL
- `phenomenon` — filter by phenomenon (ECLIPSE, DORMANCY, EXODUS, COVERT_MODE, CATALYST, REVERSAL, CONVERGENCE)
- `era` — filter by era slug (resolves through archive_folders → archive_folder_items junction)
- `date_from` / `date_to` — date range filter on published_at/collected_at

Returns full metadata for each match: id, title, snippet, source_type, primary_cycle, phenomena (normalized array), era, geo_scope, published_at, collected_at, source_name, source_url. Default limit 10, max 30.

#### Tool 2: cycle_distribution (line 3447)
**Implementation:** `cycleDistributionTool(env, filters)` (line 3569)

Groups archive records by publication date and counts how many records fall into each microcycle per date. This shows how cycle dominance shifts over time. Supports filtering by source_type, era, primary_cycle, date_from, date_to.

Returns: `cycle_totals` (overall counts per cycle), `distribution` (array of per-date objects with counts for each cycle + total), `date_count`, and `filters_applied`.

This is the tool that makes cycle pattern analysis possible. Instead of searching for text about "cycles," Ida can see the actual distribution of classified records over time.

#### Tool 3: detect_transitions (line 3462)
**Implementation:** `transitionDetectionTool(env, filters)` (line 3624)

Performs sliding-window analysis on the cycle distribution to detect points where the dominant microcycle shifts. Calls `cycleDistributionTool()` first, then slides a window (configurable size 1-7, default 3) across the dates, summing cycle counts per window, identifying the dominant cycle per window, and detecting where the dominant cycle changes between consecutive windows.

Returns: `window_analyses` (per-window dominant cycle, sums, dominance ratio), `transitions` (array of shift points with preceding/succeeding windows, shift description like "FREEDOM_SUPPRESSION → ADVANCEMENT_RESISTANCE", transition date range), `transition_count`, and the full distribution data.

This is the tool that allows Ida to answer "when did the pattern change" and "what triggered a cycle shift" — the exact capability that was missing before the upgrade and caused her to fail the microcycle investigation.

#### Tool 4: web_search (line 3480)
**Implementation:** `webSearchTool(query)` (line 3688)

Runs an external web search via DuckDuckGo HTML scraping. Used for supplemental research beyond the archive — identifying real-world events that match transition dates, finding news context, etc.

#### Tool 5: list_project_folders (line 3486)
**Implementation:** `listProjectFoldersTool(env)` (line 3729)

Lists the owner's existing project folders from `workspace_project_folders`. Used when Ida needs to know where to file her findings.

#### Tool 6: append_project_note (line 3490)
**Implementation:** `appendProjectNoteTool(env, folderName, title, content)` (line 3740)

Saves a note to a project folder in `workspace_notes`. If the named folder doesn't exist, it's created. This is how Ida persists her research findings for JU to review later.

### Tool Dispatcher

**`runTerminalTool(env, name, args)`** (line 3778) — Routes tool calls to their implementations:
- `search_archive` → `archiveSearchTool()`
- `cycle_distribution` → `cycleDistributionTool()`
- `detect_transitions` → `transitionDetectionTool()`
- `web_search` → `webSearchTool()`
- `list_project_folders` → `listProjectFoldersTool()`
- `append_project_note` → `appendProjectNoteTool()`

### The Agentic Loop

**`callGeminiAgentic(env, history, userText)`** (line 3791) — This is the heart of Ida's intelligence. It implements an agentic loop:

1. Builds the `contents` array from conversation history + new user message
2. Loops up to `maxTurns = 12` times (upgraded from 5 in the base version)
3. Each turn: sends the conversation + system prompt + tools to Gemini's `generateContent` API
4. If Gemini returns function calls: executes each tool via `runTerminalTool()`, records the step, and feeds the results back to Gemini
5. If Gemini returns text (no function calls): that's the final response — break the loop
6. If the loop exhausts all 12 turns without a text response: returns a "did not reach final summary" message
7. Returns `{ text, model, provider, steps }` — the final text plus the trace of all tool steps

The `steps` array is what the terminal UI renders as the tool trace — each step shows the tool name, arguments, and result.

### Non-Agentic Gemini Call

**`callGemini(env, history, userText)`** (line 3857) — A simpler Gemini call without tools. Used for non-terminal contexts where tool use isn't needed. Sends the conversation + system prompt to Gemini and returns the text response directly.

---

## LAYER 8: WORKSPACE API (`/api/workspace/*`)

The workspace API handles conversations, messages, notes, and project folders. All endpoints require owner session authentication (the `hasOwnerSession` check at the top of `handleWorkspace`).

### Route Handler

**`handleWorkspace(request, env)`** (line 3891) — Routes based on URL path:
- `/api/workspace/conversations` → `handleConversationsList()`
- `/api/workspace/conversations/{id}` → `handleConversationById()`
- `/api/workspace/conversations/{id}/messages` → `handleConversationMessages()`
- `/api/workspace/notes` → `handleNotesList()`
- `/api/workspace/notes/{id}` → `handleNoteById()`
- `/api/workspace/project-folders` → `handleProjectFoldersList()`
- `/api/workspace/project-folders/{id}` → `handleProjectFolderById()`

### Handlers

| Line | Function | Purpose |
|------|----------|---------|
| 3932 | `handleConversationsList(request, env)` | GET: lists all conversations. POST: creates a new conversation. PATCH: renames. DELETE: deletes. All scoped by `owner_key = IDA_OWNER_KEY` |
| 3966 | `handleConversationById(request, env, id)` | GET: returns a conversation with its messages. DELETE: deletes a conversation and its messages |
| 3992 | `handleConversationMessages(request, env, conversationId)` | POST: sends a message. Saves the user message to `workspace_messages`, calls `callGeminiAgentic()` to get Ida's response (with tool steps), saves Ida's response to `workspace_messages`, and returns the response with steps. This is the endpoint the Ida Terminal calls when you send a message |
| 4067 | `handleNotesList(request, env)` | GET: lists all notes. POST: creates a note. PATCH: updates. DELETE: deletes |
| 4099 | `handleNoteById(request, env, id)` | GET: returns a note. DELETE: deletes a note |
| 4126 | `handleProjectFoldersList(request, env)` | GET: lists folders. POST: creates a folder. PATCH: renames. DELETE: deletes |
| 4154 | `handleProjectFolderById(request, env, id)` | GET: returns a folder. PATCH: updates folder metadata |
| 4179 | `slugifyFolderName(name)` | Converts a folder name to a URL-safe slug |

---

## LAYER 9: MAIN ROUTER (Worker Entry Point)

**`ida_archive_worker_default`** (line 4279) — The main worker export. The `fetch(request, env)` method is the entry point for all requests. It routes by pathname:

```
/api/archive                    → handleArchive()
/api/pens/methodology           → handlePens()
/api/pens/methodology/training/* → handlePens()
/api/login                      → handleLogin()
/api/logout                     → handleLogout()
/api/workspace/*                → handleWorkspace()
/main-navy.js                   → serveEnhancedMainScript()
/sw.js                          → 410 "Service worker retired"
* (everything else)             → serveAsset()
```

The `serveAsset()` fallback handles all HTML page requests — it fetches from Cloudflare Pages, rewrites the content, injects the client script, and serves the result.

---

## LAYER 10: RESPONSE UTILITIES

| Line | Function | Purpose |
|------|----------|---------|
| 2581 | `json(body, status)` | Creates a JSON `Response` with standard headers |
| 2586 | `jsonWithHeaders(body, status, headers)` | Creates a JSON `Response` with additional headers (used for setting cookies) |

---

## COMPLETE API ENDPOINT MAP

### Public Endpoints (no auth)
| Method | Path | Handler | Purpose |
|--------|------|---------|---------|
| GET | `/api/archive?type=stats` | `archiveStats` | Archive statistics |
| GET | `/api/archive?type=health` | `archiveHealth` | System health |
| GET | `/api/archive?type=activity` | `archiveActivity` | Collection activity feed |
| GET | `/api/archive?type=cycle-state` | `archiveCycleState` | Current cycle dominance state |
| GET | `/api/archive?type=folders` | `archiveFolders` | System folders (eras) |
| GET | `/api/archive?type=folder-items&folder={id}` | `archiveFolderItems` | Items in a folder |
| GET | `/api/archive?type=items&{filters}` | `archiveItems` | Filtered archive items |
| GET | `/api/archive` (no filters) | `proxyOriginalArchive` | Original archive proxy |
| GET | `/api/pens/methodology` | `pensMethodology` | PENS methodology list |
| GET | `/main-navy.js` | `serveEnhancedMainScript` | Enhanced nav script |
| GET | `/*` | `serveAsset` | HTML pages (proxied from Pages) |

### Authenticated Endpoints (requires owner session)
| Method | Path | Handler | Purpose |
|--------|------|---------|---------|
| POST | `/api/login` | `handleLogin` | Two-step authentication |
| POST | `/api/logout` | `handleLogout` | Clear session |
| GET/POST/PATCH/DELETE | `/api/workspace/conversations` | `handleConversationsList` | Conversation CRUD |
| GET/DELETE | `/api/workspace/conversations/{id}` | `handleConversationById` | Single conversation |
| POST | `/api/workspace/conversations/{id}/messages` | `handleConversationMessages` | Send message to Ida |
| GET/POST/PATCH/DELETE | `/api/workspace/notes` | `handleNotesList` | Notes CRUD |
| GET/DELETE | `/api/workspace/notes/{id}` | `handleNoteById` | Single note |
| GET/POST/PATCH/DELETE | `/api/workspace/project-folders` | `handleProjectFoldersList` | Folder CRUD |
| GET/PATCH | `/api/workspace/project-folders/{id}` | `handleProjectFolderById` | Single folder |

### Operations-Auth Endpoints (requires ops token or owner session)
| Method | Path | Handler | Purpose |
|--------|------|---------|---------|
| GET | `/api/archive?type=audit` | `archiveAudit` | Audit workload |
| GET | `/api/pens/methodology/training/{id}` | `pensTrainingAnalysis` | Specific PENS study |

---

## WHAT THE UPGRADE ADDED (vs. THE BASE VERSION)

The 5 tool upgrades that SuperNinja implemented and deployed are part of the versioned copies, not the base master. For reference:

1. **search_archive upgraded** — Added metadata field filtering (primary_cycle, source_type, phenomenon, era, date_from, date_to) and rich results (full metadata per record). Base version was text-only search with minimal results.
2. **cycle_distribution added** — New tool. Groups records by date, counts per cycle. Did not exist in the base version.
3. **detect_transitions added** — New tool. Sliding-window transition detection. Did not exist in the base version.
4. **maxTurns increased** — Changed from 5 to 12. Base version had maxTurns=5, which was insufficient for multi-step forensic analysis.
5. **TERMINAL_SYSTEM_PROMPT rewritten** — Added metadata field query methodology, tool descriptions, LIVE-first default, and explicit warning against text-searching for cycle terms. Base version had a minimal prompt.

These changes are documented in detail in `/workspace/dis_update_proposals/06_ida_tool_upgrade.md`.

---

## SUMMARY: CONCEPT → CODE MAPPING

| DIS Concept | Code That Implements It |
|-------------|------------------------|
| Boot Terminal (auth) | `handleLogin()`, `createOwnerSession()`, `hasOwnerSession()`, `constantTimeEqual()` |
| Archive (980 records) | `allBrowsableArchiveItems()`, `archiveItems()`, `archiveStats()`, `filterDerivedBrowsableItems()` |
| 4 Microcycles (C1-C4) | `primary_cycle` field in `archive_items` table; filtered by `archiveSearchTool()`, counted by `cycleDistributionTool()`, analyzed by `transitionDetectionTool()` |
| 7 Phenomena | `phenomena` field in `archive_items` table; normalized by `normalizePhenomena()`; filtered by `archiveSearchTool()`; defined in `PHENOMENON_DEFINITIONS` (client-side, line 92) |
| 11 Eras | `archive_folders` + `archive_folder_items` tables; served by `archiveFolders()` and `archiveFolderItems()`; filtered by `archiveSearchTool()` via era slug resolution |
| Cycle State (dominance) | `cycle_snapshots` table; served by `archiveCycleState()`; computed independently by `cycleDistributionTool()` and `transitionDetectionTool()` |
| Ida (AI analyst) | `callGeminiAgentic()`, `callGemini()`, `IDA_SYSTEM_PROMPT`, `TERMINAL_SYSTEM_PROMPT`, `GEMINI_MODEL` |
| Ida's Tools | `TERMINAL_TOOLS` array, `runTerminalTool()` dispatcher, 6 tool implementation functions |
| PENS (Private Workspace) | `handlePens()`, `pensMethodology()`, `pensTrainingAnalysis()`; data in `ida_pens_methodologies` + `ida_pens_training_analyses` tables; accessed through `openPensMethodology()` client function |
| Workspace (conversations) | `handleWorkspace()`, `handleConversationsList()`, `handleConversationMessages()`; data in `workspace_conversations` + `workspace_messages` |
| Workspace (notes) | `handleNotesList()`, `handleNoteById()`; data in `workspace_notes` |
| DIS Tracker (collection) | `archiveHealth()`, `archiveActivity()`, `disTrackerCollectorHealth()`, `archiveMirrorHealth()`; data in `collection_log`, `archive_sync_runs` |
| HTML serving | `serveAsset()`, `rewriteRetiredWorkspaceLanguage()`, `stripPwaShell()`, `BPHEN_PAGES_ORIGIN` |
| Supabase integration | `supabase()`, `selectHeaders()`, `SUPABASE_URL`, `SUPABASE_SERVICE_KEY` |

---

This document, together with `DIS_MASTER_COPY_CONCEPTUAL.md`, constitutes the complete master copy of the DIS. The conceptual document explains what the DIS is and why each component exists. This code document explains how each concept is implemented in the actual JavaScript. With both documents, the DIS can be fully reconstructed, understood, and reverted if needed.