PAL - Personal AI Live-Assistant

Overview

PAL is a single-page, voice-first AI assistant that runs in the browser and talks to Claude. The user clicks a mic button (or starts a continuous "call") and speaks, and PAL transcribes the speech, sends it to Claude, and reads the reply back out loud. A settings drawer exposes voice, speech rate, pitch, listening language (English or Arabic), and how long PAL waits for silence before treating an utterance as finished.

Approach

The whole client is one HTML file (pal.html) with no framework and no build step, paired with a tiny Node server (server.js) that does two jobs: serve the static file and proxy requests to the Anthropic API so the browser never has to make a cross-origin call directly. The UI is built around an explicit state machine (idle, listening, thinking, speaking), which drives both a canvas-based animated "orb" visualization and the mic button's styling, so the interface always reflects what PAL is doing.

Architecture

• pal.html: all UI markup, styling, and client-side logic (Web Speech API for speech-to-text, speechSynthesis for text-to-speech, an animated node graph rendered on canvas as the visual "brain").
• server.js: a plain http server with no external dependencies that serves static assets from disk (with a path-traversal guard), proxies POST /api/chat to the Anthropic Messages API (forwarding the API key or falling back to ANTHROPIC_API_KEY, passing through SSE streaming), and handles POST /api/claude-code, which spawns the claude CLI with a prompt and JSON output and parses the result back into a short spoken reply.
• Two operating modes selectable in Settings: "Chat" (direct streamed Claude API calls) and "Claude Code" (drives the CLI against a configured working directory, keeping a session-key to session-id map in memory so multi-turn CLI sessions can be resumed).

Tech stack

• Vanilla JavaScript, HTML, CSS (no frameworks, no bundler)
• Node.js built-ins only (http, https, fs, path, child_process), no npm dependencies
• Browser Web Speech API (SpeechRecognition / webkitSpeechRecognition) for STT and speechSynthesis for TTS
• Anthropic Messages API (model configurable in Settings) via a server-side proxy
• Claude Code CLI, invoked as a local subprocess for the "Claude Code" mode

AI work

• Designed a system prompt tuned specifically for voice output: short sentences, no markdown or code blocks, headline-first answers, since the reply is read aloud by TTS rather than displayed as text.
• Built a sentence-boundary chunker that speaks completed sentences as they stream in from Claude rather than waiting for the full response, so PAL starts talking before the model has finished generating.
• Wrote a separate autonomy directive for Claude Code mode instructing the agent to make reasonable assumptions and use its file tools to implement changes, then give a one or two sentence spoken summary, since a voice user cannot easily answer follow-up clarifying questions.
• Used prompt caching (ephemeral cache_control on the system block) on the chat-mode API calls.

Engineering highlights

• No-build, no-dependency architecture: the entire app runs with node server.js and zero npm installs.
• Session continuity for Claude Code mode via an in-memory sessions map keyed by project working directory, so a project's voice conversation persists across turns without a database.
• A "Call" mode that loops listen, think, speak automatically using a configurable silence timeout, versus a manual one-shot mic press for single questions.
• Basic security handling on the static file server (path normalization plus a directory-escape check) and a clear separation of the API key, kept client-side in localStorage or supplied via environment variable, never hardcoded.