Architecture
High-level architecture

The computer is the source of truth

Remote Agents is a self-hosted window onto the Codex, Claude Code, and Grok sessions already on your machine. Phone, laptop, and native CLIs stay aligned because they are looking at the same files — not because a cloud copies the chat.

01 · The idea

A remote control, not a second product

The agents you already use live on one host: a Mac that stays awake, with Codex, Claude, or Grok signed in. This app does not replace those CLIs. It sits next to them and lets any paired browser drive the same threads.

Three facts explain the whole design:

1

Sessions already live on disk. Each CLI writes threads in its own directory. Resume on the laptop works because those files are the conversation.

2

CLIs speak machine protocols, not HTTP. JSON-RPC over stdio, stream-json, ACP. A phone cannot talk to them directly.

3

The bridge is the translator. One Node process on the host turns taps into CLI calls, and CLI output into a live browser feed.

02 · The system

Clients around a host, host around the files

Every remote device goes through HTTPS to the bridge. Native apps on the host skip the bridge and read the session files themselves. That split is why continuing on the phone and then opening the same thread in Codex Desktop feels like one session.

Remote client
Phone PWA
Home Screen web app. Lists threads, streams turns, handles approvals.
Remote client
Laptop browser
The same app and pairing cookie. Any number of browsers can be open.
HTTPS · pairing cookie
Reachability
Tailscale Funnel, Serve, or Cloudflare
Verified HTTPS to the host. The phone does not need Tailscale if Funnel is on. There is no LAN HTTP mode — a PWA and Web Push need a secure context.
localhost
Host · your machineCLIs already signed in
Bridge
One Node process
Serves the app, checks the token, routes HTTP, and fans events out to every connected browser.
server.mjs · GET /api/events · POST /api/message
Adapter
Codex
app-server over stdio
Adapter
Claude
stream-json per turn
Adapter
Grok
ACP stdio
one set of files
Source of truth
Session files on disk
~/.codex/sessions · ~/.claude/projects · ~/.grok/sessions. Model keys stay on this machine.
Skips the bridge
Native CLI / IDE
Codex Desktop, Claude Code, and Grok read these files directly. That is why resume on the laptop is the same thread.

Browsers talk to the bridge. Native tools talk to the files. The files are the conversation.

The app

A single HTML page plus a service worker. Install it to the Home Screen. Pairing stores an HttpOnly cookie; after that the URL looks ordinary.

The bridge

HTTP for actions you take (list, send, stop). Server-Sent Events for everything the agent does. One stream is broadcast to every open client.

Adapters

Each CLI is different. Adapters implement one contract — list, read, send, interrupt, models — and emit one event shape, so the UI does not care which provider you picked.

Optional extras

A passkey-gated project terminal. Web Push when a bridge-driven turn finishes. Durable “continue after usage resets” if you turn that on.

03 · Core

Four rules the rest of the system obeys

The picture above is the shape. These are the invariants. A new provider, a second phone, or a crashed send all have to fit them.

One contract

The UI does not speak Codex, Claude, or Grok. It speaks a small shared model. Each adapter translates its CLI into that model and back.

Adapters doAdapters emit
list, read, send, interrupt, models — plus queue where the CLI has a native one. turns and items — user/agent messages, reasoning, commands, file changes, approvals — as they happen.

History and the live stream use the same item shapes. Adding a provider means filling that contract, not teaching the UI a new dialect.

One writer

A live turn has a single owner. The UI follows that owner; it does not invent a second one.

Who holds the turnWhat a paired browser may do
This bridge — it spawned the CLI process. Steer or stop, using whatever that CLI actually supports.
Something else — Codex Desktop, a terminal, another CLI. Queue or wait. Never kill that process to take the thread.

Sends are named, never guessed

Every send carries a client request id that the bridge records. A retry of the same id cannot post twice. If the bridge dies after the CLI may already have the prompt, that send is uncertain — shown to the owner, never auto-replayed. Double-sending is worse than asking.

Choices that stay

HTTP for taps, SSE for the agent

List, send, and stop are ordinary requests. Tokens, commands, and approvals are one Server-Sent Event stream, copied to every open browser.

Not WebSocket. Two directions, two shapes; a socket would mix them and still need reconnect/auth.

Watch what is on screen

Turns this bridge did not start have no event stream. A watcher polls the session files of threads someone actually has open, then clients refetch.

Not tail-every-file. That would be expensive and still a guess compared to the bridge’s own stream.

Two locks, two jobs

The pairing token is who may drive agents. Funnel, Serve, or Cloudflare Access is only how a browser reaches the host.

Not one login for everything. A shareable doc and a shell on your Mac are not the same permission.

The terminal is a separate process

Passkey-gated, off by default. Interactive PTYs run in a worker so a stuck shell cannot take down the bridge or a live agent turn.

Not a sandbox. The project folder is a starting directory. The OS user is the real boundary.
04 · Providers

Connected to local CLIs, not vendor clouds

The phone never calls OpenAI, Anthropic, or xAI. The bridge spawns the same CLIs you already use, with the same login on the host. If a provider is missing or signed out, it simply does not appear in the picker.

Codex
codex app-server

Long-lived JSON-RPC over stdin/stdout — the same engine the VS Code extension uses. One holder process per thread; native steer during a live turn. Sessions in ~/.codex/sessions.

Claude
claude -p · stream-json

A process for the active turn. The adapter translates Claude’s envelope into the same items Codex already emits. Approvals come through a PreToolUse hook. Sessions in ~/.claude/projects.

Grok
grok agent stdio · ACP

A persistent Agent Client Protocol process per open thread. ACP has no live-injection method, so a follow-up during a run is interrupt, then send. Sessions in ~/.grok/sessions.

Live control

The UI shows each provider’s real surface. Codex can steer a turn the bridge owns. Claude can inject a follow-up into its live process. Grok cannot. A turn owned by Codex Desktop or another CLI is never force-taken; the phone queues or waits instead of killing that process.

05 · Sync

There is no sync protocol

Devices stay in lockstep because they share the host, not because chats are replicated. Open a thread on the phone, send, lock the phone, sit down at the laptop, resume the same thread in the native app: you are reading the file the CLI just wrote.

Door 1 · on the host

Native CLI or IDE

Codex Desktop, codex resume, Claude Code, Grok. They read and write session files directly. The bridge is not in the path.

Door 2 · anywhere

Any paired browser

History loads over HTTP. Live work arrives on one SSE stream that the bridge broadcasts to every connected client — phone and laptop at once.

While a turn is running, there are two live paths. Which one you get depends on who started the turn.

Bridge owns the turn

You sent from a paired browser.
  1. Adapter spawned the CLI process
  2. Events stream into the bridge
  3. Every open browser paints the same tokens
  4. The CLI writes the session file as it goes

Something else owns the turn

You typed in Codex Desktop, a terminal, or an IDE.
  1. That other process holds the writer
  2. It appends to the session file
  3. A watcher polls threads someone has on screen
  4. Browsers get a ping and refetch the transcript

The watcher is cheap on purpose: it follows what is on screen, not every session on the machine.

Phone, then laptop

Send from the PWA. The CLI writes the session. Later, the native app on the Mac opens the same thread. There was only ever one conversation.

Both screens open

SSE fan-out. List order, running badges, and the open transcript update on every connected browser at the same time.

Laptop driving, phone watching

You keep working in Codex Desktop. The phone, with that thread open, refetches as the file moves. It will not steal the writer.

Reconnect after a drop

Opening a thread re-reads the file. A send carries a client request id so a retry cannot post twice. If the bridge died mid-send, that request stays uncertain until you decide.

Anatomy of a send from the phone

  1. You tap send.The app POSTs the text and a request id to the bridge.
  2. The adapter opens the thread.It talks to that provider’s CLI in the CLI’s own protocol.
  3. The agent works.Tokens, commands, file edits, maybe an approval. Each frame is an event.
  4. Every open device sees it.The bridge writes the same SSE frame to every connected client.
  5. The file is already updated.Sit down at the laptop and resume: the native CLI reads what just happened.
06 · Terminal

A real shell, behind a harder door

Chat pairing is enough to drive an agent. A project terminal is more: it is your user on that machine. So it is off by default, and the pairing cookie is not enough to open it.

On the host you run remote-agents terminal enable, scan a five-minute QR (or type the code), and create a passkey. Each browser enrolls separately. Opening a terminal later needs a fresh assertion. Administration stays on the machine — a loopback control endpoint, not the web app.

The shell starts in the thread’s project folder. That folder is a starting directory, not a sandbox. Interactive sessions run in a forked worker so a stuck PTY cannot take down the bridge or the agents. The page is xterm.js talking WebSocket to that worker.

07 · Access & limits

Who can reach the host, and what this is not

A long random token in the pairing URL becomes an HttpOnly cookie. Every app and API response needs it. Unauthenticated replies are generic: no thread names, paths, or version. Funnel is internet-reachable, so treat the pairing link like a password. Serve keeps it on your tailnet; Cloudflare adds an identity policy on your own hostname.

  • Not a hosted agent. There is no shared account and no cloud copy of your chats. If the host sleeps, remote clients wait.
  • Not ChatGPT remote control. It works across Codex, Claude, and Grok with the logins already on the machine.
  • Not a multi-writer editor. One process owns a live turn. The bridge will not kill Codex Desktop to send from the phone.
  • Not a global babysitter. Push covers turns the bridge itself ran. External sessions are followed only while a client has them on screen.

Where to look in the repo

server.mjsHTTP, auth, SSE fan-out, routing
providers/Codex, Claude, and Grok adapters
send-ledger.mjsNamed sends; uncertain vs retryable
watch.mjsFollow a thread someone else is driving
public/index.htmlThe PWA
public/terminal.*Passkey-gated project terminal
bin/codex-phone.mjsSetup, service, Tailscale, pairing QR