Browser Profiles & Agent Browser

A browser profile is a dedicated, persistent browser identity (cookies, local storage, session state) owned by exactly one user. Profiles are personal: every API and agent tool only ever sees the caller's own profiles.

• Local profiles (default provider) keep their Chromium user-data-dir on the instance under the shared workspace volume (.browser-profiles/<id>), so login state never leaves the Clapilot server.
• Browser Use Cloud profiles (browser-use provider, admin-configured) live at Browser Use and are only used by browser_use_run.

Profiles are created and managed in Settings → App Connections (/settings/app-verbindungen), in the browser profiles card. Creating one requires explicit consent, which is audited.

Creating a local profile is all-or-nothing: Clapilot first creates the profile's user-data directory (<CLAPILOT_WORKSPACE_DIR>/.browser-profiles/<id>) and only then stores the profile together with its profile_created audit event in a single database statement. If the directory cannot be created (for example because CLAPILOT_WORKSPACE_DIR is unset or not writable on that server), nothing is saved and the settings card shows "profile storage on this server is not writable" (HTTP 500); if saving fails (for example a duplicate name), the new directory is removed again. A failed attempt therefore never leaves a half-created profile behind. The same applies when the agent auto-creates its default Agent profile.

Signing in once (interactive live view)

Local profiles sign in through an interactive live view: Clapilot launches the profile's Chromium inside the web container and streams it into a dialog where the user types, clicks, and pastes like in a normal browser. Finishing the sign-in closes the browser so the state is flushed to disk and marks the profile ready. The same dialog opens from an agent chat card when the agent calls request_browser_signin (login, 2FA, or CAPTCHA the agent cannot supply) while it has no browser session open; with an open session the user takes over the agent's own browser instead (see Taking over the agent's browser).

The agent's browser

The agent drives a real Chromium on the instance with the local browser tools (browser_open, browser_navigate, browser_snapshot, browser_scroll, browser_click, browser_type, browser_read, browser_close) and can sign itself in with saved logins (browser_credentials_list, browser_fill_credential; see Saved logins). Each user has at most one agent browser session at a time, on one of their local profiles; it closes after 5 minutes without activity or after 20 minutes of agent time in total (time the user spends controlling the session does not count). When browser_open gets an explicit profile_id, that profile is used as given. Without one, the agent picks the user's most recently updated local profile whose sign-in is confirmed (ready); only if there is none does it fall back to the most recent profile still awaiting sign-in, and only if there is no such profile either does it create a default Agent profile. Disabled profiles are never picked automatically, and a newer profile that was never signed in cannot displace the one the user logged in with. See Agent tool contracts for the tool details.

Saved logins (automatic re-sign-in)

Profile sessions expire: a site can log the agent out after some days or weeks. So the agent does not have to ask the user every time, users can save website logins for it in Settings → App Connections → Saved logins (/settings/app-verbindungen, next to the browser profiles card; also in the native iOS/macOS connections settings). Each login is a website (domain or login URL), a username or email address, a password, and an optional label.

• Personal: like profiles, saved logins belong to one user and are only filled into that user's own agent browser.
• Write-only passwords: passwords are encrypted at rest (AES-256-GCM) and never shown again — not in the settings, not in any API response, and not to the agent. Editing a login with an empty password field keeps the stored password.
• Bound to the website: a login saved for example.com (or https://www.example.com/login) is only filled into pages on example.com and its subdomains, over https (http only if the saved address explicitly uses http://), and never into a form that submits to another website. This keeps a manipulated page or prompt from getting the agent to fill the password somewhere else.
• How the agent uses it: when a page shows a login form, the agent calls browser_credentials_list and then browser_fill_credential, which fills username and password directly from the server into the form; snapshots show password fields only as [filled]. For 2FA, CAPTCHAs, or a wrong password it still hands over to the user with request_browser_signin.
• Audit: every created, changed, or deleted login and every fill (or refused fill, with the page host) is recorded; the settings list shows when an agent last used each login.

Live agent browser window

While the agent has a browser session open, a floating window shows that browser live so the user can follow along.

• What it shows: a live picture of the page the agent is working on (CDP screencast, JPEG, at most ~12 frames per second), a live indicator with "Agent steuert den Browser", the browser profile name, and the current URL (read-only, truncated; hover shows the full address).
• View only until taken over: while the agent drives, the window never sends input to the browser ("Nur Ansicht"). The Übernehmen button hands the same session to the user (see below).
• When it appears: the web app checks for an active agent session every few seconds while the tab is visible (slower after a period without sessions, paused while the tab is hidden). Chat surfaces also trigger an immediate check as soon as a browser_* tool event streams in, so the window usually opens with the agent's first browser step.
• Controls (circular icon buttons in the header, plus the primary Übernehmen / An Agent zurückgeben button while the session is live):
  • Minimize collapses the window to a small pill with the live indicator and the current host name; clicking the pill restores it.
  • Maximize / Restore size toggles between the compact window (about 520 px wide) and a near-fullscreen view. Esc leaves the maximized view.
  • Close hides the window for the current agent session. It stays hidden for that session (remembered per browser tab); the next agent browser session opens it again.
• Placement: by default the window sits bottom-right, next to the floating chat dock. It can be dragged by its header; the position is remembered in the browser and kept inside the viewport. On phones (< 768 px) it starts as the minimized pill above the bottom tab bar and expands into a full-width sheet.
• When the session ends (the agent closes the browser, the session idles out, the agent switches profiles, or the profile is handed over for a separate human sign-in) the window stays open: it keeps the last frame and shows "Browser geschlossen" with the reason in a status strip until the user closes it. If the agent opens a new browser session meanwhile, the same window switches to it and keeps its current size (normal, minimized, or maximized).
• Privacy: sessions are keyed by the authenticated user. Only the owner of an agent browser session can see its status, watch it, or take it over; other users, including admins, never get another user's session. Watching does not count as activity and does not extend the session.
• Cost: the screencast only runs while at least one window is connected; an unwatched agent session costs nothing extra.

The window is available in the classic and the tab layout of the web app.

iOS and macOS

The native Apple clients show the same live view and take-over natively (same endpoints, same rules):

• Discovery: while the app is active and signed in, the client checks GET /api/browser-profiles/agent-session every 4 seconds (every 15 seconds after five checks without a session, paused in the background). A streamed browser_* tool event in the native chat triggers an immediate check.
• macOS: the live view opens in its own resizable Agent-Browser window with the native close, minimize, and zoom controls. A new agent session opens the window on its own unless the user closed it for that session. After the session ends, the window keeps the last frame with "Browser geschlossen" and the reason until the user closes it. Closing the window while in control hands the browser back.
• iOS/iPadOS: a compact floating pill (live indicator plus host name, or "Du steuerst den Browser") sits above the bottom inset. Tapping it opens a full-screen viewer. The chevron button minimizes it back to the pill, and the close button (on the pill or in the viewer) dismisses it for that session, handing control back first if the user holds it. After the session ends, the pill and viewer keep the last frame until the user closes them.
• Viewer: the frame is shown aspect-fit and letterboxed. The header shows the live indicator, the title ("Agent steuert den Browser" / "Du steuerst den Browser" / "Browser geschlossen"), the profile name, and the URL bar (read-only "Nur Ansicht" while the agent drives, editable while the user drives: Return navigates). The actions are Übernehmen / An Agent zurückgeben, plus Agent fortsetzen lassen for about 60 seconds after handing back, which sends the localized continue prompt through the native chat.
• Input while in control: on iOS/iPadOS, tap = click (double taps count as double clicks), one- or two-finger drag or trackpad scrolling = wheel, and pointer hover = mouse move. The keyboard button opens the software keyboard: typed text is inserted, Return/Tab/Backspace are sent as keys, and hardware keyboards add arrows, Escape, Home/End, Page Up/Down, and Delete. On macOS, mouse buttons with native click counts, drags, hover, and the scroll wheel/trackpad are forwarded. Keys are forwarded from the focused picture: printable ASCII as keys, umlauts and other characters as text, and Shift/Control/Command as modifiers (Command acts as the server's shortcut modifier for Command-A/C/X/Z/Y). Command-V pastes the local clipboard as text, and other Command shortcuts stay native. Points are mapped to frame pixels with the same letterboxing math as the web window, and input is sent in ordered batches of about 50 ms.
• Streaming: the SSE stream (/stream) runs only while the viewer is visible (the macOS window or the iOS full-screen viewer). It reconnects with backoff (1, 2, 4, 8, then 15 s) and decodes JPEG frames off the main thread, dropping stale frames. While only the iOS pill is visible, the status poll keeps the URL and controller up to date.
• Chat cards: takeOverActions buttons open the viewer for the current agent session and switch it into user control, falling back to the sign-in viewer when the session is gone. signInActions buttons start an interactive sign-in session (POST /api/browser-profiles/login-sessions) on the given profile, or on the most recent local profile (creating the default Agent profile if there is none). They then open the same viewer in sign-in mode, which is always interactive, navigates to the card URL once the first frame arrives, and offers Fertig (saves the login and marks the profile ready) and Abbrechen (discards the session).

Taking over the agent's browser

The user can take over the agent's live browser session — same page, cookies, login, and cart — to sign in, solve a CAPTCHA, or review a cart and place an order, and then hand it back.

• Who can: only the signed-in owner of the session, from their own live window. Sessions are keyed by user id; an optional session id sent by the window must match the current session, so a stale window can never act on a newer session.
• How to start: press Übernehmen in the window header (on phones, in the bottom bar of the sheet or next to the minimized pill), or press the Browser übernehmen button on the chat card the agent posts with request_browser_signin (take-over mode). The card opens or restores the window for the current session — even one the user had closed — and switches it into user control. If the session has already ended by then, the card falls back to the separate sign-in live view on the same profile. Taking over from the minimized pill restores the window. Take-over is not offered once the session has ended.
• While the user controls it: the header reads "Du steuerst den Browser", the window gets a primary focus ring, and the picture becomes interactive — click, double-click, drag, scroll (mouse wheel or touch drag), and type (including Enter, Tab, Backspace, arrow keys, shortcuts, umlauts and other non-ASCII characters, and pasting from the clipboard). Cmd on macOS acts as the server browser's shortcut modifier. The URL bar becomes editable; Enter navigates (http/https only, private hosts rejected). Popups the page opens (for example an OAuth sign-in window) are followed and the view returns to the main tab when they close. On phones, tap = click, drag = scroll, and a small typing field below the picture opens the keyboard and streams text into the page.
• What happens to the agent: its page-changing tools (browser_open, browser_navigate, browser_click, browser_type, browser_fill_credential, browser_scroll, browser_close) fail with a clear error ("The user has taken control of the browser. Do not act in the browser now; …"), so the agent tells the user it continues once control is handed back. browser_snapshot and browser_read keep working (read-only).
• Timeouts: user input counts as activity; the session still closes after 5 minutes without input, but the 20-minute limit is paused while the user is in control (only agent time counts toward it).
• Handing back: An Agent zurückgeben returns control (any keys or mouse buttons still pressed are released). For about 60 seconds the window then offers Agent fortsetzen lassen, which sends a chat message ("Ich habe den Browser zurückgegeben – mach bitte dort weiter.", localized) to the agent via the chat dock — queued if the chat is minimized. Closing the window while in control, or leaving the page from the tab that took over, also hands control back.
• Security: input is only accepted while the user holds control (409 otherwise); events are validated and capped server-side; navigation uses the same private-host blocklist as the agent and the sign-in view; each take-over is audited (profile_used, via: "agent_browser_takeover"). The agent itself still never submits orders or payments — the user does, in the taken-over window.

Agent access

Watching needs no agent tool: the window visualizes the agent's own browser_* activity. Handing the browser to the user is the agent's request_browser_signin tool: with an open session it returns mode: "takeover" and posts a card with a takeOverActions button; without one it returns mode: "signin" and posts the sign-in card (see Agent tool contracts). The native iOS and macOS chats render both takeOverActions and signInActions as buttons that open the native viewer (see iOS and macOS). The endpoints are listed in the API reference (/api/browser-profiles/agent-session, /stream, /control, /input, and /navigate).