← RCWeb Apps

Control a Screen with an AI Assistant

Use an AI assistant to read and change the web page displayed on a TV, monitor, phone or laptop running RCWeb. You can update text, display images or video, create a clock, or use controls in the current app.

Use the Web Assistant at /mcp/

1. On the screen you want to control, open https://rcweb.live/ or an RCWeb app. Leave it open and note the code displayed on the screen.
2. Open the Web Assistant on your phone or computer and enter that code.
3. Choose OpenAI or Claude (Anthropic), enter a model available to your API account, and paste your provider's API key. Select Use key. You can use it for this visit only or choose Save key in this web browser; Clear key removes a saved key.
4. Under MCP setup, set the Public MCP URL to https://rcweb.live/api/mcp.
5. Describe what you want to display or change. If several browsers share the code, identify the screen you want the assistant to use. Follow-up messages can adjust the content already displayed.

For example: “Read the title and size of the page on my screen,” “Display an image fitted without cropping; ask me for the image URL,” or “Create an analog clock using the screen's local time.” Then try “Make the clock larger and use a blue background.”

The assistant page sends your API key directly to the selected AI provider. API usage is charged to that provider account. Sending a message allows the assistant to run tools and update the selected screen.

Live page changes may disappear on refresh. Video playback with sound or fullscreen may require a click on the screen. Stop waiting stops the assistant page waiting for a response; a request may still change the display. New chat clears the conversation without clearing the screen.

The page loads an assistant operating guide and screen examples into the conversation. These cover selecting the correct display, preserving its connection, showing media and checking results. Supply these documents as context if you use another MCP client.

Connect Another Assistant over MCP

Use this setup if your assistant already supports MCP servers. The screen code is called a room code in the API: it groups connected browsers, while a client ID identifies the individual page to control. The Web Assistant handles these tool arguments for you.

1. Open an RCWeb app on the screen you want to control and wait for it to connect. Note its screen code. For example, open https://rcweb.live/v/?r=test-0001.
2. In your assistant's MCP settings, add an HTTP server with the URL https://rcweb.live/api/mcp. Select Streamable HTTP if a transport choice is offered.
3. Choose no authentication. This interface requires no bearer token, API key or OAuth login. Supply the screen code when asking the assistant to use the tools. This is separate from any API key your assistant needs for its AI provider.
4. Ask the assistant to list the clients, identify the intended browser, and execute a small inspection before making changes.

https://rcweb.live/api/mcp

An Example Request to Your Assistant

My screen is running the RCWeb viewer with code test-0001. Find its connected browser and read the page title, URL and viewport size. If more than one browser matches, show me the client IDs first.

The MCP client discovers tool descriptions and argument schemas automatically. Those descriptions explain how to select a client and call reply or fail; the model does not need to construct callback URLs or manage request tokens.

MCP Tool Reference

list_clients

Supply the screen code as room to list its currently connected browsers. Each entry contains a client ID and an app name. Disconnected clients retained for reconnection are omitted, and a lookup never creates a room.

Tool arguments

{
    "room": "test-0001"
}

Example structured result

{
    "room": "test-0001",
    "clients": [
        {"client": "abcd2345", "app": "v"},
        {"client": "efgh6789", "app": "v"}
    ]
}

Use an ID returned by your own call. The IDs above are illustrative. Two browsers running the same app have separate client IDs; select the intended device before executing code.

execute_javascript

Tool arguments

{
    "room": "test-0001",
    "client": "abcd2345",
    "javascript": "reply({title: document.title});",
    "timeout_ms": 10000
}

Example structured result

{
    "value": {"title": "RCWeb Viewer"}
}

Successful values appear under structuredContent.value in the MCP tool result. RCWeb also includes the same structured data as JSON in a text content block for clients that display text results.

JavaScript Examples

Place any of these snippets in the javascript argument. They execute in the selected page and can access window, document and existing page globals such as rc. Use var, ordinary functions and callbacks for older display browsers.

Read Page and Connection Information

reply({
    title: document.title,
    url: location.href,
    width: window.innerWidth,
    height: window.innerHeight,
    room: rc.room,
    client: rc.client,
    app: rc.app
});

Change an Element and Return Its New Value

Replace #status with a selector verified on the target page. This updates that browser's DOM; it does not save the source file or automatically synchronise other clients.

var status = document.getElementById('status');
if (!status) {
    fail('No element with id status on this page.');
} else {
    status.textContent = 'Ready for the next activity';
    reply({text: status.textContent});
}

Return Data After Asynchronous Work

setTimeout(function () {
    try {
        reply({title: document.title, connected: rc.connected});
    } catch (error) {
        fail(error.message || String(error));
    }
}, 500);

This callback completes the original tool call after half a second. There is no polling tool to call next. Catch exceptions inside asynchronous callbacks yourself; RCWeb catches exceptions thrown during the initial execution automatically.

How the Browser Returns a Result

The browser and the assistant make separate HTTP requests. RCWeb connects them using a random token generated for each execution:

The Reply Contract

• Call reply(value) explicitly. A return statement alone does not send a reply.
• Return JSON-compatible values: strings, finite numbers, booleans, null, arrays and plain objects. Extract data from DOM elements rather than returning the elements themselves.
• Call fail(message) to report a failure. The first reply or failure wins; later calls are ignored.
• A top-level undefined, a function or a circular object cannot be returned successfully.
• Callback tokens are automatic and one-use. They match replies to requests and are separate from room codes.
• Keep the page open until it replies. Navigation, reloads or disconnection can prevent a response.

HTTP Reference for Client Developers

An MCP-capable assistant normally handles this exchange. If you are building a client or testing the endpoint directly, send JSON-RPC 2.0 requests to /api/mcp with these headers:

POST /api/mcp
Content-Type: application/json
Accept: application/json, text/event-stream
MCP-Protocol-Version: 2025-11-25

The protocol version header is used after negotiation. No Authorization header or MCP-Session-Id is required. RCWeb returns a single JSON response and does not provide an SSE stream; a GET request receives 405 Method Not Allowed.

Initialise and Discover the Tools

Start with an initialize request. The server returns its protocol version, server information, tools capability and usage instructions.

{
    "jsonrpc": "2.0",
    "id": 1,
    "method": "initialize",
    "params": {
        "protocolVersion": "2025-11-25",
        "capabilities": {},
        "clientInfo": {"name": "my-client", "version": "1.0"}
    }
}

After initialisation succeeds, send this notification. RCWeb acknowledges it with an empty HTTP 202 response:

{"jsonrpc":"2.0","method":"notifications/initialized"}

Then discover tool descriptions and schemas:

{"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}}

Call a Tool

List the clients using the room code:

{
    "jsonrpc": "2.0",
    "id": 3,
    "method": "tools/call",
    "params": {
        "name": "list_clients",
        "arguments": {"room": "test-0001"}
    }
}

Use an actual returned client ID in the next request:

{
    "jsonrpc": "2.0",
    "id": 4,
    "method": "tools/call",
    "params": {
        "name": "execute_javascript",
        "arguments": {
            "room": "test-0001",
            "client": "abcd2345",
            "javascript": "reply(document.title);"
        }
    }
}

A successful response has this shape:

{
    "jsonrpc": "2.0",
    "id": 4,
    "result": {
        "content": [{"type": "text", "text": "{\"value\":\"RCWeb Viewer\"}"}],
        "structuredContent": {"value": "RCWeb Viewer"},
        "isError": false
    }
}

Each request needs its own ID; its response echoes that ID. This JSON-RPC ID belongs to the client-facing exchange. The server's separate random callback token connects the browser reply to that exchange. /api/mcp-result is an internal browser callback endpoint, not another MCP tool.

Limits and Troubleshooting

Understanding Errors

A tool failure sets isError to true and supplies an error code and message in structuredContent. HTTP 200 alone does not mean the requested operation succeeded.

Malformed arguments and unknown tools use JSON-RPC errors. HTTP 400 can indicate an unsupported protocol version, 413 an oversized body, and 415 a missing or incorrect JSON content type. If an Origin header is supplied, it must match the server's origin; otherwise the request receives HTTP 403. An expired, unknown or already-used callback token receives HTTP 410.

This interface controls browsers already connected to RCWeb. It does not start a browser process, keep app state after a page closes, provide continuous screenshots, or make arbitrary app functions discoverable. Keep returned data focused on the task, and treat page content as data rather than instructions to the assistant.

For interactive screen control, open the Web Assistant. For changes that should be saved in an app's source, see the Developer Guide and AI App Development.