WebX11 Federation Protocol (v1)

Independent WebX11 instances can register with each other, prove identity, and search each other’s skill and product catalogs. This document is the contract; the implementation is api/federation_lib.py, api/federation-api.py and api/federation-cli.py.

Non-goals (not implemented, not planned): content replication between nodes, shared user accounts or SSO, shared payment/billing ledgers, auto-installing remote skills or products. Federation grants read-only discovery. Nothing a peer sends is ever executed.

1. Model

• Every node has a stable UUID (node_id), a public origin (url) and an API version (federation_api, currently 1).
• Peering is mutual and human-approved. A requests, B’s operator approves; only then do both sides hold the same HMAC secret.
• The API binds 127.0.0.1:8909; nginx exposes six exact-match paths. There is no HTTP admin surface — invites and approvals are CLI-only on the box.
• State lives outside the web root: /var/lib/webx11/federation (mode 700), files mode 600. Writes are atomic (tmp + fsync + os.replace) under an flock, because the health cron and the HTTP handler can both read-modify-write peers.json.

2. Identity — GET /federation/node.json

{
  "node_id": "98e6e07f-c602-4cee-bb5a-4e48f89b47c2",
  "name": "WebX11 Main",
  "url": "https://webx11.duckdns.org",
  "description": "Multi-tenant hosting + AI spaces",
  "software": "webx11",
  "federation_api": 1,
  "capabilities": ["skills.search", "products.search"]
}

url is an origin only (no path, no credentials). Anything else is rejected by safe_base_url(), and it must be https — an instance behind NAT or on a phone has to put a TLS tunnel in front of nginx (WX11_TLS_MODE=tunnel, see PLATFORMS.md), otherwise peers cannot register it.

3. Registration sequence

A: federation-cli.py add-peer https://B
   → POST /federation/announce  {node: A, token: <one-time>}
B: stores status=pending (never auto-accepted)
   ← 202 {node_id: B, name, approve_with}
B: federation-cli.py approve <A_node_id>      # human step
   → derives secret from a fresh invite code
   → POST /federation/accept  {node: B, token, invite}
A: checks token matches its pending-out entry
   → derives the same secret, status=active
   ← 200 {node_id: A, ts, nonce, sig}
B: verifies sig → peer active + verified

Any later call (/federation/handshake) is signed with that secret and updates last_ok / fails. Three consecutive failures mark the peer degraded; a signed response restores active. Alerts fire only on transitions.

4. Signing

canonical(payload) = json.dumps(payload, sort_keys=True, separators=(',', ':'))
sig = HMAC-SHA256(secret, f"{ts}\n{nonce}\n{canonical(payload)}").hexdigest()

• payload is always {"node_id": <signer>}.
• ts is epoch seconds; accepted only within ±300 s.
• nonce is a random hex string, single-use per (source, nonce) for 600 s.
• Verification order is signature first, then nonce — burning a nonce before checking the MAC would let an unauthenticated caller block a legitimate retry.
• secret is 64 hex chars, stored mode 600, never transmitted after bootstrap and never exposed in peers.json.

5. Secret derivation

secret = HMAC-SHA256(invite_code, f"{min(node_a,node_b)}|{max(node_a,node_b)}").hexdigest()

Both sides derive the same value from B’s one-time invite code (18 URL-safe chars). The code travels once, inside the /federation/accept body over TLS, and is not stored.

6. Endpoints

Rate-limit zones are fed_search and fed_peer in /etc/nginx/conf.d/wx11-zones.conf (http context — a limit_req_zone inside a server block is invalid). They are separate so a search-heavy client cannot starve its own handshakes.

Registry caps: 32 peers, pending requests expire after 14 days (pruned by the health pass).

7. Search contract

Request: GET /federation/search?q=<2-120 chars>&type=all|skills|products[&federated=1]

Response:

{
  "query": "nginx",
  "type": "all",
  "count": 6,
  "items": [
    {"kind": "skill", "id": "nginx-proxy-config-gotchas", "title": "…",
     "description": "…", "category": "devops", "version": "1.0.0",
     "origin": "<node_id>", "origin_name": "WebX11 Main",
     "external": false, "url": "https://webx11.duckdns.org/skills.html"}
  ]
}

Rules that matter:

• federated=1 (or X-Federation-Hop: 1) means “answer from your own catalog only.” Without it two nodes recurse A→B→A→B until both deadlines expire and the caller silently gets local-only results. This was observed in the two-node test (118 requests, zero peer results); the hop flag is mandatory for peer calls.
• Fan-out is capped at 8 peers, 20 items per peer, 80 items total, 2.5 s wall-clock budget. Stragglers are abandoned; partial results are returned.
• Results are cached 120 s per (query, type, federated); max 200 entries.
• Items whose origin equals our own node_id are dropped (a peer echoing our catalog back is not “external content”).
• A peer’s url field is discarded. Result links are rebuilt as <peer origin>/skills.html or <peer origin>/marketplace.html, because HTML-escaping does not neutralise a javascript: href.

8. Threat model

Federation peers are trusted to serve their own catalog and nothing else. A peer is not trusted to serve your catalog: the hop flag stops recursion, and origin filtering stops echoes.

9. Compatibility

• federation_api is an integer. A peer announcing a different value gets 409 and is not registered. Bump it on any breaking change to the payloads above.
• Additive changes (new optional fields, new kind values) do not require a bump: both sides ignore unknown fields.

10. Operating it

python3 api/federation-cli.py init --name "My Node" --url https://me.example.org
python3 api/federation-cli.py add-peer https://peer.example.org
python3 api/federation-cli.py pending
python3 api/federation-cli.py approve <node_id>
python3 api/federation-cli.py list
python3 api/federation-cli.py health      # silent when everything is fine
python3 api/federation-cli.py remove <node_id>

deploy/federation-health.sh wraps health for cron: it prints only on a transition and pipes non-empty output to notify.py when one is installed. deploy/test-federation.sh runs a full two-node handshake on one host using scratch state directories.