# WebX11 Federation Protocol (v1)

Independent WebX11 instances can register with each other, prove identity, and search each
other's skill and product catalogs. This document is the contract; the implementation is
`api/federation_lib.py`, `api/federation-api.py` and `api/federation-cli.py`.

**Non-goals (not implemented, not planned):** content replication between nodes, shared user
accounts or SSO, shared payment/billing ledgers, auto-installing remote skills or products.
Federation grants **read-only discovery**. Nothing a peer sends is ever executed.

---

## 1. Model

- Every node has a stable UUID (`node_id`), a public origin (`url`) and an API version
  (`federation_api`, currently `1`).
- Peering is **mutual and human-approved**. A requests, B's operator approves; only then do
  both sides hold the same HMAC secret.
- The API binds `127.0.0.1:8909`; nginx exposes six exact-match paths. There is **no HTTP
  admin surface** — invites and approvals are CLI-only on the box.
- State lives outside the web root: `/var/lib/webx11/federation` (mode 700), files mode 600.
  Writes are atomic (`tmp` + `fsync` + `os.replace`) under an `flock`, because the health
  cron and the HTTP handler can both read-modify-write `peers.json`.

## 2. Identity — `GET /federation/node.json`

```json
{
  "node_id": "98e6e07f-c602-4cee-bb5a-4e48f89b47c2",
  "name": "WebX11 Main",
  "url": "https://webx11.duckdns.org",
  "description": "Multi-tenant hosting + AI spaces",
  "software": "webx11",
  "federation_api": 1,
  "capabilities": ["skills.search", "products.search"]
}
```

`url` is an origin only (no path, no credentials). Anything else is rejected by
`safe_base_url()`, and it **must be https** — an instance behind NAT or on a phone has to put a
TLS tunnel in front of nginx (`WX11_TLS_MODE=tunnel`, see `PLATFORMS.md`), otherwise peers
cannot register it.

## 3. Registration sequence

```
A: federation-cli.py add-peer https://B
   → POST /federation/announce  {node: A, token: <one-time>}
B: stores status=pending (never auto-accepted)
   ← 202 {node_id: B, name, approve_with}
B: federation-cli.py approve <A_node_id>      # human step
   → derives secret from a fresh invite code
   → POST /federation/accept  {node: B, token, invite}
A: checks token matches its pending-out entry
   → derives the same secret, status=active
   ← 200 {node_id: A, ts, nonce, sig}
B: verifies sig → peer active + verified
```

Any later call (`/federation/handshake`) is signed with that secret and updates
`last_ok` / `fails`. Three consecutive failures mark the peer `degraded`; a signed response
restores `active`. Alerts fire only on transitions.

## 4. Signing

```
canonical(payload) = json.dumps(payload, sort_keys=True, separators=(',', ':'))
sig = HMAC-SHA256(secret, f"{ts}\n{nonce}\n{canonical(payload)}").hexdigest()
```

- `payload` is always `{"node_id": <signer>}`.
- `ts` is epoch seconds; accepted only within ±300 s.
- `nonce` is a random hex string, single-use per `(source, nonce)` for 600 s.
- Verification order is **signature first, then nonce** — burning a nonce before checking the
  MAC would let an unauthenticated caller block a legitimate retry.
- `secret` is 64 hex chars, stored mode 600, never transmitted after bootstrap and never
  exposed in `peers.json`.

## 5. Secret derivation

```
secret = HMAC-SHA256(invite_code, f"{min(node_a,node_b)}|{max(node_a,node_b)}").hexdigest()
```

Both sides derive the same value from B's one-time invite code (18 URL-safe chars). The code
travels once, inside the `/federation/accept` body over TLS, and is not stored.

## 6. Endpoints

| Path | Method | Auth | Limit | Body cap | Returns |
|---|---|---|---|---|---|
| `/federation/node.json` | GET | none | — | — | node identity (cached 60 s) |
| `/federation/peers.json` | GET | none | — | — | active peers, **no secrets** |
| `/federation/search` | GET | none | 30 r/m, burst 20 | — | aggregated items |
| `/federation/announce` | POST | none (schema-validated) | 12 r/m, burst 5 | 16 KB | `202 pending` / `409` / `429` |
| `/federation/accept` | POST | accept token | 12 r/m, burst 5 | 16 KB | signed ack / `403` / `404` |
| `/federation/handshake` | POST | HMAC | 12 r/m, burst 10 | 16 KB | signed ack |
| `/federation/status` | GET | **loopback only** | — | — | health + counters (404 publicly) |

Rate-limit zones are `fed_search` and `fed_peer` in `/etc/nginx/conf.d/wx11-zones.conf`
(http context — a `limit_req_zone` inside a server block is invalid). They are separate so a
search-heavy client cannot starve its own handshakes.

Registry caps: 32 peers, pending requests expire after 14 days (pruned by the health pass).

## 7. Search contract

Request: `GET /federation/search?q=<2-120 chars>&type=all|skills|products[&federated=1]`

Response:

```json
{
  "query": "nginx",
  "type": "all",
  "count": 6,
  "items": [
    {"kind": "skill", "id": "nginx-proxy-config-gotchas", "title": "…",
     "description": "…", "category": "devops", "version": "1.0.0",
     "origin": "<node_id>", "origin_name": "WebX11 Main",
     "external": false, "url": "https://webx11.duckdns.org/skills.html"}
  ]
}
```

Rules that matter:

- **`federated=1` (or `X-Federation-Hop: 1`) means "answer from your own catalog only."**
  Without it two nodes recurse A→B→A→B until both deadlines expire and the caller silently
  gets local-only results. This was observed in the two-node test (118 requests, zero peer
  results); the hop flag is mandatory for peer calls.
- Fan-out is capped at 8 peers, 20 items per peer, 80 items total, 2.5 s wall-clock budget.
  Stragglers are abandoned; partial results are returned.
- Results are cached 120 s per `(query, type, federated)`; max 200 entries.
- Items whose `origin` equals our own `node_id` are dropped (a peer echoing our catalog back
  is not "external content").
- A peer's `url` field is **discarded**. Result links are rebuilt as
  `<peer origin>/skills.html` or `<peer origin>/marketplace.html`, because HTML-escaping does
  not neutralise a `javascript:` href.

## 8. Threat model

| Threat | Control |
|---|---|
| SSRF via peer URL | `https` only, no credentials, no path/query, every DNS answer must be public |
| DNS rebinding between validation and fetch | the validated IP is pinned; TLS SNI and `Host` still use the hostname |
| Replay | ±300 s window, single-use nonce, signature verified before the nonce is consumed |
| Pending-peer hijack (`node_id` is public) | a second announce with a **different** URL for a pending node is rejected `409` |
| Registry flooding | 12 r/m, 32-peer cap, human approval, 14-day pending TTL |
| Click-XSS from peer content | peer `url` ignored, links rebuilt from origin, every field length-capped and HTML-escaped, `external` badge |
| Amplification through us | fan-out ≤ 8 peers, 2.5 s deadline, response cache |
| Admin takeover | no admin endpoints at all; invites/approvals need shell access on the node |

Federation peers are trusted to serve **their own** catalog and nothing else. A peer is not
trusted to serve *your* catalog: the hop flag stops recursion, and origin filtering stops
echoes.

## 9. Compatibility

- `federation_api` is an integer. A peer announcing a different value gets `409` and is not
  registered. Bump it on any breaking change to the payloads above.
- Additive changes (new optional fields, new `kind` values) do not require a bump: both sides
  ignore unknown fields.

## 10. Operating it

```bash
python3 api/federation-cli.py init --name "My Node" --url https://me.example.org
python3 api/federation-cli.py add-peer https://peer.example.org
python3 api/federation-cli.py pending
python3 api/federation-cli.py approve <node_id>
python3 api/federation-cli.py list
python3 api/federation-cli.py health      # silent when everything is fine
python3 api/federation-cli.py remove <node_id>
```

`deploy/federation-health.sh` wraps `health` for cron: it prints only on a transition and
pipes non-empty output to `notify.py` when one is installed. `deploy/test-federation.sh`
runs a full two-node handshake on one host using scratch state directories.