Cocoon — Confidential Inference

Connection Flow

1. Client opens a WebSocket to /v1/cocoon/stream and sends its Ed25519 public key

2. TEE responds with its Ed25519 public key and a TDX attestation quote

3. ECDH key exchange — both sides convert Ed25519 → X25519, compute shared secret

4. AES-256-GCM session key is derived: key = SHA-256(shared_secret)

5. Client encrypts the inference request and sends the ciphertext

6. TEE decrypts, runs the model, encrypts response chunks

7. Client decrypts each chunk as it arrives (streaming)

8. TEE sends a signed usage attestation at the end

The Shroud gateway only sees:

• Encrypted blobs (ciphertext + nonce)

• Model name (from the init message)

• Token counts (from the signed usage report)

List Worker Types — Cocoon-native

A near-direct mirror of the upstream Cocoon client.workerTypesV2 reply. For each model that the gateway has registered in its catalog, returns the list of currently live workers with their per-worker coefficient (price multiplier) and load (active_requests/max_active_requests). The shape is intentionally raw so SDKs can pick a worker, compute capacity, or render a load graph without the gateway pre-aggregating.

The top-level object discriminator is always the literal string "cocoon.workerTypes". When the upstream Cocoon proxy is unreachable the gateway responds with HTTP 503 and a Retry-After: 5 header; clients should back off rather than retry tightly.

List Models — OpenAI shim

A minimal OpenAI-compatible model catalog. Returns only id, object, and owned_by per entry — no pricing, no worker counts, no timestamps. Use this endpoint when an OpenAI SDK is enumerating models; use /v1/cocoon/workers when you need pricing or load.

The unprefixed /v1/models returns the union across every enabled Cocoon network, so the same model id can appear once per network with a distinct owned_by value. The network-prefixed forms (/cocoon-classic/v1/models, /cocoon-alpha/v1/models) return only the matching network. The created field that OpenAI's spec includes is intentionally omitted: Cocoon does not expose a meaningful creation timestamp for a registered model, so synthesizing one would mislead.

Streaming Inference (WebSocket)

See Wire Protocol for the detailed message format.

Network-prefixed routes

Each of the routes above is also reachable under a network-prefixed form (/cocoon-classic/v1/... or /cocoon-alpha/v1/...) that pins the call to a specific Cocoon network instead of the deployment default. See Cocoon networks for the full grid and SDK pinning patterns.

What the Gateway Can See

Attestation Verification

The SDKs verify attestation automatically when an AttestationPolicy is configured:

1. Parse the TDX quote (versions 3, 4, or 5)

2. Verify report_data == SHA-512(tee_public_key) — ensures the key belongs to this TEE

3. Compute image hash: SHA-256(MRTD || MR_CONFIG_ID || MR_OWNER || MR_OWNER_CONFIG || RTMR[0..3] || zeros[64])

4. Check image hash against the allowed list in the policy

For the end-to-end trust chain (open-source TEE proxy, reproducible build, image-hash allowlist, Intel DCAP signature path) see How attestation works. For the three audience-shaped verification workflows — default policy, custom allowlist, reproduce-the-build — see Verification paths. For what attestation does and does not protect against, see Threat model.

Usage Attestation

After inference completes, the TEE signs a usage report:

1. TEE serializes usage data to JSON

2. Computes hash = SHA-256(usage_json)

3. Signs with Ed25519: signature = Ed25519.sign(hash, tee_private_key)

4. Client verifies the signature using the TEE's public key from the session

This ensures token counts cannot be tampered with by the gateway.

Default-disable-thinking allowlist

When a /v1/chat/completions request targets a model whose name starts with any prefix in this list and the caller did not set chat_template_kwargs.enable_thinking, the TEE proxy injects chat_template_kwargs.enable_thinking=false into the decrypted request before forwarding to the worker. This is the platform-wide default that produces the clean-content behavior described in Reasoning content and <think> tags.

Matching is case-sensitive prefix matching. Each comma-separated entry is trimmed; empty entries are dropped.

Examples:

The empty-string value is the documented escape hatch: it leaves all models at their native worker default, which for current Qwen3 weights is "thinking on". Use this if a deployment needs to expose chain-of-thought behavior to all callers without each caller setting chat_template_kwargs.enable_thinking=true.

Caller-set values always win over the platform default. If a request explicitly sets chat_template_kwargs.enable_thinking to either true or false, the proxy passes the value through unchanged regardless of the allowlist.