In one sentence. A GPU behind a home router has no address the internet can dial, so ComputeFlux has the machine call out and hold the line open — then sends inference work back down the same wire.
Picture it like this. You can't phone someone whose line has no number. But if they call you and stay on, you can talk all day. That's Edge Route: a call placed by the home machine, kept alive with a heartbeat every ten seconds, carrying hundreds of simultaneous conversations over the one connection.
Why it matters. This is where "decentralized" stops being branding. If only data centers can participate, you've rebuilt the cloud with extra steps. Turning a consumer GPU into a first-class network participant is a networking problem long before it's an economics problem — and it's the part most projects hand-wave.
Want the mechanics? Skip to the tunnel lifecycle.
The hardest problem in decentralized compute is not scheduling, pricing, or reputation. It's networking. A GPU behind a residential NAT gateway is simply unreachable from the public internet: no static IP, no port forwarding, and often several layers of carrier-grade NAT between it and any prospective client.
ComputeFlux's Edge Route answers this with a reverse tunnel — WebSocket transport, yamux stream multiplexing, and on-chain identity verification. It's the networking layer beneath the ProviderPersonal type introduced in Article 6.
The NAT Problem and Why Reverse Tunnels
The traditional approach to exposing a local service — port forwarding — fails for the target audience of personal GPU providers. Many ISPs use carrier-grade NAT (CGNAT), where hundreds of customers share a single public IPv4 address. Even when a public IP is available, home routers require manual configuration most users get wrong, and UPnP is unreliable and disabled on security-conscious networks.
The alternative is a reverse tunnel. The local machine opens an outbound connection to a publicly reachable Gateway and holds it open. When an external client wants the local GPU, the Gateway sends the request down that existing tunnel. The connection direction inverts: the server dials the client, not the other way around.
The cost is that all traffic passes through the Gateway, which adds a hop. Concretely: a client in Virginia reaching a GPU in California through a Gateway in Oregon pays roughly 10–20ms of extra round-trip time. That would matter a great deal for most latency-sensitive work. Here it doesn't, because LLM generation takes seconds to tens of seconds and swallows the difference whole.
Why yamux Multiplexing Over WebSocket
The tunnel could use raw TCP or TLS connections — simple, well-understood, low-overhead. Two factors drove the choice of yamux over that simpler approach:
Stream multiplexing: A single GPU can serve multiple concurrent inference requests. Without multiplexing, each concurrent request needs a separate TCP connection, multiplying open file descriptors and TLS handshake overhead. yamux lets a single TCP connection carry hundreds of logical streams, each independently flow-controlled. The Gateway opens a new yamux stream per incoming client request; the local GPU server accepts streams and routes them to the inference engine.
Firewall friendliness: To a middlebox, a WebSocket connection looks like an HTTP upgrade. Corporate firewalls, hotel Wi-Fi, and ISP filters that block raw TCP on non-standard ports let it through, because it opens as a legitimate HTTP request with standard headers. That matters more here than it would elsewhere: per Article 6's registration model, these providers are individuals contributing idle hardware from whatever network they happen to be on, not operators of dedicated infrastructure.
The yamux configuration is tuned for tunnel stability. KeepAliveInterval: 10s pings often enough to detect a dead connection before NAT timeouts bite, since most home routers drop idle TCP connections after 60–120 seconds. ConnectionWriteTimeout: 5s kills the session when a write blocks that long, which means the connection is stuck. AcceptBacklog: 256 lets up to 256 streams queue before the server accepts them, absorbing bursts from concurrent users.
The Tunnel Lifecycle
The lifecycle has four phases, each with distinct failure modes.
Phase 1: Connect. The Local Tunnel Client (on the GPU owner's machine) opens a WebSocket connection to wss://gateway:19000/tunnel — standard TLS-encrypted WebSocket, providing transport-layer security without custom cryptographic protocols. Connection failure triggers exponential backoff starting at 1 second, capping at 30 seconds.
Phase 2: Authenticate. The client sends a Handshake frame carrying the provider's on-chain identity and an API key. The Gateway checks three things against the registry from Article 6: that the provider exists on-chain, that its providerType is ProviderPersonal, and that the API key matches one registered for the provider's models. Together those keep unauthorized machines out of the routing table. Authentication failures skip the fast retry and wait the full backoff — bad credentials rarely become good in a hurry, and hammering the Gateway would only waste its resources.
Phase 3: Register. The client then sends a ModelControlFrame for each model the GPU can serve, naming the model and its base URL — typically http://localhost:11434 for Ollama or http://localhost:8000 for vLLM. The Gateway creates a PoolEntry binding the yamux session to the provider ID and model name. Registration starts with Available: false and a 10-second warm-up, so no traffic reaches a tunnel whose inference engine is still loading weights into GPU memory. EdgeModelRunningStatus reads the same Available flag the router uses, so during warm-up it reports the model as not yet running. It is not a separate signal that could drift out of sync with the routing table.
Phase 4: Serve. When warm-up ends, PoolEntry.Available flips to true and the endpoint filter admits the model into active routing. ForwardRequest handles each incoming request: open a yamux stream, write the HTTP request (method, path, headers, body), read the response back. The stream closes once the response completes and returns to the pool.
Security Considerations
The tunnel architecture introduces attack surface that doesn't exist for traditional API proxies.
Tunnel authentication: The Handshake frame is JSON carrying providerID and apiKey, with no cryptographic challenge-response. The key travels in plaintext inside the TLS-encrypted WebSocket. TLS provides confidentiality, so this holds up in ordinary conditions — but a compromised Gateway, or a man-in-the-middle holding a valid Gateway certificate, can read those keys straight off the wire. The defense-in-depth fix is to sign the handshake with the provider's on-chain key, proving possession without transmitting anything.
Traffic encryption: The outer TLS connection encrypts all tunnel traffic. The yamux streams inside it aren't separately encrypted; they inherit the parent connection's security. That's efficient — one handshake, one encryption context — but it means a compromised Gateway can read everything. Encrypting each stream independently would double the encryption CPU cost to defend a threat model that is already catastrophic on its own.
Rate limiting local providers: Tunnel traffic gets the same rate limiting as everything else. MaxConcurrency from the provider's on-chain registration (Article 6) caps concurrent yamux streams, and the ModelSemaphore system, shared with remote API providers, adds soft and hard concurrency limits. Without them, one malicious provider could open hundreds of streams and exhaust the Gateway's file descriptors.
Session hijacking: Intercept a yamux session's TCP connection — from a compromised router between provider and Gateway, say — and you can inject or modify streams. yamux offers no stream-level authentication or integrity of its own; it leans entirely on TLS underneath. The defense is TLS with valid certificates required at both ends.
Failure Modes and Recovery
Tunnel failures sort by what they do to in-flight requests.
A clean disconnect — a WebSocket close frame or a graceful TCP termination — makes the Gateway drop the session's PoolEntry and fail in-flight requests with a connection error.
A dirty disconnect is a connection that vanishes without a close frame, from a power cut or a network outage. The keepalive timer eventually catches it, and in-flight requests time out on the Gateway's configured request timeout — a couple of minutes, not the tunnel's much shorter keepalive interval.
A reconnection race happens when the client reconnects before the Gateway has noticed the old session died. The resolution is deliberately conservative. The Pool refuses an incoming registration for a route key that still has a live session, and only lets a new session take over once the old one is confirmed closed. So a legitimately reconnecting client can briefly be turned away instead of silently swapped in. That trades slower recovery for a guarantee that two sessions never race over the same route.
Monitoring and Observability
The tunnel exposes metrics through a ForwardMetrics struct — total requests forwarded, failed requests, cumulative latency — alongside a pool-level struct tracking total connections, active connections, and rejected handshakes. Together they let an operator tell two very different problems apart. Active connections dropping to zero means the provider went offline. Failed requests climbing while connections hold steady means the provider's GPU is overloaded or misbehaving.
GPU owners get the same up/down visibility directly through the EdgeModelRunningStatus GraphQL query (Article 15). That feedback loop matters: without it, a provider can sit there believing they're earning revenue hours after their tunnel quietly died.
Key Takeaways
- The hard problem in decentralized compute is networking, not scheduling or pricing. Most home internet connections physically cannot accept an incoming connection. Everything else is downstream of solving that.
- The tunnel dials out and dresses as ordinary web traffic. That's what lets it survive hotel Wi-Fi, corporate firewalls, and carrier-grade NAT. The cost is 10–20ms of extra latency, which is noise beside the seconds an LLM takes to generate a response — an easy trade, made deliberately.
- One connection, many simultaneous requests. Each request gets its own independently flow-controlled lane, so a single GPU serves several users without opening a connection per user.
- A ten-second warm-up before any traffic arrives. A model still loading into video memory would fail every request sent to it, so the network waits before advertising the GPU as available.
- Two weaknesses stated rather than hidden. The tunnel ends at a gateway — a centralized element in a decentralized system, accepted because no production-ready alternative exists. And the handshake transmits an API key over an encrypted channel rather than proving key possession cryptographically, which a compromised gateway could exploit.
Edge Route resolves the central tension of decentralized compute: making ephemeral, residential machines as routable as cloud VMs. The yamux-over-WebSocket design picks connectivity over latency — it works through NAT and firewalls — and the on-chain identity check from Article 6 keeps unregistered machines from injecting routes.
Its main weakness is real and acknowledged: tunnel termination depends on a centralized Gateway. True peer-to-peer GPU serving would mean solving NAT traversal at scale, and no production-ready decentralized answer to that exists today.
Once a request can reach any endpoint at all — an Edge Route tunnel or a remote API provider — the old question is replaced by a new one. Out of everything available, where should this particular request go?
Next — Article 8: Sticky Balancer Scheduling: the counterintuitive answer, and why spreading load evenly is exactly the wrong instinct for AI inference.