WebSocket Architecture: Real-Time Communication, Scaling, and Fallback Strategies

When HTTP Isn't Enough

HTTP is request-response. The client asks, the server answers, the connection sits idle. For most web applications, that's fine. But for features where the server needs to push data to clients in real-time — chat, live notifications, collaborative editing, multiplayer games, live dashboards — the request-response model breaks down.

You could poll. Every 2 seconds, the client makes an HTTP request asking "anything new?" For 1,000 connected clients, that's 500 requests per second to mostly receive "nope, nothing." It works, but it's wasteful and adds 0-2 seconds of latency to every update.

WebSocket gives you a persistent, bidirectional connection. Both sides can send data at any time without the overhead of establishing new connections or sending HTTP headers with every message.

WebSocket Protocol Basics

A WebSocket connection starts as an HTTP request with an Upgrade header. The server responds with 101 Switching Protocols, and from that point on, the connection speaks the WebSocket protocol — framed binary or text messages over a persistent TCP connection.

# WebSocket handshake
GET /ws HTTP/1.1
Host: example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Version: 13

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=

After the handshake, both sides can send messages independently. Messages are framed with a small header (2-14 bytes) that indicates the message type (text, binary, ping/pong, close) and payload length. Compare that to HTTP where every request carries headers that might be 500+ bytes.

Server-Side Implementation

For Python, I'd recommend using websockets library (for standalone servers) or channels (for Django integration). For Node.js, the ws package is the standard — Socket.IO adds features on top but also adds complexity.

This connects to the ideas in Python Type Hints and Mypy: Gradually Typing a Large Codebas.

# Python WebSocket server with websockets library
import asyncio
import websockets
import json

CONNECTIONS = set()

async def handler(websocket):
 CONNECTIONS.add(websocket)
 try:
 async for message in websocket:
 data = json.loads(message)
 # Broadcast to all other connected clients
 others = CONNECTIONS - {websocket}
 if others:
 await asyncio.gather(
 *(ws.send(json.dumps({
 "type": "message",
 "user": data.get("user"),
 "text": data.get("text")
 })) for ws in others)
 )
 finally:
 CONNECTIONS.remove(websocket)

async def main():
 async with websockets.serve(handler, "0.0.0.0", 8765):
 await asyncio.Future() # Run forever

asyncio.run(main())

Scaling WebSocket Servers

Here's where things get interesting. With HTTP, a load balancer can route any request to any server. With WebSocket, a client's connection is pinned to one specific server for its lifetime. This creates two problems.

Problem 1: Connection Distribution

If server A has 10,000 connections and server B has 100, they're doing very different amounts of work. You need sticky sessions or a connection-aware load balancer. Nginx supports WebSocket proxying — the proxy_pass with Upgrade header forwarding works, but you lose the ability to redistribute connections without disconnecting clients.

Problem 2: Cross-Server Messaging

User A is connected to server 1. User B is connected to server 2. User A sends a message to user B. Server 1 has the message but server 2 has user B's connection. You need a pub/sub layer between servers.

Redis Pub/Sub is the go-to solution for this. Every WebSocket server subscribes to relevant channels. When a message needs to reach clients on other servers, it's published to Redis, and all subscribed servers relay it to their connected clients.

# Multi-server WebSocket with Redis Pub/Sub
import aioredis

redis = aioredis.from_url("redis://redis-host:6379")

async def subscribe_to_channels(websocket, room_id):
 pubsub = redis.pubsub()
 await pubsub.subscribe(f"room:{room_id}")

 async for message in pubsub.listen():
 if message["type"] == "message":
 await websocket.send(message["data"])

async def broadcast_to_room(room_id, data):
 # All servers with clients in this room will receive this
 await redis.publish(f"room:{room_id}", json.dumps(data))

Connection Limits

Each WebSocket connection holds an open file descriptor and some memory (roughly 10-50KB per connection depending on your server and buffer sizes). A single server can handle tens of thousands of concurrent connections — I've seen Go servers hold 100K+ with proper tuning.

This connects to the ideas in Database Backup and Recovery: Point-in-Time Recovery, WAL Ar.

The bottleneck is usually message fan-out, not connections. Broadcasting a message to 50,000 connected clients means serializing and writing to 50,000 sockets. At that scale, you want connection pools organized by rooms/topics so each broadcast only touches the relevant connections.

Server-Sent Events: The Simpler Alternative

If your communication is one-directional — server pushes updates to clients, but clients only send data via regular HTTP — Server-Sent Events (SSE) are worth considering.

SSE uses a plain HTTP connection that the server keeps open. The server sends events as specially formatted text. The browser's EventSource API handles reconnection automatically, including resuming from the last received event ID.

# SSE endpoint in Flask
from flask import Response, stream_with_context

@app.route('/events')
def events():
 def generate():
 pubsub = redis.pubsub()
 pubsub.subscribe('updates')
 for message in pubsub.listen():
 if message['type'] == 'message':
 data = message['data'].decode()
 yield f"id: {message_id}\ndata: {data}\n\n"

 return Response(
 stream_with_context(generate()),
 mimetype='text/event-stream',
 headers={
 'Cache-Control': 'no-cache',
 'X-Accel-Buffering': 'no' # Disable Nginx buffering
 }
 )

SSE vs WebSocket

SSE advantages: Works over standard HTTP (no upgrade needed), automatic reconnection with event ID tracking, works through HTTP/2 multiplexing, simpler to implement, no special load balancer configuration.

SSE limitations: Server-to-client only (clients use regular HTTP POST/PUT for sending), maximum 6 connections per domain in HTTP/1.1 (not an issue with HTTP/2), text-only (no binary frames).

See also: Building Design Systems at Scale: Tokens, Components, and Go.

For live dashboards, news feeds, notifications, and stock tickers, SSE is often the better choice. It's simpler and "just works" with existing HTTP infrastructure. I'd reach for WebSocket only when you need bidirectional communication — chat, collaborative editing, gaming.

Connection Lifecycle Management

Real-world WebSocket connections aren't persistent forever. Networks drop, mobile devices switch between WiFi and cellular, laptops go to sleep. Your server needs to detect dead connections and clean up resources.

Use ping/pong frames. The WebSocket protocol has built-in ping and pong frames for keepalive. Send a ping every 30 seconds; if you don't receive a pong within 10 seconds, consider the connection dead and close it.

On the client side, implement reconnection with exponential backoff. When the connection drops, wait 1 second, then 2, then 4, up to a maximum (say 30 seconds). Add jitter to prevent all clients from reconnecting simultaneously after a server restart. And maintain a message queue on the client so messages sent during reconnection aren't lost.