TCP & UDP Fundamentals

Every avoidable round trip is a visible stall

The single most important latency fact in networking: setting up a TCP connection costs a round trip before any application data flows, and if you also do TLS you pay more. On a fast intra-datacenter link (RTT under 1ms) nobody notices. On a mobile user in Jakarta hitting a US server (RTT 200ms+), every avoidable round trip is a visible stall.

The handshake and slow start

The 3-way handshake: client sends SYN, server replies SYN-ACK, client sends ACK. Only after that ACK can the client send the request. That is 1 RTT of pure setup. Layer TLS 1.3 on top and you add roughly 1 more RTT (TLS 1.2 added 2). So a brand-new HTTPS connection is about 2 RTTs before the first request byte, then more RTTs for the response. Over a 200ms link that is 400ms+ of overhead spent on nothing but ceremony.

TCP earns that cost by giving reliability: every byte has a sequence number, the receiver ACKs what it got, and unacknowledged data is retransmitted, so the application sees an ordered, gap-free stream. It also runs congestion control: a new connection starts in slow start with a small congestion window and ramps up as ACKs return, probing for available bandwidth. This is why throughput on a fresh connection is low at first and climbs, and why short-lived connections never reach full speed: they die in slow start before the window opens up. Reusing a warmed connection means you keep the opened window.

The fixes for a chatty API

If each of 30 API calls opens a new connection, you pay the handshake and restart slow start 30 times. The fixes, none of which touch business logic:

• Keep-alive and connection pooling: reuse one warm connection for many requests, amortizing the handshake and keeping the congestion window open. HTTP keep-alive and client pools (a database connection pool is the same idea) do this.
• HTTP/2 multiplexing: many concurrent requests share one connection, so you pay one handshake and one slow start for all of them.
• Move the endpoint closer: an edge POP or CDN near the user shrinks the RTT itself, so every round trip is cheaper. A TLS terminator at a nearby POP means the expensive handshakes happen over a short hop, and the long-haul leg is a reused warm connection.

When UDP is right

UDP is the other L4 protocol: connectionless, no handshake, no ordering, no retransmission, no congestion control by default. You send a datagram and hope. That sounds worse, but it is exactly right when late data is useless: real-time voice and video (a retransmitted audio packet arrives after the moment it was needed, so drop it and move on), gaming, DNS (one small request/reply, a handshake would double the cost), and high-volume telemetry where losing a few samples is fine. QUIC (HTTP/3) is built on UDP precisely to escape TCP's handshake and head-of-line-blocking constraints while rebuilding reliability itself.

Interview nuance: at scale, watch TIME_WAIT and ephemeral-port exhaustion. A client that opens and closes connections rapidly leaves each in TIME_WAIT (about 60s) holding an ephemeral port; a single source IP has ~28k usable ports, so a busy proxy talking to one backend IP can run out and start failing to connect. Connection reuse fixes this too. This is another reason pooling is not optional at scale.

Recap: the TCP handshake costs a round trip (plus TLS) before data and starts slow in congestion control, so reuse connections (keep-alive, pooling, HTTP/2) and move endpoints closer to cut RTTs; reach for UDP when late data is worthless and watch TIME_WAIT/port exhaustion under churn.