TCP/IP Networking

Programs on the same machine communicate through shared memory, pipes, or local sockets. Network programming adds complexity because the communicating programs run on different machines, potentially thousands of miles apart, connected by unreliable links with varying latency.

When two programs exchange data, they need to agree on the rules: How do you start a conversation? How do you know when a message ends? What happens if data gets corrupted? These rules form a protocol.

The Internet uses a family of protocols called TCP/IP. Corosio implements the TCP portion, giving your programs reliable communication channels over the network.

At the lowest level, networks transmit bits as voltage changes on copper wire, light pulses in fiber optic cables, or radio waves for wireless. These physical signals have limitations: maximum distance, susceptibility to interference, and finite propagation speed.

A Network Interface Card (NIC) connects your computer to the network. Each NIC has a globally unique Media Access Control (MAC) address—48 bits, typically written as six hexadecimal pairs like 00:1A:2B:3C:4D:5E.

MAC addresses identify devices on the local network segment. They matter for switches and local routing but are invisible to your application code.

Ethernet is the dominant local area network (LAN) technology. It defines how devices share a network medium, how frames are formatted, and how collisions are handled. Modern Ethernet runs at 1 Gbps or faster over twisted-pair cables.

Your home router typically combines a switch (for local devices) with a router (connecting to your ISP).

Layering provides modularity. You can replace Ethernet with Wi-Fi without changing your application. TCP can run over any network technology that supports IP. Each layer has a well-defined interface to the layers above and below.

The Open Systems Interconnection (OSI) model defines seven layers:

The OSI model is useful for discussion but doesn’t perfectly map to real protocols.

The Internet uses a four-layer model that better reflects actual implementation:

Corosio operates at the Transport layer, providing TCP sockets. Your application logic sits above, sending and receiving data through Corosio’s abstractions.

When you send data, each layer wraps it with its own header:

The receiving side reverses this process, stripping headers at each layer until the application data emerges.

An IP address identifies a host on the network.

IPv4 addresses are 32 bits, written as four decimal numbers separated by dots: 192.168.1.100. There are about 4 billion possible addresses—not enough for the modern Internet.

IPv6 addresses are 128 bits, written as eight groups of hexadecimal digits separated by colons: 2001:0db8:0000:0000:0000:0000:0000:0001. Leading zeros can be omitted, and consecutive groups of zeros can be replaced with ::, so this becomes 2001:db8::1.

Networks are divided into subnets using a netmask. The netmask defines which bits of the address identify the network versus the host.

CIDR (Classless Inter-Domain Routing) notation combines the address and prefix length: 192.168.1.0/24 means the first 24 bits identify the network, leaving 8 bits (256 addresses) for hosts.

Common prefix lengths:

Some address ranges are reserved for private networks and aren’t routable on the public Internet:

Your home devices likely use 192.168.x.x addresses.

Network Address Translation (NAT) allows multiple devices with private addresses to share a single public IP address. Your router maintains a translation table, mapping internal (private IP + port) to external (public IP + port).

NAT complicates peer-to-peer connections because devices behind NAT can’t directly receive incoming connections without port forwarding or NAT traversal techniques.

IP transmits data in packets (also called datagrams). Each packet is independent and may take a different route to the destination.

If a packet is too large for a network link, IP can fragment it into smaller pieces. The receiving host reassembles fragments before delivering to the transport layer. Fragmentation adds overhead and can cause problems if any fragment is lost.

Routers forward packets toward their destination based on routing tables. Each router examines the destination IP address and forwards the packet to the next hop. The path from source to destination may traverse many routers.

The Time To Live (TTL) field prevents packets from circulating forever due to routing loops. Each router decrements TTL; when it reaches zero, the packet is discarded. The sender sets the initial TTL (commonly 64 or 128).

Tools like traceroute exploit TTL to discover the network path by sending packets with increasing TTL values and observing which router discards each one.

TCP establishes a connection before transferring data. Both sides maintain state about the connection: sequence numbers, window sizes, timers. This contrasts with UDP, which just sends packets without setup.

TCP connections begin with a three-way handshake:

After the handshake, both sides can send and receive data.

TCP numbers each byte in the stream. The sender assigns sequence numbers; the receiver acknowledges which bytes it has received. This enables detection of lost, duplicated, or reordered data.

If the sender doesn’t receive an acknowledgment within a timeout, it retransmits. The receiver uses sequence numbers to put out-of-order data back in sequence and discard duplicates.

The receiver advertises a window size: how many bytes it can buffer. The sender must not have more unacknowledged bytes in flight than the receiver’s window allows. This prevents a fast sender from overwhelming a slow receiver.

The window size dynamically adjusts. When the receiver processes data, it opens the window; when it falls behind, the window shrinks.

TCP also limits sending rate to avoid overwhelming the network itself. Algorithms like slow start, congestion avoidance, and fast retransmit probe for available bandwidth.

When packet loss occurs (detected by missing acknowledgments), TCP assumes network congestion and reduces its sending rate. This is why TCP throughput can vary significantly depending on network conditions.

When TCP detects packet loss (no acknowledgment received), it retransmits. The retransmission timeout (RTO) is calculated from measured round-trip times. On a slow or variable network, RTO can be several seconds.

This is why network operations can take a long time to fail—TCP may retry multiple times before giving up.

TCP connections close with a four-way handshake:

Either side can initiate the close. The FIN indicates "I have no more data to send" but the connection remains half-open—the other side can still send.

A RST (reset) immediately terminates the connection without the graceful handshake, used when something goes wrong.

A TCP connection moves through states:

The TIME_WAIT state lasts 2× the maximum segment lifetime (typically 1-2 minutes). This prevents old packets from a closed connection being mistaken for a new connection on the same port. It’s why restarting a server may fail with "address already in use."

UDP doesn’t establish connections. You just send a datagram to a destination address and port. There’s no handshake, no state maintained, no guaranteed delivery.

If a UDP packet is lost, the sender doesn’t know (unless the application builds its own acknowledgment mechanism).

UDP is appropriate when:

TCP is appropriate for most applications where correctness matters.

UDP includes a checksum covering the header and data. The receiver verifies the checksum and discards corrupted packets. Unlike TCP, UDP doesn’t retransmit—the data is simply lost.

Corosio currently supports only TCP. UDP may be added in future versions.

A port is a 16-bit number (0–65535) identifying an application on a host.

Standard services use well-known ports:

A socket endpoint is the combination of an IP address and port number. A TCP connection is uniquely identified by four values:

This means a server on port 80 can have thousands of simultaneous connections from different clients—each connection has a unique tuple.

A listening socket waits for incoming connections on a specific port. When a client connects, the operating system creates a new connected socket for that specific connection. The listening socket continues accepting more connections.

The traditional socket API has operations that map to TCP concepts:

Corosio wraps these operations in coroutine-friendly abstractions.

When you connect to www.example.com, your system must find its IP address. This involves querying DNS servers, which form a hierarchical, distributed database.

DNS responses often include multiple addresses (for load balancing or redundancy). Your application should try each address until one succeeds.

Corosio provides the resolver class for asynchronous DNS lookups:

The loopback interface (127.0.0.1 or ::1) allows a machine to communicate with itself. Traffic never leaves the host, making it useful for testing and inter-process communication.

Binding to loopback restricts access to local connections only—external hosts cannot connect.

Firewalls filter network traffic based on addresses and ports. Corporate networks often block all incoming connections and restrict outgoing to specific ports (80, 443). Your application may need to work within these constraints.

TCP keep-alive sends periodic probes on idle connections to detect if the peer has disappeared. Without keep-alive, a connection to a crashed host may appear open indefinitely.

Operating systems have configurable keep-alive intervals (often defaulting to hours). Applications needing faster detection should implement application-level heartbeats.

Nagle’s algorithm batches small writes. Instead of sending each small piece immediately, TCP waits briefly to accumulate more data, sending fewer, larger packets.

This improves efficiency for bulk transfers but adds latency for interactive applications (like games or terminals) that send small, frequent messages.

The TCP_NODELAY socket option disables Nagle’s algorithm, sending data immediately regardless of size. Use this for latency-sensitive applications where you’re sending small packets that shouldn’t be delayed.

When a socket closes, its address may remain in TIME_WAIT state for minutes. SO_REUSEADDR allows a new socket to bind to an address that’s in TIME_WAIT.

This is essential for servers that restart—without it, the restart fails with "address already in use" until TIME_WAIT expires.

SO_REUSEPORT (available on some operating systems) allows multiple sockets to bind to the same address and port. The operating system distributes incoming connections among them.

This enables multi-process server architectures where each process has its own listening socket.

TCP’s byte-stream nature means read_some() may return fewer bytes than you requested, and write_some() may send fewer bytes than you provided.

Always loop or use composed operations (read(), write()) when you need exact amounts:

If no server is listening on the target port, connect fails with connection_refused. Always handle this error—it’s common during development and when servers restart.

A server that terminates and immediately restarts may fail to bind because the OS keeps the old socket in TIME_WAIT state. Production servers typically enable SO_REUSEADDR.

Long-running computations in a coroutine block other operations. For CPU-bound work, dispatch to a separate thread pool.