The network protocol specifies how programs send data across the network.
Below, we'll look in detail at HTTP and binary data exchange.
HTTP is the protocol used by web pages (that's why URLs usually start with "http://"). HTTP servers listen on port 80 by default, but you can actually use the :port syntax to connect to any port you like (for example, "https://lawlor.cs.uaf.edu:8888/some_url").
An HTTP client, like a web browser, starts by doing a DNS lookup on the server name. That's the "resolving host name" message you see in your browser. The browser then does a TCP connection to that port on the server ("Connecting to server").
Once connected, the HTTP client usually sends a "GET" request. Here's the simplest possible GET request:
"GET / HTTP/1.0\r\n\r\n"
Note the DOS newlines, and the extra newline at the end of the request. You can list a bunch of optional data in your GET request, like the languages you're willing to accept ("Accept-Language: en-us\r\n") and so on. HTTP 1.1 (not 1.0) requires a Host to be listed in the request ("Host: www.foobar.com\r\n"), which is used by virtual hosts.
The HTTP server then sends back some sort of reply. Officially, this is supposed to be a "HTTP/1.1 200 OK\r\n" followed by another set of line-oriented ASCII optional data, such as the Content-Length in bytes ("Content-Length: 187\r\n"). Because the receiver knows how many bytes are coming, after the ASCII header you can efficiently exchange binary data without worrying about newlines or malicious comments that mimic HTTP headers. This is an important point:
Do not exchange payload data using ASCII: you'll never get it right. Send a byte count and switch to binary for the arbitrary payload.
Here's an example of a real HTTP exchange between Firefox and Apache:
Firefox connects to server. Apache accepts the connection.
Firefox, the client, sends this ASCII data, with DOS newlines:
GET /my_name_is_url.html HTTP/1.1
Host: lawlor.cs.uaf.edu:8888
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.9) Gecko/20070126 Ubuntu/dapper-security Firefox/1.5.0.9
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Cookie: __utmz=62224958.1163103248.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none); __utma=62224958.570638686.1163103248.1163107343.1164832326.3
<- blank line at end of HTTP request headers
Apache, the server, sends this data back:
HTTP/1.1 200 OK
Date: Fri, 06 Apr 2007 20:20:50 GMT
Server: Apache/2.0.55 (Ubuntu)
Accept-Ranges: bytes
Content-Length: 9443
Connection: close
Content-Type: text/html; charset=UTF-8
<- blank line at end of HTTP response headers
<html><head><title>UAF Department of ... rest of web page, total of 9443 bytes after blank line
This is a pretty simple, ASCII-based protocol. The only binary data is the contents of the web resource, transmitted by the server after the blank line. The "Content-Length:" field tells the client how many bytes to expect.
A typical very simple custom web client might look like this:
#include "osl/socket.h"
#include "osl/socket.cpp"
int foo(void) {
skt_ip_t ip=skt_lookup_ip("137.229.25.247"); // lawlor.cs.uaf.edu
SOCKET s=skt_connect(ip,80,2);
/* Send off HTTP request to server (with URL) */
const char *req=
"GET / HTTP/1.1\r\n" // the "/" is the URL I'm requesting
"Host: lawlor.cs.uaf.edu\r\n" // hostname is required for HTTP 1.1
"User-Agent: Raw socket example code (lawlor@alaska.edu)\r\n" // web browser ID string
"\r\n"; // blank line == end of HTTP request
skt_sendN(s,req,strlen(req));
/* Receive HTTP response headers, up to the newline */
std::string response;
int length=0;
while ((response=skt_recv_line(s))!="")
{
std::cout<<response<<"\n";
if (response.substr(0,15)=="Content-Length:")
length=atoi(response.substr(16).c_str());
}
/* Receive HTTP response data, and print it */
std::cout<<"-- bottom line: "<<length<<" bytes of data\n";
if (length>0 && length<10000) { // sanity check
std::vector<char> page(length); // place to store data
skt_recvN(s,&page[0],length); // grab data from server
for (int i=0;i<length;i++) std::cout<<page[i]; // print to screen
}
skt_close(s);
return 0;
}
A typical similarly simplified web server might look like this. Note it just keeps being a webserver forever:
#include "osl/socket.h"
#include "osl/socket.cpp"
int foo(void) {
/* Make a TCP socket listening on port 8888 */
unsigned int port=8888;
SERVER_SOCKET serv=skt_server(&port);
/* Keep servicing clients */
while (1) {
std::cout<<"Waiting for connections on port "<<port<<"\n";
skt_ip_t client_ip; unsigned int client_port;
SOCKET s=skt_accept(serv,&client_ip,&client_port);
std::cout<<"Connection from "<<skt_print_ip(client_ip)<<":"<<client_port<<"!\n";
// Grab HTTP request line, typically GET /url HTTP/1.1
std::string req=skt_recv_line(s);
// Grab rest of HTTP header info (mostly useless)
std::string hdr;
while ((hdr=skt_recv_line(s))!="") std::cout<<"Client header: "<<hdr<<"\n";
// Prepare HTTP response header
std::string page="<html><body>IT WORKED!</body></html>";
char response[1024];
sprintf(response, // needed to get the page length into the string
"HTTP/1.1 200 OK\r\n"
"Server: Random example code (lawlor@alaska.edu)\r\n"
"Content-Length: %d\r\n"
"\r\n" // blank line: end of header
,(int)page.length());
skt_sendN(s,&response[0],strlen(response));
skt_sendN(s,&page[0],page.length());
skt_close(s);
}
return 0;
}
Note that using these for real production servers sounds like a terrible idea (use a lightweight C++ library like mongoose, or a real server like Apache, nginx, or node.js), but they do work for simple testing.
One recurring issue in exchanging binary data is there are no standards: x86 and ARM machines store data to memory starting with the little end, so 0x4321 gets stored as 0x21, then 0x43. Other CPUs like PowerPC (by default, it's switchable) store data starting with the big end, which was once the standard, and so is called "network byte order". I've programmed on machines where "int" is 2 bytes, most set it to 4 bytes for now, but there are hints of an eventual transition to 8 bytes. This means you might send a binary "int", and get any of three sizes in either of two orders! The protocol fix is to just specify, such as "big-endian 32 bit unsigned integer", and then everybody will know what to use.
How do you send a 32-bit big endian integer from a little endian machine? There are several ways such as htonl() to handle byte order, and other ways such as <stdint.h> uint32_t to handle sizes, but my favorite fix is to solve both at once by specifying the bytes manually, via a special class.
class Big32 { //Big-endian (network byte order) 32-bit integer
typedef unsigned char byte;
byte d[4];
public:
Big32() {}
Big32(unsigned int i) { set(i); }
operator unsigned int () const { return (d[0]<<24)|(d[1]<<16)|(d[2]<<8)|d[3]; }
unsigned int operator=(unsigned int i) {set(i);return i;}
void set(unsigned int i) {
d[0]=(byte)(i>>24);
d[1]=(byte)(i>>16);
d[2]=(byte)(i>>8);
d[3]=(byte)i;
}
};
The cool part about this class is it's always the correct size, and it never needs alignment padding, so you can stick together a long struct to represent the network message order and it will match byte for byte. My osl/socket.h header includes this class (and Big16) by default.
Consider two processes exchanging more complex binary data over a network socket. To make it easy to run, I'll start both processes myself, using fork.
#include <sys/wait.h> /* for wait() */
#include "osl/socket.h"
#include "osl/socket.cpp"
/* Run child process's code. Socket connects to parent. */
void run_child(SOCKET s) {
cout<<"Child alive! Sending data to parent."<<std::endl;
std::string str="Cybertron";
skt_sendN(s,&str,sizeof(str));
}
/* Run parent process's code. Socket connects to child */
void run_parent(SOCKET s) {
cout<<"Parent alive! Getting data from child"<<std::endl;
std::string str="";
skt_recvN(s,&str,sizeof(str));
cout<<"Parent done. got="<<str<<std::endl;
}
int foo(void) {
unsigned int port=12345; cout.flush();
int newpid=fork();
if (newpid!=0) { /* I'm the parent */
SERVER_SOCKET serv=skt_server(&port);
SOCKET s=skt_accept(serv,0,0); /* connection from child */
run_parent(s);
skt_close(s);
int status=0;
wait(&status); /* wait for child to finish */
} else { /* I'm the child */
SOCKET s=skt_connect(skt_lookup_ip("127.0.0.1"),port,1); /* connect to parent */
usleep(1000); /* slow down child, to avoid corrupted cout! */
run_child(s);
skt_close(s);
exit(0); /* close out child process when done */
}
return 0;
}
Drat! This crashes! Yet it works fine if we send and receive integers, floats, or simple flat classes. The problem with sending a std::string (or std::vector, or map, etc) is this basic fact:
The problem is my pointer is a reference to a place in my memory. If we've each got our own separate memory, then you dereferencing my pointer is not going to work--the best you could hope for is a crash.
And inside a std::string is a pointer to the data. On my machine, this pointer is "basic_string::_M_dataplus._M_p", in the nearly unreadable /usr/include/c++/4.4.3/bits/basic_string.h. Inside std::vector? Also a pointer.
This is really annoying, because real applications use complicated data structures like std::vector<std::string> all over the place, and you'd like to just send them, not break them up into little sendable pointer-free pieces.
For example, here's one correct way to send a string: first send the length, then send the data.
// Send side:
std::string str="Cybertron";
int length=str.length();
skt_sendN(s,&length,sizeof(length)); // OK because length is an integer
skt_sendN(s,&str[0],length); // OK because now we're sending the string *data*
// Receive side:
std::string str="";
int length=0;
skt_recvN(s,&length,sizeof(length)); // OK because length is an integer
str.resize(length);
skt_recvN(s,&str[0],length); // OK because we reallocated the string
This works fine, but:
Luckily, there's a design pattern that solves all these issues called "pup", which stands for pack/unpack. The basic idea is a single function named "pup" can both send or receive an object. Since it's the same code on both sides, a mismatch between send and receive is much harder, fixing (1). We fix (2) by using "structural recursion" to break up complex compound objects: for an object A with parts B and C, A's pup function just calls B's pup function first, then calls C's pup function. For example:
class A {
B b;
C c;
public:
...
friend void pup(...,A &a) {
pup(...,a.b);
pup(...,a.c);
}
};
Here's a complete example:
/*********** network pup "library" code ****************/
/* Sends data out across the network immediately */
class send_PUPer {
SOCKET s;
public:
send_PUPer(SOCKET s_) :s(s_) {}
// The global "pup" function just sends basic types across the network.
friend void pup(send_PUPer &p,int &v) { skt_sendN(p.s,&v,sizeof(v)); }
friend void pup(send_PUPer &p,char &v) { skt_sendN(p.s,&v,sizeof(v)); }
// and so on for bool, float, etc. You can convert to network byte order too!
};
/* Receives data from the network immediately */
class recv_PUPer {
SOCKET s;
public:
recv_PUPer(SOCKET s_) :s(s_) {}
// The global "pup" function just sends basic types across the network.
friend void pup(recv_PUPer &p,int &v) { skt_recvN(p.s,&v,sizeof(v)); }
friend void pup(recv_PUPer &p,char &v) { skt_recvN(p.s,&v,sizeof(v)); }
// and so on for bool, float, etc. You can convert to network byte order too!
};
// Explain how to pup a std::string.
// This is a little mind-bending, since the same code is used for both send and recv.
template <class PUPer>
void pup(PUPer &p,std::string &v) {
int length=v.length(); // send: actual length. recv: initial length.
pup(p,length);
v.resize(length); // send: does nothing. recv: reallocates array
for (int i=0;i<length;i++) pup(p,v[i]);
}
/************ user code ***********/
/* Run child process's code. Socket connects to parent. */
void run_child(SOCKET s) {
send_PUPer p(s);
cout<<"Child alive! Sending data to parent."<<std::endl;
std::string str="Cybertron";
pup(p,str);
}
/* Run parent process's code. Socket connects to child */
void run_parent(SOCKET s) {
recv_PUPer p(s);
cout<<"Parent alive! Getting data from child"<<std::endl;
std::string str="";
pup(p,str);
cout<<"Parent done. got="<<str<<std::endl;
}
For a bare string, this isn't very convincing. Let's add std::vector support.
// Explain how to pup a std::vector. This just recurses to the element pup functions,
// so we can automatically pup std::vector<int>, std::vector<string>, std::vector<std::vector<char>>, etc.
template <class PUPer,typename T>
void pup(PUPer &p,std::vector<T> &v) {
int length=v.size(); // send: actual size. recv: initial size.
pup(p,length);
v.resize(length); // send: does nothing. recv: reallocates storage
for (int i=0;i<length;i++) pup(p,v[i]); // might even be recursive!
}
/************ user code ***********/
/* Run child process's code. Socket connects to parent. */
void run_child(SOCKET s) {
send_PUPer p(s);
cout<<"Child alive! Sending data to parent."<<std::endl;
std::vector<std::string> strs;
strs.push_back("Cybertron"); strs.push_back("G6");
pup(p,strs);
}
/* Run parent process's code. Socket connects to child */
void run_parent(SOCKET s) {
recv_PUPer p(s);
cout<<"Parent alive! Getting data from child"<<std::endl;
std::vector<std::string> strs;
pup(p,strs);
for (unsigned int i=0;i<strs.size();i++)
cout<<"got="<<strs[i]<<std::endl;
}
OK, but what if we want to do one big send instead of dozens of smaller sends? (Often the network either sends one packet per send, which is very inefficient for lots of small sends, or buffers up data using Nagle's algorithm, which adds tons of latency.) To accumulate the data, we just need to write a new PUPer that accumulates data before sending it.
/* Sends data out across the network in one big block */
class send_delayed_PUPer {
std::vector<char> data;
public:
send_delayed_PUPer() {}
// The global "pup" function just accumulates our data.
friend void pup(send_delayed_PUPer &p,char &v) { p.data.push_back(v); }
friend void pup(send_delayed_PUPer &p,int &v) {
int i=p.data.size(); // store our old end
p.data.resize(i+sizeof(int)); // make room for one more int
*(int *)&(p.data[i]) = v; // write new value into data buffer
}
// and so on for bool, float, etc.
// send off all our buffered data, and clear it
void send(SOCKET s) {
skt_sendN(s,&data[0],data.size());
data.resize(0);
}
};
... everything else as before ...
/************ user code ***********/
/* Run child process's code. Socket connects to parent. */
void run_child(SOCKET s) {
send_delayed_PUPer p;
cout<<"Child alive! Sending data to parent."<<std::endl;
std::vector<std::string> strs;
strs.push_back("Cybertron"); strs.push_back("G6");
pup(p,strs); // accumulates data locally
p.send(s); // sends across network
}
Things I've done with PUPers include:
It's a surprisingly flexible trick! If the PUPer objects confuse you, here's a function-oriented version, where we just swap out the send or receive functions:
#include "osl/socket.h"
#include <sys/wait.h>
#ifdef _WIN32
# include <winsock.h> /* windows sockets */
# pragma comment (lib, "wsock32.lib") /* link with winsock library */
#else /* non-Windows: Berkeley sockets */
# include <sys/types.h>
# include <sys/socket.h> /* socket */
# include <arpa/inet.h> /* AF_INET and sockaddr_in */
#endif
#define errcheck(code) if (code!=0) { perror(#code); return 0; }
struct recvData {
char type; // 'L' for lawlor data
Big32 length; // length of data
};
struct sockaddr build_addr(int port,unsigned char ip3,unsigned char ip2,unsigned char ip1,unsigned char ip0)
{
struct sockaddr_in addr;
addr.sin_family=AF_INET;
addr.sin_port=htons(port); // port number in network byte order
unsigned char bytes4[4]={ip3,ip2,ip1,ip0}; // IP address
memcpy(&addr.sin_addr,bytes4,4);
return *(struct sockaddr *)&addr;
}
int build_socket(void)
{
int s=socket(AF_INET,SOCK_STREAM,0);
// Allow us to re-open a port within 3 minutes
int on = 1; /* for setsockopt */
setsockopt(s,SOL_SOCKET, SO_REUSEADDR,&on,sizeof(on));
return s;
}
/* Send or receive an int across the network. Always uses big-endian 32-bit mode */
template <typename sendOrRecv_t>
void network(int socket, sendOrRecv_t sendOrRecv, int &i)
{
Big32 b=i;
sendOrRecv(socket,&b,sizeof(b),0);
i=b;
}
/* Send or receive a string across the network */
template <typename sendOrRecv_t>
void network(int socket, sendOrRecv_t sendOrRecv, std::string &str)
{
int length=str.size();
network(socket,sendOrRecv,length);
str.resize(length);
sendOrRecv(socket,&str[0],length,0);
}
/* Send or receive a vector across the network */
template <typename sendOrRecv_t,class T>
void network(int socket, sendOrRecv_t sendOrRecv, std::vector<T> &vec)
{
int length=vec.size();
network(socket,sendOrRecv,length);
vec.resize(length);
for (int i=0;i<(int)length;i++)
network(socket,sendOrRecv,vec[i]);
}
long foo(void) {
if (fork())
{
// Create a socket
int server=build_socket();
// Make listening socket
struct sockaddr addr=build_addr(1025, 0,0,0,0);
errcheck(bind(server,&addr,sizeof(addr)));
errcheck(listen(server,10));
// Wait for client to connect to me!
int s=accept(server,0,0);
// Receive binary data from client
std::vector<std::string> b;
network(s,recv,b);
std::cout<<"Got string value "<<b[0]<<" ("<<b[0].size()<<" bytes)\n";
// Send HTTP response:
std::string req="HTTP/1.1 200 OK\r\n"
"Content-Length: 5\r\n"
"\r\n"
"YES!!";
send(s,&req[0],req.size(),0);
close(s);
int status;
wait(&status);
}
else
{
// Create a socket
int s=build_socket();
// Connect to server above
struct sockaddr addr=build_addr(1025, 127,0,0,1);
errcheck(connect(s,&addr,sizeof(addr)));
// Send binary request:
std::vector<std::string> b; b.push_back("philly");
network(s,send,b);
// Receive HTTP reply. Hangs due to HTTP keepalive
char c;
while (1==recv(s,&c,1,0)) {
printf("%c",c);
}
printf("Closed!?\n");
}
return 0;
}
Most scripting languages don't need a design idiom like "pup" because the language allows you to loop over the pieces of any object. For example, here's structural recursion in JavaScript:
/* This recursive function dumps everything inside v */
function printIt(v) {
if (typeof v === "object")
{ /* it's got subobjects */
print("{");
for (f in v) { /* loop over the fields in the object */
print(f+":"); /* print string name of object subfield */
var newV=v[f]; /* extract object subfield's value */
printIt(newV); /* structural recursion! */
}
print("}");
}
else { /* it's a primitive type, like int or string */
print(v+",\n");
}
}
/* Build a complicated object */
var d = {x:3, y:4};
d.woggle={clanker:"ping", z:8};
/* Print it */
printIt(d);
Like pup, this approach allows you to disassemble and reassemble arbitrarily complex objects. But unlike C++, no per-object support is needed.
CS 441 Lecture Note, 2014, Dr. Orion Lawlor, UAF Computer Science Department.