The standard UNIX process creation call fork() creates a new program that is a complete copy of the old program, including a new copy of everything in the address space of the old program, including global variables. UNIX does this using copy-on-write pages, so you can safely fork a 10GB process on a 16GB machine, at least as long as you don't change all the in-memory data on both processes.
Note that the variable "who" lives at the same address on both processes, but modifying one copy doesn't change the other copy: this is the essence of distributed memory! (Annoyingly, simultaneous cout statements will overwrite each other on NetRun due to how I redirect stdout into a file.)
#include <unistd.h> /* for fork. Only exists on UNIX systems. */
std::string delay;
std::string who="original"; // typical global variable
long foo(void) {
std::cout<<"Original: "<<&who<<", "<<who<<"\n";
if (fork()) // <- makes two copies of the process
{ // parent process
who="parent";
for (int i=0;i<1000*1000;i++) delay+=".";
std::cout<<"Parent: "<<&who<<", "<<who<<"\n";
}
else
{ // child process
std::cout<<"Child: "<<&who<<", "<<who<<"\n";
who="child";
}
std::cout<<"Return: "<<&who<<", "<<who<<"\n";
return 0;
}
The hard part about distributed memory is we have perfect process isolation. But we're working on the same problem--how would we allow communication between parent and child? It is possible to precreate shared-memory areas that both parent and child can read and write using mmap. This essentially rediscovers shared memory, including both the good parts (anybody can read or write any data, making sharing easy), and the bad parts (synchronization and race conditions are a problem).
#include <unistd.h> /* for fork. Only exists on UNIX systems. */
#include <sys/mman.h> /* for mmap. Windows equivalent is VirtualAlloc. */
std::string delay;
int *who=(int *)mmap(0, // suggested address (0 means pick one)
4096, // bytes to allocate (multiple of 4K page size)
PROT_READ|PROT_WRITE|PROT_EXEC, // memory access
MAP_SHARED|MAP_ANONYMOUS, // shared across fork, no file
-1,0 // file and offset, not used for MAP_ANONYMOUS
);
long foo(void) {
*who=0;
std::cout<<"Original: "<<who<<", "<<*who<<"\n";
if (fork()) // <- makes two copies of the process
{ // parent process
*who=1;
for (int i=0;i<1000*1000;i++) delay+=".";
std::cout<<"Parent: "<<who<<", "<<*who<<"\n";
}
else
{ // child process
*who=7;
std::cout<<"Child: "<<who<<", "<<*who<<"\n";
}
std::cout<<"Return: "<<who<<", "<<*who<<"\n";
return 0;
}
At least with mmap-and-fork, memory is only shared when you explicitly ask for it, instead of the shared-everything (even when it hurts!) model of threads.
One standard way to communicate between distributed-memory processes is via network sockets. These have the advantage that they're standardized via the Berkeley sockets interface, which works everywhere.
Brian Hall, or "Beej", maintains the definitive readable introduction to Berkeley sockets programming, Beej's Guide to Network Programming. He's got a zillion examples and a readable style. Go there.
Sadly, bare Berkeley sockets are fairly tricky and ugly, especially for creating connections. The problem is Berkeley sockets support all sorts of other protocols, addressing modes, and other features like "raw sockets" (used for packet sniffers). But when I write network code, I find it a lot easier to use my own little library of public domain utility routines called "osl/socket.h". I'll give examples here using my library.
There are hundreds of network "protocols", different languages you can speak via sockets, but there are only a few I'd recommend for new applications.
If you don't know any better, you should probably use TCP or HTTP to send your data across the network.
Here's a simple example of a TCP client and server.
#include "osl/socket.h" /* <- Dr. Lawlor's funky networking library */
#include "osl/socket.cpp"
#include <unistd.h> /* for fork. Only exists on UNIX systems. */
long foo(void) {
skt_ip_t ip=skt_lookup_ip("127.0.0.1"); // my IP address
unsigned int port=5678; // random port number (above 1024)
if (fork()) // <- makes two copies of the process
{ // parent process acts as network server
SERVER_SOCKET srv=skt_server(&port);
SOCKET s=skt_accept(srv,0,0);
std::cout<<"Parent got connection\n";
std::string str=skt_recv_line(s); // read a line of text
std::cout<<"Parent got string "<<str<<"\n";
skt_close(s);
}
else
{ // child process acts as network client
std::cout<<"Child connecting\n";
SOCKET s=skt_connect(ip,port,2);
std::string send="Hello!\n";
std::cout<<"Child sending string\n";
skt_sendN(s,&send[0],send.size());
skt_close(s);
}
return 0;
}
If you'd prefer not to use my osl/socket library, you can also write programs using raw sockets. Aside from the ugly bit with address handling, it's basically similar.
#ifdef _WIN32
# include <winsock.h> /* windows sockets */
# pragma comment (lib, "wsock32.lib") /* link with winsock library */
#else /* non-Windows: Berkeley sockets */
# include <sys/types.h>
# include <sys/socket.h> /* socket */
# include <arpa/inet.h> /* AF_INET and sockaddr_in */
#endif
#define errcheck(code) if (code!=0) { std::cout<<#code<<"\n"; return 0; }
long foo(void) {
// Create a socket
int s=socket(AF_INET,SOCK_STREAM,0);
// Allow us to re-open a port within 3 minutes
int on = 1; /* for setsockopt */
setsockopt(s,SOL_SOCKET, SO_REUSEADDR,&on,sizeof(on));
// Connect socket to lawlor.cs, port 80 (http port)
struct sockaddr_in addr;
addr.sin_family=AF_INET;
addr.sin_port=htons(80); // port number in network byte order
unsigned char bytes4[4]={137,229,25,247}; // lawlor IP address
memcpy(&addr.sin_addr,bytes4,4);
errcheck(connect(s,(const struct sockaddr *)&addr,sizeof(addr)));
// Send HTTP request:
std::string req="GET / HTTP/1.1\r\n"
"Host: lawlor.cs.uaf.edu\r\n"
"\r\n";
send(s,&req[0],req.size(),0);
// Receive HTTP reply. Hangs due to HTTP keepalive
char c;
while (1==recv(s,&c,1,0)) {
printf("%c",c);
}
printf("Closed!?\n");
return s;
}
CS 441 Lecture Note, 2014, Dr. Orion Lawlor, UAF Computer Science Department.