9.7 Performing Interprocess Communication Using Sockets

9.7.1 Problem

You have two or more processes running on the same machine that need to communicate with each other.

9.7.2 Solution

Modern operating systems support a variety of interprocess communications primitives that vary from system to system. If you intend to make your program portable among different platforms and even different implementations of Unix, your best bet is to use sockets. All modern operating systems support the Berkeley socket interface for TCP/IP at a minimum, while most—if not all—Unix implementations also support Unix domain sockets.

9.7.3 Discussion

Many operating systems support various methods of allowing two or more processes to communicate with each other. Most systems (including both Unix and Windows) support anonymous and named pipes. Many Unix systems (including BSD) also support message queues, which have their origins in AT&T's System V Unix. Windows systems have a similar construct known as mailslots. Unix systems also have Unix domain sockets, which share the Berkeley socket interface with TCP/IP sockets. Here's an overview of common mechanisms:

Anonymous pipes

Anonymous pipes are useful for communication between a parent and a child process. The parent can create the two endpoints of the pipe before spawning the child process, and the child process will inherit the file descriptors from the parent. There are ways on both Unix and Windows for two otherwise unrelated processes to exchange file descriptors, but this is rarely done. On Unix, you can use Unix Domain sockets. On Windows, you can use the OpenProcess( ) and DuplicateHandle( ) Win32 API functions.

Named pipes

Instead of using anonymous pipes between unrelated processes, a better solution may be to use named pipes. With named pipes, a process can create a pipe that has a name associated with it. Another process that knows the name of the pipe can then open the pipe. On Unix, named pipes are actually special files created in the filesystem, and the name used for the pipe is the name of that special file. Windows uses a special namespace in the kernel and does not actually use the filesystem at all, although the restrictions on the names given to pipes are similar to those for files. Pipes work well when there are only two processes involved, but adding additional processes to the mix quickly complicates matters. Pipes were not designed for use by more than two processes at a time and we strongly advise against attempting to use pipes in this way.

Message queues (Unix)

Unix message queues are named with an arbitrary integer value called a key. Often a file is created, and that file's inode is used as the key for the message queue. Any process that has permission to read from the message queue can do so. Likewise, any process with the proper permissions may write to the message queue. Message queues require cooperation among the processes that will use the queues. A malicious program can easily subvert that cooperation and steal messages away from the queue. Message queues are also limited such that they can only handle small amounts of data.

Mailslots (Windows)

Windows mailslots can be named just as named pipes can be, though there are two separate and distinct namespaces. Mailslots are a one-way communication mechanism. Only the process that creates the mailslot can read from it; other processes can only write to it. Mailslots work well when there is a single process that needs to receive information from other processes but does not need to send anything back.

Sockets

It is difficult these days to find an operating system that does not support the Berkeley socket interface for TCP/IP sockets. While most TCP/IP connections are established over a network between two different machines, it is also possible to connect two processes running on the same machine without ever touching a network using TCP/IP. On Unix, the same interface can also be used for Unix Domain sockets, which are faster, can be used to exchange file descriptors, and can also be used to exchange credentials (see Recipe 9.8).

Using TCP/IP sockets for interprocess communication (IPC) is not very different from using them for network communications. In fact, you could use them in exactly the same way and it would work just fine, but if your intent is to use the sockets strictly for local IPC, there are a couple of additional things that you should do, which we discuss in the following paragraphs.

If you are using TCP/IP sockets for local IPC, the most important thing for you to know is that you should always use the loopback address. When you bind a socket, do not bind to INADDR_ANY, but instead use 127.0.0.1. If you do this, you will only be able to connect to the port using the 127.0.0.1 address. This means that the server will be unreachable from any other machine, whether or not you have blocked the port in your firewall.

On Windows systems, the following code will strictly use TCP/IP sockets, but on Unix systems, we have made an optimization to use Unix sockets if the loopback address of 127.0.0.1 is used. We have created a wrapper around the socket descriptor that keeps track of the type of socket (Unix or TCP/IP) and the address to which it is bound. This information is then used in spc_socket_accept( ), spc_socket_sendto( ), and spc_socket_recvfrom( ), which act as wrappers around accept( ), sendto( ), and recvfrom( ), respectively.

Remember that on Windows you must call WSAStartup( ) before you can use any socket functions. You should also be sure to call WSACleanup( ) when you have finished using sockets in your program.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
   
#ifndef WIN32
#include <errno.h>
#include <netdb.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/un.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#define INVALID_SOCKET  -1
#define closesocket(x)  close((x))
#else
#include <windows.h>
#include <winsock2.h>
#endif
   
#define SPC_SOCKETFLAG_BOUND  0x1
#define SPC_SOCKETFLAG_DGRAM  0x2
   
typedef struct {
#ifdef WIN32
  SOCKET            sd;
#else
  int               sd;
#endif
  int               domain;
  struct sockaddr   *addr;
  int               addrlen;
  int               flags;
} spc_socket_t;
   
void spc_socket_close(spc_socket_t *);
   
static int make_sockaddr(int *domain, struct sockaddr **addr, char *host,
                         int port) {
  int                 addrlen;
  in_addr_t           ipaddr;
  struct hostent      *he;
  struct sockaddr_in  *addr_inet;
   
  if (!host) ipaddr = INADDR_ANY;
  else {
    if (!(he = gethostbyname(host))) {
      if ((ipaddr = inet_addr(host)) =  = INADDR_NONE) return 0;
    } else ipaddr = *(in_addr_t *)he->h_addr_list[0];
    endhostent(  );
  }
   
#ifndef WIN32
  if (inet_addr("127.0.0.1") =  = ipaddr) {
    struct sockaddr_un  *addr_unix;
   
    *domain = PF_LOCAL;
    addrlen = sizeof(struct sockaddr_un);
    if (!(*addr = (struct sockaddr *)malloc(addrlen))) return 0;
    addr_unix = (struct sockaddr_un *)*addr;
    addr_unix->sun_family = AF_LOCAL;
    snprintf(addr_unix->sun_path, sizeof(addr_unix->sun_path),
             "/tmp/127.0.0.1:%d", port);
#ifndef linux
    addr_unix->sun_len = SUN_LEN(addr_unix) + 1;
#endif
    return addrlen;
  }
#endif
   
  *domain = PF_INET;
  addrlen = sizeof(struct sockaddr_in);
  if (!(*addr = (struct sockaddr *)malloc(addrlen))) return 0;
  addr_inet = (struct sockaddr_in *)*addr;
  addr_inet->sin_family = AF_INET;
  addr_inet->sin_port = htons(port);
  addr_inet->sin_addr.s_addr = ipaddr;
  return addrlen;
}
   
static spc_socket_t *create_socket(int type, int protocol, char *host, int port) {
  spc_socket_t  *sock;
   
  if (!(sock = (spc_socket_t *)malloc(sizeof(spc_socket_t)))) return 0;
  sock->sd    = INVALID_SOCKET;
  sock->addr  = 0;
  sock->flags = 0;
  if (!(sock->addrlen = make_sockaddr(&sock->domain, &sock->addr, host, port)))
    goto error_exit;
  if ((sock->sd = socket(sock->domain, type, protocol)) =  = INVALID_SOCKET)
    goto error_exit;
  return sock;
   
error_exit:
  if (sock) spc_socket_close(sock);
  return 0;
}
   
void spc_socket_close(spc_socket_t *sock) {
  if (!sock) return;
  if (sock->sd != INVALID_SOCKET) closesocket(sock->sd);
  if (sock->domain =  = PF_LOCAL && (sock->flags & SPC_SOCKETFLAG_BOUND))
    remove(((struct sockaddr_un *)sock->addr)->sun_path);
  if (sock->addr) free(sock->addr);
  free(sock);
}
   
spc_socket_t *spc_socket_listen(int type, int protocol, char *host, int port) {
  int           opt = 1;
  spc_socket_t  *sock = 0;
   
  if (!(sock = create_socket(type, protocol, host, port))) goto error_exit;
  if (sock->domain =  = PF_INET) {
    if (setsockopt(sock->sd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt)) =  = -1)
      goto error_exit;
    if (bind(sock->sd, sock->addr, sock->addrlen) =  = -1) goto error_exit;
  } else {
    if (bind(sock->sd, sock->addr, sock->addrlen) =  = -1) {
      if (errno != EADDRINUSE) goto error_exit;
      if (connect(sock->sd, sock->addr, sock->addrlen) != -1) goto error_exit;
      remove(((struct sockaddr_un *)sock->addr)->sun_path);
      if (bind(sock->sd, sock->addr, sock->addrlen) =  = -1) goto error_exit;
    }
  }
  sock->flags |= SPC_SOCKETFLAG_BOUND;
  if (type =  = SOCK_STREAM && listen(sock->sd, SOMAXCONN) =  = -1) goto error_exit;
  else sock->flags |= SPC_SOCKETFLAG_DGRAM;
  return sock;
   
error_exit:
  if (sock) spc_socket_close(sock);
  return 0;
}
   
spc_socket_t *spc_socket_accept(spc_socket_t *sock) {
  spc_socket_t  *new_sock = 0;
   
  if (!(new_sock = (spc_socket_t *)malloc(sizeof(spc_socket_t))))
    goto error_exit;
  new_sock->sd      = INVALID_SOCKET;
  new_sock->domain  = sock->domain;
  new_sock->addrlen = sock->addrlen;
  new_sock->flags   = 0;
  if (!(new_sock->addr = (struct sockaddr *)malloc(sock->addrlen)))
    goto error_exit;
   
  if (!(new_sock->sd = accept(sock->sd, new_sock->addr, &(new_sock->addrlen))))
    goto error_exit;
  return new_sock;
   
error_exit:
  if (new_sock) spc_socket_close(new_sock);
  return 0;
}
   
spc_socket_t *spc_socket_connect(char *host, int port) {
  spc_socket_t  *sock = 0;
   
  if (!(sock = create_socket(SOCK_STREAM, 0, host, port))) goto error_exit;
  if (connect(sock->sd, sock->addr, sock->addrlen) =  = -1) goto error_exit;
  return sock;
   
error_exit:
  if (sock) spc_socket_close(sock);
  return 0;
}
   
int spc_socket_sendto(spc_socket_t *sock, const void *msg, int len, int flags,
                      char *host, int port) {
  int             addrlen, domain, result = -1;
  struct sockaddr *addr = 0;
   
  if (!(addrlen = make_sockaddr(&domain, &addr, host, port))) goto end;
  result = sendto(sock->sd, msg, len, flags, addr, addrlen);
   
end:
  if (addr) free(addr);
  return result;
}
   
int spc_socket_recvfrom(spc_socket_t *sock, void *buf, int len, int flags,
                        spc_socket_t **src) {
  int result;
   
  if (!(*src = (spc_socket_t *)malloc(sizeof(spc_socket_t)))) goto error_exit;
  (*src)->sd      = INVALID_SOCKET;
  (*src)->domain  = sock->domain;
  (*src)->addrlen = sock->addrlen;
  (*src)->flags   = 0;
  if (!((*src)->addr = (struct sockaddr *)malloc((*src)->addrlen)))
    goto error_exit;
  result = recvfrom(sock->sd, buf, len, flags, (*src)->addr, &((*src)->addrlen));
  if (result =  = -1) goto error_exit;
  return result;
   
error_exit:
  if (*src) {
    spc_socket_close(*src);
    *src = 0;
  }
  return -1;
}
   
int spc_socket_send(spc_socket_t *sock, const void *buf, int buflen) {
  int nb, sent = 0;
  
  while (sent < buflen) {
    nb = send(sock->sd, (const char *)buf + sent, buflen - sent, 0);
    if (nb =  = -1 && (errno =  = EAGAIN || errno =  = EINTR)) continue;
    if (nb <= 0) return nb;
    sent += nb;
  }
   
  return sent;
}
   
int spc_socket_recv(spc_socket_t *sock, void *buf, int buflen) {
  int nb, recvd = 0;
   
  while (recvd < buflen) {
    nb = recv(sock->sd, (char *)buf + recvd, buflen - recvd, 0);
    if (nb =  = -1 && (errno =  = EAGAIN || errno =  = EINTR)) continue;
    if (nb <= 0) return nb;
    recvd += nb;
  }
   
  return recvd;
}

9.7.4 See Also

Recipe 9.8