Unix System Programming

1   This Course

1.1   Syllabus

  • gcc, make, vi
  • File I/O
  • Process management
  • Pipes
  • Signals
  • Sockets

2   Introduction

2.1   Getting help

2.2   System calls

  • Specific procedure, as processor needs to switch to supervisor mode
  • The standard C library provides C-wrappers
  • Documented in section 2; see also section 3 and section 7
  • 380 system calls for Linux 3.2.0, see syscalls(2)

2.3   Error handling

  • System calls almost always return -1 (or NULL) when an error occurs

  • C library makes available an explanation through variable

    extern int errno;
    

    defined in <errno.h>, see errno(3)

Variables errno(3)
Functions strerror(3), perror(3)
See also err(3) (non-standard BSD functions, but very handy)

2.4   First example

#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <err.h>

int main(int argc, char *argv[])
{
   fprintf(stderr, "EACCES: %s\n", strerror(EACCES));
   errno = ENOENT;
   warn ("%s: The error ENOENT", argv[0]);
   exit(0);
}
  • Use exit(3) or simply return from the main function to terminate the program. The return code of the program is the value passed to exit or returned from main.

2.5   Compiling with gcc

Assume you have a small project consisting on two C files, main.c, containing the main() function of the program, and aux.c, containing auxiliary code.

Compiling and linking with one command:

gcc aux.c main.c -o myprog

Compiling into object code + linking:

gcc aux.c        # outputs aux.o
gcc main.c       # outputs main.o
gcc aux.o main.o -o myprog

The advantage of the second method is that it requires compiling only the files that were modified since last compilation (+ linking)

Here are some frequent options:

-g
-Wall
-Wextra
-O1, -O2, -O3
-lm
-lMY_LIB
-I INCLUDE_DIR
-D MACRO=VALUE

2.6   Using make

  • The tool make computes which parts of a program need to be recompiled and calls the compiler accordingly

  • Information about the source code is in the file Makefile (or makefile)

  • Target-oriented rules of the form:

    target : prerequisite1 prerequisite2
    <TAB> shell commands using the prerequisites and generating the target
    <TAB> more shell commands
    
  • Observe that the commands need to be prefixed with a tab (!)

  • The target of the first rule in the Makefile becomes the default target

First example of a Makefile:

myprog: main.c aux.c
    gcc -Wall -Wextra -g aux.c main.c -o myprog

A more advanced version:

myprog: main.o aux.o
    gcc main.o aux.o -o myprog

main.o: main.c
    gcc -Wall -Wextra -g -c main.c

aux.o: aux.c
    gcc -Wall -Wextra -g -c aux.c

Now the same but using pattern rules and automatic variables:

myprog: main.o aux.o
    gcc $^ -o $@

%.o: %.c
    gcc -Wall -Wextra -g -c $^ -o $@

In fact, make already knows the pattern rule above, not only for C but also for many other languages! Such built-in rules (which you can display running make -p) are parametrized with variables. An even shorter version of our Makefile would be:

CC=gcc
CFLAGS=-Wall -Wextra -g

myprog: main.o aux.o
    gcc $^ -o $@

Some frequently used options of make are

-n
-j N
-p

3   File I/O

System call Description
open, creat open and possibly create a file or device
close close a file descriptor
lseek reposition read/write file offset
dup, dup2, dup3 duplicate a file descriptor
read read from a file descriptor
write write to a file descriptor
stat, fstat, lstat get file status
sync, syncfs commit buffer cache to disk
fsync, fdatasync synchronize a file's in-core state with storage device
ioctl control device
truncate, ftruncate truncate a file to a specified length
remove remove a file or directory
mkdir create a directory
select synchronous I/O multiplexing
poll, ppoll wait for some event on a file descriptor
mmap, munmap map or unmap files or devices into memory
msync synchronize a file with a memory map

4   Processes

Process management:

System call Description
getpid, getppid get process identification
getuid, geteuid get user identity
setuid set user identity
setgid set group identity
fork create a child process
exit cause normal process termination
wait, waitpid, waitid wait for process to change state
execl, execv execute a file
system execute a shell command

Basic thread management:

System call Description  
pthread_create create a new thread
pthread_attr_init, pthread_attr_destroy initialize and destroy thread attributes object
pthread_detach detach a thread
pthread_exit terminate calling thread
pthread_join join with a terminated thread
pthread_mutex_destroy, pthread_mutex_init destroy and initialize a mutex
pthread_mutex_lock, pthread_mutex_trylock, pthread_mutex_unlock lock and unlock a mutex

5   Pipes

System call Description
pipe, pipe2 create a half-duplex pipe
popen, pclose pipe stream to or from a process
mkfifo make a FIFO special file (a named pipe)

6   Signals

Signal management:

System call Description
signal ANSI C signal handling
kill send signal to a process
raise send a signal to the caller
Table of signals:
-- From the signal(7) manual page
Signal Value Action Comment
SIGHUP 1 Term Hangup detected on controlling terminal or death of controlling process
SIGINT 2 Term Interrupt from keyboard
SIGQUIT 3 Core Quit from keyboard
SIGILL 4 Core Illegal Instruction
SIGABRT 6 Core Abort signal from abort(3)
SIGFPE 8 Core Floating point exception
SIGKILL 9 Term Kill signal
SIGSEGV 11 Core Invalid memory reference
SIGPIPE 13 Term Broken pipe: write to pipe with no readers
SIGALRM 14 Term Timer signal from alarm(2)
SIGTERM 15 Term Termination signal
SIGUSR1 30,10,16 Term User-defined signal 1
SIGUSR2 31,12,17 Term User-defined signal 2
SIGCHLD 20,17,18 Ign Child stopped or terminated
SIGCONT 19,18,25 Cont Continue if stopped
SIGSTOP 17,19,23 Stop Stop process
SIGTSTP 18,20,24 Stop Stop typed at tty
SIGTTIN 21,21,26 Stop tty input for background process
SIGTTOU 22,22,27 Stop tty output for background process

7   Other Inter-Process Communication (IPC) Primitives

8   Network

Network programming with POSIX sockets enables the initiation, handling and termination of TCP and UDP streams, among others. The interface primitives are thus coupled with the operations associated to these protocols.

The following image illustrates the flow of operations that client and server shall carry out to maintain a TCP connection.

fig/sockets.png

Both server and client will initially create a socket, specifying they want to use the internet protocols (AF_INET), in particular a TCP stream (SOCK_STREAM). The server will then bind the socket to a service point, i.e., a couple (IP address, port) to which the client will later connect.

When the server invokes listen, the socket is set as a listening socket, that is, a socket that can only be used to derive new data sockets. No data will be transmitted through a listening socket. After calling accept, the server will block until a client executes connect. The return value of accept will be a new data socket, that can now be used to transmit data.

Subsequent calls to write and read on the client or server will transfer data to the client or server. It is not true that the data chunk passed to write at one side of the connection will necessarily be the one that read returns at the other side. Each TCP stream is best thought as a data pipe, where write pushes data at one side and read extracts data from the other side. Often, read will return as soon as any data is available. This may be triggered, for instance, by the arrival of an IP network packet encapsulating a TCP segment. Realize that boundaries created the TCP segmentation algorithm will often not be boundaries associated to the write calls, although it may be the case. [1]

Invoking close at the client or server will close the connection in both ways. It is also possible to partially close the connection in any direction, using shutdown. Closing the connection at one side will trigger different events on the other side. If the other side writes, the process will receive the signal SIGPIPE [2], and write will fail with EPIPE. [3] If the other side reads, read will return 0 bytes, indicating an end of file.

System call Description
socket create an endpoint for communication
bind bind a nam to a socket
listen listen for connections on a socket
accept accept a connection on a socket
connect initiate a connection on a socket
shutdown shut down part of a full-duplex connection
htonl, htons, ntohl, ntohs convert values between host and network byte order
send, sendto, sendmsg send a message on a socket
recv, recvfrom, recvmsg receive a message from a socket
getaddrinfo, freeaddrinfo, gai_strerror network address and service translation
getsockopt, setsockopt get and set options on sockets

8.1   Exercise, the command tcpcat

The goal of this exercise is writing a simplified version of the netcat(1) command. The output should be called tcpcat, and it will be able to both open TCP connexions as a client and act as a TCP server, with the following syntax and functionality:

  • $ tcpcat HOST PORT
    where HOST is either an IP address or an URL, and PORT is a port number.

    This command will open a connection to the port PORT of the host HOST. It will send anything written to its standard input and write on standard output anything received from the connection.

  • $ tcpcat -l PORT
    where PORT is a port number.

    This command will open a listening TCP socket accepting incoming connections from any network card. It will accept one client, will send to it anything read from its standard input, and will write to the standard output anything received from the connection.

Note: you do not have the right to use threads to implement this program, use either poll(2) or select(2).

9   Notes

[1]If, for instance, your application introduces large enough amounts of time between each write.
[2]And will be killed, unless the default handler for this signal is modified.
[3]The write primitive, as usual, will return -1 and the variable errno will be equal to the value EPIPE.