System Programming in the .NET Framework

(last update: Thu Jan 5 19:53:32 CET 2017)

1   Course Information

This course is a 3-day intensive introduction to C# system programming under the .NET environment. The main objectives of the course is learning the fundamentals of multithreaded and network programming. The course includes a small programming project.

1.1   Instructor

César Rodríguez (http://lipn.univ-paris13.fr/~rodriguez)

1.2   Syllabus and Program

Day 1: Multithreaded programming

  • Creating and destroying threads
  • Passing data to threads
  • Thread synchronization: Monitors, locks, WaitHandles, CountdownEvent, the ThreadPool

Day 2: Network programming

  • The Sockets API: client API, server API
  • Above the socket layer: classes TcpClient, TcpServer, UdpClient

Day 3: Windows

  • The Windows registry
  • Event logs
  • Windows Services

1.3   Attendance, Grading, and other Policies

Our school publishes every year an updated version of the document "Modalités de contrôle de connaissances" (MCC). The MCC contains rules and policies applicable to this course, read them (starting from p. 106). The rules below in this section recall the basic principles of the MCC and extend the MCC when appropriate for this course.

Attendance. Attendance to all the course sessions is mandatory and will be accounted for the final grade of the course. The absence to a session (lecture or practical) can be justified with appropriate documents, such as medical certificates, delay certificates from the SNCF, or others. Those justifications shall be delivered to the "sécretariat", not the instructor.

Delays. A student will be considered late if he/she enters the classroom up to 30 min after the stating time (usually 8.30am or 1.45pm). A delay will be considered unjustified unless the student provides appropriate justifying documents (see previous paragraph) or explanations to the instructor (not the "sécretariat"). Each student has the right to 1 unjustified delay per course (6 sessions = 3 days). Each unjustified delay starting from the 2nd will reduce 5% the overall course grade.

Being late for more than 30 min will be considered an absence for the entire session.

Grading. The final grade will be determined as follows:

Final course grade = Academic Grade * Attendance coef. * Delay coef.

Attendance coef.   = 1 - 0.5  * (Unjustified absences / Number of course sessions)
Delay coefficient  = 1 - 0.05 * (Unjustified delays - 1)

Academic grade     = 0.3 * Continuous assessment +
                     0.2 * Exercises
                     0.5 * Project

Collaboration. Discussion, exchange of ideas, and mutual help for understanding complex concepts are key aspects of the learning process inside of a classroom, and as such they are strongly encouraged in this course. However, any submitted piece of work (exercise solution, program code, report, etc.) displaying the student's name must be the result of the student's original and individual work. Helping your peer to independently reach the solution to a problem or difficulty is acceptable. Passing or receiving the solution from your peer is not.

Collaboration between students (working in groups of two or more) shall be explicitly authorised by the instructor.

All program code produced for the various course assignments is expected to be your own creation. Verbatim copying from Internet webpages or any other sources is entirely disallowed. Taking inspiration to write your own code from code in manpages, tutorials, etc. is allowed under the condition that the source is cited in comments.

Inside the classroom. Using mobile phones, music players, or headphones is disallowed inside of the classroom.

1.4   Evaluation of your code

Here are some criteria that will be taken into account for the evaluation of the source could that you are expected to submit in this course:

  • Homogeneous and reasonable indentation and spacing (paragraphs, empty lines, spaces between words).
  • Your code is as simple as possible, but not simpler.
  • The code is clear, easy to read.
  • Your code is robust and well tested. It rather displays error messages complaining about non-treated corner cases than it produces run-time crashes (for instance, segmentation faults).
  • You use comments to explain your code. You are strongly advised to structure your code in paragraphs, and introduce every paragraph with a one-line comment. You can draw inspiration on every single code example you will see in this web page or the MSDN webpages.

1.5   Exercises and Project

1.6   How to Submit Exercises / Code for Evaluation

You are expected to submit your source code using the ENT.

  1. Last day. Exercises to be submitted until the last day of the course. Deadline: 11.55pm on January 11, 2017. Click here to submit. You are expected to submit exactly the following files:

    ThreadExercises.cs
    Client.cs
    Server.cs
    

    Please submit exactly those files. Do not change any file name. Do not add other files unless it is strictly necessary. Failure to follow these instructions will negatively impact your grade.

  2. Programming project. The Survey system. Deadline: 11.55pm on TBD. Click here to submit.

2   Multithreaded Programming

Almost all you will find here has been extracted from the MSDN. A good starting point is the section Threading Objects and Features of the .NET Framework Development Guide.

2.1   Introduction

  • What is a thread?
  • Threads vs processes
  • Preemtive multitasking, time slices
  • When to use multiple threads?
    • Reactive user interaction when long calculations are needed
    • Communication
    • Priority

2.2   Delegates

Intuitively, a delegate is the type of a pointer to a function. A delegate also acts like a class. Each instance of the class is a pointer to a function of the right type (the type of the delegate). We can also "call" the instances, and that will indirectly call the function pointed.

Example:

class Test
{
   // the delegate
   private delegate int Comparer (object o1, object o2, int i);

   // a function with the same "type"
   private static int string_cmp (object ob1, object ob2, int i)
   {
     return i + ((string) ob1).Length - ((string) ob2).Length;
   }

   // another function with the same "type"
   private static int string_cmp (object ob1, object ob2, int i)
   {
       return 123;
   }

   static void Main ()
   {
     // cmp is an instance of the Comparer delegate; it points to string_cmp
     Comparer cmp = new Comparer (string_cmp);

     // you can "call" the variable cmp
     Console.WriteLine (cmp ("bye", "hello", 3));

     // will compile, but will throw an exception. Which?
     Console.WriteLine (cmp ("bye", 4.5f, 3));
   }
}

2.3   Creating a Thread, Minimal Example

  • Threads are created via delegates, or parametrized delegates (so as to pass data to the thread)
  • More examples at the Examples section from the class Thread.
using System;
using System.Threading;

class Example
{
   static void MyThread ()
   {
      for (int i = 0; i < 10; i++)
      {
         Console.WriteLine ("Thread: i = {0}", i);
         Thread.Sleep (200); // in ms
      }
   }

   static void Main (string [] args)
   {
      ThreadStart fun;
      Thread t;

      // we create an instance of the System.Threading.ThreadStart delegate,
      // then we create the thread
      fun = new ThreadStart (MyThread);
      t = new Thread (fun);

      // start the thread (this will not block the Main)
      t.Start();

      // do something on the main function
      for (int i = 0; i < 5; i++)
      {
         Console.WriteLine ("Main: i = {0}", i);
         Thread.Sleep (100);
      }

      // block until the thread terminates
      t.Join ();
      Console.WriteLine ("Main: end!");
   }
}

2.4   Passing data to the thread

There is two fundamental ways to pass data to the new thread:

  1. Executing an instance method from a newly created object that contains the data. See 2nd example here.
  2. Using the ParameterizedThreadStart delegate (observe that the Thread object has 4 constructors). In this case the function executed by the thread will have one argument.

Example of the first method:

class ThreadWithState {
   public int n;

   public ThreadWithState (int n_) {
      n = n_;
   }

   public void Work () {
      int orig = n;

      for (int i = 0; i < 2; i++)
         n *= orig;

      Console.WriteLine ("Thread: n = {0}, n^3 = {0}", orig, n);
   }
}

class Example2 {
   static void Main (string [] args) {
      ThreadWithState tws;
      Thread t;

      // we "pass" number 10 to the thread
      tws = new ThreadWithState (10);
      t = new Thread (new ThreadStart (tws.Work));

      t.Start ();
      t.Join ();

      // we "read" the results of the computation
      Console.WriteLine ("Main: tws.n = {0}", tws.n);
   }
}

Exercises: ThreadExercises.cs, exercises 1 and 2.

2.5   Thread Destruction

Calling method Thread.Abort causes the execption ThreadAbortException to be thrown in the thread. The thread can catch it. After the execution of the exeception handler, the thread cannot be restarted.

For more information, see Destroying threads.

2.6   Pausing and Resuming Threads

Calling method Thread.Interrupt will cause the execption ThreadInterruptedException to be thrown in the thread. The time at which the exception is thrown varies:

  • If the thread is currently blocked on a blocking call (e.g., Thread.Sleep), it will be thrown immediately, and the blocking call will return immediately
  • If the thread is currently running, it will be thrown the next time that the threads tries to call to a blocking function.

For more information, see Pausing and Resuming Threads.

The Thread class includes two methods, Thread.Suspend and Thread.Resume, for pausing and resuming a thread. However, the use of these methods is not recommended.

2.7   Other Generalities

  • As a general overview, see the Managed Threading Basics section of the guide
  • On single-processor machines, threads are executed during time slots of 20ms
  • Executing Thread.Sleep (0) context-switches to a different executable thread and terminates the time slot given to this one
  • Exceptions in thread generally terminate the process, with the exceptions mentioned here
  • So do handle exceptions on threads, as they can have unexpected consequences if your threads synchronize in any way
  • Foreground and background threads and the IsBackground property

2.8   Synchronization

When a program makes use of multiple threads, in almost all cases threads need to interact. Common interactions include to exchange data, to wait for each other when performing some task, to avoid modifying the same variables at the same time, to consume the data produced by another thread, etc.

Threads achieve these interactions using synchronization primitives, that is, low-level operations provided by the underlying programming language (C# in our case) to perform the elementary steps of an interaction between threads.

We already know at least two synchronization primitives: the methods Start and Join in the class Thread. The synchronization they produce (the constraints that they impose on the execution of the overall program) are as follows:

  • The thread code will be executed only after the call to the Start method.
  • The thread code will terminate always before the method Join returns.

These two obvious but important guarantees help the programmer to organize the interaction of the threads in order to achieve the goals of the application. In this section we will study the primitives that C# offers to write more complex thread interactions.

When thread interaction is not adequately performed, the application can display unexpected behavior and results. We shall start by studying a small program that illustrates what can go wrong when improper or no synchronization is used between threads:

class Bank {
   public int [] account;

   public Bank () {
      account = new int[4];
      for (int i = 0; i < 4; i++) account[i] = 100;
   }

   public void Transfer (int src, int dst, int quantity) {
      int balance;

      balance = account[src];
      Thread.Sleep (0);

      if (balance < quantity) {
         Console.WriteLine ("error: src {0} balance {1} quantity {2}",
            src, balance, quantity);
         return;
      }

      account[src] -= quantity;
      account[dst] += quantity;
   }

   public void Print () {
      for (int i = 0; i < 4; i++)
         Console.WriteLine ("account {0} balance {1}", i, account[i]);
   }

   public void Branch1 () {
      Transfer (0, 1, 50);
   }

   public void Branch2 () {
      Transfer (0, 2, 60);
   }
}

class Example3 {
   static void Main (string [] args) {
      Bank b = new Bank ();
      Thread t1 = new Thread (new ThreadStart (b.Branch1));
      Thread t2 = new Thread (new ThreadStart (b.Branch2));

      t1.Start ();
      t2.Start ();
      t1.Join ();
      t2.Join ();

      b.Print ();
   }
}

Pay close attention to the method Transfer. What can go wrong when two transfers are concurrently initiated from two different Bank branches (threads)?

A good conceptual overview of the synchronization primitives in C# can be found in the section Overview of Synchronization Primitives. We can divide all synchronization primitives that we will learn in this course in two classes:

  • Locking: Locking primitives help your threads guaranteeing mutual exclusion. That is, only one thread (or a specific number of them) access one resource at a time.

    Locking primitives include the lock statement, Monitor, Mutex, SpinLock, ReaderWriterLock, Semaphore. Whenever possible the lock statement should be used.

  • Signaling: Signaling consist on informing from one thread to one or more other threads of a specific condition.

    Signaling Primitives: EventWaitHandle and friends (auto reset, manual reset), Mutex, Semaphore, Barriers. All classes but Barrier derive from WaitHandle.

Exercises: ThreadExercises.cs, exercise 3.

2.8.1   Monitors (locking)

Monitors are perhaps the most elementary synchronization primitives in C#. A monitor is a synchronization device on which we can perform 4 basic operations. Each object in C# has a monitor. We access the monitor of an object using the methods of the class Monitor, which is static (that is, all methods are static):

System.Threading.Monitor.Enter (obj);
System.Threading.Monitor.Exit (obj);
System.Threading.Monitor.Wait (obj);
System.Threading.Monitor.Pulse (obj);

We will first focus on the first two. Intuitively, a monitor behaves like a room, it can be in two states: busy or available. Initially a monitor is available. When a thread calls Enter on a monitor, two things can happen:

  • If the monitor is available, it will now become busy and the call will return immediately.
  • If the monitor is busy, the calling thread will block and wait until the monitor becomes available. It will then turn the monitor into the "busy" state, and the call to Enter will return.

The thread that acquired the monitor (that is, Enter returned the control to the calling thread) can release the monitor (make it available again) by calling Exit.

Methods Wait and Pulse make possible that one thread that had acquired the monitor temporarily releases it, allowing another thread to acquire it. When that second thread acquires the monitor and calls Pulse, the first thread is scheduled to re-acquire the monitor, which may happen as soon as the second thread releases the monitor (when it calls Exit). Calls tu Pulse do not accumulate, that is, they are not remembered. If a thread calls Pulse and no thread is Wait-ing, the call is lost, and next time a thread calls Wait it may wait forever.

2.8.2   The lock statement

The lock statement is a keyword in C# that allows to more easily use the Monitor of an object to synchronize multiple threads.

It has the following syntax:

lock (obj)
{
   // the code here
}

Internally, the compiler will translates the code above by the following code:

System.Threading.Monitor.Enter (obj);
// the code here
System.Threading.Monitor.Exit (obj);

You can now see that the lock statement is a just a simpler way to use the Monitor of an object. See the C# reference for more information.

Exercises: ThreadExercises.cs, exercise 4.

2.8.3   Wait Handles (signaling and locking)

  • Conceptual overview here
  • All signaling primitives (EventWaitHandle, AutoResetEvent, Mutex...) inherit from the class WaitHandle !!
  • Encapsulates Win32 synchronization handle
  • Can be less portable and less efficient in terms of resource requirements than monitors/locks
  • It is abstract, so it cannot be instantiated

Important methods:

System.Threading.WaitHandle.WaitOne ()
System.Threading.WaitHandle.WaitAll (WaitHandle [])
System.Threading.WaitHandle.WaitAny (WaitHandle [])

Derived classes differ in thread affinity (who can signal/release the wait handle, the thread that got it, or anyone?); mutexes have thread affinity, others have not

2.8.4   EventWaitHandle, AutoResetEvent, ManualResetEvent (signaling, but also locking)

All the three classes inherit from WaitHandle. We focus on EventWaitHandle, as the behavior of AutoResetEvent and ManualResetEvent can be explained in terms of EventWaitHandle.

Intuitively, an EventWaitHandle behaves like a traffic light. It has two states, green and red. When the light is green cars can cross without waiting. When the light is red, cars trying to cross get blocked.

  • Method Set turns the traffic light to green.
  • Method Reset turns the traffic light to red.
  • Method WaitOne lets the car cross the traffic light. If the light is green, the calling thread (car) will cross without getting blocked. If it is red, then the calling thread will get blocked.

We can create an EventWaitHandle as follows:

bool           init = ?? // true or false
EventResetMode mode = ?? // EventResetMode.AutoReset or EventResetMode.ManualReset

h = new EventWaitHandle (init, mode);

The first parameter gives the initial state of the EventWaitHandle:

  • true = green
  • false = red

The second parameter determines how the traffic light will change the state when threads cross it. Two modes are possible:

  • AutoReset: Each car that crosses the traffic light (a thread calling WaitOne), turns the light to red (after having crossed the traffic light).
  • ManualReset: Whenever the traffic light is green, every car (thread) can cross freely (WaitOne does not block). If you want to block cars, you need to manually turn the light red (call the method Reset)

AutoResetEvent and ManualResetEvent are classes deriving from EventWaitHandle. They behave like EventWaitHandle when it is initialized passing the associated value to the second parameter of the constructor.

More on auto-reset and manual-reset wait handles.

Exercises: ThreadExercises.cs, exercises 5 and 6.

2.8.5   CountdownEvent (signaling)

Consider the following code:

CountdownEvent cde = new CountdownEvent (12)
  • It creates a CountdownEvent
  • This is an object that behaves like a traffic semaphore with two integer variables associated to it:
    • the CurrentCount, it will act as a countdown counter, and
    • the InitialCount, the 12 passed to the constructor.
  • The value of InitialCount can only be modified on the constructor
  • You can modify the value of CurrentCount with
    • Signal(), decrements by 1
    • AddCount(), increments by 1
    • Reset(), sets it to the value of InitialCount
  • A CountdownEvent has a WaitHandle associated to it, as well.
  • When CurrentCount is 0, it sets the handle, that is, the semaphore becomes green.
  • Otherwise the handle is closed (semaphore red)
  • To block until the CurrentCount becomes 0, call the instance method Wait()
  • If the handle is set (the CurrentCount is 0), calling Signal or AddCount will raise an exception. You can only call Reset.

2.8.6   Other Synchronization Primitives

2.9   The Thread Pool

  • Interesting information in the Remarks section of the ThreadPool Class
  • System.Threading.ThreadPool is static
  • Threadpool.QueueUserWorkItem is the routine to execute start a thread
  • You may pass an object to the function; the thread's function will receive it in its (single) argument (it will receive null if you did not provide an argument to QueueUserWorkItem)
  • All threads are background threads

Exercises: ThreadExercises.cs, exercises 7 and 8.

2.10   Timers

See this article on the development guide.

3   Network Programming

As before, we will heavily rely on the section Network Programming in the .NET Framework of the .NET Framework Development Guide.

In this chapter we will study the API available in the System.Net.Sockets namespace.

3.1   Sockets

We focus on connective sockets, using TCP. The first step is creating the socket.

3.1.1   The Client Side

3.1.2   The Server Side

While the client usually only establishes one connection at a time, the server needs to deal with potentially several ones. As in probably any other language you know, we have a select function for checking the status of several threads and blocking until we need to do some work.

The static method Socket.Select takes three lists of (references to) Socket instances and can block until there is something to read (first list), something to write (second list) or an error was reported (third list). A call to Select will return when at least one socket is ready to do work with it, which we can check with the Poll method of the Socket class.

If a socket is a listening socket, i.e., it is being used to accept new clients, place it in the reading list. If a client has arrived, and Accept will return immediately with its associated Socket, then the Poll method will return true when invoked with the SelectMode.SelectRead switch.

  • Example 1: SimpleTcpClient.cs, illustrating how to create a client socket and receiving data form it.

3.1.3   UDP Sockets

  • The client do not need to call Connect, it can rather send data using the SendTo and ReceiveFrom methods from the Socket class, which allow to directly provide the destination address
  • If, however, it calls Connect on a UDP socket, incoming packages from a different IP address and port will be discarded.
  • They client may also want to Bind the socket to an specific local IP and port before sending data
  • As for the server, there is no need to call Listen (as you are not accepting connections), just create the socket and bind it to a local IP and port

Exercises: Client.cs, exercise 9; Server.cs, exercises 11, 12.

3.2   Above the Sockets interface: TCP and UDP Services

The classes TcpClient, TcpListener, and UdpClient help you opening data connections over TCP and UDP, as well as accepting incoming TCP connections. They abstract away the details of setting up and using sockets and provide a simplified access to the network.

Their use is straightforward, see

Exercises: Client.cs, exercise 10.

4   The Windows Registry

The Windows registry is a database where applications can store settings, configuration, and general information that shall be preserved after the application is closed.

It offers an interface to store and retrieve information with a tree-like structure, where each branch of the tree is technically called a key and each node of the tree can store zero or more pairs of the form

(name, value)

For instance, the well known application Notepad seems to store some settings under the key

KEY_CURRENT_USER\Software\Microsoft\Notepad

where we find multiple name-value pairs storing certain settings, such as:

("IfFaceName", "Fixedsys")
("iWindowPosX", 0x00000012)
("iWindowPosY", 0x0000002f)

You can visualize (and modify!) the Windows registry using the tool regedit.exe (you may want to type the name of the program at the Start > Execute dialog).

In .NET, the two main classes to access the registry are the (static) class Registry and the class RegistryKey, both located within the namespace Microsoft.Win32.

Important entry points to the MSDN documentation:

Two examples:

object obj;
obj = Registry.GetValue("HKEY_CURRENT_USER\\Software\\WinRAR\\ArcHistory", "1", null);

RegistryKey k;

k = Registry.CurrentUser;

Console.WriteLine (k);
Console.WriteLine (k.Name);
Console.WriteLine (k.ValueCount);
Console.WriteLine (k.SubKeyCount);

foreach (var s in k.GetValueNames()) Console.WriteLine(s);
foreach (var s in k.GetSubKeyNames()) Console.WriteLine(s);

There are several standard entry points to the registry, they correspond to the nodes of the tree hanging just below the root of the tree (see the static fields of class Registry).

Also, each value in a name-value pair has a type associated to it (see the method Registry.SetValue and the enumeration RegistryValueKind).

5   Windows Event Logs

The Windows event log is a place where both Windows and applications running on top of it record important software and hardware events. Each entry in the log contains informations such as the machine where it was generated, the time it was generated, a source, and of course a text message.

The source identifies, conceptually, the application originating the event, although a source does not need to designate an application. Both the log where the event will be stored as well as the source need to be created within the system well before events can be logged.

Several standard event logs are already present in the machine. In particular, the log Application will be of interest for us. You can view the contents of existing event logs using the Event Viewer (Start > Control Panel > System and Security > Administrative Tools > Event Viewer, administrative permission will be required). Many informations about the existing event logs are stored at the registry, see here.

It is possible to create new event logs and sources. This should be done well before you start using the log, as Windows apparently needs some time to update certain internal files (...). Ideally this should be done during the installation of your application.

In .NET, we can

6   Windows Services

A Windows service is a notion analog to that of a daemon in Unix: a long-running process, executing in background, and offering some service to someone else. See this introduction to service applications from the MSDN library.

Unlike a Unix daemon, which often only offers one service, a single Windows program can contain several Windows services. They must be installed on the server before they run. That is, we need to carry out some specific procedure by which the Service Control Manager (SCM) will get to know the executable binary where our services are contained and the services themselves. Once this is done, we will be able to start, stop, or pause the service from the SCM.

Services, thus, function differently from regular applications. The overall procedure to write and run a service coded in .NET is as follows:

  1. A service will be a class inheriting from ServiceBase. You will override the methods OnStart, OnStop and optionally OnPause and OnContinue.
  2. The Main method of your program will instantiate all services in your program and pass each of them to the static method ServiceBase.Run.
  3. The binary program needs to define an installer class, inheriting from System.Configuration.Install.Installer. Within its constructor, construct and initialize instances of the classes ServiceProcessInstaller and ServiceInstaller. These will tell the SCM how to invoke, respectively, the binary program containing the service, and the services themselves.
  4. To install and uninstall the service, use the tool installutil.exe. Administrative rights are required.

Here you have a minimal example (warning, I didn't compile it!):

using System.ServiceProcess;
using System.Configuration.Install;

static class Program {
 static void Main() {
       ServiceBase.Run (new PingService ());
 }
}

public class PingService : ServiceBase {
 public PingService() {
       ServiceName = "PingService";
       CanPauseAndContinue = true;
 }

 protected override void OnStart(string[] args) {
       // do something
 }

 protected override void OnStop() {
       // do something
 }

 protected override void OnPause() {
       // do something
 }

 protected override void OnContinue() {
       // do something
 }
}

/* for the install utility to recognize Setup as a valid installer */
[RunInstaller(true)]
public class Setup : System.Configuration.Install.Installer {
 private ServiceProcessInstaller proc_inst;
 private ServiceInstaller serv_inst;

 public Setup() {
       proc_inst = new ServiceProcessInstaller();
       serv_inst = new ServiceInstaller();

       proc_inst.Account = ServiceAccount.LocalSystem;
       proc_inst.Password = null;
       proc_inst.Username = null;

       serv_inst.Description = "A long description of your service";
       serv_inst.DisplayName = "The name you will see in the SCM";
       serv_inst.ServiceName = "PingService";
       serv_inst.StartType = ServiceStartMode.Automatic;

       Installers.Add (proc_inst);
       Installers.Add (serv_inst);
 }
}

Several good entry points to the subject on the MSDN library:

Finally, to start the service, open the SCM (Start > Programs > Administrative Tools > Services), search for your service (the column "Name" is the DisplayName property of the ServiceInstaller object created during installation) and click on Start, Stop (or Pause). You can also manually do it from code, using the ServiceController class.