Operating Systems course guide To follow along with "Three Easy Pieces"

We start by looking at the question of running processes. One of the magic things that an operating system does is to allow multiple programs to run all at the same time while being oblivious of each other. Take a look at the program in ostep-code/intro/cpu.c. You can compile it and run it (in the codespace’s normal operating system) using the following instructions:

gcc -o cpu cpu.c -Wall
./cpu "A"

You should see the program print the letter A over and over.

Hit Ctrl-C to stop the process.

So far nothing very interesting. You wrote a program, ran it, seeing its output. But now what happens if we have multiple programs running? We can try this out by running this same program multiple times, with the following command:

./gpu "A" & ./gpu "B" & ./gpu "C" & ./gpu "D" &

What this does is to start 4 different instances of the program, then has them run in the "background" (that’s what th & sign does). You will now be seeing all 4 letters printed out, in some order. Not necessarily the same order each time.

Since these processes run in the background, you cannot stop them by hitting Ctrl-C. To stop the processes, click the trash looking icon at the top of the terminal window, to close this shell. Then do Ctrl-J or Cmd-J to open up a new terminal. What is really happening here, using concepts we’ll discuss later, is that all the four processes are "children" of the "shell process", and clicking the trash icon closes the shell process, and thus terminates all its children as well.

The second big theme of the course is what the operating system does with the computer’s memory.

Take a look at the example from Figure 2.3 in the book, which you can find in ostep-code/intro/mem.c. You should compile it and execute it with:

gcc -o mem mem.c -Wall
./mem 0

This program demonstrates some of the magic that an operating system does with regards to the computer’s memory. It is also a way to demonstrate some of the differences between C and C++ and prepare us for writing C programs. Note that in C we don’t allocate memory by using the new key word. Instead we use malloc, a shorthand for "memory allocate". malloc takes as a parameter the number of bytes to allocate, in this instance sizeof(int) which is the number of bytes that integers take in this particular system, whatever that number is (probably 4 in more systems). What we get back is a pointer to the appropriate memory location (or the 0 pointer if the allocation failed).

Read that again: memory allocation may fail, and in fact in this course you need to contend with a lot of operations failing, and your code needing to check everything. In this instance the very next line says "I am asserting that the pointer is to an actual memory position (and not 0) and we should fail otherwise".

So next we print out a message showing the memory address that we reserved. Then we make sure to keep incrementing the integer value in that location, and printing the new value each time. Note the atoi C function which is a simple way to convert a string to an integer. This takes the first argument we passed to the call above (./mem 0) and turns it into the actual number 0, and uses it as the initial value.

So far nothing particularly interesting is going on here. The interesting thing comes when we try to run two versions of this program. But before we do this we need to turn off some awesome features of modern operating systems that we’ll come back to later, and also to recompile. Start with the recompilation step:

gcc -o mem mem.c -Wall -fno-pie -no-pie

What this does is to instruct the compiler not produce a "position-independent executable". Don’t worry about what this means for the moment. Next we need to tell the system to not randomize when allocating memory, with the following command (which will start a new shell, so you may need to use cd after it to get back to the intro directory):

setarch `uname -m` -R /bin/bash

This instructs the operating system to start a new bash shell but with the -R option enabled, which makes it so that Linux uses a particular personality called ADDR_NO_RANDOMIZE, which for us simply means that the system will be using a predictable memory address when it allocates the memory. Try to run the ./mem 0 program a few times now, and you’ll see that each time it uses the same memory address to store the integer counter.

Great! With that under our belt, let’s run three copies of the program:

./mem 0 & ./mem 50 & ./mem 100 &

You should be seeing that all three programs (identified by their process id number) are using the exact same memory location for their numbers. But notice that each of them also has a completely independent counter. What value one of them sees does not affect what value the other one sees. Yet they all think they are using the same memory address.

So this is then the second big theme that we will consider, namely virtualization of memory. The operating system, with the help of the hardware, sets mechanisms in place so that each process thinks they have complete usage of the whole system memory, while in reality they each talk to a completely different location. The base mechanism for this is address translation and we will be taking a closer look at it at a later point.

While one part of the the OS work is to protect processes from messing with each other, another part is allowing a single process to have multiple threads of computation happening seemingly simultaneously, and possibly operating on the same parts of the memory. In such a situation we need to make sure that we have correctly accounted for the various interactions between these threads. The job of the operating system in the process is ensure these switches between threads happen effortlessly and safely, and that appropriate primitives are in place to enforce synchronization patterns. While we will discuss these items more n subsequent sections, for now we will see what happens with the simple program of Figure 2.5 in the book, which you can find in the ostep/intro/threads.c file. Let’s compile and run it:

gcc -o threads threads.c -Wall
./threads 100

What this program is meant to do is create two threads that both update the same memory location, incrementing it 100 times total each. So we would expect the final answer to be 200. Run that command a few times, what answers do you get? Also try with a bigger initial value, e.g. 10000.

Recognizing such problems and providing adequate structures to prevent them will occupy a lot of our attention during the concurrency part of this course.

Read the rest of the introduction chapter to learn about the design goals for an operating system as well as a bit of the historical development of operating systems, then answer the following questions.