1. What is Advanced Operating Systems

General

Looking at the course title, Advanced Operating Systems, we can infer something about the content of the course:

Advanced

This is an advanced course, it has prerequisites! If you haven't had a prior course in the subject, you should be very hesitant about taking this course!

Appropriate prerequisites include courses with names such as Introduction to System Software and Introductory Operating Systems. The content of such courses is not well standardized, but you should know a fair amount about at least one operating system, and you should have at least a theoretical understanding of process management and interrupt-driven input/output.

Operating Systems

This is a course about operating systems. Not just about one operating system (imagine Advanced Windows, or Advanced Linux) but about advanced subjects in the field of operating systems.

Continuing this word-by-word breakdown, we can learn something about the subject of the course from the phrase operating systems.

Operating

The use of the word operating implies a subject (something that operates) and an object (something that is operated). What are these?

The object is clearly the computer. The operating system operates the computer.

The indirect subject is clear. The operating system operates on behalf of its human user or on behalf of its human user community.

The direct subject is less clear. Generally, the human users interact with applications programs of one kind or another, and the operating system, in turn, provides services to and operates on those applications.

Systems

The word systems says quite a bit. A system is composed of components of some kind, frequently not primitive components, but subsystems in their own right. What are these subsystems, and does the organization of the operating system relate to their ogranization?

What is an Operating System?

Posing the question "what is an operating system?" to a class typically generates the following kinds of responses:

• An operating system may provide a uniform interface across multiple platforms. This is certainly true of modern Unix (and its many variants such as Linux, HP-UX, IRIX, AIX, Solaris), although Unix was not originally designed with this in mind; Unix as we know it today began as an operating system for the 16-bit PDP-11 computer, although an earlier version was written in assembly language for the 18-bit PDP-7.

  Other modern operating systems, notably MS-Windows and Mac OS were not developed with platform independence as a design goal, although Mac OS, in its evolution, has migrated from the 68000 processor used in early Macintosh computers to the Power PC architecture of modern systems, and with MacOS X, it is built on top of the extremely portable Mach kernel, a kernel originally developed to support a reimplementation of Unix.

  While Microsoft's Windows family of operating systems was originally intended only for use on the Intel X86 family of architectures, Windows NT was ported to a variety of non-Intel platforms in its early days, before Microsoft made a deliberate to focus exclusively on the Intel platform. made a deliberate decision to cease supporting other platforms. (thereby leading to the near termination of research into computer architectures that are not Intel compatable).

• An operating system provides abstractions such as files, windows, timers and other tools that programmers would like to use.

  There is no theoretical need for abstraction! In theory, any program can be written on a bare machine, but for any machine with more than a few kilobytes of program and data memory, this is extremely inconvenient, so programmers develop standard libraries of abstractions that, taken together, constitute an operating system.

• An operating system controls the allocation of resources:
  • One or more CPUs.
  • The main memory.
  • The I/O channels.
  • The secondary memory (typically disk).

  This resource management model was at the forefront of most thinking about operating systems in the 1970's, when most computer systems were shared between many users who were in competition for the scarce resources of the systems.

  As a result, with the advent of personal computers, there was even talk about the irrelevance of operating systems, since with only one user, it was easy to imagine that there were no resource contention problems that needed to be solved.

  As it turns out, personal computers are not isolated. We exchange disks, we attach them to each other with networks, and we use some of them to run server software. Because of this, resource contention problems, particularly security, are even more severe today than they were in the 1970's when large centralized computer systems were dominant. Many of the huge security loopholes in the personal computer operating systems of today are the result of the first 20 years of the personal computer era, when many of the system designers believed that resource management was a non-issue.

• An operating system provides mechanisms for enforcing security policies.

  A good operating system will provide general purpose mechanisms that can be used to enforce a variety of policies; most systems in use today, from Unix to Windows were designed for environments where security was not a big issue, so to the extent they support security, their mechanisms are generally special purpose, limiting the range of policies that may be enforced.

  Much of the operating systems research in the 1970's focused on creating general purpose mechanisms for enforcing security policies, but the incorporation of these mechanisms into today's commercial operating systems is, at best, irregular.

• The combination of hardware plus operating system software provides a virtual machine to the users.

  The resources of this virtual machine are more closely related to the needs of applications programmers than the resources of the bare hardware. In speaking of a virtual machine, we treat the abstractions provided by the system as if they were hardware facilities. We may even consider the combination of programming language implementation, standard library, operating system and hardware as a virtual machine.

To most users, the operating system remains a mysterious piece of expensive and often quirky software. Most users simply define the operating system as "the software that comes with the computer." More sophisticated users define the operating system as "the software that establishes the look and feel of the system", focusing on the user interface, and particularly on the organization of the display screen.

An Example

I visited the John Deere corporate data processing center in the fall of 2001. Here's a very rough idea of the kind of system they run, giving an idea of the kind of environment in which modern operating systems are expected to run.

• Mainframes: 5 IBM systems, 5 CPU's each. These machines were each about the size of a large upright refrigerator; at their heart is a modest sized water-cooled aluminum brick to which the chips are all mounted. This brick has on it the CPU chips, cache chips, bus interface chips and peripheral I/O manager chips. Each CPU is about the size and complexity of a pentium, but designed to run IBM's mainframe instruction set that has grown from the classical IBM System 360 in of the mid 1960's, the first significant 32-bit computer to hit the marketplace. The machines runs a modern descendant of IBM's VM operating system, the ancestor of which was developed in the very late 1960's.

  Each mainframe has dedicated fiber-optic connections to a 4-terabyte fully redundant RAID disk system on the floor below. Fully redudnant means that the actual capacity of the disk system is over 8 terabytes.

• Over 250 IBM RS-6000 machines -- high end dual processor 32-bit RISC systems running AIX, IBM's version of Unix, and served by a file server that appears to have involved another 4-terabyte RAID system.

• Several hundred Windows NT machines, either sharing the same file server as the AIX machines, or with their own dedicated server. This was unclear, but it doesn't really matter to the story.

• An automated tape library subsystem with 400 terabytes of capacity. This appeared to back the mainframes; it appears that there was another 100 terabyte automated tape library backing the Unix and Windows machines. Each 100 terabyte module was a "silo" about the size of my office, with a robotic arm that could access any tape in about 2 seconds. The tapes looked like 8mm videocassettes. The 4 silos making up the 400 terabyte unit were interconnected, so that each could pass tapes to the other, allowing any tape to be mounted on any drive in the system.

In operation, this system serves basically two purposes. First, some fraction of the Windows NT machines were used as E-mail servers, both gateway machines and machines that could respond by E-mail to specially formatted E-mail from remote application software to deliver information services to off-web users.

Second, some fraction o the machines served as dial-up hosts for remote sites that dial directly into the corporate system. Apparently, these were mostly at dealerships that don't yet use the internet.

Third, most of the Windows NT and Unix power is providing web service, to both public corporate web sites, private corporate sites accessed over the public internet, and private corporate sites accessed over the corporate intranet. Most new applications seem to be web-based, with various server-side applications (both Unix and Windows NT based) providing the required access to corporate resources. Many of these services, in turn, submit jobs on the mainframes to do the actual operatons on corporate databases. The mainframes are, in effect, back-end database machines, although they also run a number of purely mainframe-based applications.

So, the fun puzzle is, how do you design operating systems to run in this kind of environment! This is very different from the world of the purely personal computer with a single CPU and one end user, yet the operating systems they use, Windows NT, a version of Unix, and a version of IBM's old VM system, are all systems that have also been used in single user settings (curiously, VM was even supported by one short-lived model in the PC family, the PC-XT/370).

WARNING

This is a hard class. The prerequisites for studying this subject include the following:

• A course on elementary computer architecture.
  You are expected to have a fair understanding of computer architecture at two levels. First, you should be familiar with processor-memory-switch level descriptions of computers, including questions of basic bus structure, direct memory access I/O, and memory management units. Second, you should be familiar with the instruction-set-processor level, specifically with issues of memory addressing, register utilization, and typical input/output instructions.

• A course on elementary operating systems.
  You are expected to have a fair understanding of process, memory and input-output management from the user's perspective. File system organization and security, input-output buffering, process creation and synchronization issues should be familiar.

• An introductory course on analysis of algorithms.
  Graph algorithms play a major part in this course. Specifically, algorithms for finding connected components of a graph, algorithms for depth first and breadth first traversal of a graph, and algorithms for finding shortest paths in a graph. Knowledge of data structures and recursive programming sufficient to understand such algorithms is essential.

• Significant programming skills.
  We will illustrate a number of concepts from operating systems with exercises involving the Unix system and with exercises involving a low-level portable thread package written in C and usable in C, C++ and possibly in other languages. While we do not specify the particular programming language you should use, use of the required thread package is likely to be very difficult in languages other than C or C++; it is possible that the routines in the package may operate correctly if called using external linkage from languages such as Pascal or Ada, but there are no guarantees of this!