Background

Today there are a wide choice of OSs available which provide functionality of a variety of degrees: for simple household appliances to mainframes in data centers and everything in-between. Selecting an OS is usually determined by a number of factors, such as: whether the OS supports particular hardware architecture(s); the number of services and features are made available to the user by the OS itself; and the degree of ease it is to develop new software applications for the OS. In this section, two concepts are covered. The first, architecture support, decides the approach to developing the core of the OS. The second is the services made available by the OS, which in turn offers the ability to provide new software applications on the OS.

System Architecture

In order to develop the fundamental components of an OS, the architecture of the CPU must be considered. Many of the tasks the OS performs are based off the functionality of the CPU itself. As a result, considering what instructions, services, and processing strategies the CPU architecture makes available is the key to developing an OS. With the number of available OSs made available – proprietary, open-source, and hobbyist – infrastructure for using an OS when a general-purpose personal computer starts has been widely debated. The approach taken in here is to initially study the booting process to prepare for implementation approaches.

x86

As described by Intel, x86 is a family of CPU processors which provide “extensive support for … system-level architecture [consisting] of a set of registers, data structures, and instructions designed to support basic system-level operations such as memory management, interrupt and exception handling, task management, and control of multiple processors.”¹ The Intel Architectures Software Developer’s Manual provides an extensive and detailed explanation of the functionality of this group of processors. Its underlying motive is to provide a means for an OS based on this architecture to be compatible with future processors as the underlying registers, data structures and instructions will remain the same.

Selecting x86 as choice of architecture for an simple OS, therefore, will make it compatible with many existing systems that exist today. General-purpose personal computers typically offer CPU processors based on this architecture and so providing for it makes it largely compatible with many computers today. Embedded systems, however, vary dramatically as discussed by Silberschatz, G. S. et al. as they are generally built with a discrete set of functionality, using “application-specific integrated circuits that perform their tasks without an operating system.”²

Finally, x86 offers hierarchical protection domains, enabling the execution of programs on the CPU a degree of security³. This provided in the form of code segmentation with attributes that dictate read and write permissioning (see Global Descriptor Table). This gives programs a privilege level defined as either “user space” and “kernel space”. Programs executed at either one of these levels determines the access rights it has to memory.

Bootloading

Historically, general-purpose personal computers have provided a Basic Input/Output System (BIOS)⁴ which is provided by the manufacturer of the computer for controlling the boot sequence and other hardware specific initialisations. The BIOS firmware is typically written to its own physical microchip located on the motherboard of the computer. When the computer boots, its the BIOS that loads the first instructions of the OS into the CPU from a secondary storage medium such as a hard drive or USB stick.

A bootloader “is the first software program that runs when a computer starts”⁵ and is defined as the first 512KB written to the storage medium,⁶ loaded by the BIOS, responsible initialising the CPU to accept more instructions. It is typically defined with a number of flags, including a boot signature and an byte sequence at the end of the sector. A bootloader is therefore highly dependent on knowing the CPU’s architecture. This includes, for instance, knowing the addresses of various memory locations, known as registers, the maximum length of bits the CPU can accept as input, and Global Descriptor Tables.

Providing the different instruction sets for different CPU architectures has been greatly simplified with tools such as the GNU Assembler (simply, AS)⁷. Without it, a version of the bootloader and any other architecture-specific instructions would have to be written for all the different CPU architectures the OS wishes to be compatible for. Instead, AS can be used to compile GNU Assembly instructions into machine code by specifying the architecture during compilation. Whilst this helps increase the compatibility of OS for different CPU architectures, it does not solve it completely. Taggart, M. outlines the on-going problems and possible solutions to ‘multiarch’ OS support. Whilst generally providing a 32-bit instruction set may be the most compatible method of creating a cross-platform OS this may not be the most efficient. With advent 64-bit CPUs there are differences in orders of magnitude of the processing speed of the CPU and so providing only 32-bit support may not be highly desired. In the case of this implementation, providing this distinguishment is not a requirement.

The process of packaging the entirety of the OS into a binary image ready to be written to a storage medium requires linking this first 512KB bootloader with the rest of the compiled OS. This is where systems such as the GNU GRand Unified Bootloader (GRUB) provide additional support. The GRUB manual, as detailed by Matzigkeit, Y. K., et al., outlines the specifications of a ‘multiboot’ image format. This format describes the flags, headers, and offsets of additional code and data in the image. The benefits of GRUB include standardising the image format, helping increase the distributability of file, and offering extensive documentation, tips, and guides for development of an OS bootloader. GRUB can also be bundled with the OS during image compilation in order to provide these features. Finally, the GRUB multiboot header is supported by a number of emulation tools such as QEMU (see Development Environment Setup) for testing purposes.

Kernel

When considering the overall approach to designing an OS, and at its very core, a smaller system exists for continuing the process of executing more program instructions after the bootloader. Known as the Kernel, it is responsible for sending instructions directly to the CPU, managing the memory of the kernel itself and for the user, and managing hardware peripherals such as a keyboard and screen. In the case of defining a minimum set of functionality needed to provide abstractions to the user, this type of kernel is known as a microkernel.⁸

A number of existing kernels have already been developed, such as the Linux Kernel, GNU/Hurd, and the Windows NT Kernel.⁹ Many of the concepts behind these kernels, however, have been standardised by the IEEE Computer Society known as the Portable Operating System Interface (POSIX).¹⁰ This standard provides compatibility of software between OSs, specifically for the purpose of incorporating the ANSI C Programming Language (commonly referred to as simply C). A kernel conforming to the POSIX standard, therefore, will provide a number of ‘Standard C Libraries’¹¹ containing predefined methods or functions. For example, the library entitled <string.h> must contain a function called strlen()¹². How this method is implemented, however, is defined by the kernel itself. In order to provide the abstractions needed for running user-defined programs and higher-level languages, these libraries must be provided.

Memory Management

Providing memory to a program for the purpose of storing temporary data as it performs calculations has a number of caveats. The foremost problem is that a program run in ‘user space’ should never be able to read or write from that of ‘kernel space’ memory. This is to prevent the user from disrupting the operations of the OS itself. The second main responsibility of memory management is to provide the allocation of memory in an efficient and robust manner, so as not to waste the memory made available by the hardware itself, known as external fragmentation, and to prevent unnecessary redistribution of memory based on the needs of additional processes.

The strategies involved in managing both the memory and the processor of a CPU is widely discussed¹³¹⁴. The general techniques involved here will be to implement the use of paging. This is a memory management technique which offers the ability to allocate memory from the physical address space, offered by the RAM, as virtual addresses in a non-contiguous manner.¹⁵ In addition, the x86 architecture offers native support for paging, specified by bit 31 of CR0.¹⁶

Virtual File System (VFS)

A file system is a hierarchical mechanism which offers an extensible and dynamic method of mapping the location and size of a piece of data, or file, on a physical storage medium with a retrievable index. The size of the file in the context of the user is variable and as a result, the file system implementation must deal with non-contiguous chunks which represent the file as a whole as providing a space on the storage medium to fit the file in its exactitude is inefficient. Many mechanisms and approaches have been developed over time to deal with this problem succinctly, including EXT2, NTFS, and FAT.¹⁷ UNIX and UNIX-like OSs such as Linux and macOS have preferred extensible journaling systems such as EXT2 for its simplicity, ability to map data to the physical address on the storage media, and resilience to system failures with a log of file update transactions.¹⁸.

The main mechanism featured behind these file systems are their approach in mapping the index to the correct file in its entirety and providing a mechanism for reducing and increasing the file size. These approaches are abstracted by the OS in what is known as the Virtual File System.¹⁹ As an OS encounters new storage mediums, it should provide these mechanisms for the provisioning of the storage’s contents.

In addition, the file systems mentioned include a varying degree of metadata such as permissioning features which offer the OS the ability prevent its users from either reading or writing to the file.

Supporting a Higher-Level Language

Higher-level languages provide a greater degree of abstractions for software developers and oftentimes help simplify the task of writing programs. This can decrease the development time of writing new software applications and, depending on the popularity of the language itself, increase the amount of software made available to the OS. In this case, providing resources for higher-level languages in an OS is a requirement.

There are a wide variety of higher-level languages that exist for different purposes. JavaScript is a high-level interpreted language and was selected based on the following factors:

• It is a widely accessible language, whose syntax is simple, concise, and easily intelligible;
• It is a widely used language, popularised by its incorporation by web-browsers; and
• It is a highly maintained language. The API for the language is continuously developed and specified by the European Computer Manufacturers Association (ECMA). The implementation of this API, known as ECMAScript, is continuously implemented by software companies providing support for JavaScript or JavaScript-like languages. Their implementations and feedback perpetuates further development of the ECMAScript API.

JavaScript Engines

Much like providing compatibility for different hardware architectures, JavaScript is made available in a number of different fashions depending on its desired functionality. This has resulted in the development of a number of JavaScript ‘engines’, such as Google’s V8²⁰, NodeJS²¹, and Duktape²².

DOM-focused Engines

Google’s V8, Mozilla’s and Apple’s WebKit²³ are examples of JavaScript engines that provide support for the rendering a Document Object Model (DOM) such as HyperText Markup Language (HTML). This is considered the classic JavaScript use case, where websites provide web applications seamlessly to the browser by linking to relevant sources: the markup and stylesheets which describe the User Interface (UI); and, the JavaScript files which describe the functionality of the application.

Eccentric Engines

The idea of using JavaScript in a server-side model has recently become popularised through engines such as NodeJS. This greatly increases the level of functionality as it reduces JavaScript to simply processing inputs and providing outputs. Whilst this concept may seem counter intuitive, providing JavaScript in this manner gives the developer the ability to create a larger set of functions non-typical to the DOM. In this manner, JavaScript, for instance, can be used to process requests, server-side, in addition rendering them, client-side. In addition, tools such as the Node Package Manager (NPM) assist in the means of distributing JavaScript applications that can be run analogous to classical software distribution tools such as Advanced Packaging Tool (APT) and independently of each other.

Embedded Engines

Duktape is an open-source and “embeddable JavaScript engine, with a focus on portability and compact footprint.”²⁴ Like NodeJS, it offers a means for running JavaScript within the server-side model, however, unlike NodeJS it attempts to reduce its standalone footprint in order to provide JavaScript in the form of a single, distributable library implemented in C.

This implementation is a more practical approach in providing a method for interpreting the language. As part of its configuration, ECMAScript features can be toggled; a particular architecture and compiler can be specified; and, a number of Standard C Libraries can be swapped. It offers extensive control over its memory, with the ability to define the execution of programs by providing sets buffers and stacks to program execution. Finally, it offers the ability to map native C bindings to JavaScript, making it the only engine capable of performing kernel operations.