Indigo2 and POWER Indigo2 Technical Report

IRIX 6.0.1 is an upwardly compatible revision of IRIX 5 that incorporates substantial functionality from UNIX System V, Release 4.1. IRIX 6.0.1 complies with the Applications Binary Interface described in the System V Interface Definition, Issue 3 (SVID 3), the defining document for System V Release 4 (SVR4). This compliance places IRIX 6.0.1 in the mainstream of UNIX products, and in a leadership position in supplying an SVR4 UNIX with 64 bit features.

Outside of extensions to support 64 hits, the features or IRIX 6.0.1 are inherited directly from IRIX 5.3, so the kernel technology, file system, processor scheduler enhancements, memory management policies, and I/O enhancements have the same behavior as they do in IRIX 5.3.

This chapter explains:

• kernel architecture

• fine-grained kernel locking

• memory management

• process management (scheduling)

• synchronization primitives

• lightweight file services

• file services

• I/O performance improvement options

• 64-bit API for 32-bit programs

• NFS

• MIPS Applications Binary Interface (ABI)

FIGURE 21 IRIX 6.0.1 Kernel Architecture.

Because the IRIX 6.0.1 kernel itself is a 64-bit entity, all pointers and longs are treated as 64-bit quantities. Table 7 summarizes other structures that were extended to provide for 64-bit support.

/sys/                 Other

<sys/ipc.h>           <dirent.h>
<sys/msg.h>           <fcntl.h>
<sys/procset.h>       <ftw.h>
<sys/resource.h>      <grp.h>
<sys/sem.h>           <locale.h>
<sys/shm.h>           <math.h>
<sys/siginfo.h>       <netconfig.h>
<sys/stat.h>          <netdir.h>
<sys/statvfs.h>       <nl_types.h>
<sys/time.h>          <poll.h>
<sys/times.h>         <pwd.h>
<sys/tiuser.h>        <rpc.h>
<sys/types.h>         <search.h>
<sys/uio.h>           <setjmp.h>
<sys/utime.h>         <sigaction.h>
<sys/utsname.h>       <signal.h>
                      <stddef.h>
                      <stdio.h>
                      <stdlib.h>
                      <stropts.h>
                      <termios.h>
                      <ucontext.h>

The effects of these adjustments are:

• User processes can grow to a maximum supported size of 1 TB (2^40 bits).

• Up to 255 swap devices are supported, with a maximum of 2^24 pages per device. (Pages are 16 K on IRIX 6.0.1)

  
  

It is possible to add and subtract swap partitions on a running system. One can also swap to NFS partitions, and to mirrored partitions, and to regular files. Another feature of IRIX is that one can overcommit swap allocation, although this practice is not recommended and requires good knowledge of the application's working set.

Numerous tunable variables that affect memory management policies and performance are documented in the systune() man page.

To support multiple processes, IRIX implements a process-scheduling algorithm that assures a fairly equitable division of processor time between all processes. This algorithm is said to be nonpreemptive, that is, the running process cannot be preempted by another process (although it can be preempted by the kernel).

The running process can yield to another process "voluntarily," by making a system call (such as an I/O request) that causes it to sleep, in which case another process is selected to run. Also, the running process can be preempted by the kernel to handle an exception, in which case execution returns to the process after the exception handler has finished its business.

The kernel also enforces a limit on the amount of time a process can monopolize the processor. When this specified time has elapsed, an exception is generated, and the exception handler selects a new process to run and executes a context switch.

While the process management model discussed represents an apparently equitable resource sharing model, the effect of scheduling processes in a simplistic model does not maximize the computing capability of a multiprocessor system. These effects increase with the number of processors, and are central to the issue of how well a multiprocessor system scales in performance when additional processors are added.

These and other effects are accounted for in sophisticated scheduling algorithms and include:

• A periodically occurring deadline, by which time the process must receive a specified amount of CPU time. For example, the system designer can specify that this process is to receive 10,000 nanoseconds of CPU time every 200,000 nanoseconds of elapsed time. Additional facilities permit the process to block waiting the CPU time allocation or to yield the CPU if the required work is completed before the allocation has been used. This mechanism is much more efficient and easier to use than schemes based on conventional mechanisms, such as timers and signals.

• Scheduling by processor cache affinity management, where the potential affinity of processes for processors clue to the likelihood that the context for the process may reside in a specific processors cache is taken into account.

• Gang scheduling, where the user has the ability to schedule related processes as a group.

• Processor sets, where one can designate a set of processors to run specific tasks. This system permits flexible scheduling of processes at a global level. The UNIXrunon command and others can be used for individual processor control.

A shared process group can be constructed from sproc calls from a common ancestor. In addition to virtual address space, members of a share group can share other attributes such as file tables, current working directories, effective userids, and so on.

Traditional POSIX threads are available in the Silicon Graphics ADA product, and are planned as a feature for general release on a future version of IRIX.

IRIX 6.0.1 uses the Virtual File System interface known as the vnode interface. The name vnode is derived from the name of the data structure used in the interface. This interface was developed to facilitate the incorporation of different file systems in the system. A de facto standard, this interface has been used by third-party providers of file system technology, such as the enhanced file system found in the CASEVision Tracker product. It facilitates the inclusion of additional file system types into IRIX 6.0.1.

The file system types supported for IRIX include:

• proc, a file system that provides access to the image of each active process in the system

• Both 64-bit and 32-bit /proc clients can operate and issue standard /proc ioctls. Additional ioctls were added for 32-bit debuggers to support 64-bit clients.

• NFS, the mount path on the server of the directory to be served via NFS

• iso9660, a file system type used to mount CD-ROM discs in ISO 9660 (with or without Rock Ridge extensions) and High Sierra formats

• DOS: The file system driver supports 5.25-inch floppy drives in three standard formats when used with the freestanding SCSI floppy drive. The standard single and double density dual sided 3 1/2" drive and the 3 1/2" 20.1MB floptical drive is also supported.

• swap, which can be either a file or block device to be used as a swap resource.

• EFS, a file system on a block special device (for example, /dev/root).

  
  

EFS offers much higher performance than conventional UNIX file systems because it uses extents of up to 64 Kbytes instead of the 8-KByte (or smaller) blocks commonly found in other systems. This arrangement can dramatically reduce the number of I/O operations compared to other file systems. EFS also permits more files in a file system through the use of 32-bit internal handles (inode number). EFS file systems can be up to 8 GBytes (an individual file is limited to 2 GBytes). Conventional UNIX file system structures are constrained by the use of a 16-bit inode number, and thus are limited to 64 K files in a file system and to 2 GBytes for both file size and file system size.

8.7.1 File System Reorganizer

The fsr (file system reorganizer) program improves the organization of mounted file systems. The reorganization algorithm operates on one file at a time, compacting or otherwise improving the layout of the file extents (contiguous blocks of file data) while simultaneously compacting the file system free space. The intended usage is to call for from crontab at a regular time; the default is once per week.

• asynchronous I/O

• memory-mapped I/O

• direct I/O

  
  

Traditionally, UNIX systems have allowed a given process to have only one I/O operation in progress at a time. This situation has forced applications like database servers that required several concurrent I/O operations to adopt complex, multiprocess architectures. IRIX 6.0.1 includes support for multiple concurrent I/O operations within a single process. The interfaces comply with Draft 12 of the POSIX P1003.4 specifications. These features allow simpler, more efficient implementations of subsystems requiring multiple concurrent I/O operations. Since they are standards-compliant, applications that use them are potentially portable between systems using the same interfaces.

IRIX 6.0.1 supports the SVR4 interfaces for memory-mapped I/O, which can make disk files visible in the address space of the program and allow them to be accessed without explicit I/O operations. Behind the scenes, the kernel arranges to bring in the necessary pages similarly to the way in which it brings in the pages of an executable, using the demand paging features of the virtual memory subsystem.

mmap() permits the implementation of systems that are potentially more efficient by avoiding the copying of bytes to (from) user space from (to) I/O buffers in the kernel. IRIX 6.0.1 supports the auxiliary operations for memory-mapped files as defined in SVID 3, such as control of copying modified pages back to the disk. Full support for memory-mapped I/O has required internal changes to the virtual memory subsystem in the kernel to support page level protection.

Like memory-mapped I/O, direct I/O also accomplishes the I/O directly from (to) the disk to (from) the user address space. Direct I/O, however, requires substantially less change to an existing application than memory-mapped I/O. Direct I/O requires that the program implement its own read-ahead and write-behind policies. Using both direct and asynchronous I/O, the program can have complete control of its I/O while retaining the benefits of the file system. Previously, this level of control was only available to programs using the raw disk interfaces; direct I/O provides performance approaching that of the raw disk. The interfaces for direct I/O have been defined by Silicon Graphics.

The need for these extensions is only to allow 32-bit applications to access 64-bit files. They are not needed by 64-bit applications, or by 32-bit applications that have no need to deal with 64-bit files.

A 32-bit program either recognizes the existence of large files or it does not. Those that do use new interfaces to access these files. Those that do not see all files as having a maximum size of 2 GB. Data beyond offset 2GB in a file will be inaccessible to applications that do not recognize the existence of large files. A file size greater than 2GB will not be visible to an application that does not know of such files.

Sun Microsystems has proposed 64-bit extensions to NFS as part of the ONC+ package. These extensions are required because the current offset is explicit in each NFS request and is a 32-bit quantity. Silicon Graphics will implement ONC+ features as soon as they are made available.

• a generic document that is relevant for all processor types.

• a processor-specific document detailing how the generic ABI is implemented on a particular processor family.

Compliance with the SVR4 ABI is tested with a variety of commercial and public-domain test packages. One of the most comprehensive is the Generic ABI Test Suite (gABI) developed by UNISOFT, a comprehensive test of the compliance of the system with the SVR4 ABI as defined by the System V Interface Definition Issue 3.0 (SVID 3.0). The test suite consists of over 7,800 tests of the details of ABI compliance.

No amount of care in writing specifications and in implementing systems based on them can ensure that binary executables will indeed execute on all systems. To address this issue, the community building SVR4 systems for MIPS processors has joined together in an organization called the MIPS ABI Group. The goal of the group has been to iron out the subtle inconsistencies between their systems to ensure that they do indeed support binary compatibility. The effort includes major Independent Software Vendors (ISVs) who are porting their packages to the systems. Silicon Graphics has supported this effort, including the setting up of laboratory space on the Mountain View campus to house a collection of systems from the member companies for testing of interoperability. IRIX 6.0.1 meets the ABI and can run the "shrink-wrapped" versions of the ISV packages. The reaction from the ISV community to this effort has been very positive, which will translate into a broader range of applications becoming available for IRIX, allowing us to better serve expanded markets.

• System V Applications Binary Interface, Revised Edition, UNIX Press (Prentice Hall), 1990, ISBN 0-13-880410-9. Defines the Generic ABI, mostly by reference to SVID 3.

• System V Application Binary Interface, MIPS Processor Supplement, UNIX Press (Prentice Hall) 1990, ISBN 0-13-880170-3. Defines the MIPS processor implementation of the Generic ABI.

• System V Interface Definition, Third Edition, AT&T/Prentice Hall 1989. (Five Volumes). Commonly referred to as SVID3, this set of documents define the SVR4 API.

• MIPS Processor ABI Porting and Conformance Guide, drafts available from Silicon Graphics. Also called "The Black Book", this clarifies the implementation of the ABI and discusses how to produce ABI compliant programs.

• Processor Segmentation: A resource management facility for Shared Memory Multiprocessors, available from Silicon Graphics.

• Parallel throughput performance of IRIX 5.X, available from Silicon Graphics.

• Silicon Graphics Targets SMP Leadership, D.H. Born Associates, Inc.