mapleson@gamers.org Tel: (+44) (0)1772 893297 Fax: (+44) (0)1772 892913 WWW: http://www.futuretech.vuurwerk.nl/
The UNIX operating system (OS) is widely used around the world, eg.
In reality, a great deal of the everyday computing world relies on UNIX-based systems running on computers from a wide variety of vendors such as Compaq (Digital Equipment Corporation, or DEC), Hewlett Packard (HP), International Business Machines (IBM), Intel, SGI (was Silicon Graphics Inc., now just 'SGI'), Siemens Nixdorf, Sun Microsystems (Sun), etc.
In recent years, many companies which previously relied on DOS or Windows have begun to realise that UNIX is increasingly important to their business, mainly because of what UNIX has to offer and why, eg. portability, security, reliability, etc. As demands for handling data grow, and companies embrace new methods of manipulating data (eg. data mining and visualisation), the need for systems that can handle these problems forces companies to look at solutions that are beyond the Wintel platform in performance, scalability and power.
Oil companies such as Texaco [1] and Chevron [2] are typical organisations which already use UNIX systems extensively because of their data-intensive tasks and a need for extreme reliability and scalability. As costs have come down, along with changes in the types of available UNIX system (newer low-end designs, eg. Ultra5, O2, etc.), small and medium-sized companies are looking towards UNIX solutions to solve their problems. Even individuals now find that older 2nd-hand UNIX systems have significant advantages over modern Wintel solutions, and many companies/organisations have adopted this approach too [3].
This course serves as an introduction to UNIX, its history, features,
operation, use and services, applications, typical administration
tasks, and relevant related topics such as the Internet, security and
the Law. SGI's version of UNIX, called IRIX, is used as an example
UNIX OS. The network of SGI Indys and an SGI Challenge S server I
admin is used as an example UNIX hardware platform.
The course lasts three days, each day consisting of a one hour lecture followed by a two hour practical session in the morning, and then a three hour practical session in the afternoon; the only exceptions to this are Day 1 which begins with a two hour lecture, and Day 3 which has a 1 hour afternoon lecture.
Detailed notes are provided for all areas covered in the lectures and the practical sessions. With new topics introduced step-by-step, the practical sessions enable first-hand familiarity with the topics covered in the lectures.
As one might expect of an OS which has a vast range of features, capabilities and uses, it is not possible to cover everything about UNIX in three days, especially the more advanced topics such as kernel tuning which most administrators rarely have to deal with. Today, modern UNIX hardware and software designs allow even very large systems with, for example, 64 processors to be fully setup at the OS level in little more than an hour [4]. Hence, the course is based on the author's experience of what a typical UNIX user and administrator (admin) has to deal with, rather than attempting to present a highly compressed 'Grand Description of Everything' which simply isn't necessary to enable an admin to perform real-world system administration on a daily basis.
For example, the precise nature and function of the Sendmail email system on any flavour of UNIX is not immediately easy to understand; looking at the various files and how Sendmail works can be confusing. However, in the author's experience, due to the way UNIX is designed, even a default OS installation without any further modification is sufficient to provide users with a fully functional email service [5], a fact which shouldn't be of any great surprise since email is a built-in aspect of any UNIX OS. Thus, the presence of email as a fundamental feature of UNIX is explained, but configuring and customising Sendmail is not.
Key: BTL = Bell Telephone Laboratories GE = General Electric WE = Western Electric MIT = Massachusetts Institute of Technology BSD = Berkeley Standard DomainSummary History:
1957: BTL creates the BESYS OS for internal use. 1964: BTL needs a new OS, develops Multics with GE and MIT. 1969: UNICS project started at BTL and MIT; OS written using the B language. 1970: UNICS project well under way; anonymously renamed to UNIX. 1971: UNIX book published. 60 commands listed. 1972: C language completed (a rewritten form of B). Pipe concept invented. 1973: UNIX used on 16 sites. Kernel rewritten in C. UNIX spreads rapidly. 1974: Work spreads to Berkeley. BSD UNIX is born. 1975: UNIX licensed to universities for free. 1978: Two UNIX styles, though similar and related: System V and BSD. 1980s: Many companies launch their versions of UNIX, including Microsoft. A push towards cross-platform standards: POSIX/X11/Motif Independent organisations with cross-vendor membership control future development and standards. IEEE included. 1990s: 64bit versions of UNIX released. Massively scalable systems. Internet springs to life, based on UNIX technologies. Further standardisation efforts (OpenGL, UNIX95, UNIX98).
Detailed History.
UNIX is now nearly 40 years old. It began life in 1969 as a combined project run by BTL, GE and MIT, initially created and managed by Ken Thompson and Dennis Ritchie [6]. The goal was to develop an operating system for a large computer which could support hundreds of simultaneous users. The very early phase actually started at BTL in 1957 when work began on what was to become BESYS, an OS developed by BTL for their internal needs.
In 1964, BTL started on the third generation of their computing resources. They needed a new operating system and so initiated the MULTICS (MULTIplexed operating and Computing System) project in late 1964, a combined research programme between BTL, GE and MIT. Due to differing design goals between the three groups, Bell pulled out of the project in 1969, leaving personnel in Bell's Computing Science and Research Center with no usable computing environment.
As a response to this move, Ken Thompson and Dennis Ritchie offered to design a new OS for BTL, using a PDP-7 computer which was available at the time. Early work was done in a language designed for writing compilers and systems programming, called BCPL (Basic Combined Programming Language). BCPL was quickly simplified and revised to produce a better language called B.
By the end of 1969 an early version of the OS was completed; a pun at previous work on Multics, it was named UNICS (UNIplexed operating and Computing System) - an "emasculated Multics". UNICS included a primitive kernel, an editor, assembler, a simple shell command interpreter and basic command utilities such as rm, cat and cp. In 1970, extra funding arose from BTL's internal use of UNICS for patent processing; as a result, the researchers obtained a DEC PDP-11/20 for further work (24K RAM). At that time, the OS used 12K, with the remaining 12K used for user programs and a RAM disk (file size limit was 64K, disk size limit was 512K). BTL's Patent Department then took over the project, providing funding for a newer machine, namely a PDP-11/45. By this time, UNICS had been abbreviated to UNIX - nobody knows whose idea it was to change the name (probably just phonetic convenience).
In 1971, a book on UNIX by Thompson and Ritchie described over 60 commands, including:
In 1972, Ritchie and Thompson rewrote B to create a new language called C. Around this time, Thompson invented the 'pipe' - a standard mechanism for allowing the output of one program or process to be used as the input for another. This became the foundation of the future UNIX OS development philosophy: write programs which do one thing and do it well; write programs which can work together and cooperate using pipes; write programs which support text streams because text is a 'universal interface' [6].
By 1973, UNIX had spread to sixteen sites, all within AT&T and WE. First made public at a conference in October that year, within six months the number of sites using UNIX had tripled. Following a publication of a version of UNIX in 'Communications of the ACM' in July 1974, requests for the OS began to rapidly escalate. Crucially at this time, the fundamentals of C were complete and much of UNIX's 11000 lines of code were rewritten in C - this was a major breakthrough in operating systems design: it meant that the OS could be used on virtually any computer platform since C was hardware independent.
In late 1974, Thompson went to University of California at Berkeley to teach for a year. Working with Bill Joy and Chuck Haley, the three developed the 'Berkeley' version of UNIX (named BSD, for Berkeley Software Distribution), the source code of which was widely distributed to students on campus and beyond, ie. students at Berkeley and elsewhere also worked on improving the OS. BTL incorporated useful improvements as they arose, including some work from a user in the UK. By this time, the use and distribution of UNIX was out of BTL's control, largely because of the work at Berkeley on BSD.
Developments to BSD UNIX added the vi editor, C-based shell interpreter, the Sendmail email system, virtual memory, and support for TCP/IP networking technologies (Transmission Control Protocol/Internet Protocol). Again, a service as important as email was now a fundamental part of the OS, eg. the OS uses email as a means of notifying the system administrator of system status, problems, reports, etc. Any installation of UNIX for any platform automatically includes email; by complete contrast, email is not a part of Windows3.1, Win95, Win98 or WinNT - email for these OSs must be added separately (eg. Pegasus Mail), sometimes causing problems which would not otherwise be present.
In 1975, a further revision of UNIX known as the Fifth Edition was released and licensed to universities for free. After the release of the Seventh Edition in 1978, the divergence of UNIX development along two separate but related paths became clear: System V (BTL) and BSD (Berkeley). BTL and Sun combined to create System V Release 4 (SVR4) which brought together System V with large parts of BSD. For a while, SVR4 was the more rigidly controlled, commercial and properly supported (compared to BSD on its own), though important work occurred in both versions and both continued to be alike in many ways. Fearing Sun's possible domination, many other vendors formed the Open Software Foundation (OSF) to further work on BSD and other variants. Note that in 1979, a typical UNIX kernel was still only 40K.
Because of a legal decree which prevented AT&T from selling the work of BTL, AT&T allowed UNIX to be widely distributed via licensing schemas at minimal or zero cost. The first genuine UNIX vendor, Interactive Systems Corporation, started selling UNIX systems for automating office work. Meanwhile, the work at AT&T (various internal design groups) was combined, then taken over by WE, which became UNIX System Laboratories (now owned by Novell). Later releases included Sytem III and various releases of System V. Today, most popular brands of UNIX are based either on SVR4, BSD, or a combination of both (usually SVR4 with standard enhancements from BSD, which for example describes SGI's IRIX version perfectly). As an aside, there never was a System I since WE feared companies would assume a 'system 1' would be bug-ridden and so would wait for a later release (or purchase BSD instead!).
It's worth noting the influence from the superb research effort at Xerox Parc, which was working on networking technologies, electronic mail systems and graphical user interfaces, including the proverbial 'mouse'. The Apple Mac arose directly from the efforts of Xerox Parc which, incredibly and much against the wishes of many Xerox Parc employees, gave free demonstrations to people such as Steve Jobs (founder of Apple) and sold their ideas for next to nothing ($50000). This was perhaps the biggest financial give-away in history [7].
One reason why so many different names for UNIX emerged over the years was the practice of AT&T to license the UNIX software, but not the UNIX name itself. The various flavours of UNIX may have different names (SunOS, Solaris, Ultrix, AIX, Xenix, UnixWare, IRIX, Digital UNIX, HP-UX, OpenBSD, FreeBSD, Linux, etc.) but in general the differences between them are minimal. Someone who learns a particular vendor's version of UNIX (eg. Sun's Solaris) will easily be able to adapt to a different version from another vendor (eg. DEC's Digital UNIX). Most differences merely concern the names and/or locations of particular files, as opposed to any core underlying aspect of the OS.
Further enhancements to UNIX included compilation management systems such as make and Imake (allowing for a single source code release to be compiled on any UNIX platform) and support for source code management (SCCS). Services such as telnet for remote communication were also completed, along with ftp for file transfer, and other useful functions.
In the early 1980s, Microsoft developed and released its version of UNIX called Xenix (it's a shame this wasn't pushed into the business market instead of DOS). The first 32bit version of UNIX was released at this time. SCO developed UnixWare which is often used today by Intel for publishing performance ratings for its x86-based processors [8]. SGI started IRIX in the early 1980s, combining SVR4 with an advanced GUI. Sun's SunOS sprang to life in 1984, which became widely used in educational institutions. NeXT-Step arrived in 1989 and was hailed as a superb development platform; this was the platform used to develop the game 'Doom', which was then ported to DOS for final release. 'Doom' became one of the most successful and influential PC games of all time and was largely responsible for the rapid demand for better hardware graphics systems amongst home users in the early 1990s - not many people know that it was originally designed on a UNIX system though. Similarly, much of the development work for Quake was done using a 4-processor Digital Alpha system [9].
During the 1980s, developments in standardised graphical user interface elements were introduced (X11 and Motif) along with other major additional features, especially Sun's Networked File System (NFS) which allows multiple file systems, from multiple UNIX machines from different vendors, to be transparently shared and treated as a single file structure. Users see a single coherant file system even though the reality may involve many different systems in different physical locations.
By this stage, UNIX's key features had firmly established its place in the computing world, eg. multi-tasking and multi-user (many independent processes can run at once; many users can use a single system at the same time; a single user can use many systems at the same time). However, in general, the user interface to most UNIX variants was poor: mainly text based. Most vendors began serious GUI development in the early 1980s, especially SGI which has traditionally focused on visual-related markets [10].
From the point of view of a mature operating system, and certainly in the interests of companies and users, there were significant moves in the 1980s and early 1990s to introduce standards which would greatly simplify the cross-platform use of UNIX. These changes, which continue today, include:
There are also commercial versions of Linux which have additional features and services, eg. Red Hat Linux and Calderra Linux. Note that many vendors today are working to enable the various UNIX variants to be used with Intel's CPUs - this is needed by Intel in order to decrease its dependence on the various Microsoft OS products.
In a move to unify the marketplace, SGI and Microsoft signed a deal in the late 1990s to merge DirectX and Direct3D into OpenGL - the project, called Fahrenheit, will eventually lead to a single unified graphics programming interface for all platforms from all vendors, from the lowest PC to the fastest SGI/Cray supercomputer available with thousands of processors. To a large degree, Direct3D will simply either be phased out in favour of OpenGL's methods, or focused entirely on consumer-level applications, though OpenGL will dominate in the final product for the entertainment market.
OpenGL is managed by the OpenGL Architecture Review Board, an independent organisation with member representatives from all major UNIX vendors, relevant companies and institutions.
Many important features of UNIX do not exist in other OSs such as WindowsNT and will not do so for years to come, if ever. These include guaranteeable reliability, security, stability, extreme scalability (thousands of processors), proper support for advanced multi-processing with unified shared memory and resources (ie. parallel compute systems with more than 1 CPU), support for genuine real-time response, portability and an ever-increasing ease-of-use through highly advanced GUIs. Modern UNIX GUIs combine the familiar use of icons with the immense power and flexibility of the UNIX shell command line which, for example, supports full remote administration (a significant criticism of WinNT is the lack of any real command line interface for remote administration). By contrast, Windows2000 includes a colossal amount of new code which will introduce a plethora of new bugs and problems.
A summary of key UNIX features would be:
In recent years, one aspect of UNIX that was holding it back from spreading more widely was cost. Many vendors often charged too high a price for their particular flavour of UNIX. This made its use by small businesses and home users prohibitive. The ever decreasing cost of PCs, combined with the sheer marketing power of Microsoft, gave rise of the rapid growth of Windows and now WindowsNT. However, in 1993, Linus Torvalds developed a version of UNIX called Linux (he pronounces it rather like 'leenoox', rhyming with 'see-books') which was free and ran on PCs as well as other hardware platforms such as DEC machines. In what must be one of the most astonishing developments of the computer age, Linux has rapidly grown to become a highly popular OS for home and small business use and is now being supported by many major companies too, including Oracle, IBM, SGI, HP, Dell and others.
Linux does not have the sophistication of the more traditional UNIX variants such as SGI's IRIX, but Linux is free (older releases of IRIX such as IRIX 6.2 are also free, but not the very latest release, namely IRIX 6.5). This has resulted in the rapid adoption of Linux by many people and businesses, especially for servers, application development, home use, etc. With the recent announcement of support for multi-processing in Linux for up to 8 CPUs, Linux is becoming an important player in the UNIX world and a likely candidate to take on Microsoft in the battle for OS dominance.
However, it'll be a while before Linux will be used for 'serious' applications since it does not have the rigorous development history and discipline of other UNIX versions, eg. Blue Mountain is an IRIX system consisting of 6144 CPUs, 1500GB RAM, 76000GB disk space, and capable of 3000 billion floating-point operations per second. This level of system development is what drives many aspects of today's UNIX evolution and the hardware which supports UNIX OSs. Linux lacks this top-down approach and needs a lot of work in areas such as security and support for graphics, but Linux is nevertheless becoming very useful in fields such as render-farm construction for movie studios, eg. a network of cheap PentiumIII machines, networked and running the free Linux OS, reliable and stable. The film "Titanic" was the first major film which used a Linux-based render-farm, though it employed many other UNIX systems too (eg. SGIs, Alphas), as well as some NT systems.
EDIT (2008): Linux is now very much used for serious work, running most of the planet's Internet servers, and widely used in movie studios for Flame/Smoke on professional x86 systems. It's come a long way since 1999, with new distributions such as Ubuntu and Gentoo proving very popular. At the high-end, SGI offers products that range from its shared-memory Linux-based Altix 4700 system with up to 1024 CPUs, to the Altix ICE, a highly expandable XEON/Linux cluster system with some sites using machines with tens of thousands of cores.
UNIX has come a long way since 1969. Thompson and Ritchie could never have imagined that it would spread so widely and eventually lead to its use in such things as the control of the Mars Pathfinder probe which last year landed on Mars, including the operation of the Internet web server which allowed millions of people around the world to see the images brought back as the Martian event unfolded [13].
Today, from an administrator's perspective, UNIX is a stable and reliable OS which pretty much runs itself once it's properly setup. UNIX requires far less daily administration than other OSs such as NT - a factor not often taken into account when companies form purchasing decisions (salaries are a major part of a company's expenditure). UNIX certainly has its baggage in terms of file structure and the way some aspects of the OS actually work, but after so many years most if not all of the key problems have been solved, giving rise to an OS which offers far superior reliability, stability, security, etc. In that sense, UNIX has very well-known baggage which is absolutely vital to safety-critical applications such as military, medical, government and industrial use. Byte magazine once said that NT was only now tackling OS issues which other OSs had solved years before [14].
Thanks to a standards-based and top-down approach, UNIX is evolving to remove its baggage in a reliable way, eg. the introduction of the NSD (Name Service Daemon) to replace DNS (Domain Name Service), NIS (Network Information Service) and aspects of NFS operation; the new service is faster, more efficient, and easier on system resources such as memory and network usage.
However, in the never-ending public relations battle for computer systems and OS dominance, NT has firmly established itself as an OS which will be increasingly used by many companies due to the widespread use of the traditional PC and the very low cost of Intel's mass-produced CPUs. Rival vendors continue to offer much faster systems than PCs, whether or not UNIX is used, so I expect to see interesting times ahead in the realm of OS development. Companies like SGI bridge the gap by releasing advanced hardware systems which support NT (eg. the Visual Workstation 320 [15]), systems whose design is born out of UNIX-based experience.
One thing is certain: some flavour of UNIX will always be at the forefront of future OS development, whatever variant it may be.
http://www.sgi.com/origin/successes/buffalo.html
http://virtual.park.uga.edu/hc/unixhistory.html
http://pantheon.yale.edu/help/unixos/unix-intro.html
http://www.sis.port.ac.uk/~briggsjs/csar4/U2.htm
http://osiris.staff.udg.mx/man/ingles/his.html
http://www.pasc.org/abstracts/posix.htm
http://www.specbench.org/
http://doomgate.gamers.org/dhs/dhs/usavisit/dallas.html
http://www.futuretech.vuurwerk.nl/pcw9-90pi4d25.html
IRIS Indigo Brochure, SGI, 1991 [HLW-BRO-01 (6/91)].
"Smooth Operator", CGI Magazine, Vol4, Issue 1, Jan/Feb 1999, pp. 41-42.
Digital Media World '98 (Film Effects and Animation Festival, Wembley Conference Center, London). Forty six pieces of work were submitted to the conference magazine by company attendees. Out of the 46 items, 43 had used SGIs; of these, 34 had used only SGIs.
http://www.lanl.gov/asci/ http://www.lanl.gov/asci/bluemtn/ASCI_fly.pdf http://www.lanl.gov/asci/bluemtn/bluemtn.html http://www.lanl.gov/asci/bluemtn/t_sysnews.shtml http://www.lanl.gov/orgs/pa/News/111298.html#anchor263034
http://visual.sgi.com/
(Have you found this page via a web search? Examine the index pages to see all the available items!) [Future Technology Research Index] [SGI Tech/Advice Index] [Nintendo64 Tech Info Index]