Fwd: Why Free Software? (Long!)

[Seth Johnson]

This is a multi-part message in MIME format.
--------------6FF298A85FA76866B9A469FF
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit



-------- Original Message --------
Date: Wed, 27 Feb 2002 03:01:35 -0500
From: Seth Johnson <seth.johnson@RealMeasures.dyndns.org>
Organization: Real Measures
To: C-FIT_Community@RealMeasures.dyndns.org,
C-FIT_Release_Community@RealMeasures.dyndns.org,
fairuse-discuss@mrbrklyn.com


(A truly exhaustive treatment of this question, with piles
of quantitative measures and qualitative analyses.  -- Seth)

http://www.dwheeler.com/oss_fs_why.html


Why Open Source Software / Free Software (OSS/FS)? Look at
the Numbers!

by David A. Wheeler
Revised as of February 22, 2002
--------------6FF298A85FA76866B9A469FF
Content-Type: text/plain; charset=iso-8859-1;
 name="Why Free.txt"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline;
 filename="Why Free.txt"


(A truly exhaustive treatment of this question, with piles of quantitativ=
e measures and qualitative analyses.  -- Seth)

http://www.dwheeler.com/oss_fs_why.html


Why Open Source Software / Free Software (OSS/FS)? Look at the Numbers!

by David A. Wheeler
Revised as of February 22, 2002


Open Source Software / Free Software (OSS/FS) has risen to prominence. If=
 you're not sure what these terms mean, you can get an explanation of the=
se terms and related information from my list of Open Source Software / F=
ree Software (OSS/FS) references at http://www.dwheeler.com/oss_fs_refs.h=
tml. Briefly, OSS/FS programs are programs whose licenses permit users th=
e freedom to run the program for any purpose, to modify the program, and =
to redistribute the original or modified program (without payment or rest=
riction on who they can redistribute the program to). Those who use the t=
erm ``open source software'' tend to emphasize technical advantages of su=
ch software (such as better reliability and security), while those who us=
e the term ``Free Software'' tend to emphasize freedom from control by an=
other. The opposite of OSS/FS is ``closed'' or ``proprietary'' software. =
Note that many OSS/FS programs are commercial programs, so don't make the=
 mistake of calling OSS/FS software ``non-commercial.'' Almost no OSS/FS =
programs are in the ``public domain'' (which has a specific legal meaning=
), so avoid that term as well. Some call OSS/FS software ``public service=
 software'', since these licenses are designed to serve the public at lar=
ge.

Some sites provide a few anecdotes on why you should use OSS/FS, but for =
many that's not enough information to justify using OSS/FS. Instead, this=
 paper emphasizes quantitative measures (such as experiments and market s=
tudies) on why using OSS/FS products is, in a number of circumstances, a =
reasonable or even superior approach. I should note that while I find muc=
h to like about OSS/FS, I'm not a rabid advocate; I use both proprietary =
and OSS/FS products myself. Vendors of proprietary products often work ha=
rd to find numbers to support their claims; this page provides a useful a=
ntidote of hard figures to aid in comparing these proprietary products to=
 OSS/FS.

Note that this paper's goal is not to show that all OSS/FS is better than=
 all proprietary software. There are those who believe this is true from =
ethical, moral, or social grounds, but no numbers could prove such a broa=
d statement. Instead, I'll simply compare commonly-used OSS/FS software w=
ith commonly-used proprietary software, to show that at least in certain =
situations and by certain measures, OSS/FS is at least as good or better =
than its proprietary competition.

I'll emphasize the GNU/Linux operating system (which some abbreviate as `=
`Linux'') and the Apache web server, since these are some of the most vis=
ible OSS/FS projects. I'll also primarily compare it to Microsoft Windows=
, since Windows has significant market share and Microsoft is one of prop=
rietary software's strongest proponents. I'll mention Unix systems in pas=
sing as well, though the situation with Unix is more complex; many Unix s=
ystems include a number of OSS/FS components or software primarily derive=
d from OSS/FS components. Thus, comparing proprietary Unix systems to OSS=
/FS systems (when examined as entire systems) is often not as clear-cut. =
Both Unix and GNU/Linux are ``Unix-like'' systems. The most recent Apple =
Macintosh operating system (MacOS OS X) presents the same kind of complic=
ations; older versions of MacOS were entirely proprietary, but Apple's op=
erating system has been redesigned so that it's now based on a Unix syste=
m with a substantial contribution from OSS/FS programs. Indeed, Apple is =
now openly encouraging collaboration with OSS/FS developers. I'll include=
 data over a series of years, not just the past year; I believe that all =
relevant data should be considered when making a decision, instead of arb=
itrarily ignoring older data, and the older data shows that OSS/FS has a =
history of many positive traits.

Below is data discussing market share, reliability, performance, scaleabi=
lity, security, and total cost of ownership. I close with a brief discuss=
ion of non-quantitative issues, unnecessary fears, other sites providing =
related information, and conclusions.


Market Share

Many people believe that a product is only a winner if it has significant=
 market share. This is lemming-like, but there's some rationale for this:=
 products with big market shares get applications, trained users, and mom=
entum that reduces future risk. Some writers argue against OSS/FS or GNU/=
Linux as ``not being mainstream'', but if their use is widespread then su=
ch statements reflect the past, not the present. There's excellent eviden=
ce that OSS/FS has significant market share in numerous markets:

Apache is the #1 web server on the public Internet, and has been since Ap=
ril 1996. Netcraft's statistics on webservers have consistently shown Apa=
che (an OSS/FS web server) dominating the public Internet web server mark=
et ever since Apache became the #1 web server in April 1996. For example,=
 in November 2001, Netcraft polled all the web sites they could find (tot=
aling 36,458,394 sites), and found that of all the sites they could find,=
 Apache had 56.81% of the market, Microsoft had 29.74% of the market, iPl=
anet (aka Netscape) had 3.59%, and Zeus had 2.20%.

Market Share for Top Web Servers, August 1995 - November 2001

(Graph)


More recently, Netcraft has been trying to discount ``inactive'' sites, s=
ince recently many sites have been deployed that are simply ``placeholder=
'' sites not being actively used. When counting only active sites, Apache=
 does even better; by November 2001, Apache had 61.88% of the web server =
market, Microsoft had 26.40%, iPlanet 1.48%, and Zeus 0.04%.

Market Share for Active Web Servers, June 2000 - November 2001

(Graph)


The same overall result has been determined independently by E-soft - the=
ir report published July 1, 2001 surveyed 3,178,197 web servers and found=
 that Apache was #1 (59.84%), with Microsoft IIS being #2 (28.42%). Obvio=
usly these figures fluctuate monthly; see Netcraft and E-soft for the lat=
est survey figures.


GNU/Linux is the #2 web serving operating system on the public Internet (=
counting by IP address), according to a study surveying March and June 20=
01. Some of Netcraft's surveys have also included data on operating syste=
ms; two 2001 surveys (their June 2001 and September 2001 surveys) found t=
hat GNU/Linux is the #2 operating system for web servers when counting by=
 IP address (and has been consistently gaining market share since Februar=
y 1999). Every connection to a network accessible to the public is given =
a unique number called the IP address, and this survey uses IP addresses =
(their web server survey counts by name, not by IP address). A quick asid=
e: Netcraft dates their information based on the web server surveys (not =
the publication date), and they only report operating system summaries fr=
om an earlier month. Thus, the survey dated ``June 2001'' was published i=
n July and covers operating system survey results of March 2001, while th=
e survey dated ``September 2001'' was published in October and covers the=
 operating system survey results of June 2001. Here's a summary of Netcra=
ft's study results:

OS group | Percentage (March) | Percentage (June) | Composition
Windows | 49.2% | 49.6% | Windows 2000, NT4, NT3, Windows 95, Windows 98
[GNU/]Linux | 28.5% | 29.6% | [GNU/]Linux
Solaris | 7.6% | 7.1% | Solaris 2, Solaris 7, Solaris 8
BSD | 6.3% | 6.1% | BSDI BSD/OS, FreeBSD, NetBSD, OpenBSD
Other Unix | 2.4% | 2.2% | AIX, Compaq Tru64, HP-UX, IRIX, SCO Unix, SunO=
S 4 and others
Other non-Unix | 2.5% | 2.4% | MacOS, NetWare, proprietary IBM OSs
Unknown | 3.6% | 3.0% | not identified by Netcraft operating system detec=
tor

Much depends on what you want to measure. Several of the BSDs (FreeBSD, N=
etBSD, and OpenBSD) are OSS/FS as well; so at least a portion of the 6.1%=
 for BSD should be added to GNU/Linux's 29.6% to determine the percentage=
 of OSS/FS operating systems being used as web servers. Thus, it's likely=
 that approximately one-third of web serving computers use OSS/FS operati=
ng systems. There are also regional differences, for example, GNU/Linux l=
eads Windows in Germany, Hungary, the Czech Republic, and Poland.

Well-known web sites using GNU/Linux include the White House, Google, and=
 Rackspace. Other well-known web sites also use OSS/FS operating systems;=
 for example, Yahoo and Sony Japan use FreeBSD.

If you really want to know about the web server market breakdown of ``Uni=
x vs. Windows,'' you can find that also in this study. All of the various=
 Windows operating systems are rolled into a single number (even Windows =
95/98 and Windows 2000/NT4/NT3 are merged together, although they are fun=
damentally very different systems). Merging all the Unix-like systems in =
a similar way produces a total of 44.8% for Unix-like systems (compared t=
o Windows' 49.2%) in March 2001.

Note that these figures would probably be quite different if they were ba=
sed on web addresses instead of IP addresses; in such a case, the clear m=
ajority of web sites are hosted by Unix-like systems. As stated by Netcra=
ft, ``Although Apache running on various Unix systems runs more sites tha=
n Windows, Apache is heavily deployed at hosting companies and ISPs who s=
trive to run as many sites as possible on a single computer to save costs=
=2E''


GNU/Linux is the #1 server operating system on the public Internet (count=
ing by domain name), according to a 1999 survey of primarily European and=
 educational sites. The first study that I've found that examined GNU/Lin=
ux's market penetration is a survey by Zoebelein in April 1999. This surv=
ey found that, of the total number of servers deployed on the Internet in=
 1999 (running at least ftp, news, or http (WWW)) in a database of names =
they used, the #1 operating system was GNU/Linux (at 28.5%), with others =
trailing. It's important to note that this survey, which is the first one=
 that I've found to try to answer questions of market share, used existin=
g databases of servers from the .edu (educational domain) and the RIPE da=
tabase (which covers Europe , the Middle East, parts of Asia, and parts o=
f Africa), so this isn't really a survey of ``the entire Internet'' (e.g.=
, it omits ``.com'' and ``.net''). This is a count by domain name (e.g., =
the text name you would type into a web browser for a location) instead o=
f by IP address, so what it's counting is different than the Netcraft Jun=
e 2001 operating system study. Also, this study counted servers providing=
 ftp and news services (not just web servers).

Here's how the various operating systems fared in the study:

Market Share | Operating System | Composition
GNU/Linux | 28.5% | GNU/Linux
Windows | 24.4% | All Windows combined (including 95, 98, NT)
Sun | 17.7% | Sun Solaris or SunOS
BSD | 15.0% | BSD Family (FreeBSD, NetBSD, OpenBSD, BSDI, ...)
IRIX | 5.3% | SGI IRIX

A portion of the BSD family is also OSS/FS, so the OSS/FS operating syste=
m total is even higher; if over 2/3 of the BSDs are OSS/FS, then the tota=
l share of OSS/FS would be about 40%. Advocates of Unix-like systems will=
 notice that the majority (around 66%) were running Unix-like systems, wh=
ile only around 24% ran a Microsoft Windows variant.


GNU/Linux is the #2 server operating system sold in 1999 and 2000, and is=
 the fastest-growing. According to a June 2000 IDC survey of 1999 license=
s, 24% of all servers (counting both Internet and intranet servers) insta=
lled in 1999 ran GNU/Linux. Windows NT came in first with 36%; all Unixes=
 combined totaled 15%. Again, since some of the Unixes are OSS/FS systems=
 (e.g., FreeBSD, OpenBSD, and NetBSD), the number of OSS/FS systems is ac=
tually larger than the GNU/Linux figures. Note that it all depends on wha=
t you want to count; 39% of all servers installed from this survey were U=
nix-like (that's 24%+15%), so ``Unix-like'' servers were actually #1 in i=
nstalled market share once you count GNU/Linux and Unix together.

IDC released a similar study on January 17, 2001 titled ``Server Operatin=
g Environments: 2000 Year in Review''. On the server, Windows accounted f=
or 41% of new server operating system sales in 2000, growing by 20% - but=
 GNU/Linux accounted for 27% and grew even faster, by 24%. Other major Un=
ixes had 13%.

Data such as these (and the TCO data shown later) have inspired statement=
s such as this one from IT-Director on November 12, 2001: ``Linux on the =
desktop is still too early to call, but on the server it now looks to be =
unstoppable.''

These measures do not measure all server systems installed that year; som=
e Windows systems are not paid for (they're illegally pirated), and OSS/F=
S operating systems such as GNU/Linux and the BSDs are often downloaded a=
nd installed on multiple systems (since it's legal and free to do so).


An Evans Data survey published in November 2001 found that 48.1% of inter=
national developers and 39.6% of North Americans plan to target most of t=
heir applications to GNU/Linux. The November 2001 edition of the Evans Da=
ta International Developer Survey Series reported on in-depth interviews =
with more than 400 developers representing over 70 countries, and found t=
hat when asked which operating system they plan to target with most of th=
eir applications next year, 48.1% of international developers and 39.6% o=
f North Americans stated that they plan to target most of their applicati=
ons to GNU/Linux. This is particularly surprising since only a year earli=
er less than a third of the international development community was writi=
ng GNU/Linux applications. The survey also found that 37.8% of the intern=
ational development community and 33.7% of North American developers have=
 already written applications for GNU/Linux, and that more than half of t=
hose surveyed have enough confidence in GNU/Linux to use it for mission-c=
ritical applications.

Microsoft sponsored its own research to ``prove'' that GNU/Linux isn't as=
 widely used, but this research has been shown to be seriously flawed. Mi=
crosoft sponsored a Gartner Dataquest report claiming only 8.6% of server=
s shipped in the U.S. during the third quarter of 2000 were Linux-based. =
However, it's worth noting that Microsoft (as the research sponsor) has e=
very incentive to create low numbers, and these numbers are quite differe=
nt from IDC's research in the same subject. IDC's Kusnetzky commented tha=
t the likely explanation is that Gartner used a very narrow definition of=
 "shipped"; he thought the number was "quite reasonable" if it only surve=
yed new servers with Linux , "But our research is that this is not how mo=
st users get their Linux. We found that just 10 to 15 percent of Linux ad=
option comes from pre-installed machines... for every paid copy of Linux,=
 there is a free copy that can be replicated 15 times." Note that it's qu=
ite difficult to buy a new x86 computer without a Microsoft operating sys=
tem (Microsoft's contracts with computer makers ensure this), but that do=
esn't mean that these operating systems are used. Gartner claimed that it=
 used interviews to counter this problem, but its final research results =
(when compared to known facts) suggest that Gartner did not really counte=
r this effect. For example, Gartner states that Linux shipments in the su=
percomputer field were 'zero'. In fact, Linux is widely used on commodity=
 parallel clusters at many scientific sites, including a number of high-p=
rofile sites. Many of these systems were assembled in-house, showing that=
 Gartner's method of defining a ``shipment'' does not appear to correlate=
 to working installations. The Register's article, ``No one's using Linux=
'' (with its companion article ``90% Windows..'') discusses this further.=
 In short, Microsoft-sponsored research reported low numbers, but these n=
umbers are quite suspect.

GNU/Linux had 80% as many client shipments in 1999 as Apple's MacOS. Acco=
rding to the June 2000 IDC survey of 1999 licenses (5.0% for Mac OS, 4.1%=
 for GNU/Linux), there were almost as many client shipments of GNU/Linux =
as there were of MacOS - and no one doubts that MacOS is a client system.=


Currently, GNU/Linux is a relatively new contender in the client operatin=
g system (OS) market, and has relatively little market penetration. This =
should not be surprising; unlike the wealth of server and developer appli=
cations, GNU/Linux has relatively few OSS/FS client applications and many=
 of those client applications are still maturing. There are commercial cl=
ient applications (e.g. Corel's Word Perfect), but they aren't OSS/FS and=
 are available on other platforms as well. Without strong OSS/FS client a=
pplications (in particular an office suite), GNU/Linux has no strong adva=
ntage over other client platforms. After all, GNU/Linux combined with a p=
roprietary office suite still lacks the freedoms and low cost of purely O=
SS/FS systems, and it has to compete with established proprietary systems=
 which have more applications available to them. Microsoft Office is esse=
ntially a monopoly controlling the office suite arena, and it isn't avail=
able on GNU/Linux (the second-largest player in office suites is Corel; s=
ince Corel is partly owned by Microsoft, Corel cannot really be considere=
d a competitor to Microsoft). Various OSS/FS office application programs =
are only now maturing, such as StarOffice / OpenOffice), and alternatives=
 include AbiWord, Gnumeric, and the KOffice suite. In addition, the cost =
of switching all those desktops in the face of compatibility issues with =
an entrenched monopoly makes it difficult to use anything other than Wind=
ows and Office on clients.

Nevertheless, in spite of weaker OSS/FS client applications in several ke=
y areas at the time, an IDC study determined that GNU/Linux captured 4% o=
f the market in 1999. More generally, IDC determined that there were 98.6=
 million shipments of client operating systems in 1999, of which Windows =
3.x, 95 and 98 accounted for 66%, Windows NT had 21%, MacOS had 5%, and G=
NU/Linux 4%.

There are some interesting hints that this may change in the future. Some=
 organizations, such as TrustCommerce and the city of Largo, Florida, rep=
ort that they've successfully transitioned to using Linux on the desktop.=
 It will be interesting to see if these numbers begin to change in 2002-2=
003, when I anticipate OSS/FS client applications (e.g., AbiWord and Open=
Office) will be available that will begin to rival their proprietary comp=
etitors in both useability and in the functionality that people need. The=
re's already some evidence that others anticipate this; Richard Thwaite, =
director of IT for Ford Europe, stated in 2001 that an open source deskto=
p is their goal, and that they expect the industry to eventually go there=
 (he controls 33,000 desktops, so this would not be a trivial move). It c=
ould be argued that this is just a ploy for negotiation with Microsoft - =
but such ploys only work if they're credible.


Businesses plan to increase their use of GNU/Linux. A Zona Research study=
 found that more than half of the large enterprise respondents expected i=
ncreases of up to 25% in the number of GNU/Linux users in their firm, whi=
le nearly 20% expected increases of more than 50%. In small companies, mo=
re than one third felt that GNU/Linux usage would expand by 50%. The most=
 important factors identified that drove these decisions were reliability=
, lower price, speed of applications, and scaleability. Here are the numb=
ers:

Expected GNU/Linux Use | Small Business | Midsize Business | Large Busine=
ss | Total
50% increase | 21.0% | 16% | 19.0% | 19%
10-25% increase | 30.5% | 42% | 56.5% | 44%
No growth | 45.5% | 42% | 24.5% | 36%
Reduction | 3.0% | 0% | 0% | 1%

You can see more about this study in ``The New Religion: Linux and Open S=
ource'' (ZDNet) and in InfoWorld's February 5, 2001 article ``Linux light=
s up enterprise: But concerns loom about OS vendor profitability.''

The global top 1000 Internet Service Providers expect GNU/Linux use to in=
crease by 154%, according to Idaya's survey conducted January through Mar=
ch 2001. A survey conducted by Idaya of the global top 1000 ISPs found th=
at they expected GNU/Linux to grow a further 154% in 2001. Also, almost t=
wo thirds (64%) of ISPs consider the leading open source software meets t=
he standard required for enterprise level applications, comparable with p=
roprietary software. Idaya produces OSS/FS software, so keep that in mind=
 as a potential bias.

IBM found a 30% growth in the number of enterprise-level applications for=
 GNU/Linux in the six month period ending June 2001. At one time, it was =
common to claim that ``Not enough applications run under GNU/Linux'' for =
enterprise-level use. However, IBM found there are more than 2,300 GNU/Li=
nux applications (an increase in 30% over 6 months) available from IBM an=
d the industry's top independent software vendors (ISVs). A Special repor=
t by Network Computing on Linux for the Enterprise discusses some of the =
strengths and weaknesses of GNU/Linux, and found many positive things to =
say about GNU/Linux for enterprise-class applications.

A 2001 survey found that 46.6% of IT professionals were confident that th=
eir organizations could support GNU/Linux, a figure larger than any OS ex=
cept Windows. A TechRepublic Research survey titled Benchmarks, Trends, a=
nd Forecasts: Linux Report found that ``support for Linux runs surprising=
ly deep'' when it surveyed IT professionals and asked them how confidentl=
y their organizations could support various operating systems. Given Wind=
ows' market dominance on the desktop, it's not surprising that most were =
confident that their organizations could support various versions of Wind=
ows (for Windows NT the figure was 90.6%; for Windows 2000, 81.6%). Howev=
er, GNU/Linux came in third, at 46.4%; about half of those surveyed respo=
nded that their organizations were already confident in their ability to =
support GNU/Linux! This is especially shocking because GNU/Linux beat oth=
er well-known products with longer histories including Unix (42.1%), Nove=
ll Netware (39.5%), Sun Solaris (25.7%), and Apple (13.6%). TechRepublic =
suggested that there are several possible reasons for this surprisingly l=
arge result:

GNU/Linux is considered to be a rising technology; many IT professionals =
are already studying it and learning how to use it, assuming that it will=
 be a marketable skill in the near future.

Many IT professionals already use GNU/Linux at home, giving GNU/Linux an =
entree into professional organizations.

Since GNU/Linux is similar to Unix, IT professionals who are proficient i=
n Unix can easily pick up GNU/Linux.

TechRepublic suggests that IT executives should inventory their staff's s=
kill sets, because they may discover that their organization can already =
support GNU/Linux if they aren't currently using it.

Sendmail, an OSS/FS program, is the leading email server. A survey betwee=
n 2001-09-27 and 2001-10-03 by D.J. Bernstein of one million random IP ad=
dresses successfully connected to 958 SMTP (email) servers (such servers =
are also called mail transport agents, or MTAs). Bernstein found that Uni=
x Sendmail had the largest market share (42% of all email servers), follo=
wed by Windows Microsoft Exchange (18%), Unix qmail (17%), Windows Ipswit=
ch IMail (6%), Unix smap (2%), UNIX Postfix (formerly VMailer, 2%) and Un=
ix Exim (1%). Note that Bernstein implements one of Sendmail's competitor=
s (qmail), so he has a disincentive to identify Sendmail's large market s=
hare. Qmail is not OSS/FS, because derivatives of Qmail cannot be freely =
redistributed; Qmail is ``source viewable,'' so some people are confused =
into believing that Qmail is OSS/FS. However, Sendmail, Postfix, and Exim=
 are all OSS/FS. Indeed, not only is the leading program (Sendmail) OSS/F=
S, but that OSS/FS program has more than twice the installations of its n=
earest competition.

A survey in the second quarter of 2000 found that 95% of all reverse-look=
up domain name servers (DNS) used bind, an OSS/FS product. The Internet i=
s built from many mostly-invisible infrastructure components. This includ=
es domain name servers (DNSs), which take human-readable machine names (l=
ike ``yahoo.com'') and translate them into numeric addresses. Publicly ac=
cessible machines also generally support ``reverse lookups'', which conve=
rt the numbers back to names; for historical reasons, this is implemented=
 using the hidden ``in-addr.arpa'' domain. By surveying the in-addr domai=
n, you can gain insight into how the entire Internet is supported. Bill M=
anning has surveyed the in-addr domain and found that 95% of all name ser=
vers (in 2q2000) performing this important Internet infrastructure task a=
re some version of ``bind.'' Bind is an OSS/FS program.


Reliability

There are a lot of anecdotal stories that OSS/FS is more reliable, but fi=
nally there is quantitative data confirming that mature OSS/FS programs a=
re more reliable:

Equivalent OSS/FS applications are more reliable, according to a 1995 stu=
dy. The 1995 ``Fuzz Revisited'' paper measured reliability by feeding pro=
grams random characters and determining which ones resisted crashing and =
freeze-ups. Some researchers scoff at this measure, since this approach i=
s unlikely to find subtle failures, but the study authors note that their=
 approach still manages to find many errors in production software and is=
 a useful tool for finding software flaws.

OSS/FS had higher reliability by this measure. It states in section 2.3.1=
 that:

It is also interesting to compare results of testing the commercial syste=
ms to the results from testing "freeware" GNU and Linux. The seven commer=
cial systems in the 1995 study have an average failure rate of 23%, while=
 Linux has a failure rate of 9% and the GNU utilities have a failure rate=
 of only 6%. It is reasonable to ask why a globally scattered group of pr=
ogrammers, with no formal testing support or software engineering standar=
ds can produce code that is more reliable (at least, by our measure) than=
 commercially produced code. Even if you consider only the utilities that=
 were available from GNU or Linux, the failure rates for these two system=
s are better than the other systems.

There is evidence that Windows NT applications have similar reliability t=
o the proprietary Unix software (e.g., less reliable than the OSS/FS soft=
ware). A later paper, ``An Empirical Study of the Robustness of Windows N=
T Applications Using Random Testing'', found that with Windows NT GUI app=
lications, they could crash 21% of the applications they tested, hang an =
additional 24% of the applications, and could crash or hang all the teste=
d applications when subjecting them to random Win32 messages. Thus, there=
's no evidence that proprietary Windows software is more reliable than OS=
S/FS by this measure.

Although this experiment was done in 1995, nothing that's happened since =
suggests that proprietary software has become much better than OSS/FS pro=
grams since then. Indeed, since 1995 there's been an increased interest a=
nd participation in OSS/FS, resulting in far more ``eyeballs'' examining =
and improving the reliability of OSS/FS programs.

Now be careful: OSS/FS is not magic pixie dust; beta software of any kind=
 is still buggy! However, the 1995 experiment measured mature OSS/FS to m=
ature proprietary software, and the OSS/FS software was more reliable und=
er this measure.


GNU/Linux is more reliable than Windows NT, according to a 10-month ZDnet=
 experiment. ZDnet ran a 10-month test for reliability to compare Caldera=
 Systems OpenLinux, Red Hat Linux, and Microsoft's Windows NT Server 4.0 =
with Service Pack 3. All three used identical (single-CPU) hardware, and =
network requests were sent to each server in parallel for standard Intern=
et, file, and print services. The result: NT crashed an average of once e=
very six weeks, each taking about 30 minutes to fix; that's not bad, but =
neither GNU/Linux server ever went down. This ZDnet article also does a g=
ood job of identifying GNU/Linux weaknesses (e.g., desktop applications a=
nd massive SMP).

GNU/Linux is more reliable than Windows NT, according to a one-year Bloor=
 Research experiment. Bloor Research had both operating systems running o=
n relatively old Pentium machines. In the space of one year, GNU/Linux cr=
ashed once because of a hardware fault (disk problems), which took 4 hour=
s to fix, giving it a measured availability of 99.95 percent. Windows NT =
crashed 68 times, caused by hardware problems (disk), memory (26 times), =
file management (8 times), and a number of odd problems (33 times). All t=
his took 65 hours to fix, giving an availability of 99.26 percent. It's i=
ntriguing that the only GNU/Linux problem and a number of the Windows pro=
blems were hardware-related; it could be argued that the Windows hardware=
 was worse, or it could be argued that GNU/Linux did a better job of avoi=
ding and containing hardware failures. In any case, file management failu=
re can be blamed on Windows, and the ``odd problems'' appear to be caused=
 by Windows as well, indicating that GNU/Linux is far more reliable than =
Windows. Gnet summarized this as saying ``the winner here is clearly Linu=
x.''

Sites using Microsoft's IIS web serving software have more than double th=
e time offline (on average) than sites using the Apache software, accordi=
ng to a 3-month Swiss evaluation. These are the results of Syscontrol AG'=
s analysis of website uptime (announced February 7, 2000) They measured o=
ver 100 popular Swiss web sites over a three-month period, checking from =
4 different locations every 5 minutes (it'd be interesting to see what a =
larger sample would find!). You can see their report (in German), or a Ba=
belfish (machine) translation of the report. Here's their entire set of p=
ublished data on ``average down-time per hour per type of server'', plus =
a 3-month average that I've computed:

Downtime | Apache | Microsoft | Netscape | Other
September | 5.21 | 10.41 | 3.85 | 8.72
October | 2.66 | 8.39 | 2.80 | 12.05
November | 1.83 | 14.28 | 3.39 | 6.85
Average | 3.23 | 11.03 | 3.35 | 9.21

It's hard not to notice that Apache (the OSS web server) had the best res=
ults over the three-month average (and with better results over time, too=
). Indeed, Apache's worst month was better than Microsoft's best month. I=
 believe the difference between Netscape and Apache is statistically insi=
gnificant - but this still shows that the freely-available OSS/FS solutio=
n (Apache) has a reliability at least as good as the most reliable propri=
etary solution. The report does note that this might not be solely the fa=
ult of the software's quality, since there were several Microsoft IIS sit=
es that had short interruptions at the same time each day (suggesting reg=
ular restarts). However, this still begs the question -- why did the IIS =
sites require so many more regular restarts than the Apache sites? Every =
outage, even if preplanned, results in a service loss (and for e-commerce=
 sites, a potential loss of sales).


According to a separate uptime study by Netcraft, OSS/FS does very well; =
as of August 3, 2001, of the 50 sites with the highest uptimes, 92% use A=
pache and 50% run on OSS/FS operating systems. Netcraft keeps a track of =
the 50 often-requested sites with the longest uptimes at http://uptime.ne=
tcraft.com. Looking at the August 3, 2001 uptime report, I found that 92%=
 (46/50) of the sites use Apache; one site's web server was unknown, and =
three others were not Apache. Of those three, only one reported to be Mic=
rosoft IIS, and that one instance is suspicious because its reported oper=
ating system is BSD/OS (this apparant inconsistency can be explained in m=
any ways, e.g., perhaps there is a front-end BSD/OS system that ``masks''=
 the IIS web site, or perhaps the web server is lying about its type to c=
onfuse attackers). In this snapshot, 50% (25/50) ran on an open source op=
erating system, and only Unix-like operating systems had these large upti=
mes (no Windows systems were reported as having the best uptimes).

As with all surveys, this one has weaknesses, as discussed in Netcraft's =
Uptime FAQ. Their techniques for identifying web server and operating sys=
tems can be fooled. Only systems for which Netcraft was sent many request=
s were included in the survey (so it's not ``every site in the world''). =
Any site that is requested through the ``what's that site running'' query=
 form at Netcraft.com is added to the set of sites that are routinely sam=
pled; Netcraft doesn't routinely monitor all 22 million sites it knows of=
 for performance reasons. Many operating systems don't provide uptime inf=
ormation and thus can't be included; this includes AIX, AS/400, Compaq Tr=
u64, DG/UX, MacOS, NetWare, NT3/Windows 95, NT4/Windows 98, OS/2, OS/390,=
 SCO UNIX, Sony NEWS-OS, SunOS 4, and VM. Thus, this uptime counter can o=
nly include systems running on BSD/OS, FreeBSD (but not the default confi=
guration in versions 3 and later), recent versions of HP-UX, IRIX, GNU/Li=
nux 2.1 kernel and later (except on Alpha processor based systems), MacOS=
 X, recent versions of NetBSD/OpenBSD, Solaris 2.6 and later, and Windows=
 2000. Note that Windows NT systems cannot be included in this survey (be=
cause their uptimes couldn't be counted), but Windows 2000 systems are in=
cluded in this survey. Again, no Windows system actually made it into the=
 top 50 for uptimes in this snapshot. Note that HP-UX, (many versions of)=
 GNU/Linux, Solaris and recent releases of FreeBSD cycle back to zero aft=
er 497 days, exactly as if the machine had been rebooted at that precise =
point. Thus it is not possible to see an HP-UX, (most) GNU/Linux, or Sola=
ris system with an uptime measurement above 497 days, and in fact their u=
ptimes can be misleading (they may be up for a long time, yet not show it=
). Still, this survey does compare Windows 2000, GNU/Linux (up to 497 day=
s usually), FreeBSD, and several other operating systems, and a vast numb=
er of web servers, and OSS/FS does quite well.

Of course, there are many anecdotes about Windows reliability vs. Unix. F=
or example, the Navy's ``Smart Ship'' program caused a complete failure o=
f the entire USS Yorktown ship in September 1997. Anthony DiGiorgio (a wh=
istle-blower) stated that Windows is ``the source of the Yorktown's compu=
ter problems.'' Ron Redman, deputy technical director of the Fleet Introd=
uction Division of the Aegis Program Executive Office, said ``there have =
been numerous software failures associated with [Windows] NT aboard the Y=
orktown.'' Redman also said ``Because of politics, some things are being =
forced on us that without political pressure we might not do, like Window=
s NT... If it were up to me I probably would not have used Windows NT in =
this particular application. If we used Unix, we would have a system that=
 has less of a tendency to go down.''

One problem with reliability measures is that it takes a long time to gat=
her data on reliability in real-life circumstances. Nevertheless, the ava=
ilable evidence suggests that OSS/FS has a significant edge in reliabilit=
y.


Performance

Comparing GNU/Linux and Microsoft Windows performance on equivalent hardw=
are has a history of contentious claims and different results based on di=
fferent assumptions. I think that OSS/FS has at least shown that it's oft=
en competitive, and in many circumstances it beats the competition.

Performance benchmarks are very sensitive to the assumptions and environm=
ent, so the best benchmark is one you set up yourself to model your inten=
ded environment. Failing that, you should use unbiased measures, because =
it's so easy to create biased measures.

First, here are a few recent studies suggesting that some OSS/FS systems =
beat their proprietary competition in at least some circumstances:

PC Magazine's November 2001 performance tests for file servers found that=
 Linux with Samba significantly outperformed Windows 2000. PC Magazine's =
article Performance Tests: File Server Throughput and Response Times foun=
d that Linux with Samba significantly outperformed Windows 2000 Server wh=
en used as a file server for Microsoft's own network file protocols. This=
 was true regardless of the number of simultaneous clients (they tested a=
 range up to 30 clients), and it was true on the entire range on computer=
s they used (Pentium II/233MHz with 128MiB RAM, Pentium III/550MHz with 2=
56MiB RAM, and Pentium III/1GHz with 512MiB RAM, where MiB is 2^20 bytes)=
=2E Indeed, as the machines became more capable the absolute difference b=
ecame more pronounced. On the fastest hardware while handling largest num=
ber of clients, GNU/Linux's throughput was about 130 MB/sec vs. Windows' =
78 MB/sec (GNU/Linux was 78% faster).

In performance tests by Sys Admin magazine, GNU/Linux beat Solaris (on In=
tel), Windows 2000, and FreeBSD. The article ``Which OS is Fastest for Hi=
gh-Performance Network Applications?'' in the July 2001 edition of Sys Ad=
min magazine examined high-performance architectures and found that GNU/L=
inux beat its competition when compared with Solaris (on Intel), FreeBSD =
(an OSS/FS system), and Windows 2000. They intentionally ran the systems =
``out of the box'' (untuned), except for increasing the number of simulta=
neous TCP/IP connections (which is necessary for testing multi-threaded a=
nd asynchronous applications). They used the latest versions of operating=
 systems and the exact same machine. They reported (by operating system) =
the results of two different performance tests.

The FreeBSD developers complained about these tests, noting that FreeBSD =
by default emphasizes reliability (not speed) and that they expected anyo=
ne with a significant performance need would do some tuning first. Thus, =
Sys Admin's re-did the tests for FreeBSD after tuning FreeBSD. One change=
 they made was switching to ``asynchronous'' mounting, which makes a syst=
em faster (though it increases the risk of data loss in a power failure) =
- this is the GNU/Linux default and easy to change in FreeBSD, so this wa=
s a very small and reasonable modification. However, they also made many =
other changes, for example, they found and compiled in 17 FreeBSD kernel =
patches and used various tuning commands. The other operating systems wer=
en't given the chance to ``tune'' like this, so comparing untuned operati=
ng systems to a tuned FreeBSD isn't really fair.

In any case, here are their two performance tests:

Their ``real-world'' test measured how quickly large quantities of email =
could be sent using their email delivery server (MailEngine). Up to 100 s=
imultaneous sends there was no difference, but as the number increased th=
e systems began showing significant differences in their hourly email del=
ivery speed. By 500 simultaneous sends GNU/Linux was clearly faster than =
all except FreeBSD-tuned, and GNU/Linux remained at the top. FreeBSD-tune=
d had similar performance to GNU/Linux when running 1000 or less simultan=
eous sends, but FreeBSD-tuned peaked around 1000-1500 simultaneous connec=
tions with a steady decline not suffered by GNU/Linux, and FreeBSD-tuned =
had trouble going beyond 3000 simultaneous connections. By 1500 simultane=
ous sends, GNU/Linux was sending 1.3 million emails/hour, while Solaris m=
anaged approximately 1 million, and Windows 2000 and FreeBSD-untuned were=
 around 0.9 million.

Their ``disk I/O test'' created, wrote, and read back 10,000 identically-=
sized files in a single directory, varying the size of the file instances=
=2E Here Solaris was the slowest, with FreeBSD-untuned the second-slowest=
=2E FreeBSD-tuned, Windows 2000, and GNU/Linux had similar speeds at the =
smaller file sizes (in some cases FreeBSD-tuned was faster, e.g., 8k and =
16k file size), but when the file sizes got to 64k to 128k the operating =
systems began to show significant performance differences; GNU/Linux was =
the fastest, then Windows 2000, then FreeBSD. At 128k, FreeBSD was 16% wo=
rse than Windows 2000, and 39% worse than GNU/Linux; all were faster than=
 FreeBSD-untuned and Solaris. When totaling these times across file sizes=
, the results were GNU/Linux: 542 seconds, Windows 2000: 613 seconds, Fre=
eBSD-tuned: 630 seconds, FreeBSD-untuned: 2398 seconds, and Solaris: 3990=
 seconds.

GNU/Linux with TUX has produced better SPEC values than Windows/IIS in se=
veral cases, even when given inferior drive configurations. One organizat=
ion that tries to develop unbiased benchmarks is the SPEC Consortium, whi=
ch develops and maintains a whole series of benchmarks. We can compare Mi=
crosoft Windows versus GNU/Linux by comparing SPECweb99 results (which me=
asure web server performance) on identical hardware if both have undergon=
e the same amount of performance optimization effort. Alas, things are no=
t so simple; rarely are the same basic hardware platforms tested with bot=
h operating systems, and even when that occurs, as of July 13, 2001 no ex=
actly identical configurations have been tested (they differ in ways such=
 as using a different number of hard drives, or including some faster har=
d drives). Using all results available by July 13, 2001, there were three=
 hardware configurations, all from Dell, which ran both GNU/Linux (using =
the TUX web server/accelerator) and Windows (using IIS) on exactly the sa=
me underlying hardware. Here are the SPECweb99 results as of July 13, 200=
1 (larger is better), noting configuration differences:

System | Windows SPEC Result | Linux SPEC Result
Dell PowerEdge 4400/800, 2 800MHz Pentium III Xeon | 1060 (IIS 5.0, 1 net=
work controller) 2200 (TUX 1.0, 2 network controllers)
Dell PowerEdge 6400/700, 4 700MHz Pentium III Xeon | 1598 (IIS 5.0, 7 9GB=
 10KRPM drives) 4200 (TUX 1.0, 5 9GB 10KRPM drives)
Dell PowerEdge 8450/700, 8 700MHz Pentium III Xeon | 7300/NC (IIS 5.0, 1 =
9Gb 10KRPM and 8 16Gb 15KRPM drives) then 8001 (IIS 5.0, 7 9Gb 10KRPM and=
 1 18Gb 15KRPM drive) 7500 (TUX 2.0, 5 9Gb 10KRPM drives)

The first row (the PowerEdge 4400/800) doesn't really prove anything. The=
 IIS system has lower performance, but it only had one network controller=
 and the TUX system has two - so while the TUX system had better performa=
nce, that could simply be because it had two network connections it could=
 use.

The second entry (the PowerEdge 6400/700) certainly suggests that GNU/Lin=
ux plus TUX really is much better - the IIS system had two more disk driv=
es available to it (which should increase performance), but the TUX syste=
m had more than twice the IIS system's performance.

The last entry for the PowerEdge 8450/700 is even more complex. First, th=
e drives are different - the IIS systems had at least one drive that revo=
lved more quickly than the TUX systems (which should give IIS higher perf=
ormance overall, since the transfer speed is almost certainly higher). Al=
so, there were more disk drives (which again should give IIS still higher=
 performance). When I originally put this table together showing all data=
 publicly available in April 2001 (covering the third quarter of 1999 thr=
ough the first quarter of 2001), IIS 5.0 (on an 8-processor Dell PowerEdg=
e 8450/700) had a SPECweb99 value of 7300. Since that time, Microsoft cha=
nged the availability of Microsoft SWC 3.0, and by SPECweb99 rules, this =
means that those test results are ``not compliant'' (NC). This is subtle;=
 it's not that the test itself was invalid, it's that Microsoft changed w=
hat was available and used the SPEC Consortium's own rules to invalidate =
a test (possibly because the test results were undesirable to Microsoft).=
 A retest then occurred, with yet another disk drive configuration, at wh=
ich point IIS produced a value of 8001. However, both of these figures ar=
e on clearly better hardware - and in one circumstance the better hardwar=
e didn't do better.

Thus, in these configurations the GNU/Linux plus TUX system was given inf=
erior hardware yet still sometimes won on performance. Since other factor=
s may be involved, it's hard to judge - there are pathological situations=
 where ``better hardware'' can have worse performance, or there may be an=
other factor not reported that had a more significant effect. Hopefully i=
n the future there will be many head-to-head tests in a variety of identi=
cal configurations.

Note that TUX is intended to be used as a ``web accelerator'' for many ci=
rcumstances, where it rapidly handles simple requests and then passes mor=
e complex queries to another server (usually Apache). I've quoted the TUX=
 figures because they're the recent performance figures I have available.=
 As of this time I have no SPECweb99 figures or other recent performance =
measures for Apache on GNU/Linux, or for Apache and TUX together; I also =
don't have TUX reliability figures. I expect that such measures will appe=
ar in the future.


Low-level benchmarks by IBM found that GNU/Linux had better performance t=
han Windows for pipes (an input/output mechanism), and also process and t=
hread creation. Ed Bradford (manager of Microsoft Premier Support for IBM=
 Software group) published in October 2001 the study Pipes in Linux, Wind=
ows 2000, and Windows XP. In this study he examined the the performance o=
f pipes, a common low-level mechanism for communicating between program p=
rocesses. He found the pipes in Red Hat 7.1 (with Linux kernel version 2.=
4.2) had a peak I/O rate of around 700 MB/sec, with a steady state at nea=
r 100 MB/sec for very large block sizes. In contrast, Windows 2000 peaked=
 at 500 MB/sec, with a large block steady state of 80 MB/sec. Windows XP =
Professional (evaluation version) was especially disappointing; its peak =
I/O rate was only 120 MB/sec, with a stead state of 80 MB/sec, all on the=
 same platform and all running a GUI.

In February 2002 he published Managing processes and threads, in which he=
 compared the performance of Red Hat Linux 7.2, Windows 2000 Advanced Ser=
ver ("Win2K"), and Windows XP Professional ("WinXP"), all on a Thinkpad 6=
00X with 320MiB of memory. Linux managed over to create 10,000 threads/se=
cond, while Win2K didn't quite manage 5,000 threads/second and WinXP only=
 created 6,000 threads/second. In process creation, Linux managed 330 pro=
cesses/second, while Win2K managed less than 200 processes/second and Win=
XP less than 160 processes/second.

Fastcenter ran Oracle performance tests and found that show Microsoft Win=
dows 2000 Server was capable of only 85% of throughput compared to SuSE L=
inux Enterprise 7 on the same hardware. Fastcenter ran a benchmark of dat=
abase performance tests using Oracle on exactly the same hardware (twin 1=
 GHz processors, 2 Gibibytes of ram, and two 10k rpm 160mb/sec disks). Th=
ey compared Microsoft Windows 2000 Server (Version 5.0.2195, build 2195, =
no patches) with SuSE Enterprise Linux Server 7 (standard installation). =
Both operating systems were installed essentially as "stock" installation=
s; see FastCenter's benchmarks for more information. The Windows 2000 ser=
ver instance build only achieved 70% of the performance of the Linux (on =
Windows 2000 server this action took 5 minutes and 56 seconds, while on S=
uSE Linux the same action took 4 minutes and 12 seconds). Once installed,=
 Microsoft's Windows 2000 server averaged 85% of the throughput of SuSE L=
inux (it varied depending on the load, but in all cases SuSE Linux had be=
tter performance). Fastcenter is a partner with SuSE Corporation - a Linu=
x distributor - so take that into account. For more information, see Fast=
center's paper.

All operating systems in active development are in a constant battle for =
performance improvements over their rivals. The history of comparing Wind=
ows and GNU/Linux helps put this in perspective:


Ziff-Davis found that GNU/Linux with Apache beat Windows NT 4.0 with IIS =
by 16%-50% depending on the GNU/Linux distribution. Ziff-Davis compared L=
inux and Windows NT's performance at web serving. They found that ``Linux=
 with Apache beats NT 4.0 with IIS, hands down. SuSE, the least effective=
 Linux, is 16% faster than IIS, and Caldera, the leader, is 50% faster.''=


Mindcraft released a report in April 1999 that claimed that Microsoft Win=
dows NT Server 4.0 is 2.5 times faster than Linux (kernel 2.2) as a File =
Server and 3.7 times faster as a Web Server when running on a 4-CPU SMP s=
ystem. Several people and organizations, such Linux Weekly News (LWN) and=
 Dan Kegel, identified serious problems with this study. An obvious issue=
 was that NT was specially tuned by Microsoft's NT experts, at Microsoft,=
 while GNU/Linux was not tuned at all. Another issue is that the price/pe=
rformance wasn't considered (nor was total expenditure kept constant - fo=
r the same amount of money, the GNU/Linux system could have had better ha=
rdware). Mindcraft claimed they asked for help, but they didn't use the d=
ocumented methods for getting help nor did they purchase a support contra=
ct. Many were especially offended that even though this study was funded =
by Microsoft (one of the contestants) and held at their facility, neither=
 Mindcraft's initial announcement nor its paper made any mention of this =
conflict-of-interest - and it could be easily claimed that their configur=
ation was designed to put GNU/Linux at a disadvantage. Their configuratio=
n was somewhat bizarre - it assumed all web pages were static (typical bi=
g sites tend to use many dynamically generated pages) and that there were=
 100 or so clients connected via 100baseT (in 1999 a more typical situati=
on would be that most clients are using slower 28.8 or 56 Kbps modems).

Careful examination of the benchmark did find some legitimate Linux kerne=
l problems, however. These included a TCP bug, the lack of ``wake one'' s=
emantics, and SMP bottlenecks (see Dan Kegel's pages for more information=
). The Linux kernel developers began working on the weaknesses identified=
 by the benchmark.


PC Week confirmed that Windows did indeed do better in this less probable=
 configuration. In June 30, 1999, Mindcraft released their Open Benchmark=
 in conjunction with PC Week. While this didn't excuse Mindcraft's biases=
, it did make a convincing case that there were legitimate problems in th=
e Linux kernel and Apache that made GNU/Linux a poorer-performing product=
 in this somewhat improbable configuration (serving static web pages to c=
lients with high-speed connections). Note that this configuration was con=
siderably different than Ziff-Davis's, so the benchmarks don't necessaril=
y conflict; it's merely that different assumptions can produce different =
results (as I've already stressed).

Network Computing found that GNU/Linux with Samba ran at essentially the =
same speed as Windows for file serving. In their article ``Is it Time for=
 Linux'', Network Computing compared Red Hat Linux v5.2 running Samba 2.0=
=2E3 against Microsoft Windows NT Server Enterprise Edition on a Pentium =
II-based HP NetServer LPr, stressing the machine with multiple reads and =
writes of small, medium and large files over the course of several hours.=


For file serving, they discovered only ``negligible performance differenc=
es between the two for average workloads... [and] depending on the degree=
 of tuning performed on each installation, either system could be made to=
 surpass the other slightly in terms of file-sharing performance.'' Red H=
at Linux slightly outperformed NT on file writes, while NT edged out Red =
Hat Linux on massive reads. Note that their configuration was primarily n=
etwork-limited; they stated ``At no point were we able to push the CPUs m=
uch over 50-percent utilization--the single NIC, full duplex 100BASE-T en=
vironment wouldn't allow it.''

They also noted that ``examining the cost difference between the two lice=
nses brings this testing into an entirely new light... the potential savi=
ngs on licenses alone is eye-opening. For example, based on the average s=
treet price of $30 for a Windows NT client license, 100 licenses would co=
st around $3,000, plus the cost of an NT server license (around $600). Co=
mpare this to the price of a Red Hat Linux CD, or perhaps even a free dow=
nload, and the savings starts to approach the cost of a low-end workgroup=
 server. Scale that up to a few thousand clients and you begin to see the=
 savings skyrocket.'' See this paper's section on total cost of ownership=
=2E


The German magazine c't found that web sites with NT was better at static=
 content and dual network connections, but GNU/Linux was better for sites=
 with dynamic content and single connections. Their article Mixed Double:=
 Linux and NT as Web Server on the Test Bed examined Windows NT with IIS =
against GNU/Linux (kernel 2.2.9) with Apache on a machine with four Penti=
um II Xeon CPUs. They found that the performance winner depended on the s=
ituation (by now that should not be a surprise). If the web server primar=
ily served static web pages through two high-performance network cards, N=
T's performance was better. However, they also noted that in sophisticate=
d web sites this result didn't apply, because such sites tend to have pri=
marily dynamic content, and that few sites had this kind of dual-network =
connection (when only one network board was available, GNU/Linux generall=
y had an edge). See their paper for more figures and background.

The Linux developers' various efforts to improve performance appear to ha=
ve paid off. In June 2000, Dell measured the various SPECweb99 values not=
ed above.

There are other benchmarks available, but I've discounted them on various=
 grounds:

A more recent set of articles from eWeek on June 2001, shows some eye-pop=
ping performance numbers for GNU/Linux with TUX. However, although they c=
ompare it to Microsoft IIS, they don't include Microsoft's SWC (Scaleable=
 Web Cache), Microsoft's response to TUX - and omitting it makes this com=
parison unfair in my opinion. You can read more at ``Tux: Built for Speed=
'', ``Smart Coding pays off Big'', and Kegel's detailed remarks.

The ZDNet article Take that! Linux beats MS in benchmark test, loudly tru=
mpeted that GNU/Linux was the May 2001 performance leader in the TPC-H de=
cision support (database) benchmark (``100Gb'' category). However, this r=
esult should not be taken very seriously; the hardware that Linux ran on =
was more powerful than that of the runner-up (Windows 2000). Frankly, the=
 more surprising fact than its top score (which can be easily explained b=
y the hardware) is its mere measurement at all with this benchmark - trad=
itionally only Microsoft's numbers were reported for this benchmark at th=
is range. For more information, see the TPC results.

More information on various benchmarks is available from Kegel's NT vs. L=
inux Server Benchmark Comparisons, SPEC, and the dmoz entry on benchmarki=
ng.

Remember, in benchmarking everything depends on the configuration and ass=
umptions that you make. Many systems are constrained by network bandwidth=
; in such circumstances buying a faster computer won't help at all. Even =
when network bandwidth isn't the limitation, neither Windows nor GNU/Linu=
x do well in large-scale symmetric multiprocessing (SMP) configurations; =
if you want 64-way CPUs with shared memory, neither are appropriate (Sun =
Solaris, which is not OSS/FS, does much better in this configuration). On=
 the other hand, if you want massive distributed (non-shared) memory, GNU=
/Linux does quite well, since you can buy more CPUs with a given amount o=
f money. If massive distribution can't help you and you need very high pe=
rformance, Windows isn't even in the race; today Windows 2000 only runs o=
n Intel x86 compatible chips, while GNU/Linux runs on much higher perform=
ance processors as well as the x86.


Scaleability

Which brings us to the topic of scaleability, a simple term with multiple=
 meanings:

GNU/Linux and NetBSD (both OSS/FS) support a wider range of hardware plat=
forms and performance than any other operating system. Many people mean b=
y ``scaleability'' to answer the question, ``can you use the same softwar=
e system for both small and large projects?'' Often the implied issue is =
that you'd like to start with a modest system, but have the ability to gr=
ow the system as needs demand without expensive modifications. Here OSS/F=
S is unbeatable; because many people can identify scaleability problems, =
and because its source code can be optimized for its platform, the scalea=
bility of many OSS/FS products is amazing. Let's specifically look at GNU=
/Linux. GNU/Linux works on PDAs (including the Agenda VR3), obsolete hard=
ware (so you needn't throw the hardware away), common modern PC hardware,=
 over a dozen different chipsets (not just Intel x86s), mainframes, massi=
ve clusters, and a number of supercomputers. GNU/Linux can be used for ma=
ssive parallel processing; a common approach for doing this is the Beowul=
f architecture. Sandia's ``CPlant'' runs on a set of systems running GNU/=
Linux, and it's the forty-second most powerful computer in the world as o=
f June 2001 (number 42 on the TOP 500 Supercomputer list, June 2001). The=
re's even a prototype implementation of GNU/Linux on a wrist watch, And G=
NU/Linux runs on a vast number of different CPU chips, including the x86,=
 Intel Itanium, ARM, Alpha, IBM AS/400 (midrange), SPARC, MIPS, 68k, and =
Power PC. Another operating system that widely scales to many other hardw=
are platforms is NetBSD.

Thus, you can buy a small GNU/Linux or NetBSD system and grow it as your =
needs grow; indeed, you can replace small hardware with massively paralle=
l or extremely high-speed processors or very different CPU architectures =
without switching operating systems. Windows CE/ME/NT scales down to smal=
l platforms, but not to large ones, and it only works on x86 systems. Man=
y Unix systems (such as Solaris) scale well to specific large platforms b=
ut not as well to distributed or small platforms. These OSS/FS systems ar=
e some of the most scaleable programs around.


OSS/FS development processes can scale to develop large software systems.=
 At one time it was common to ask if the entire OSS/FS process is ``scale=
able,'' that is, if OSS/FS processes could really develop large-scale sys=
tems. Bill Gates' 1976 ``Open Letter to Hobbyists'' asked rhetorically, `=
`Who can afford to do professional work for nothing? What hobbyist can pu=
t three man-years into programming, finding all bugs, documenting his pro=
duct, and distribute it for free?'' He presumed these were unanswerable q=
uestions - but he was wrong. See my reports estimating GNU/Linux's size. =
For Red Hat Linux 6.2, I found the size to be over 17 million source line=
s of code (SLOC). Implemented traditionally it would have taken 4,500 per=
son-years and over $600 million to implement this distribution. For Red H=
at Linux 7.1, I found it to have over 30 million SLOC, representing 8,000=
 person-years or $1 billion (a ``Gigabuck''). Most developers ascribe to =
the design principle that components should be divided into smaller compo=
nents where practical - a practice also applied to GNU/Linux - but some c=
omponents aren't easily divided, and thus some components are quite large=
 themselves (e.g., over 2 million lines of code for the kernel, mostly in=
 device drivers). Thus, it's no longer reasonable to argue that OSS/FS ca=
nnot scale to develop large systems -- because it clearly can.


Security

Quantitatively measuring security is very difficult. However, here are a =
few attempts to do so, and they suggest that OSS/FS is often superior to =
proprietary systems. I'll concentrate in particular on comparing OSS/FS t=
o Windows systems.

J.S. Wurzler Underwriting Managers' ``hacker insurance'' costs 5-15% more=
 if Windows is used instead of Unix or GNU/Linux for Internet operation. =
At least one insurance company has indicated that Windows NT is less secu=
re than Unix or GNU/Linux systems, resulting in higher premiums for Windo=
ws-based systems. It's often difficult to find out when a company has bee=
n successfully cracked; companies often don't want to divulge such inform=
ation to the public for a variety of reasons. Indeed, if consumers or bus=
iness partners lost trust in a company, the resulting loss might be much =
greater than the original attack. However, insurance companies that insur=
e against cracking can require that they get such information (as a condi=
tion of coverage), and can compute future premiums based on that knowledg=
e. According to CNET, Okemos, Mich.-based J.S. Wurzler Underwriting Manag=
ers, one of the earliest agencies to offer ``hacker insurance'' (and thus=
 more likely to have historical data for premium calculation), has begun =
charging its clients anywhere from 5 to 15 percent more if they use Micro=
soft's Windows NT software instead of Unix or GNU/Linux for their Interne=
t operations. Walter Kopf, senior vice president of underwriting, said th=
at ``We have found out that the possibility for loss is greater using the=
 NT system.'' He also said the decision is based on findings from hundred=
s of security assessments the company has done on their small and midsize=
 business clients over the past couple of years.

Most defaced web sites are hosted by Windows, and Windows sites are dispr=
oportionately defaced more often than explained by its market share. Anot=
her way to look at security is to look at the operating system used by de=
faced web sites, and compare them to their market share. A ``defaced'' we=
b site is a site that has been broken into and has its content changed (u=
sually in a fairly obvious way, since subtle modifications are often not =
reported). The advantage of this measure is that unlike other kinds of se=
curity break-ins (which are often ``hushed up''), it's often very difficu=
lt for victims to hide the fact that they've been successfully attacked. =
Historically, this information was maintained by Attrition.org. A summary=
 can be found in James Middleton's article, with the actual data found in=
 Attrition.org's web site. Attrition.org's data showed that 59% of deface=
d systems ran Windows, 21% Linux, 8% Solaris, 6% BSD, and 6% all others i=
n the period of August 1999 through December 2000. Thus, Windows systems =
have has nearly 3 times as many defacements as GNU/Linux systems. This wo=
uld make sense if there were 3 times as many Windows systems, but no matt=
er which figures you use, that's simply not true.

Of course, not all sites are broken through their web server and OS - man=
y are broken through exposed passwords, bad web application programming, =
and so on. But if this is so, why is there such a big difference in the n=
umber of defacements based on the operating system? No doubt some other r=
easons could be put forward (this data only shows a correlation not a cau=
se), but this certainly suggests that OSS/FS can have better security.

Attrition.org has decided to abandon keeping track of this information du=
e to the difficulty of keeping up with the sheer volume of broken sites, =
and it appeared that tracking this information wouldn't be possible. Howe=
ver, defaced.alldas.de has decided to perform this valuable service. Thei=
r recent reports show that this trend has continued; on July 12, 2001, th=
ey report that 66.09% of defaced sites ran Windows, compared to 17.01% fo=
r GNU/Linux, out of 20,260 defaced websites.


The Bugtraq vulnerability database suggests that the least vulnerable ope=
rating system is OSS/FS, and that all the OSS/FS operating systems in its=
 study were less vulnerable than Windows in 1999-2000. One approach to ex=
amining security is to use a vulnerability database; an analysis of one d=
atabase is the Bugtraq Vulnerability Database Statistics page. As of Sept=
ember 17, 2000, here are the total number of vulnerabilities for some lea=
ding operating systems:

OS | 1997 | 1998 | 1999 | 2000
Debian GNU/Linux | 2 | 2 | 30 | 20
OpenBSD | 1 | 2 | 4 | 7
Red Hat Linux | 5 | 10 | 41 | 40
Solaris | 24 | 31 | 34 | 9
Windows NT/2000 | 4 | 7 | 99 | 85

You shouldn't take these numbers very seriously. Some vulnerabilities are=
 more important than others (some may provide little if exploited or only=
 be vulnerable in unlikely circumstances), and some vulnerabilities are b=
eing actively exploited (while others have already been fixed before expl=
oitation). Open source operating systems tend to include many application=
s that are usually sold separately in proprietary systems (including Wind=
ows and Solaris) - for example, Red Hat 7.1 includes two relational datab=
ase systems, two word processors, two spreadsheet programs, two web serve=
rs, and a large number of text editors. In addition, in the open source w=
orld, vulnerabilities are discussed publicly, so vulnerabilities may be i=
dentified for software still in development (e.g., ``beta'' software). Th=
ose with small market shares are likely to have less analysis. The ``smal=
l market share'' comment won't work with GNU/Linux, of course, since we'v=
e already established that GNU/Linux is the #1 or #2 server OS (depending=
 on how you count them). Still, this clearly shows that the three OSS/FS =
OSs listed (Debian GNU/Linux, OpenBSD, and Red Hat Linux) did much better=
 by this measure than Windows in 1999 and (so far) in 2000. Even if a biz=
arre GNU/Linux distribution was created explicitly to duplicate all vulne=
rabilities present in any major GNU/Linux distribution, this intentionall=
y bad GNU/Linux distribution would still do better than Windows (it would=
 have 88 vulnerabilities in 1999, vs. 99 in Windows). The best results we=
re for OpenBSD, an OSS/FS operating system that for years has been specif=
ically focused on security. It could be argued that its smaller number of=
 vulnerabilities is because of its rarer deployment, but the simplest exp=
lanation is that OpenBSD has focused strongly on security - and achieved =
it better than the rest.

This data is partly of interest because one journalist, Fred Moody, faile=
d to understand his data sources - he used these figures to try to show s=
how that GNU/Linux had worse security. He took these numbers and then add=
ed the GNU/Linux ones so each Linux vulnerability was counted at least tw=
ice (once for every distribution it applied to plus one more). By using t=
hese nonsensical figures he declared that GNU/Linux was worse than anythi=
ng. If you read his article, you also need to read the rebuttal by the ma=
nager of the Microsoft Focus Area at SecurityFocus to understand why the =
journalist's article was so wrong.

In 2002, another journalist (James Middleton) made the same mistake, appa=
rently not learning from previous work. Middleton counted the same Linux =
vulnerability up to four times. What's bizarre is that he even reported t=
he individual numbers showing that specific Linux systems were actually m=
ore secure by using Bugtraq's vulnerability list through August 2001, and=
 somehow he didn't realize what it meant. He noted that Windows NT/2000 s=
uffered 42 vulnerabilities, while Mandrake Linux 7.2 notched up 33 vulner=
abilities, Red Hat Linux 7.0 suffered 28, Mandrake 7.1 had 27 and Debian =
2.2 had 26. In short, all of the GNU/Linux distributions had significantl=
y fewer vulnerabilities by this count. It's not entirely clear what was b=
eing considered as being ``in'' the operating system in this case, which =
would of course make a difference; there are some hints that vulnerabilit=
ies in some Windows-based products (such as Exchange) weren't counted whi=
le vulnerabilities in the same functionality (e.g., sendmail) were counte=
d. It also appears that many of the Windows attacks were more dangerous (=
which were often remote attacks actively exploited), as compared to the G=
NU/Linux ones (which were often local attacks, found by looking at source=
 and not actively exploited at the time). I would appreciate links to som=
eone who's analyzed these issues more carefully. The funny thing is that =
given all these errors, the paper gives evidence that the GNU/Linux distr=
ibutions were more secure.

Indeed, as noted in Bruce Schneier's Crypto-gram of September 15, 2000, v=
ulnerabilities are affected by other things such as how many attackers ex=
ploit the vulnerability, the speed at which a fix is released by a vendor=
, and the speed at which they're applied by administrators. Nobody's syst=
em is invincible.

A more recent analysis by John McCormick in Tech Republic compared Window=
s and Linux vulnerabilities using numbers through September 2001. This is=
 an interesting analysis, showing that although Windows NT lead in the nu=
mber of vulnerabilities in 2000, using the 2001 numbers through September=
 2001, Windows 2000 had moved to the ``middle of the pack'' (with some Li=
nux systems having more, and others having fewer, vulnerabilities). Howev=
er, it appears that in these numbers, bugs in Linux applications have bee=
n counted with Linux, while bugs in Windows applications haven't - and if=
 that's so, this isn't really a fair comparison. As noted above, typical =
Linux distributions bundle many applications that are separately purchase=
d from Microsoft.


Red Hat (an OSS/FS vendor) responded more rapidly than Microsoft or Sun t=
o advisories; Sun had fewer advisories to respond to yet took the longest=
 to respond. Another data point is that SecurityPortal has compiled a lis=
t of the time it takes for vendors to respond to vulnerabilities. They co=
ncluded that:

How did our contestants [fare]? Red Hat had the best score, with 348 rece=
ss days on 31 advisories, for an average of 11.23 days from bug to patch.=
 Microsoft had 982 recess days on 61 advisories, averaging 16.10 days fro=
m bug to patch. Sun proved itself to be very slow, although having only 8=
 advisories it accumulated 716 recess days, a whopping three months to fi=
x each bug on average.

Their table of data for 1999 is as shown:

1999 Advisory Analysis

Vendor | Total Days of Hacker Recess | Total Advisories | Days of Recess =
per Advisory
Red Hat | 348 | 31 | 11.23
Microsoft | 982 | 61 | 16.10
Sun | 716 | 8 | 89.50


Clearly this table uses a different method for counting security problems=
 than the previous table. Of the three noted here, Sun's Solaris had the =
fewest vulnerabilities, but it took by far the longest to fix security pr=
oblems identified. Red Hat was the fastest at fixing security problems, a=
nd placed in the middle of these three in number of vulnerabilities. It's=
 worth noting that the OpenBSD operating system (which is OSS/FS) had few=
er reported vulnerabilities than all of these. Clearly, having a propriet=
ary operating system doesn't mean you're more secure - Microsoft had the =
largest number of security advisories, by far, using either counting meth=
od.


Apache has a better security record than Microsoft's IIS, as measured by =
reports of serious vulnerabilities. Eweek's July 20, 2001 article ``Apach=
e avoids most security woes'' examined security advisories dating back to=
 Apache 1.0. They found that Apache's last serious security problem (one =
where remote attackers could run arbitrary code on the server) was announ=
ced in January 1997. A group of less serious problems (including a buffer=
 overflow in the server's logresolve utility) was announced and fixed in =
January 1998 with Apache 1.2.5. In the three and a half years since then,=
 Apache's only remote security problems have been a handful of denial-of-=
service and information leakage problems (where attackers can see files o=
r directory listings they shouldn't).

In contrast, in the article ``IT bugs out over IIS security,'' eWeek dete=
rmined that Microsoft has issued 21 security bulletins for IIS from Janua=
ry 2000 through June 2001. Determining what this number means is a little=
 difficult, and the article doesn't discuss these complexities, so I've e=
xamined Microsoft's bulletins myself to find their true significance. Not=
 all of the bulletins have the same significance, so just stating that th=
ere were ``21 bulletins'' doesn't give the whole picture. However, it's c=
lear that several of these bulletins discuss dangerous vulnerabilities th=
at allow an external user to gain control over the system. I count 5 bull=
etins on such highly dangerous vulnerabilities for IIS 5.0 (in the period=
 from January 2000 through June 2001), and previous to that time, I count=
 3 such bulletins for IIS 4.0 (in the period of June 1998 through Decembe=
r 1999). Feel free to examine the bulletins yourself; they are MS01-033, =
MS01-026, MS01-025, MS01-023, MS00-086, MS99-025, MS99-019, and MS99-003.=
 The Code Red worm, for example, exploited a vast number of IIS sites thr=
ough the vulnerabilities identified in the June 2001 security bulletin MS=
01-033.

In short, by totaling the number of reports of dangerous vulnerabilities =
(that allow attackers to execute arbitrary code), I find a total of 8 bul=
letins for IIS from June 1998 through June 2001, while Apache had zero su=
ch vulnerabilities for that time period. Apache's last such report was in=
 January 1998, and that one affected the log analyzer not the web server =
itself. As was noted above, the last such dangerous vulnerability in Apac=
he itself was announced in January 1997.

Even this doesn't give the full story, however; a vulnerability in IIS is=
 far more dangerous than an equivalent vulnerability in Apache, because A=
pache wisely follows the good security practice of ``least privilege.'' I=
IS is designed so that anyone who takes over IIS can take over the entire=
 system, performing actions such as reading, modifying, or erasing any fi=
le on the system. In contrast, Apache is installed with very few privileg=
es by default, so even taking over Apache gives attackers relatively few =
privileges. For example, cracking Apache does not give attackers the righ=
t to modify or erase most files. This is still not good, of course, and a=
n attacker may be able to find another vulnerability to give them complet=
e access, but an Apache system presents more challenges to an attacker th=
an IIS.

The article claims there are four reasons for Apache's strong security, a=
nd three of these reasons are simply good security practices. Apache inst=
alls very few server extensions by default (a ``minimalist'' approach), a=
ll server components run as a non-privileged user (supporting ``least pri=
vilege'' as noted above), and all configuration settings are centralized =
(making it easy for administrators to know what's going on). However, the=
 article also claims that one of the main reasons Apache is more secure t=
han IIS is that its ``source code for core server files is well-scrutiniz=
ed,'' a task that is made much easier by being OSS/FS, and it could be ar=
gued that OSS/FS encourages the other good security practices.

Simple counts of vulnerability notices aren't necessarily a good measure,=
 of course. A vendor could intentionally release fewer bulletins - but si=
nce Apache's code and its security is publicly discussed, it seems unlike=
ly that Apache is releasing fewer notices. Fewer vulnerability notices co=
uld result if the product isn't well scrutinized or is rarely used - but =
this simply isn't true for Apache. Even the trend line isn't encouraging =
- using the months of the bulletins (2/99, 6/99, 7/99, 11/00, three in 5/=
01, and 6/01), I find the time in months between new major IIS vulnerabil=
ity announcements to be 4, 1, 18, 6, 0, 0, 1, and 3 as of September 2001;=
 this compares to 12 and 44 as of September 2001 for Apache. Given these =
trends, it looks like IIS's security is slowly improving, but it has litt=
le likelihood of meeting Apache's security in the near future. Indeed, th=
ese vulnerability counts are corroborated by other measures such as the w=
eb site defacement rates.

The issue here isn't whether or not a particular program is invincible (w=
hat nonsense!) - the issue here is which is more likely to resist future =
attacks, based on past performance. It's clear that Apache has much a bet=
ter security record than IIS, so much so that Gartner Group decided to ma=
ke an unusual recommendation (described next).


The Gartner Group is recommending that businesses switch from Microsoft I=
IS to Apache or iPlanet due to IIS's poor security track record, noting t=
hat enterprises had spent $1.2 billion simply fixing Code Red (IIS-relate=
d) vulnerabilities by July 2001. Microsoft's IIS has such a bad security =
record that in September 2001, Gartner Group announced a recommendation t=
hat ``businesses hit by both Code Red and Nimda immediately investigate a=
lternatives to IIS, including moving Web applications to Web server softw=
are from other vendors such as iPlanet and Apache. Although those Web ser=
vers have required some security patches, they have much better security =
records than IIS and are not under active attack by the vast number of vi=
rus and worm writers.'' Microsoft is sometimes a Gartner Group customer, =
so this announcement is especially surprising.

In a background document by Gartner, they discuss Code Red's impacts furt=
her. By July 2001, Computer Economics (a research firm) estimated that en=
terprises worldwide had spent $1.2 billion fixing vulnerabilities in thei=
r IT systems that Code Red could exploit (remember, Code Red is designed =
to only attack IIS systems; systems such as Apache are immune). To be fai=
r, Gartner correctly noted that the problem is not just that IIS has vuln=
erabilities; part of the problem is that enterprises using IIS are not ke=
eping their IT security up to date, and Gartner openly wondered why this =
was the case. However, Gartner also asked the question, ``why do Microsof=
t's software products continue to provide easily exploited openings for s=
uch attacks?'' This was prescient, since soon after this the ``Nimba'' at=
tack surfaced which attacked IIS, Microsoft Outlook, and other Microsoft =
products.

A brief aside is in order here. Microsoft spokesman Jim Desler tried to c=
ounter Gartner's recommendation, trying to label it as ``extreme'' and sa=
ying that ``serious security vulnerabilities have been found in all Web s=
erver products and platforms.. this is an industrywide challenge.'' While=
 true, this isn't the whole truth. As Gartner points out, ``IIS has a lot=
 more security vulnerabilities than other products and requires more care=
 and feeding.'' It makes sense to select the product with the best securi=
ty track record, even if no product has a perfect record.


The majority of the most serious security problems only apply to Microsof=
t's products, and not to OSS/FS products, as suggested by the CERT/CC's `=
`most frequent, high-impact types of security incidents and vulnerabiliti=
es'' and the ICAT database. Some security vulnerabilities are more import=
ant than others, for a variety of reasons. Thus, some analysis centers tr=
y to determine what's ``most important,'' and their results suggest that =
OSS/FS just doesn't have as many vulnerabilities.

The CERT Coordination Center (CERT/CC) is federally funded to study secur=
ity vulnerabilities and perform related activities such as publishing sec=
urity alerts. I sampled their list of ``current activity'' of the most fr=
equent, high-impact security incidents and vulnerabilities on September 2=
4, 2001, and found yet more evidence that Microsoft's products have poor =
security compared to others (including OSS/FS). Four of the six most impo=
rtant security vulnerabilities were specific to Microsoft: W32/Nimda, W32=
/Sircam, cache corruption on Microsoft DNS servers, and ``Code Red'' rela=
ted activities. Only one of the six items primarily affected non-Microsof=
t products (a buffer overflow in telnetd); while this particular vulnerab=
ility is important, it's worth noting that many open source systems (such=
 as Red Hat 7.1) normally don't enable this service (telnet) in the first=
 place and thus are less likely to be vulnerable. The sixth item (``scans=
 and probes'') is a general note that there is a great deal of scanning a=
nd probing on the Internet, and that there are many potential vulnerabili=
ties in all systems. Thus, 4 of 6 issues are high-impact vulnerabilities =
are specific to Microsoft, 1 of 6 are vulnerabilities primarily affecting=
 Unix-like systems (including OSS/FS operating systems), and 1 of 6 is a =
general notice about scanning. Again, it's not that OSS/FS products never=
 have security vulnerabilities - but they seem to have fewer of them.

The ICAT system provides a searchable index and ranking for the vulnerabi=
lities cross-references by CVE. I sampled its top ten list on December 19=
, 2001; this top ten list is defined by the number of requests made for a=
 particular vulnerability in ICAT (and including only vulnerabilities wit=
hin the last year). In this case, 8 of the top 10 vulnerabilities only af=
fect proprietary systems (in all cases, Windows). Only 2 of 10 affect OSS=
/FS systems (#6, CAN-2001-0001, a weakness in PHP-Nuke 4.4, and #8, CVE-2=
001-0013, a new vulnerability found in an old version of BIND - BIND 4). =
Obviously, by itself this doesn't prove that there are fewer serious vuln=
erabilities in OSS/FS programs, but it is suggestive of it.


Computer viruses are overwhelmingly more prevalent on Windows than any ot=
her system. Virus infection has been a major cost to users of Microsoft W=
indows. The LoveLetter virus alone is estimated to have cost $960 million=
 in direct costs and $7.7 billion in lost productivity, and the anti-viru=
s software industry sales total nearly $1 billion annually. Dr Nic Peelin=
g and Dr Julian Satchell's Analysis of the Impact of Open Source Software=
 includes an analysis of the various data sources for virus counts. Here =
is what they said:

The numbers differ in detail, but all sources agree that computer viruses=
 are overwhelmingly more prevalent on Windows than any other system. Ther=
e are about 60,000 viruses known for Windows, 40 or so for the Macintosh,=
 about 5 for commercial Unix versions, and perhaps 40 for Linux. Most of =
the Windows viruses are not important, but many hundreds have caused wide=
spread damage. Two or three of the Macintosh viruses were widespread enou=
gh to be of importance. None of the Unix or Linux viruses became widespre=
ad - most were confined to the laboratory.

Why is Windows disproportionately vulnerable? There are three reasons, on=
e social and two technical. Windows is much the most attractive target fo=
r virus writers, simply because it is in such widespread use. For a virus=
 to spread, it has to transmit itself to other susceptible computers; on =
average, each infection has to cause at least one more. The ubiquity of W=
indows machines makes it easier for this threshold to be reached. Finally=
, Windows has had a number of design choices over the years (e.g. executi=
on of start-up macros in Word, execution of attachments in Outlook, lack =
of write protection on system directories in Windows 3.1/95/98) that have=
 allowed the execution of untrusted code, and this has made it a very eas=
y target.

While it's possible to write a virus for OSS/FS systems, their design mak=
es it more difficult for them to spread. It appears that OSS/FS developer=
s tend to select design choices that limit the damage of viruses, perhaps=
 in part because their code is subject to public inspection and comment. =
For example, OSS/FS programs generally do not support start-up macros nor=
 execution of mail attachments that can be controlled by attackers, and l=
eading OSS/FS operating systems (such as GNU/Linux and the *BSDs) have al=
ways had write protection on system directories.


According to a Network Security evaluation, an OSS/FS vulnerability scann=
er (Nessus) was found to be the best (most effective). On January 8, 2001=
, Network Computing's article Vulnerability Assessment Scanners. reported=
 an evaluation of nine network scanning tools, most of them proprietary. =
In their evaluation, Network Computing set up demonstration systems with =
17 of the most common and critical vulnerabilities; they then used the va=
rious network scanning tools to see how effectively each of the tools det=
ected these vulnerabilities. Sadly, not one product detected all vulnerab=
ilities; the best scanner was the OSS/FS program Nessus Security Scanner,=
 which found 15 of the 17 (which also received their top total score); th=
e next best was a proprietary scanner which only found 13.5 out of 17.

In their words,

Some of us were a bit skeptical of the open-source Nessus project's thoro=
ughness until [Nessus] discovered the greatest number of vulnerabilities.=
 That's a hard fact to argue with, and we are now eating our words ... [N=
essus] got the highest overall score simply because it did more things ri=
ght than the other products.

I agree with the authors that ideally a network vulnerability scanner sho=
uld find every well-known vulnerability, and that ``even one hole is too =
many.'' Still, perfection is rare in the real world. More importantly, a =
vulnerability scanner should only be part of the process to secure an org=
anization - it shouldn't be the sole activity. Still, this evaluation sug=
gests that an organization will be more secure, not less secure, by using=
 an OSS/FS program. It could be argued that this simply shows that this p=
articular OSS/FS program had more functionality - not more security - but=
 in this case, the product's sole functionality was to improve security.

One serious problem is that there are strong economic disincentives for p=
roprietary vendors to make their software secure. For example, if vendors=
 make their software more secure, they would often fail to be ``first'' i=
n a given market; this often means that they will lose that market. Since=
 it is exteremly difficult for customers to distinguish proprietary softw=
are with strong security from those with poor security, the poor products=
 tend to eliminate the good ones (after all, they're cheaper to develop a=
nd thus cost less). Governments have other disincentives as well. For a d=
iscussion of some of the economic disincentives for secure software, see =
Why Information Security is Hard - an Economic Perspective by Ross Anders=
on (Proceedings of the Annual Computer Security Applications Conference (=
ACSAC), December 2001, pp. 358-365). It's not clear that OSS/FS always av=
oids these disintentives, but it appears in at least some cases it does. =
For example, OSS/FS source code is public, so the difference in security =
is far more visible than in proprietary products.

One of the most dangerous security problems with proprietary software is =
that if intentionally malicious code is snuck into it, such code is extre=
mely difficult to find. Few proprietary vendors have other developers exa=
mine all code in great detail - their testing processes are designed to c=
atch mistakes (not malice) and often don't look at the code at all. In co=
ntrast, malicious code can be found by anyone when the source code is pub=
licly available, and with OSS/FS, there are incentives for arbitrary peop=
le to review it (such as to add new features or perform a security review=
). Thus, someone inserting malicious code to an OSS/FS project runs a far=
 greater risk of detection. Here are two examples, one confirmed, one not=
 confirmed:

Some time between 1992 and 1994, Borland inserted an intentional ``back d=
oor'' into their database server, ``InterBase'', as a secret username and=
 fixed password. This back door allowed any local or remote user to manip=
ulate any database object and install arbitrary programs, and in some cas=
es could lead to controlling the machine as ``root''. This vulnerability =
stayed in the product for at least 6 years - no one else could review the=
 product, and Borland had no incentive to remove the vulnerability. Then =
Borland released its source code on July 2000 as an OSS/FS project. The "=
Firebird" project began working with the source code, and uncovered this =
serious security problem with InterBase in December 2000 (only 5 months a=
fter release). By January 2001 the CERT announced the existence of this b=
ack door as CERT advisory CA-2001-01. What's discouraging is that the bac=
kdoor can be easily found simply by looking at an ASCII dump of the progr=
am (a common cracker trick), so it's quite possible that this vulnerabili=
ty was exploited many times in the intervening years. Once this problem w=
as found by open source developers reviewing the code, it was patched qui=
ckly.

Mohammad Afroze Abdul Razzak, arrested by Mumbai (Bombay) police Oct. 2, =
2001, claims that Osama bin Laden's Al Qaeda network were able to gain em=
ployment at Microsoft and attempted to plant "trojans, trapdoors, and bug=
s in Windows XP." This was reported to Ravi Visvesvaraya Prasad, a New De=
lhi information systems and telecommunication consultant, and then report=
ed by the Washington Post's Newsbytes division. This claim has not been c=
onfirmed; indeed, I'm somewhat skeptical. The problem, however, is that t=
his is impossible to disprove. Even if this particular case isn't true, n=
ote that this threat is unfortunately a credible threat to proprietary so=
ftware, because very few of its users can review the code. This is far le=
ss dangerous to OSS/FS software, because of the worldwide review that's p=
ossible (including the ability to see the changes made in each version).

Now it should be obvious from these figures that OSS/FS systems are not m=
agically invincible from security flaws. Indeed, some have argued that ma=
king the source code available gives attackers an advantage (because they=
 have more information to make an attack). While OSS/FS gives attackers m=
ore information, this ignores opposing forces: having the source code als=
o gives the defenders more information (because they can also examine its=
 original source code), and in addition, the defenders can improve the co=
de. For a longer description of these issues, see my discussion on open s=
ource and security (part of my book on writing secure software). However,=
 from these figures, it appears that OSS/FS systems are often better - no=
t just equal - in their resistance to attacks.


Total Cost of Ownership (TCO)

Total cost of ownership (TCO) is an important measure; it doesn't matter =
if a product starts out cheaply if it costs you more down the line. Howev=
er, TCO is extremely sensitive to the set of assumptions you make.

Indeed, whatever product you use or support, you can probably find a stud=
y to show it has the lowest TCO for some circumstance. Not surprisingly, =
both Microsoft and Sun provide studies showing that they have the lowest =
TCO (but see my comments later about Microsoft's study). Xephon has a stu=
dy determining that mainframes are the cheapest per-user (due to centrali=
zed control) at =A33450 per user per year; Centralized Unix cost =A37350 =
per user per year, and a decentralized PC environment costs =A310850 per =
user per year. Xephon appears to be a mainframe-based consultancy, though=
, and would want the results to come out this way. There are indeed situa=
tions where applying a mainframe makes sense.. but as we'll see in a mome=
nt, you can use OSS/FS in such environments too.

In short, what has a smaller TCO depends on your environment and needs. T=
o determine TCO you have to identify all the important cost drivers (the =
``cost model'') and estimate their costs. Don't forget ``hidden'' costs, =
such as administration costs, upgrade costs, technical support, end-user =
operation costs, and so on. However, OSS/FS has a number of strong cost a=
dvantages in various categories that, in many cases, will result in its h=
aving the smallest TCO.


OSS/FS costs less to initially acquire. OSS/FS costs much less to get ini=
tially. OSS/FS isn't really ``free'' in the monetary sense to get; the ``=
free'' in ``free software'' refers to freedom, not price (usually summari=
zed as ``free speech, not free beer''). You'll still spend money for pape=
r documentation, support, training, system administration, and so on, jus=
t as you do with proprietary systems. In many cases, the actual programs =
in OSS/FS distributions can be acquired freely by downloading them (linux=
=2Eorg provides some pointers on how to get distributions). However, most=
 people (especially beginners and those without high-speed Internet conne=
ctions) will want to pay a small fee to a distributor for a nicely integr=
ated package with CD-ROMs, paper documentation, and support. Even so, OSS=
/FS is far less expensive to acquire.

For example, look at some of the price differences when trying to configu=
re a server (say a public web server or an intranet file and email server=
, in which you'd like to use C++ and an RDBMS for some portions of it). T=
his is an example, of course; different missions would involve different =
components. I used the prices from ``Global Computing Supplies'' (Suwanee=
, GA), September 2000, and rounded to the nearest dollar. Here's a quick =
summary of some costs:


Microsoft Windows 2000 vs. Red Hat Linux
Operating System | $1510 (25 client) | $29 (standard), $76 deluxe, $156 p=
rofessional (all unlimited)
Email Server | $1300 (10 client) | included (unlimited)
RDBMS Server | $2100 (10 CALs) | included (unlimited)
C++ Development | $500 | included

Basically, Microsoft Windows 2000 (25 client) costs $1510; their email se=
rver Microsoft Exchange (10-client access) costs $1300, their RDBMS serve=
r SQL Server 2000 costs $2100 (with 10 CALs), and their C++ development s=
uite Visual C++ 6.0 costs $500. Red Hat Linux 6.2 (a widely-used GNU/Linu=
x distribution) costs $29 for standard (90 days email-based installation =
support), $76 for deluxe (above plus 30 days telephone installation suppo=
rt), or $156 for professional (above plus SSL support for encrypting web =
traffic); in all cases it includes all of these functionalities (web serv=
er, email server, database server, C++, and much more). A public web serv=
er with Windows 2000 and an RDBMS might cost $3610 ($1510+$2100) vs. Red =
Hat Linux's $156, while an intranet server with Windows 2000 and an email=
 server might cost $2810 ($1510+$1300) vs. Red Hat Linux's $76.

Both packages have functionality the other doesn't have. The GNU/Linux sy=
stem always comes with an unlimited number of licenses; the number of cli=
ents you'll actually use depends on your requirements. However, this cert=
ainly shows that no matter what, Microsoft's server products cost thousan=
ds of dollars more per server than the equivalent GNU/Linux system.

For another in-depth analysis comparing the initial costs GNU/Linux with =
Windows, see Linux vs. Windows: The Bottom Line by Cybersource Pty Ltd. H=
ere's a summary of their analysis (in 2001 U.S. dollars):

Company | Microsoft Solution | OSS/FS (GNU/Linux) Solution | Savings by u=
sing GNU/Linux
Company A (50 users) | $69,987 | $80 | $69,907
Company B (100 users) | $136,734 | $80 | $136,654
Company C (250 users) | $282,974 | $80 | $282,894

Consulting Times found that as the number of mailboxes got large, the thr=
ee-year TCO for mainframes with GNU/Linux became in many cases quite comp=
elling. For 50,000 mailboxes, an Exchange/Intel solution cost $5.4 millio=
n, while the Linux/IBM(G6) solution cost $3.3 million. For 5,000 mailboxe=
s, Exchange/Intel cost $1.6 million, while Groupware on IFL cost $362,890=
=2E For yet another study, see the Cost Comparison from jimmo.com. Obviou=
sly, the price difference depends on exactly what functions you need for =
a given task, but for many common situations, GNU/Linux costs far less to=
 acquire.


Upgrade costs are typically far less. Long-term upgrade costs are far les=
s for OSS/FS systems. For example, upgrading a Microsoft system will typi=
cally cost around half the original purchase. What's worse, you are essen=
tially at their mercy for long-term pricing, because there is only a sing=
le supplier (see Microsoft Turns the Screws). In contrast, the GNU/Linux =
systems can be downloaded (free), or simply re-purchased (generally for l=
ess than $100), and the single upgrade be used on every system. This does=
n't include technical support, but the technical support can be competed =
(a situation that's not practical for proprietary software). If you don't=
 like your GNU/Linux supplier (e.g., they've become too costly), you can =
switch.

OSS/FS can often use older hardware more efficiently than proprietary sys=
tems, yielding smaller hardware costs and sometimes eliminating the need =
for new hardware. OSS/FS runs faster on faster hardware, of course, but m=
any OSS/FS programs can use older hardware more efficiently than propriet=
ary systems, resulting in lower hardware costs - and in some cases requir=
ing no new costs (because ``discarded'' systems can suddenly be used agai=
n). For example, the minimum requirements for Microsoft Windows 2000 Serv=
er (according to Microsoft) are a Pentium-compatible CPU (133 MHz or high=
er), 128 MiB of RAM minimum (with 256MiB the ``recommended minimum''), an=
d a 2 GB hard drive with at least 1.0 GB free. According to Red Hat, Red =
Hat Linux 7.1 (a common distribution of GNU/Linux) requires at a minimum =
an i486 (Pentium-class recommended), 32MiB RAM (64MiB recommended), and 6=
50MB hard disk space (1.2 GB recommended).

In Scientific American's August 2001 issue, the article The Do-It-Yoursel=
f Supercomputer discusses how the researchers built a powerful computing =
platform with a large number of obsolete, discarded computers and GNU/Lin=
ux. The result was dubbed the ``Stone Soupercomputer''; by May 2001 it co=
ntained 133 nodes, with a theoretical peak performance of 1.2 gigaflops.


As the number of systems and hardware performance increases, this differe=
nce in initial and upgrade costs becomes even more substantial. As the nu=
mber of servers increases, proprietary solutions become ever more expensi=
ve. First, many proprietary systems (including Microsoft) sell per-client=
 licenses; this means that even if your hardware can support more clients=
, you'll have to pay more to actually use the hardware you've purchased. =
Secondly, if you want to use more computers, you have to pay for more lic=
enses in proprietary systems. In contrast, for most GNU/Linux distributio=
ns, you can install as many copies as you like for no additional fee, and=
 there's no performance limit built into the software. There may be a fee=
 for additional support, but you can go to competing vendors for this sup=
port.

According to Network World Fusion News, Linux is increasingly being used =
in healthcare, finance, banking, and retail because of its cost advantage=
s when large numbers of identical sites and servers are built. According =
to their calculations for a 2,000 site deployment, SCO UnixWare would cos=
t $9 million, Windows would cost $8 million, and Red Hat Linux costs $180=
=2E


There are many other factors; their effect varies on what you're trying t=
o do. There are many other factors in TCO, but it's difficult to categori=
ze their effects in general, and it's generally difficult to find justifi=
able numbers for these other effects. Windows advocates claim that system=
 administrators are cheaper and easier to find than Unix/Linux administra=
tors, while GNU/Linux and Unix advocates argue that fewer such administra=
tors are needed (because administration is easier to automate and the sys=
tems are more reliable to start with). Some GNU/Linux advocates have told=
 me that GNU/Linux lends itself to hosting multiple services on a single =
server in cases where Windows installations must use multiple servers. Li=
cense compliance administration can be costly for proprietary systems (e.=
g., time spent by staff to purchase CALS, keep track of licenses, and und=
ergo audits) - a cost that simply isn't relevant to OSS/FS.

For many circumstances, the total cost savings can be substantial; for ex=
ample, savings exceeding $250,000 per year were reported by 32% of the CT=
Os surveyed in a 2001 InfoWorld survey; 60% of these CTOs saved more than=
 $50,000 annually. The August 27, 2001 InfoWorld (pages 49-50) reported o=
n a survey of 40 CTOs who were members of the InfoWorld CTO network. In t=
his survey, 32% using OSS reported savings greater than $250,000; 12% rep=
orted savings between 100,001 and $250,000; and 16% reported saving betwe=
en $50,001 and $100,000. Indeed, only 8% reported annual savings less tha=
n $10,000 (so 92% were saving $10,000 or more annually). A chief benefit =
of OSS, according to 93% of the CTOs, was reduced cost of application dev=
elopment or acquisition; 72% said that a chief benefit was reduced develo=
pment or implementation time (multiple answers were allowed). The CTOs re=
ported using or planning to use OSS for web servers (65%), server operati=
ng systems (63%), web-application servers (45%), application development =
testing (45%), and desktop operating system (38%), among other uses. Info=
World summarized it this way: ``in early 2000, it seemed as if no one was=
 using open-source software for business-critical tasks... a vast majorit=
y of today's corporate IT executives are now using or plan to use OSS ope=
rating systems and web servers for their enterprise applications.''

Many organizations have reported significant savings when using OSS/FS. H=
ere are a few examples of specific organizations saving money through OSS=
/FS:

The paper Linux as a Replacement for Windows 2000 is an example of an ana=
lysis comparing Red Hat Linux 7.1 to Windows 2000; in this customer's cas=
e, using Linux instead of Windows 2000 saved $10,000. The reviewer came f=
rom a Windows/DOS background, and after performing an intensive hands-on =
Linux project lasting several months, determined that ``you will be stunn=
ed by the bang for the buck that ... open source software offers.''

Intel's IT Vice President, Doug Busch, reported savings of $200 million b=
y replacing expensive Unix servers with cheaper servers running GNU/Linux=
=2E

Amazon.com was able to cut $17 million in technology expenses in a single=
 quarter, largely because of a switch to Linux. Amazon spent $54 million =
on technology and content expenses in its third quarter (ending Sept. 30)=
, compared with $71 million in the year-ago quarter, and executives expec=
ted that technology costs as a portion of net sales would decrease by 20%=
 this year.

There are many other reports from those who have switched to OSS/FS syste=
ms; see http://www.dwheeler.com/oss_fs_why.html#usereports for more infor=
mation.

Microsoft's TCO study (mentioned earlier) is probably not useful as a sta=
rting point for estimating your own TCO. Their study reported the average=
 TCO at sites using Microsoft products compared to the average TCO at sit=
es using Sun systems, but although the Microsoft systems cost 37% less to=
 own, the Solaris systems handled larger databases, more demanding applic=
ations connecting to those databases, 63% more concurrent connections, an=
d 243% more hits per day. In other words, the Microsoft systems that did =
less work were less expensive. This is not a useful starting point if you=
're using TCO to help determine which system to buy -- to make a valid co=
mparison by TCO, you need to compare the TCOs of systems that both perfor=
m the job that you need to do. A two-part analysis by Thomas Pfau (see pa=
rt 1 and part 2) identifies this and many other flaws in the study.

Again, it's TCO that matters, not just certain cost categories. However, =
given these large differences, in many situations OSS/FS has a smaller TC=
O than proprietary systems. At one time it was claimed that OSS/FS instal=
lation took more time, but nowadays OSS/FS systems can be purchased pre-i=
nstalled and automatic installers result in equivalent installation labor=
=2E Some claim that system administration costs are higher, but studies l=
ike Sun's suggest than in many cases the system administration costs are =
lower, not higher, for Unix-like systems (at least Sun's). For example, o=
n Unix-like systems it tends to be easier to automate tasks (because you =
can, but do not need, to use a GUI) - thus over time many manual tasks ca=
n be automated (reducing TCO). Retraining costs can be significant - but =
now that GNU/Linux has modern GUI desktop environments, there's anecdotal=
 evidence that this cost is actually quite small (I've yet to see serious=
 studies quantitatively evaluating this issue). In short, it's often hard=
 to show that a proprietary solution's purported advantages really help o=
ffset their demonstrably larger costs in other categories when there's a =
competing mature OSS/FS product for the given function.


Does this mean that OSS/FS always have the lowest TCO? No! As I've repeat=
edly noted, it depends on its use. But the notion that OSS/FS always has =
the larger TCO is simply wrong.


Non-Quantitative Issues

In fairness, I must note that not all issues can be quantitatively measur=
ed, and to many they are the most important issues. The issues most impor=
tant to many include freedom, protection from license litigation, and fle=
xibility. Another issue that's hard to measure is innovation.


OSS/FS protects its users from the risks and disadvantages of single sour=
ce solutions. While ``free software'' advocates use the term ``freedom,''=
 and some businesses emphasize different terms such as ``multiple sources=
,'' the issue is the same: users do not want to be held hostage by any on=
e vendor. Businesses often prefer to buy products in which there is a lar=
ge set of competing suppliers, because it reduces their risk; they can al=
ways switch to another supplier if they're not satisfied or the original =
supplier goes out of business. This translates into an effect on the prod=
ucts themselves: if customers can easily choose and switch between compet=
ing products, the products' prices go down and their quality goes up. Con=
versely, if there is a near monopoly for a given product, over time the v=
endor will raise the cost to use the product and limit its uses to those =
that benefit the monopolist.

Historically, proprietary vendors eventually lose to vendors selling prod=
ucts available from multiple sources, even when their proprietary technol=
ogy is (at the moment) better. Sony's Betamax format lost to VHS in the v=
ideotape market, IBM's microchannel architecture lost to ISA in the PC ar=
chitecture market, and Sun's NeWS lost to X-windows in the networking gra=
phics market, all because customers prefer the reduced risk (and eventual=
ly reduced costs) of non-proprietary products. This is sometimes called `=
`commodification'', a term disparaged by proprietary vendors and loved by=
 users. Since users spend the money, users eventually find someone who wi=
ll provide what they want.

With OSS/FS, users can choose between distributors, and if a supplier aba=
ndons them they can switch to another supplier. As a result, suppliers wi=
ll be forced to provide good quality products and services for relatively=
 low prices, because users can switch if they don't. Users can even band =
together and maintain the product themselves (this is how the Apache proj=
ect was founded), making it possible for groups of users to protect thems=
elves from abandonment.


OSS/FS protects its users from licensing management and litigation. Propr=
ietary vendors make money from the sale of licenses, and are imposing inc=
reasingly complex mechanisms on consumers to manage these licenses. For e=
xample, Microsoft's Windows XP requires product activation - a scheme tha=
t means that an accumulation of hardware changes requires a new activatio=
n code.

Proprietary vendors also litigate against those who don't comply with the=
ir complex licensing management requirements, creating increased legal ri=
sks for users. For example, the Business Software Alliance (BSA) is a pro=
prietary software industry organization sponsored by Microsoft, Macromedi=
a, and Autodesk, and spends considerable time searching for and punishing=
 companies who cannot prove they are complying. As noted in the SF Gate (=
Feb. 7, 2002), the BSA encourages disgruntled employees to call the BSA i=
f they know of any license violations. "If the company refuses to settle =
or if the BSA feels the company is criminally negligent and deliberately =
ripping off software, the organization may decide to get a little nastier=
 and organize a raid: The BSA makes its case in front of a federal court =
in the company's district and applies for a court order. If the order is =
granted, the BSA can legally storm the company's offices, accompanied by =
U.S. marshals, to search for unregistered software."

In contrast, OSS/FS users have no fear of litigation from the use and cop=
ying of OSS/FS. Licensing issues do come up when OSS/FS software is modif=
ied and then redistributed, but to be fair, proprietary software essentia=
lly forbids this action (so it's a completely new right). Even in this ci=
rcumstance, redistributing modified OSS/FS software generally requires fo=
llowing only a few simple rules (depending on the license), such as givin=
g credit to previous developers and releasing modifications under the sam=
e license as the original program.


OSS/FS has greater flexibility. OSS/FS users can tailor the product as ne=
cessary to meet their needs in ways not possible without source code. Use=
rs can tailor the product themselves, or hire whoever they believe can so=
lve the problem (including the original developer). Some have claimed tha=
t this creates the ``danger of forking,'' that is, of multiple incompatib=
le versions of a product. This is ``dangerous'' only to those who believe=
 competition is evil - we have multiple versions of cars as well. And in =
practice, the high cost of maintaining software yourself has resulted in =
a process in which the change is contributed back to the community. If it=
's not contributed (e.g., it solves a problem that needed solving but onl=
y for a particular situation), then it's still a win for the user - becau=
se it solved a user's problem which would not have been solved otherwise.=


There are good reasons to believe OSS/FS encourages, not quashes, innovat=
ion. Microsoft publicly claims that OSS/FS (in particular its most common=
 license, the GPL) will eliminate innovation, but the facts undermine the=
se claims. Most IT managers don't believe these claims by Microsoft; in 2=
000 a Forrester Research study interviewed 2,500 IT managers and found th=
at 84% of them predicted that open source software would be the spark beh=
ind major innovations throughout the industry. Indeed, when examining the=
 most important software innovations, it's quickly discovered that Micros=
oft invented no key innovations, nor was Microsoft the first implementor =
of any of them. In contrast, a number of the key innovations were OSS/FS =
projects. For example, Tim Berners-Lee, inventor of the World Wide Web, s=
tated in December 2001 that `` A very significant factor [in widening the=
 Web's use beyond scientific research] was that the software was all (wha=
t we now call) open source. It spread fast, and could be improved fast - =
and it could be installed within government and large industry without ha=
ving to go through a procurement process.'' Note that this didn't end aft=
er the ideas were originally developed; the #1 web server in 2001 (Apache=
) is open source and the #2 web browser in 2001 (Netscape Navigator) is a=
lmost entirely open source, ten years after the original development of t=
he web. Indeed, recent court cases give strong evidence that the only rea=
son the proprietary Internet Explorer was the #1 web browser was due to y=
ears of illegal use of monopoly power by Microsoft.

This history of innovation shouldn't be surprising; OSS/FS approaches are=
 based on the scientific method, allowing anyone to make improvements or =
add innovative techniques and then make them immediately available to the=
 public. Eric Raymond has made a strong case for why innovation is more l=
ikely, not less likely, in OSS/FS projects. The Sweetcode web site report=
s on innovative free software. Here's what Sweetcode says about their sit=
e: ``Innovative means that the software reported here isn't just a clone =
of something else or a minor add-on to something else or a port of someth=
ing else or yet another implementation of a widely recognized concept... =
Software reported on sweetcode should surprise you in some interesting wa=
y.''

If Microsoft's proprietary approaches were better for research, then you =
would expect that to be documented in the research community. However, th=
e opposite is true; the paper ``NT Religious Wars: Why Are DARPA Research=
ers Afraid of Windows NT?'' found that, in spite of strong pressure by pa=
ying customers, computer science researchers strongly resisted basing res=
earch on Windows. Reasons given were: developers believe Windows is terri=
ble, Windows really is terrible, Microsoft's highly restrictive non-discl=
osure agreements are at odds with researcher agendas, and there is no cle=
ar technology transition path for operating system and network research p=
roducts built on Windows (since only Microsoft can distribute changes to =
its products). Microsoft's own secret research (later leaked as ``Hallowe=
en I'') found that ``Research/teaching projects on top of Linux are easil=
y `disseminated' due to the wide availability of Linux source. In particu=
lar, this often means that new research ideas are first implemented and a=
vailable on Linux before they are available / incorporated into other pla=
tforms.'' Stanford Law School professor Lawrence Lessig (the "special mas=
ter" in Microsoft's antitrust trial) noted that ``Microsoft was using its=
 power to protect itself against new innovation'' and that Microsoft's pr=
actices generally threaten technical innovation - not promote it.

Given an entire site dedicated to linking to innovative OSS/FS projects, =
OSS/FS's demonstrated history in key innovations, reports from IT manager=
s supporting OSS/FS, reports of dissatisfaction by researchers and others=
 about Microsoft's proprietary approaches, and Microsoft's own research f=
inding that new research ideas are often first implemented and available =
on Linux before other platforms, the claim that OSS/FS quashes innovation=
 is demonstrably false.

While I cannot quantitatively measure these issues, these issues (particu=
larly the first three) are actually the most important issues to many.


Unnecessary Fears

Some avoid OSS/FS, not because of the issues noted earlier, but because o=
f unnecessary fears of OSS/FS. Let's counter some of them:

Is proprietary software fundamentally better supported than OSS/FS? No. T=
here are actually two kinds of support for OSS/FS: traditional paid-for s=
upport and informal community support. There are many organizations who p=
rovide traditional support for a fee; since these can be competed (an opt=
ion not available for proprietary software), you can often get an excelle=
nt price for support. For example, many GNU/Linux distributions include i=
nstallation support when you purchase their distribution, and for a fee t=
hey'll provide additional levels of support. As an alternative, you can a=
lso get unpaid support from the general community of users and developers=
 through newsgroups, mailing lists, web sites, and other electronic forum=
s. While this kind of support is non-traditional, many have been very sat=
isfied with it. Indeed, in 1997 InfoWorld awarded the ``Best Technical Su=
pport'' award to the ``Linux User Community,'' beating all proprietary so=
ftware vendors' technical support. Many believe this is a side-effect of =
the Internet's pervasiveness - increasingly users and developers are dire=
ctly communicating with each other and finding such approaches to be more=
 effective than the alternatives (for more on this business philosophy, s=
ee The Cluetrain Manifesto). Using this non-traditional approach effectiv=
ely for support requires following certain rules; for more on these rules=
, consult ``How to ask smart questions''. But note that there's a choice;=
 using OSS/FS does not require you to use non-traditional support (and fo=
llow its rules), so those who want guaranteed traditional support can pay=
 for it just as they would for proprietary software.

Does proprietary software give you more legal rights than OSS/FS? No. Som=
e have commented that ``with OSS/FS you give up your right to sue if thin=
gs go wrong.'' The obvious retort is that essentially all proprietary sof=
tware licenses also forbid lawsuits - so this isn't a difference at all! =
Anyone who thinks that they can sue Microsoft or other shrink-wrap propri=
etary vendors when things go wrong is simply fooling themselves. In any c=
ase, most users aren't interested in sueing vendors - they want working s=
ystems. See ``A Senior Microsoft Attorney Looks at Open-Source Licensing'=
', where Bryan Pfaffenberger argues that ``With open-source software... y=
ou are, in principle, walking into the deal with your eyes wide open. You=
 know what you're getting, and if you don't, you can find someone who doe=
s. Open-source licenses enable the community of users to inspect the code=
 for flaws and to trade knowledge about such flaws, which they most assur=
edly do. Such licenses allow users to create derivative versions of the c=
ode that repair potentially hazardous problems the author couldn't forese=
e. They let users determine whether the program contains adequate safegua=
rds against safety or security risks. In contrast, the wealthy software f=
irms pushing UCITA are asking us to buy closed-source code that may well =
contain flaws, and even outright hazards attributable to corporate neglig=
ence -- but they won't let us see the code, let alone modify it. You don'=
t know what you're getting.'' Finally, if the software goes wrong and it'=
s very important, you can fix it yourself or pay to have it fixed; this o=
ption greatly reduces risk, and this option doesn't exist for proprietary=
 software.

Does OSS/FS expose you to greater risk of abandonment? No. Businesses go =
out of business, and individuals lose interest in products, in both the p=
roprietary and OSS/FS world. A major difference, however, is that all OSS=
/FS programs are automatically in escrow - that is, if their original dev=
eloper stops supporting the product, any person or group can step forward=
 to support it instead. This has been repeatedly demonstrated in OSS/FS. =
For example, the GIMP is a bitmapped graphical editor that was abandoned =
by its original developers (what's worse, they abandoned it before its in=
itial release and failed to arrange for anyone else to succeed them). Nev=
ertheless, even in this worst-case situation, after a period of time othe=
r users came forward and continued its development. As another example, N=
CSA abandoned its web server ``httpd'', so some of its users banded toget=
her to maintain it - its results became Apache, the world's most popular =
web server.

Is OSS/FS economically viable? Yes. There are companies that are making m=
oney on OSS/FS, and besides, looking only at them looks only at the suppl=
y side and not the demand side. Consumers are saving lots of money and ga=
ining many other benefits by using OSS/FS, so there is a strong economic =
basis for its success. Anyone who is saving money will fight to keep the =
savings, and it's often cheaper for consumers to work together to pay for=
 small improvements in an OSS/FS product than to keep paying and re-payin=
g for a proprietary product. For many, money is still involved - but it's=
 money saved, not money directly acquired as profit. Some OSS/FS vendors =
have done poorly financially - but many proprietary vendors have also don=
e poorly too. Luckily for consumers, OSS/FS products are not tied to a pa=
rticular vendor's financial situation as much as proprietary products are=
=2E

Fundamentally, software is economically different than physical goods; it=
 is infinitely replicable, it costs essentially nothing to reproduce, and=
 it can be developed by thousands of programmers working together with li=
ttle investment (driving the per-person development costs down to very sm=
all amounts). Thus, the marginal cost of deploying a copy of a software p=
ackage quickly approaches zero; this explains how Microsoft got so rich s=
o quickly, and why many OSS/FS developers can afford to give software awa=
y. See ``Open Source-onomics: Examining some pseudo-economic arguments ab=
out Open Source'' by Ganesh Prasad, which counters ``several myths about =
the economics of Open Source.''


Won't OSS/FS destroy the software industry or developers? No. It's certai=
nly possible that many OSS/FS products will eliminate their proprietary c=
ompetition, but that's the nature of competition. No one mourns the loss =
of buggy whip manufacturers, who were driven out of business by a superio=
r approach to transportation (cars). If OSS/FS approaches pose a signific=
ant threat to proprietary development approaches, then proprietary vendor=
s need to either find ways to compete or join the OSS/FS movement. As far=
 as developers go, OSS/FS doesn't require that developers work for free; =
many OSS/FS products are developed or improved by employees (whose job is=
 to do so) and/or by contract work (who contract to make specific improve=
ments in OSS/FS products). Indeed, there has been a recent shift in OSS/F=
S away from volunteer programmers and towards paid development by experie=
nced developers. Again, see Ganesh Prasad's article for more information.=


Is OSS/FS compatible with Capitalism? Yes. Years ago some tried to label =
OSS/FS as ``communistic'' or ``socialistic'' (i.e., anti-capitalist), but=
 that rhetoric has failed. One article explaining why OSS/FS and capitali=
sm are compatible is How Does the Capitalist View Open Source?. This pape=
r shows that OSS/FS is quite consistent with capitalism: it increases wea=
lth without violating principles of property ownership or free will.

Isn't OSS/FS a ``destroyer of intellectual property''? No. You can use OS=
S/FS products (e.g., a word processor) to develop private and proprietary=
 information, and you can keep the information as confidential and propri=
etary as you want. What you can't do is use someone else's material in a =
way forbidden by law.. and this is true for all software, not just OSS/FS=
=2E One interesting case is the ``General Public License'' (GPL), the mos=
t common OSS/FS license. Software covered by the GPL can be modified, but=
 any release of that modified software must include an offer for the sour=
ce code under the same GPL license.

Microsoft complains that the GPL does not allow them to take such code an=
d make changes that it can keep proprietary, but this is hypocritical. Mi=
crosoft doesn't allow others to make and distribute changes to Microsoft =
software at all, so the GPL grants far more rights to customers than Micr=
osoft does. It's true that organizations that modify GPL'ed software must=
 yield any patent and copyright rights for those additions they release, =
but such organizations do so voluntarily (no one can force anyone to modi=
fy GPL code) and with full knowledge (all GPL'ed software comes with a li=
cense clearly stating this). And such grants only apply to those particul=
ar modifications; organizations can hold other unrelated rights if they w=
ish to do so. Since organizations can't make such changes at all to propr=
ietary software in most circumstances, organizations get far more rights =
with the GPL than with proprietary licenses. And though the GPL is someti=
mes called a ``virus'' because of the way it encourages others to also us=
e the GPL license, it's only fair to note that many proprietary products =
(those with ``network effects'') also have virus-like effects. Once some =
users pick a particular product such as a proprietary operating system or=
 word processor, it becomes increasingly difficult for other users to use=
 a different product; over time the use of particular proprietary product=
 also spreads ``like a virus.'' The GPL does not ``destroy'' intellectual=
 property; instead, it creates a level playing field where people can con=
tribute improvements voluntarily without having them ``stolen'' by others=
=2E


Other Information

Here are some other related information sources:

There are many reports from users who have switched to OSS/FS that you ma=
y find useful. Oracle's Chairman and CEO, Larry Ellison, said that Oracle=
 will switch to GNU/Linux to run the bulk of its business applications no=
 later than summer 2002, replacing three Unix servers. Online brokerage E=
*Trade is moving its computer systems to IBM servers running GNU/Linux, c=
iting cost savings and performance as reasons for switching to GNU/Linux =
(the same article also notes that clothing retailer L.L. Bean and financi=
al services giant Salomon Smith Barney are switching to GNU/Linux as well=
). Linux International has a set of Linux case studies/success stories. M=
andrakesoft maintains a site recording the experiences of business users =
of the Mandrake distribution. Red Hat provides some similar information. =
Opensource.org includes some case studies. Obviously, the various large-s=
cale roll-outs that have occurred suggest that OSS/FS really is viable fo=
r enterprise deployments. Many retailer cash registers are switching to G=
NU/Linux, according to Information Week; for example, Home Depot plans to=
 roll out 90,000 terminals running Linux by 2003. According to Bob Young =
(founder of Red Hat), BP (the petroleum company) is putting 3,000 Linux s=
ervers at gas stations. There are many other reports from users, such as =
the report Linux as a Replacement for Windows 2000 and the results from A=
mazon.com. The U.S. Joint Strike Fighter (JSF) is using GNU/Linux, too. A=
 travel application service provider saved $170,000 in software costs dur=
ing the first six months of using GNU/Linux (for both servers and the des=
ktop); it also saved on hardware and reported that administration is chea=
per too. CRN's Test Center found that a GNU/Linux-based network (with a s=
erver and 5 workstations) cost 93% less in software than a Windows-based =
network, and found it to be quite capable. Adam Wiggins reports on TrustC=
ommerce's successful transition to Linux on the desktop.

Microsoft has been trying to claim that open source is somehow dangerous,=
 and indeed is its leading critic, yet the Wall Street Journal's Lee Gome=
s found that ``Microsoft Uses Open-Source Code Despite Denying Use of suc=
h Software.'' Here are some interesting quotes from his article:

=2E.. But Microsoft's statements Friday suggest the company has itself be=
en taking advantage of the very technology it has insisted would bring di=
re consequences to others. ``I am appalled at the way Microsoft bashes op=
en source on the one hand, while depending on it for its business on the =
other,'' said Marshall Kirk McKusick, a leader of the FreeBSD development=
 team.

More recently Microsoft has particularly targeted the GPL license, but ``=
it hasn't previously suggested that there were benign forms of open-sourc=
e software, and while singling out Linux for special criticism, has tende=
d to criticize all open-source with the same broad brush.'' The article c=
loses with this statement:

In its campaign against open-source, Microsoft has been unable to come up=
 with examples of companies being harmed by it. One reason, said Eric von=
 Hippel, a Massachusetts Institute of Technology professor who heads up a=
 research effort in the field, is that virtually all the available eviden=
ce suggests that open source is ``a huge advantage'' to companies. ``They=
 are able to build on a common standard that is not owned by anyone,'' he=
 said. ``With Windows, Microsoft owns them.''

There are several general information sites about OSS/FS or Unix that mig=
ht be of interest, such as the Free Software Foundation (FSF), the Open S=
ource Initiative website, and the Linux.org site. An older paper is John =
Kirch's paper Microsoft Windows NT Server 4.0 versus UNIX. The paper Our =
Open Source / Free Software Future: It's Just a Matter of Time argues tha=
t within the next few years, the standard de-facto operating system that =
nearly everyone uses, as well as much of the commodity software in widesp=
read use, will be OSS/FS. The book The Cathedral and the Bazaar by Eric R=
aymond examines OSS/FS development processes and issues. A useful collect=
ion of many OSS/FS writings, including the essay The Cathedral and the Ba=
zaar, is in the Open Source Reader.

Microsoft inadvertently advocated OSS/FS in its leaked internal documents=
, called the "Halloween" documents.

Recommendations of the Panel on Open Source Software For High End Computi=
ng; is the report of a panel created by the (U.S.) President's Informatio=
n Technology Advisory Committee (PITAC). It recommends that the ``Federal=
 government should encourage the development of open source software as a=
n alternate path for software development for high end computing''.

Several documents were written to counter Microsoft's statements such as =
those in Microsoft's "Linux Myths". This includes LWN's response and Jami=
n Philip Gray's response, and the FUD-counter site. The shared source pag=
e argues that Microsoft's ``shared source'' idea is inferior to open sour=
ce. Richard Stallman's The GNU GPL and the American Way counters the amus=
ing claim by Microsoft that the GPL was ``un-American.'' The letter Free =
Software Leaders Stand Together argues against a number of statements by =
Craig Mundie. You can find many general sites about Microsoft, including =
Cloweth's site.

``How Big Blue Fell For Linux'' is an article on how IBM transitioned to =
becoming a major backer. IBM announced that it planned to invest $1 Billi=
on in GNU/Linux in 2001 all by itself (see the IBM annual report). In 200=
2 IBM reported that they had already made almost all of the money back; I=
 and others are a little skeptical of these claims, but it's clear that I=
BM has significantly invested in GNU/Linux and seem to be pleased with th=
e results (for an example, see their Linux-only mainframe).

For a scientifically unworthy but really funny look at what people who us=
e the various operating systems say, take a look at the Operating System =
Sucks-Rules-O-Meter. It counts how many web pages make statements like ``=
Linux rocks''. It's really just an opinion poll, but if nothing else it's=
 great for a laugh.

Several studies examine developers (instead of the programs they write), =
including ``A Quantitative Profile of a Community of Open Source Linux De=
velopers'', Herman, Hertel and Niedner's study (based on questionnaires),=
 and the Who Is Doing It (WIDI) study. The Boston Consulting Group/OSDN H=
acker Survey (January 31, 2002) made some interesting observations by ran=
domly sampling SourceForge users for a survey. For example, it gives evid=
ence that open souce developers can be divided into four groups (based on=
 their motivations for writing OSS/FS software):

 - the believers, who do it because they believe source code should be op=
en (33%),
 - the skill enhancers, who do it for skill improvement (25%),
 - the fun seekers, who do it for a non-work need and intellectual stimul=
ation (21%), and
 - the professionals, who do it for work needs and professional status (2=
1%).

The study also found that the open source developers surveyed are mostly =
experienced professionals, having an average of 11 years of programming e=
xperience; the average age was 28.

Other evaluations include the Gartner Group's and GNet's evaluations.

For more general information on OSS/FS, see my list of Open Source Softwa=
re / Free Software (OSS/FS) references at http://www.dwheeler.com/oss_fs_=
refs.html


Conclusions

OSS/FS has significant market share, is often the most reliable software,=
 and in many cases has the best performance. OSS/FS scales, both in probl=
em size and project size. OSS/FS software generally has far better securi=
ty, particularly when compared to Windows. Total cost of ownership for OS=
S/FS is often far less than proprietary software, particularly as the num=
ber of platforms increases. These statements are not merely opinions; the=
se effects can be shown quantitatively, using a wide variety of measures.=
 This doesn't even consider other issues that are hard to measure, such a=
s freedom from control by a single source, freedom from licensing managem=
ent (with its accompanying litigation), and increased flexibility. I beli=
eve OSS/FS options should be carefully considered any time software or co=
mputer hardware is needed.

-------------------------------------------------------------------------=
-------

About the Author

David A. Wheeler is an expert in computer security and has a long history=
 of working with large and high-risk software systems. His books include =
Software Inspection: An Industry Best Practice (published by IEEE CS Pres=
s), Ada 95: The Lovelace Tutorial (published by Springer-Verlag), and the=
 Secure Programming for Linux and Unix HOWTO. Articles he's written inclu=
de More than a Gigabuck: Estimating GNU/Linux's Size and The Most Importa=
nt Software Innovations. Mr. Wheeler's web site is at http://www.dwheeler=
=2Ecom. You can view this article at http://www.dwheeler.com/oss_fs_why.h=
tml.


--------------6FF298A85FA76866B9A469FF--