FYI: OCLC World Wide Web characterization work

[Matthew Beacom <matthew.beacom@YALE.EDU>]

FYI

OCLC Researchers Measure the World
                         Wide Web

                         DUBLIN, Ohio, Oct. 16, 2000--In their annual review of
the World Wide Web,
                         researchers at OCLC have determined that the Web now
contains about 7 million
                         unique sites; that the public Web-sites that offer
content that is freely accessible by the
                         general public-constitutes about 40 percent of the
total Web; and that the Web
                         continues to expand at a rapid pace, but its rate of
growth is diminishing over time. 

                         According to the group's latest estimates, there were
7.1 million unique web sites, a
                         50 percent increase over the previous year's total of
4.7 million. Although the number
                         of web sites has nearly tripled in size in the last
two years, year-to-year growth rates
                         are declining, falling from almost 80 percent between
1998 and 1999, to only about 50
                         percent between 1999 and 2000. 

                         Public web sites constitute 41 percent of the Web, or
about 2.9 million sites. Private
                         sites- whose content is subject to explicit access
restrictions (e.g., Internet Protocol
                         filters or password authentication), or is not
intended for public use (e.g., web
                         interfaces to privately owned hardware devices such as
printers or routers)-comprise
                         21 percent of the Web, or 1.5 million sites. The
remaining 2.7 million sites-or about 38
                         percent of the Web-are provisional sites: their
content is in an unfinished or transitory
                         state (e.g., server default pages or "Site under
construction" notices).

                         Adult sites-those offering sexually explicit
content-now constitute about 2 percent of
                         the public Web, or 70,000 sites. The proportion of the
public Web occupied by adult
                         sites has remained unchanged since 1998. 

                         "The Web continues to grow at a substantial rate,"
said Ed O'Neill, manager of the
                         OCLC Web Characterization Project. "But a comparison
of the year-to-year growth
                         rates suggests that the Web's expansion is slowing.
This trend is even more
                         pronounced in the public Web, which grew by about 80
percent between 1997 and
                         1998 but only by about a third between 1999 and 2000.
Even in absolute terms,
                         growth seems to be slowing: the public Web increased
by 713,000 sites in the past
                         year, compared to 772,000 sites between 1998 and
1999."

                         Brian Lavoie, a research scientist working on the Web
Characterization Project, notes
                         the increasing incidence of non-public web content.
"For most people, the Web is the
                         public Web-that's where most web browsing takes place.
But there's a lot of content
                         out there that you would probably never encounter in
the course of casual browsing; in
                         other words, the private and provisional sites.
Private sites in particular have exhibited
                         steady growth relative to public sites in the past few
years, accounting for about 12
                         percent of the Web two years ago, compared to over 20
percent today."

                         The Web Characterization Project, conducted by the
OCLC Office of Research, has
                         collected a random sample of web sites annually since
1997. Current results are
                         based on analysis of the June 2000 sample. For
analytical purposes, a web site is
                         defined as content accessible through the HTTP
protocol at a given location on the
                         Internet.

                         More information on the Web Characterization Project
is on the project web site
                         <http://wcp.oclc.org/>.

Matthew Beacom
Catalog Librarian for Networked Information Resources
Yale University Library
(203) 432-4947
matthew.beacom@yale.edu
http://www.library.yale.edu/~mbeacom