Re: Spider Activity Reports from Blackwell Synergy

["James A. Robinson" <jim.robinson@stanford.edu>]

One increasingly common source of problems is the use of browser 
technology which attempts to pre-fetch pages for their user. 
For example, a person who clicks on a table-of-contents, and 
whose browser goes off and retrieves every article link on the 
page.

Sometimes people are unaware that their browser is configured to 
do this, other times people are aware and believe this is an 
acceptable use case.

It causes a problem for web site maintainers, since you all of a 
sudden have 20-40 full-text content requests being made (even if 
90% of the requested content won't be looked at).  To a publisher 
it might look like a badly behaved bot is spidering the site.

Jim

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
James A. Robinson                       jim.robinson@stanford.edu
Stanford University HighWire Press      http://highwire.stanford.edu/
+1 650 7237294 (Work)                   +1 650 7259335 (Fax)