TWiki> Community Web>WebLeftBar>StatisticsSummary (15 Feb 2006, KoenMartens)EditAttach

issuecrawler stats

  2004 2005
nr of crawls 829 1437
nr of crawls with pages 740 1200
avg crawl duration 03:43:50 02:17:09
avg nr of crawls a day 2,4 4,05
avg nr of startingpoints 22,61 30,78
page downloads 9.729.717 19.960.656
analysis by page 812 1345
analysis by site 17 92
distinct users who issued a crawl 68 152

conclusions:

  • lots of stuff has (almost or more than) doubled: the nr of crawls and nr of crawls a day, the user base, the page downloads
  • the relatively higer nr of crawls without pages is probably a cause of new users who don't know what to enter as startingpoints
  • the avg crawl duration is reduced considerably (long live the new server and the 100Mbit connection ... even though we now are a 'respectful' crawler :-))
  • the by site functionality finally works
  • we got a new top user smile

Seems like a highlight of 2005 to me smile

(compiled by Erik Borra)

-- KoenMartens - 15 Feb 2006

Edit | Attach | Print version | History: r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r1 - 15 Feb 2006 - 22:43:22 - KoenMartens
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback