A tcl robot : W3 explorer, checker and statistical data collector

This robot tries to comply to Guidelines for Robot Writers

Runs preliminary results:

  1. Run limited to hplyot's servers only : visited 378 different local pages, found 2692 different urls (1096 on hplyot, 15 on rest of obspm.fr, 76 on rest of .fr (1505 internationals)); 10 errors were found.
  2. Run on the whole observatory domain: 2686 urls visited, 7773 different urls found (5327 inside obspm.fr domain on 13 servers); 209 errors were found.

This project was very preliminary and is now quite outdated (it was written in May 1995), It was more a test shot, the source code is not very clean and thus is NOT available

The TclX source code of w3cli.tcl: a simple www client is here or one a bit more sophisticated geturl.tcl (with headers & data separated). More www fetchers & tools (newer version than the above) are on my WWW tools page

The HTTP User-agent field is set to dlw3robot/x.y (in TclX by http://hplyot.obspm.fr/~dl/), and the From field is also set. It's run from hplyot.obspm.fr.

If interested by robots, don't miss the World Wide Web Robots, Wanderers, and Spiders page.

See also some of my other W3 tools.

Maintained by dl, Last modified Sept 9th 1996.

[up] [top]