TSELA (Toxic Search Engine Log Analyser) is a perl script that analyses your (Apache) webserver logs, processes them, and saves all search engine traffic info in a MySQL database. You can then view the data with a CGI script in a nicely formatted output with lists and graphs sorted by site/day/month/etc.
TSELA is THE tool to use if you are serious about your search engine traffic, and want to be able to study data/trends and predict traffic changes or robot spidering times.
The script comes with a standard list of search engines and search engine spidering robots it looks for in the logs, but you can easily add your own search engines/user agents (for web spiders) or patterns to watch for in the logs. The patterns can be created using standard regular expressions.
You can add multiple sites to the script so it processes data for multiple domains. Each domain can be placed in a certain group/subgroup and sub-subgroup, so you can see traffic and spidering totals for separate sites, or certain groups or subgroups.
The scripts are higly customizable and scalable, and make use of a MySQL database for data storage and .PNG files for graph generation.
Click here for screenshots.
TSELA is still under development, but will be available soon. If you have any questions on specific customizations, features or pricing, feel free to contact us.