AWBot 1.1 Documentation

Install, Setup and Use AWBot

AWBot common use is made in 4 steps:

Requirements
Create a test/config file
Run AWBot
Analyze results

Requirements

To use AWBot, you must have the following :
- A computer with a command line access (DOS, Korn Shell, ...)
- A perl interpreter working on this computer with the following Perl add-on modules :
* Time:HiRes
and if you need to make SQL requests to prepare your test:
* DBI
* DBD::YourSGBD (DBD::mysql, DBD::Oracle, DBD::Sybase, DBD::ODBC...)

You can find all those free modules on the net. Remember AWBot is not a web indexing robot but a client tool to emulate visitors to your site. This free tool is great for testing the stability of your web site, be it a casino, blog, video, or most any type of site. Don't forget to consult our FAQ page if you are having trouble setting up.

Create a testconfig file

After downloading and uncompressing AWBot distribution in a directory, you will find the following files :

./docs/*	HTML documentation files (Open index.html to get the table of contents)
./bin/awbot.pl	This is AWBot main script
./bin/awbotlaunch.pl	A launcher to launch several sessions of the bot
./bin/awbot.test.conf	A sample of config file for using AWBot
./bin/lang/*	Language files

The next step is to create a test/config file:

1- Copy awbot.test.conf into awbot.mytestconfig.conf.

2- Edit the "INIT PARAMETERS" section of this config file with your own test setup:
* Change OUTPUTDIR parameter to the value of the directory that will contains all tests results (AWBot output files) and create this directory (must have write permissions).
* Change SERVER parameter with value of your domain name. For example "www.mysite.com"
* If your site need a HTTP Basic authentication, use USER and PASSWORD to setup name and password of users used by the bot when authentication is required. You will be able to overwrite those parameters when you will launch several sessions of the bot using a user/password file list (See later).
* Change DELAY parameter to set the time the bot will wait between sending each URL.
3- Edit the "LIST OF ORDERED URLS TO TEST" section of this config file with your own test setup:
* You must add here the list of all URLs you want the bot to test. All URLs between "" and "" keywords will be sent in chronological order by the bot to the server. If your URL is a page you access from a link you must declare this url with the GET keyword, if your URL need to be requested from a HTML form, you must declare the link using the POST keywords (Fields are sent wth same URL syntax than GET requests). This is an example of this 2 URLs to test: <ACTIONS>
GET "http://$SERVER/"
POST "http://$SERVER/cgi-bin/myformact?field1=xxx&field2=bbb
</ACTIONS>
Note that you can use in your testconfig file value of a parameter declared before using the "$" prefix to tell the bot to replace parameter with value ($SERVER will be replaced by "www.mysite.com"

4- If you need to catch a parameter in the response of an URL, you can use the VAR keyword and the POSIX regex syntax (like 'sed', 'awk' and a lot of Unix tools) to extract the value.
For example if you have in your testconfig file the following actions list:
<ACTIONS>
GET "http://$SERVER/"
VAR "AUTHORNAME:Welcome on this page made by (\w+)"
GET "http://$SERVER/cgi-bin/info.pl?name=$AUTHORNALE"
</ACTIONS>
This means that the bot will first sent the request of the home page for $SERVER, wait the result, will search the string "Welcome on the Web Site xxx" and will store the value of xxx in the $AUTHORNAME variable. Then this variable $AUTHORNAME is used later for next url.

5- In this same section you can also use the CHECKYES or CHECKNO keywords to ask the bot to check a particular keyword or not in the page returned by the request.
This is a simple example to start:
<ACTIONS>
GET "http://$SERVER/"
CHECKYES "Welcome on this page"
CHECKNO "access forbidden"
</ACTIONS>

6- See Directives of testconfig file page for an explanation of all other available keywords.

7- If you used SEQUENCE or SQL keywords, you must enable the Perl module for your SGBD. For this edit the awbot.pl file and uncomment the corresponding line after the "#__START_OF_SGBD_SETUP" line.

Setup is finished. You can jump to the Run AWBot section.

Run AWBot

1- You can first launch AWBot as a standalone process from the command line, like this :
awbot.pl -config=awbot.mytestconfig.conf [otheroptions]

otheroptions can be :
-server=xxx Overwrite SERVER value of testconfig file
-user=xxx Overwrite USER value of testconfig file
-password=xxx Overwrite PASSWORD value of testconfig file
-delay=n Overwrite DELAY value of testconfig file
-timeout=n Overwrite TIMEOUT value of testconfig file
-nostopiferror AWBot continues with next test when an error occurs
-verbose Output file is also reported on screen
-debug=X To add debug informations lesser than level X

Results are stored in the output file myconfigtest.conf.out in the directory defined by the OUTPUTDIR in the config file.

This is an example of outputfile you can get from a simple input file

Input file

# AWBot 1.0 Config file #----------------------------------------------------------------------------- # Note that everywhere in the file you can include : # __MONENV__ to ask AWBot to replace tag by value of environment variable MONENV # $MYVAR to ask AWBot to replace tag by value of variable already defined # in this same config file. # $STARTSESSION is defined to 1 if AWbot is ran as a standalone process, # to first session number if ran from AWBotlaunch. # $NUMSESSION is defined to 1 if AWbot is ran as a standalone process, # to session number if ran from AWBotlaunch. # $NBSESSION is defined to 1 if AWbot is ran as a standalone process, # to total number of sessions ran from AWBotlaunch. # You can also include perl code in values everywhere in the file. #----------------------------------------------------------------------------- # INIT PARAMETERS #----------------------------------------------------------------------------- # Put here all global parameters of your test configuration # # Examples: # PASSWORD=$USER #----------------------------------------------------------------------------- OUTPUTDIR = "d:/temp/output" # Directory where to store output file(s) SERVER = "www.mysite.com" # Domain name of server to test (www.mysite.com) USER = "test" # If your web application require an authentication, this is the login that awbot will use PASSWORD = "test" # If your web application require an authentication, this is the password that awbot will use BOTNAME = "AWBot" # String value to force bot name used in the user agent string DELAY = 0 # Delay beetween each HTTP request (-1 wait a key, 0 no delay, n number of seconds) MAXSIZE = 0 # Maximum size of HTTP response (0 = no limit) TIMEOUT = 120 # Timeout (when waiting for a server response after sending a request) LANG = "fr" # Choose output language # DATABASE PARAMETERS #----------------------------------------------------------------------------- # This section is required only if you added SQL request in your actions (see later) # # Examples: # BASEENGINE="mysql" # USERBASE=($USER eq "toto"?"titi":$USER) #----------------------------------------------------------------------------- BASEENGINE = "Oracle" # Here is the name of the perl DBD driver (mysql, ODBC, Oracle, Sybase...) DSN = "mybase" # Database name USERBASE = "myuser" # Database login PASSWORDBASE = "mypassword" # Database password # OTHER PARAMETERS #----------------------------------------------------------------------------- # You can add here as many parameters as you need for rest of config file. # Just use the following syntax (where n is a number between 1 and 99): # PARAMn=value # PARAMn="SELECT SQL request" # # Examples: # PARAM1="select id from employee where name='$USER'" # PARAM2=2000+$PARAM1 #----------------------------------------------------------------------------- PARAM1 = 1 # LIST OF ACTIONS TO EXECUTE BEFORE A TEST #----------------------------------------------------------------------------- # You can use one of the keyword: # SEQUENCE To set a database sequence to a value (see examples, for Oracle only) # SQL To delete/insert some records in the database (see examples) # SCRIPT To launch an external script # # Examples: # SEQUENCE "myseq" $PARAM3+1 # SQL "delete from customers where id_customers < ".($PARAM1+$PARAM2) # SCRIPT "myscript.exe" #----------------------------------------------------------------------------- <PRE ACTIONS> </PRE ACTIONS> # LIST OF ORDERED URLS TO TEST #----------------------------------------------------------------------------- # Use the following format: # GET To get an HTML page (using GET HTTP method, like HTML links) # POST To get an HTML page (using POST HTTP method, like HTML forms) # CHECKYES To check if string is found in last response (using POSIX regex) # CHECKNO To check if a string is not found in last response (using POSIX regex) # VAR To extract a parameter from last response (using POSIX regex) # SEQUENCE To set a database sequence to a value (see examples, for Oracle only) # SQL To delete/insert some records in the database (see examples) # SCRIPT To launch an external script # WRITETO Write last html response to a file # WRITETOH Write last html response with its HTTP header to a file (full response) # AUTO To make a GET on all URLs find in last HTML response page and follow # links on n level # # Examples: # GET "http://$SERVER/mypage.html?x=y&v=w" # POST "http://$SERVER/mypage.html?x=y&v=w" # CHECKYES "xxx" # CHECKNO "eee" # VAR "VARNAME:idvar=(\d)" # AUTO 1 # SEQUENCE "myseq" $PARAM3+1 # SQL "delete from customers" # SCRIPT "myscript.exe" # WRITETO "myfile.html" # WRITETOH "myfile.http" #----------------------------------------------------------------------------- <ACTIONS> GET "http://$SERVER/" </ACTIONS> # LIST OF ACTIONS TO EXECUTE AFTER A TEST #----------------------------------------------------------------------------- # You can use same rules for LIST OF ACTIONS TO EXECUTE BEFORE A TEST. #----------------------------------------------------------------------------- <POST ACTIONS> </POST ACTIONS>
Result/Output file

TEST awbot 1.0 (build 1.3) --------------------------- Config file: awbot.mytest.conf Server: chiensderace.com - User: test - Delay: 0 Botname: AWBot - TimeOut: 120 - MaxSize: 0 Date: 2002-07-03 23:12:51 Process ID: 2328 PRE ACTIONS --------------------------- ACTIONS --------------------------- 2002/07/03-23:12:51:552 URL 1 - http://chiensderace.com/ ---> OK - 390 ms POST ACTIONS --------------------------- SUMMARY --------------------------- Total requests to do: 1 Total requests sent: 1 (1 answered) Total requests duration: 390 ms Average requests response time: 390 ms/request Total Check Yes: 0/0 No: 0/0 Faster request response time: URL 1 - 390 ms Slower request response time: URL 1 - 390 ms URL 1 - Duration: 390 ms - Cumul: 390 ms - Check Yes: 0 / 0 No: 0 / 0

2- You can use the bot launcher if you need several processes (for load benchmarks):
awbotlaunch.pl -config=awbot.myconfigtest.conf -nbsessions=n

With the awbotlauncher, you will get one output file for each sessions ran. Each output file will contains the process ID of the session in its name. Example for -nbsessions=3, you will get 3 files:
myconfigtest.conf.112.out
myconfigtest.conf.113.out
myconfigtest.conf.114.out

Note 1: Don't forget that if you want to have results for n simultaneous sessions, you should set DELAY parameter to 0.
Note 2: Don't forget to purge the output directory before running a multi-session test if you want to obtain a directory with all files of last test.

If you add the parameter -authentication=@userpasswordfile when you launch awbotlaunch, each session of the bot will be launched with a different value of USER and PASSWORD whatever are their value in config file. Values for USER/PASSWORD will be thoose found in file userpasswordfile. Format for this file is:

user1 password1
user2 password2
...
usern passwordn

Analyze Results

Just open the output file. You will, for each request, time spent or the error message if URL failed.

As you can see in the preceding example, the output file contains several parts:
The first part is summary of INIT parameters

TEST awbot 1.0 (build 1.3)
---------------------------
Config file: awbot.mytest.conf
Server: chiensderace.com - User: test - Delay: 0
Botname: AWBot - TimeOut: 120 - MaxSize: 0
Date: 2002-07-03 23:12:51
Process ID: 2328

The part "PRE ACTIONS" and "POST ACTIONS" are results of all PRE and POST ACTIONS commands

The part "ACTIONS" is the list of all URLs in your "<ACTIONS>" section. You can see time when request was submitted with following format:
YYY/MM/DD-HH:MM:SS:ms (ms are milliseconds)
You have a line under each URL to test with time spent by server to send page. If a CHECKYES or CHECKNO check rule mad on response fails, you will also get an error message.

The last part is a summary of response time, and checks for global test and each URLs.

SUMMARY
---------------------------
Total requests to do: 2
Total requests sent: 2 (2 answered)
Total requests duration: 6827 ms
Average requests response time: 3413 ms/request
Total Check Yes: 1/1 No: 0/0
Faster request response time: URL 2 - 2000 ms
Slower request response time: URL 1 - 4827 ms
URL 1 - Duration: 4827 ms - Cumul: 4827 ms - Check Yes: 1 / 1 No: 0 / 0
URL 2 - Duration: 2000 ms - Cumul: 6827 ms - Check Yes: 0 / 0 No: 0 / 0