<<< Reports
Glossary of Terms >>>

How do I start working with the program?
The first step in working with the program is creation of a project. When the program is started for the first time, a blank project is created automatically. After that, every time you start the program it will load the last opened project. Now you may start creating the robots.txt file.

I have launched the generation process for robots.txt, but it doesn't work.
Each time when the file generator is launched the validity of your registration key is being verified. Check your Internet connection and proxy settings specified in the program (in case you are using a proxy server).

What is a Known User-Agents list and what is it for?
The Known User-Agents list is a list of known User-Agent values that other client programs (not spiders) use to identify themselves when visiting your site. These programs include browsers, upload/download managers, LinkCheckers, etc. This list is used during the log analysis process. The records that contain User-Agent values from this list are ignored. The more complete the list, the faster and more precise the analysis will be. This is why you may add new User-Agent values to this list as you find them. You do not necessarily have to do this, however, as the program has a large list of known User-Agents.

How can I regulate the access rights for a User-Agent which is on the list of known User-Agents?
First go to the "Options>Known User-Agents" menu and delete this User-Agent from the list. Then create a new record in the Spider List that would match this User-Agent. After that you may regulate the access rights for this User-Agent.

Why can't I save my logfile database by pressing "Save Database" ?
The logfile database can only be updated when import of new files takes place. There's a 'Save import result in project database' option at the last step of Import Wizard. When you select it, the imported files will be added to the database right after the importing and the "Save Database" button will be disabled. If you do not check this option, the button will remain enabled and you'll be able to add imported log files to the database yourself.

How can I disallow the access to the whole site for one spider/all spiders?
First of all, select the spiders you want to keep away from your site on the spider list. Then switch to the Disallow Tab and select the necessary spider(s) from the list. Then simply check the "Disallow access to all site" field and access to all pages of your site will be forbidden.

How do I restrict the access to files of a given format?
Go to the Disallow Tab . In the "Disallow" menu, select the item "Disallow File Extensions" . In the window that emerges, check the box "Disallow files with extensions" . Press "OK" . In the files/folders tree the option to select files by a given extension will appear.

How can I modify the existing robots.txt file?
Import the existing robots.txt file via menu Project>Import Robots.txt. In the Wizard that starts up, select the location of robots.txt file, import it, and specify the location of the website to which these instructions will be applied.

<<< Reports
Glossary of Terms >>>