DTP


 
Lively discussions on the graphic arts and publishing — in print or on the web


Go Back   Desktop Publishing Forum > General Discussions > Business Matters

Reply
 
Thread Tools Display Modes
Old 06-10-2007, 11:46 PM   #1
annc
Sysop
 
annc's Avatar
 
Join Date: Oct 2004
Location: Subtropical Queensland, Australia, between the mountains and the Coral Sea
Posts: 4,434
Default What percentage of your visitors are robots?

It seems that around 60% of my visitors these days are robots, which has to skew the statistics.

I sell advertising on some of my pages, and am wondering what other people do about the effect of robots on their statistics when selling advertising space. I should say that the advertisers I attract are happy to advertise because they know that their potential customers visit the site, but there is this little ethical bug that tells me I should qualify any traffic statistics if asked about actual figures.

What are your robot statistics, and what do you tell your customers?

   
__________________
annc is offline   Reply With Quote
Old 06-11-2007, 06:24 AM   #2
iamback
Member
 
iamback's Avatar
 
Join Date: Oct 2005
Location: Amsterdam, NL
Posts: 4,894
Default

Quote:
Originally Posted by annc View Post
It seems that around 60% of my visitors these days are robots, which has to skew the statistics.
Visitors, or page views, or hits?

My guess is page views: a bot requesting 10 pages is still one visitor, but generates 10 page views. A human visitor requesting 10 pages also generates 10 page views, but (unlike the majority of bots) many more hits since the browser would also request linked stylesheets and javascript files, and likely images as well: each request the browser sends to the server is a hit.

Quote:
Originally Posted by annc View Post
I sell advertising on some of my pages, and am wondering what other people do about the effect of robots on their statistics when selling advertising space. I should say that the advertisers I attract are happy to advertise because they know that their potential customers visit the site, but there is this little ethical bug that tells me I should qualify any traffic statistics if asked about actual figures.
A good statistics program will be able to filter out (most) bots from the results, and also give an idea of "unique visitors" as opposed to page views by applying a time window. If you're quoting figures to potential advertisers you should at least filter out bots (that a statistics program knows about or can be taught about).

   
__________________
Marjolein Katsma
Look through my eyes on Cultural Surfaces (soon!), My ArtFlakes shop and Flickr.
Occasionally I am also connecting online dots... and sometimes you can follow me on Marjolein's Travel Blog
iamback is offline   Reply With Quote
Old 06-11-2007, 06:37 AM   #3
Kelvyn
Staff
 
Kelvyn's Avatar
 
Join Date: Feb 2005
Location: In the Heart of the English Lake District
Posts: 1,381
Default

The servers I use have the Awstats log file analyzer installed. This shows page views by bots and "real" users, which sometimes is quite revealing. For example, me web company site gets 2-4 times more bot views that client views, while a tourism directory sees twice as many real views than bots.

   
__________________
Kelvyn

Web site design, hosting and marketing, Keswick in the UK Lake District

If you are planning a visit to Keswick then try Keswick Tourist Information website

Kelvyn is offline   Reply With Quote
Old 06-11-2007, 11:44 AM   #4
annc
Sysop
 
annc's Avatar
 
Join Date: Oct 2004
Location: Subtropical Queensland, Australia, between the mountains and the Coral Sea
Posts: 4,434
Default

I use Summary, and it differentiates between bots and real visitors (this is how I know the figure for the bots) and also between visits and hits. It also gives me the number of unique visitors.

I suppose I just have to do the division when calculating the real number of visitors.

Fortunately, vBulletin tells me how many members visit my forums every day, and I can use those as well if necessary, as all the ads are on forum pages.

I was just a bit surprised to find that the robots had reached such a high percentage.

   
__________________
annc is offline   Reply With Quote
Old 06-11-2007, 11:46 AM   #5
annc
Sysop
 
annc's Avatar
 
Join Date: Oct 2004
Location: Subtropical Queensland, Australia, between the mountains and the Coral Sea
Posts: 4,434
Default

Quote:
Originally Posted by Kelvyn View Post
The servers I use have the Awstats log file analyzer installed. This shows page views by bots and "real" users, which sometimes is quite revealing. For example, me web company site gets 2-4 times more bot views that client views, while a tourism directory sees twice as many real views than bots.
Yes, mine have too. I also have Summary installed, which gives me statistics over a longer period. While I like the AWStats, they start again every month.

   
__________________
annc is offline   Reply With Quote
Old 06-11-2007, 02:10 PM   #6
Kelvyn
Staff
 
Kelvyn's Avatar
 
Join Date: Feb 2005
Location: In the Heart of the English Lake District
Posts: 1,381
Default

Quote:
Originally Posted by annc View Post
While I like the AWStats, they start again every month.
While that does allow month to month comparison it is not optimum. I also routinely download the server logs of the sites I am marketing and run them through Weblog Analyzer Pro. That enables me to do proper month on month comparisons.

   
__________________
Kelvyn

Web site design, hosting and marketing, Keswick in the UK Lake District

If you are planning a visit to Keswick then try Keswick Tourist Information website

Kelvyn is offline   Reply With Quote
Old 06-11-2007, 10:37 PM   #7
DTP Guy
Member
 
DTP Guy's Avatar
 
Join Date: Feb 2005
Location: Milton Keynes, United Kingdom
Posts: 156
Default

Have you tried Google Analytics, Kelvin? Really good information and it can de looked at it many different ways - including setting up pages for just a specific project on your site.

The downside? A small piece of Javascript code on each page you want to monitor.

   
__________________
DTP Guy
a.k.a Brian Smithson
DTP Guy is offline   Reply With Quote
Old 06-12-2007, 06:13 AM   #8
Kelvyn
Staff
 
Kelvyn's Avatar
 
Join Date: Feb 2005
Location: In the Heart of the English Lake District
Posts: 1,381
Default

Quote:
Originally Posted by DTP Guy View Post
Have you tried Google Analytics,
Yes, I use it on most sites that I promote using Adwords campaigns, mainly because it integrates nicely with the Adwords tools, although being JavaScript based it won't work for browsers with JS turned off. I also don't think the Analytics stats on their own are as good as those obtained running the server logs through Weblog Analyzer, but it is a free service. I did have a problem a while back with page loading times being increased due to the call to Google, and I can see that this is still happening periodically.

   
__________________
Kelvyn

Web site design, hosting and marketing, Keswick in the UK Lake District

If you are planning a visit to Keswick then try Keswick Tourist Information website

Kelvyn is offline   Reply With Quote
Old 06-12-2007, 09:16 AM   #9
iamback
Member
 
iamback's Avatar
 
Join Date: Oct 2005
Location: Amsterdam, NL
Posts: 4,894
Default

Quote:
Originally Posted by Kelvyn View Post
I did have a problem a while back with page loading times being increased due to the call to Google, and I can see that this is still happening periodically.
I see this very frequently in sites I visit; so I won 't put that code on any page of mine - at least not until I stop seeing it regularly.

   
__________________
Marjolein Katsma
Look through my eyes on Cultural Surfaces (soon!), My ArtFlakes shop and Flickr.
Occasionally I am also connecting online dots... and sometimes you can follow me on Marjolein's Travel Blog
iamback is offline   Reply With Quote
Old 06-13-2007, 11:48 PM   #10
DTP Guy
Member
 
DTP Guy's Avatar
 
Join Date: Feb 2005
Location: Milton Keynes, United Kingdom
Posts: 156
Default

Hmm. I have seen the page load increase too - but not usually as a result of calling Google; more because of a call to an ad server somewhere.

   
__________________
DTP Guy
a.k.a Brian Smithson
DTP Guy is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Checking robots.txt dthomsen8 Web Site Building & Maintenance 2 10-28-2005 06:48 PM


All times are GMT -8. The time now is 06:07 AM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
Contents copyright 2004–2014 Desktop Publishing Forum and its members.