BlogHarbor Home Page
FAQFAQ  SearchSearch  MemberlistMemberlist  UsergroupsUsergroups  UsergroupsRSS   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
Reading the new web server log
 
Post new topic   Reply to topic    BlogHarbor Community Forum Index -> General Discussion
View previous topic :: View next topic  
Author Message
kazo56



Joined: 16 Sep 2005
Posts: 36

PostPosted: Wed Apr 19, 2006 7:45 pm    Post subject: Reading the new web server log Reply with quote

Hello John,

The new feature on the Dashboard - the web server log - is a great addition. I am using it at the moment to locate all 400 notices, which lead me to errors I have made on my blog.

An example of an error notice (on my blog) occurs when the visitor is looking for a post in a category, and at some time in the past I have changed the name of that category, but retained the URL of the old name (or part thereof of the name). The visitor sees the "insufficient category privileges" notice. On the web server log I see "400" and can then watch what the visitor does to try to relocate the single post. I then delete the old URL and replace it with the newer category URL. Fascinating.

But - and these are my questions:
1. how can I detect what the visitor is actually doing with the bytes? Downloading file? Saving? Linking? RSS feeding? And whatever else (I can only guess)
2. how can I detect the difference between fair use and abuse from examining the web server log?

Much appreciated, as always,
Back to top
View user's profile Send private message
john
Site Admin


Joined: 16 Mar 2004
Posts: 3434

PostPosted: Wed Apr 19, 2006 8:53 pm    Post subject: Re: Reading the new web server log Reply with quote

The log files are really best read by some log file analyzer, but I suppose you could read through them if you want...

1. how can I detect what the visitor is actually doing with the bytes? Downloading file? Saving? Linking? RSS feeding?

No hard and fast answer here... Now I think you understand why I've been saying here all along that analyzing logs is as much art as science...

You could look at the User Agent area, to see what kind of software was asking for the file. Then look at what they asked for, if it was an xml file that's probably your RSS feed... A jpg is loading as part of your page, etc.

2. how can I detect the difference between fair use and abuse from examining the web server log?

I don't think there's an easy answer to this one either... Is there some kind of abuse you are suspicious of?
Back to top
View user's profile Send private message Send e-mail Visit poster's website
kazo56



Joined: 16 Sep 2005
Posts: 36

PostPosted: Wed Apr 19, 2006 9:08 pm    Post subject: Web server log - who should be blocked? Reply with quote

Hello John,

Addenda to my previous post with this important query:

The 400 notices appearing on my web server log for the 17th April 2006 show a visitor called FunWebProducts; .NET CLR 1.1.4322. I have now blocked their IP (see my Firewall). However, I notice that this IP is actually pointing me to the "Insufficient Category Privileges" error notices on my blog, and thereby doing me a service, so should I block their IP?

In plain English, here's what is happening:
Back in May and June 2005 I posted items to categories whose names I have since altered slightly, thereby altering the URL;

For example, I changed a category called "Icons of Tasmanian Photography 1860s-1880s" to simply "Icons of Tasmanian Photography" minus the date. The FunWebProducts is actually showing me this. So I then go to the specific html of the item, and find that the trackback box is ticked (checked). I uncheck it (although there is no trackback showing), and then SAVE, not necessarily to today's date. The item then appears in its newer category.

So, is FunWebProducts malware or goodware, and how is it finding my old URLS? Why is it finding my dead URLS?
Back to top
View user's profile Send private message
john
Site Admin


Joined: 16 Mar 2004
Posts: 3434

PostPosted: Wed Apr 19, 2006 11:00 pm    Post subject: Re: Web server log - who should be blocked? Reply with quote

FunWebProducts is a user agent indicative of a modified IE browser as I understand it. Humans use it.

How might it be finding your old URLs? Search engines would be the obvious guess.

I think that banning this IP address might be going overboard, the fact that someone was using this browser and visited your site does not mean anything at all, beyond the fact that someone was using this browser and visited your site...
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Search all BlogHarbor support resources.
View previous topic :: View next topic  
Display posts from previous:   
Post new topic   Reply to topic    BlogHarbor Community Forum Index -> General Discussion All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum