my bug, nefarious scraper or a legitimate browser plugin?

I've been faced the the following symptoms for some time.

I have links coded as :post or :put, so I can make sure that bots aren't hitting particular links.

But it something is either hitting them as :get through an error I've made (like link_to not working well in some browsers?), or there's 1 or more plugins that pre load urls; or I have scrapers.

Each day I'll get 50-100 error messages - where routes aren't found - of this nature.

When I get many of these hits from the same ip address, I usually assume a scraper, then block that ip address...but I don't want to do this in very case if it's possible that a legimate (pre-loader browser plugin) is causing this to happen.

Does anybody else this kind of behaviour? How do you handle it?

thanks.
Jodi

I've been faced the the following symptoms for some time.

I have links coded as :post or :put, so I can make sure that bots
aren't hitting particular links.

But it something is either hitting them as :get through an error I've
made (like link_to not working well in some browsers?), or there's 1
or more plugins that pre load urls; or I have scrapers.

A browser with js turned off would also do this (or using a firefox
plugin like noscript to only have it on for certain websites)

Fred

ty Fred - yes. good thought - likely the simplest explanation.

will trap, then see where that leads me.

Jodi

This latest info rules out JS-off or a noscript plugin -

I've been faced the the following symptoms for some time.

I have links coded as :post or :put, so I can make sure that bots
aren't hitting particular links.

But it something is either hitting them as :get through an error I've
made (like link_to not working well in some browsers?), or there's 1
or more plugins that pre load urls; or I have scrapers.

A browser with js turned off would also do this (or using a firefox
plugin like noscript to only have it on for certain websites)

Fred

Here's the markup where the link is:

<a style=“margin-left: 0pt; padding-left: 0pt; font-size: 16px; margin-bottom: 60px;” onclick=“Windows.overlayHideEffectOptions = {duration: 3}; Dialog.info({url: ‘http://homestars.com/messages/201895-contex-roofing-company-ltd/new_company’, options: {method: ‘post’}}, {title:‘Contact Us’, className: ‘bluelighting’, width: 290, height: 450, destroyOnClose: true, draggable: true, evalScripts: true});; return false;” href="#"><img style=“margin-right: 5px;” alt=“Email Contex Roofing Company Ltd.” src="/images/contact_email.jpg"/><img style=“margin-right: 5px;” alt=“Phone Contex Roofing Company Ltd.” src="/images/contact_phone.jpg"/><span style=“font-size: 14px;”>Contact: Contex Roofing Company Ltd.</span></a>

The bot/human is reaching the url "http://homestars.com/messages/201895-contex-roofing-company-ltd/new_company", but you can see the href = '#' - so something is scanning the html, looking for urls to harvest.

So looks like either a bot or a page preloader...

I don't mind pre-loaders - so I think I'll see if I can find a patter in the plugins loaded..and if I don't find a plugin then this could work as a honeypot.

I guess I don't have a specific question - merely symptoms - hopeful that someone may have faced such a thing.

Jodi