Page 1 of 1
Google crawling of module content.
Posted: Thu Nov 17, 2005 10:53 pm
by ceilingfish
Hi,
I was reading a notice on the Google website, about the behaviour of the googlebot, apparently, according to thier
Information for webmasters, Googlebot doesn't crawl any page which has a get argument of &id=. So things like News articles, Calendar items, Guestbook entries, well just about any content generating module won't get spidered.
Anyone know a way around this?
Cheers,
Tom
Re: Google crawling of module content.
Posted: Fri Nov 18, 2005 9:44 am
by ceilingfish
Cheers Patricia,
That's fantastic!
Tom
Re: Google crawling of module content.
Posted: Fri Nov 18, 2005 11:03 am
by ceilingfish
I thought that I would share my solution for getting rid of ids in all other modules too.
I rewrote the following line in /lib/class.module.php:
Code: Select all
$text .= '/'.$goto.'?module='.$this->GetName().'&id='.$id.'&'.$id.'action='.$action;
to
Code: Select all
if($this->cms->config['assume_mod_rewrite'])
$text .= '/'.$this->GetName().'.module.'.$id.'.php?'.$id.'action='.$action;
else
$text .= '/'.$goto.'?module='.$this->GetName().'&id='.$id.'&'.$id.'action='.$action;
Then added the following rewrite rule to the .htaccess file:
Code: Select all
RewriteRule ^([A-Za-z]+)\.module\.(.+)\.php$ moduleinterface.php?module=$1&id=$2 [QSA]
which has worked pretty well for all the modules I have encountered so far. Does anyone know of any modules that over-ride the CreateLink() method?
EDIT: It has however buggered the administration. Please ignore this until I figure out some way to get around it.
Re: Google crawling of module content.
Posted: Fri Nov 18, 2005 11:12 am
by ceilingfish
Preliminarily, the following seemed to work. The code for /lib/class.module.inc.php was rewritten to:
Code: Select all
if($this->cms->config['assume_mod_rewrite'] && $returnid != '')
$text .= '/'.$this->GetName().'.module.'.$id.'.php?'.$id.'action='.$action;
else
$text .= '/'.$goto.'?module='.$this->GetName().'&id='.$id.'&'.$id.'action='.$action;
I shall test this more.
Tom