CMS Made Simple Forums

While testing a new website on our staging server we found a problem that may have affected other devs.

The scenario is that you have a site on a staging server, the site has many links to external pages that are running google analytics. This in itself is fine until you start testing the links and google finds your staging site. again not a problem if you have set your robots.txt to disallow /*, but what about if you didn't and the site has now been indexed! all 700 pages of it!

Well firstly set the robots.txt and the header of each page with: This will help any further crawls.

You can then put the site into maintenance mode in the [Site Admin->Settings - Global Settings->Maintenence Mode]

set the "Down for Maintenance" to Yes

now the site will be down for external users. You can add the testers IP addresses to the "Exclude these IP addresses" field.

Then they can test away without letting google index the site.

Now!

what happens when you want to use the CMSMS Linker plugin on the MicroTiny WYSIWYG editor, answer is it won't work in maintenance mode.

"I know that sucks" I here you say, not so, as this is a security feature after all and how secure would it be if it allow ajax calls while in maintenance mode.

The solution is simple, add the server IP to the "Exclude these IP Addresses" field this allows CMSMS Linker to make the ajax calls to the server and return the correct links.

I also discovered that the same issue befalls the PDFGenerator Module as it requires the module to make a call to build the PDF for output and returns a blank PDF, but not anymore.

I am sure there will be other occasions that this would be the case and hope that it will stop you scratching your heads and getting frustrated.

it's all good and we can move forward

Mark

Installing SitemapMgr gives you the possibility to edit your robots.txt file from out the Admin panel.
http://dev.cmsmadesimple.org/projects/sitemapmgr

You can put this line in the metadata field in Global Settings once When having the {metadata} tag in the page head, it will be automatically added to all pages.

FYI: <?php tags are not allowed or supported in any templates.

#### UPDATE ####

due to a fopar in my code addition pointed out by @Calguy1000 the following code should be replaced:

Well firstly set the robots.txt and the header of each page with:

Code: Select all

<?php header('X-Robots-Tag: noindex,noarchive,nofollow'); ?>

With

Install MetaMax and go to "Page title & Meta information" tab "Meta content" section and select the noindex, nofollow option from the dropdown.

that will do the trick.

It is possible to add the php header to the templates by using a UDT and addin if thats your preference.

happy coding