Prev: Word question (or Pages I guess)
Next: MacZot / theMacSale / SuperCard
From: Martin-S on 16 Mar 2010 14:25
In article <saramerriman-1DE5C5.16214316032010(a)news.individual.net>,
Sara <saramerriman(a)blueyonder.co.uk> wrote:

> I need to host a page on our website for a client, but I don't want it
> to show up in any web searches.

In theory that's what the robots.txt file in the web root should do:
<http://www.robotstxt.org/>

User-agent: *
Disallow: /


However there are plenty of bad bots that will ignore it. So as far as I
know, the only safe way is to password-protect it:
<http://golivecentral.com/pages/txttut/htacc.shtml>

Like the article mentions, that could be a simple checkbox operation in
your host's control panel, so check there first.

--
Martin
From: Martin-S on 16 Mar 2010 14:28
In article <cgzjmthpzs-DB8130.18250216032010(a)news.simnet.is>,
Martin-S <cgzjmthpzs(a)lzrpqi.net> wrote:

> However there are plenty of bad bots that will ignore it.

Also worth noting is, that if any other site links to any of your pages
*every bot* will find and index it.

That is the weakness of the robots.txt solution.

--
Martin
From: Duncan Kennedy on 16 Mar 2010 14:28
In message <slrnhpvc9u.2ebd.jim(a)wotan.magrathea.local>, Jim
<jim(a)magrathea.plus.com> writes
>On 2010-03-16, Sara <saramerriman(a)blueyonder.co.uk> wrote:
>>
>> I need to host a page on our website for a client, but I don't want it
>> to show up in any web searches. Is there a header tag or something I can
>> use that will have that effect?
>>
>
><http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=164734>
>
>I _think_ that should help.
>
That should fix Google but if you want the additional protection against
robots / spiders then try:

1) a meta tag in the <head> </head> area of the protected page:

<meta name="robots" content= "NOINDEX, NOFOLLOW">

2. Put a robots.txt script in your root directory:

User-agent: *
Disallow: /(directory name for the special page on its own)/
Disallow: /(directory name/filename.htm (of the special file)

It would be wise to put an index.htm file in the special directory with
just some spoof text like "test area - not in use" so that anyone
trying it on doesn't get your special file listed if there is no
index.htm (default.html if you're on a Windows web server)

--
Duncan K
Downtown Dalgety Bay
From: Jim on 16 Mar 2010 14:34
Sara <saramerriman(a)blueyonder.co.uk> wrote:

> > > I need to host a page on our website for a client, but I don't want it
> > > to show up in any web searches. Is there a header tag or something I can
> > > use that will have that effect?
> > >
> >
> > <http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=164734>
> >
> > I _think_ that should help.
> >
> Hero! Yes, thank you :-)

Ironically, I Googled it..:-)

Jim
--
"Microsoft admitted its Vista operating system was a 'less good
product' in what IT experts have described as the most ambitious
understatement since the captain of the Titanic reported some
slightly damp tablecloths." http://www.thedailymash.co.uk/
From: Sara Merriman on 16 Mar 2010 14:53
In article <1jfgm8s.1mivgj61vha0jkN%jim(a)magrathea.plus.com>,
jim(a)magrathea.plus.com (Jim) wrote:

> Sara <saramerriman(a)blueyonder.co.uk> wrote:
>
> > > > I need to host a page on our website for a client, but I don't want it
> > > > to show up in any web searches. Is there a header tag or something I
> > > > can
> > > > use that will have that effect?
> > > >
> > >
> > > <http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=16473
> > > 4>
> > >
> > > I _think_ that should help.
> > >
> > Hero! Yes, thank you :-)
>
> Ironically, I Googled it..:-)
>
> Jim

Heh.

Thanks for all the help guys, this page isn't a state secret, we just
don't want it showing up if someone does a search for the client.

--
Sara

Cuddler of rats, cats and husband
First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4
Prev: Word question (or Pages I guess)
Next: MacZot / theMacSale / SuperCard