Comments on: Controlling the Googlebot

By: Netbooks UK

Netbooks UK — Tue, 17 Mar 2009 12:41:18 +0000

Sure robots.txt might work, but what about after the pages have already been indexed. How do you get google to remove them?

By: Netbook Review

Netbook Review — Sun, 08 Mar 2009 16:05:17 +0000

I think robots.txt is a good tool, but be careful what you restrict!

By: Netbooks

Netbooks — Mon, 23 Feb 2009 20:23:57 +0000

I am a big believer in using robots.txt

By: Mark

Mark — Mon, 24 Sep 2007 15:38:52 +0000

flys – Google can take quite a while to cancel pages, however, this of course depends on the frequency of their visits.

By: What to do when you publish by mistake « Church website and blog ideas

What to do when you publish by mistake « Church website and blog ideas — Fri, 22 Sep 2006 12:18:50 +0000

[...] One webmasters comments and help when things are posted by mistake has been written about on Problogger. [...]

By: Brian

Brian — Wed, 06 Sep 2006 20:35:33 +0000

this site rocks! http://www.sis.utk.edu/Members/derik/5.html

By: adfi

adfi — Sat, 12 Aug 2006 21:31:30 +0000

Yes. Very good site!

By: flys (Marketing Lessons)

flys (Marketing Lessons) — Mon, 10 Jul 2006 16:11:50 +0000

Very interesting article.
How long Google take to cancel the pages using URL removal tool ?
bye

By: » links for 2006-07-10 Ian’s Messy Desk

» links for 2006-07-10 Ian’s Messy Desk — Mon, 10 Jul 2006 13:00:04 +0000

[...] Controlling the Googlebot (tags: blogging Search Tools Google Bots) [...]

By: Aaron Brazell

Aaron Brazell — Sun, 09 Jul 2006 12:36:10 +0000

John, I’ll presume that you realize I have no clue about Google updates. I wait like anyone else. I’ll also presume that you were not aware that there was a PR update last week.

By: John Evans (Syntagma)

John Evans (Syntagma) — Sun, 09 Jul 2006 09:53:03 +0000

Matt Mullenweg actually had the noindex tag put on his site maliciously. Now that’s a dismal thought.

Aaron, any info on when this delayed PR update is going to happen? Matt Cutts is back. What’s the holdup?

By: Michael Moncur

Michael Moncur — Sun, 09 Jul 2006 06:50:28 +0000

I always operate on the assumption that there are only two possibilities: something is private (unpublished) or it’s public (published). If I publish I assume that everyone on the world will have a chance to read everything on the site at the worst possible moment.

I haven’t had to beg Google to remove a URL yet…

Okay, that’s a lie, I’ve used Google’s URL Removal Tool a few times to correct a less dangerous mistake, but it’s one you should also watch out for:

* Always password-protect “beta” or “temporary” versions of a site.

It’s convenient to have a beta site online to test a new design or function, but be careful lest Google spider the whole thing and start substituting it for the results on your original site…

By: Nukes and Candy

Nukes and Candy — Sun, 09 Jul 2006 04:45:55 +0000

“…If you don’t want the world to see it, then don’t rely on the mechanisms listed above. Simply don’t post it.”

This phrase is paramount.

By: Brad

Brad — Sun, 09 Jul 2006 04:26:10 +0000

If I don’t want anyone to read something, then I don’t publish it on any of my websites! But, since my blog is still fairly new, I’m happy anytime I get a visit from the googlebot in hopes it helps my search engine ranking.

By: raj

raj — Sun, 09 Jul 2006 00:12:36 +0000

There is an old saying that if two people know something, it’s a secret. If three people know it, it’s public knowledge. If you need to publish sensitive info, you might want to try a password-protected site. If the info is truly life-threatening, there’s no way you should post it on the site at all. Because what if someone else links to it? [Provided you haven't changed your .htaccess file or used Google's URL remover.]