Comments on: Robots.txt & The Google Supplemental Index-Update http://blog-op.com/robotstxt-the-google-supplemental-index-update/ Opportunities, Opinions, Optimize Wed, 03 Dec 2008 20:04:57 +0000 http://wordpress.org/?v=2.6.5 By: Chris Lodge http://blog-op.com/robotstxt-the-google-supplemental-index-update/#comment-4201 Chris Lodge Tue, 26 Jun 2007 11:23:07 +0000 http://blog-op.com/robotstxt-the-google-supplemental-index-update/#comment-4201 Thanks Zaki, good tip. Thanks Zaki, good tip.

]]> By: zaki http://blog-op.com/robotstxt-the-google-supplemental-index-update/#comment-4197 zaki Tue, 26 Jun 2007 02:03:15 +0000 http://blog-op.com/robotstxt-the-google-supplemental-index-update/#comment-4197 I have implemented this before, it works but it takes some time to clear out you pages from supplemental index. From my experience, it clear out almost 7k URL in 2 weeks. Other than robots.txt, you can try using google webmaster removal URL tool I have implemented this before, it works but it takes some time to clear out you pages from supplemental index. From my experience, it clear out almost 7k URL in 2 weeks.

Other than robots.txt, you can try using google webmaster removal URL tool

]]> By: Matt Jones http://blog-op.com/robotstxt-the-google-supplemental-index-update/#comment-4175 Matt Jones Sun, 24 Jun 2007 14:13:46 +0000 http://blog-op.com/robotstxt-the-google-supplemental-index-update/#comment-4175 Hey Chris, just thought I'd let you know I'm selling my blog! All the details are on the homepage. Matt Hey Chris,
just thought I’d let you know I’m selling my blog!
All the details are on the homepage.

Matt

]]> By: Chris Lodge http://blog-op.com/robotstxt-the-google-supplemental-index-update/#comment-4174 Chris Lodge Sun, 24 Jun 2007 14:12:08 +0000 http://blog-op.com/robotstxt-the-google-supplemental-index-update/#comment-4174 <p>That's pretty good Rob.</p> <p>My feed files are about the only ones left in the supplemental as well, and I'm tempted to say that excluding the XML files would mess up accessing your Sitemap.</p> <p>From Andy Beards comments on my earlier post, supplemental isn't the end of the world, and I would suggest that getting the bulk of your posts from the supplemental to the main index is enough.</p> <p>Let us know if you decide to experiment though ;-)</p> That’s pretty good Rob.

My feed files are about the only ones left in the supplemental as well, and I’m tempted to say that excluding the XML files would mess up accessing your Sitemap.

From Andy Beards comments on my earlier post, supplemental isn’t the end of the world, and I would suggest that getting the bulk of your posts from the supplemental to the main index is enough.

Let us know if you decide to experiment though ;-)

]]> By: Rob O. http://blog-op.com/robotstxt-the-google-supplemental-index-update/#comment-4171 Rob O. Sun, 24 Jun 2007 08:37:50 +0000 http://blog-op.com/robotstxt-the-google-supplemental-index-update/#comment-4171 I've whittled my supplimental results down from around 350 to 26. And I've made a few more tweaks to my <b>robots.txt</b> file that may reduce this a smidge more. To avoid the duplicate content from counting against me, should I exclude *.xml files so Google isn't picking up my feed files? Or will this cause me grief with my <b>sitemap.xml</b> file? I’ve whittled my supplimental results down from around 350 to 26. And I’ve made a few more tweaks to my robots.txt file that may reduce this a smidge more.

To avoid the duplicate content from counting against me, should I exclude *.xml files so Google isn’t picking up my feed files? Or will this cause me grief with my sitemap.xml file?

]]>