Recent Posts Tagged With 'x-robots-tag'

• How to handle a machine-readable pandemic that search engines cannot control

  Posted on Friday June 19th, 2009 at 15:59 in google, SEO, blogging, web development, microformats, nofollow, cloaking, paid links, search quality, x-robots-tag, risky linkage

  When you’re familiar with my various rants on the ever morphing rel-nofollow microformat infectious link disease, don’t read further. This post is not polemic, ironic, insulting, or otherwise meant to entertain you. I’m just raving ...

• About the bad taste of shameless ego food

  Posted on Friday January 30th, 2009 at 17:15 in SEO, robots.txt, robots meta tags, x-robots-tag, ego food, url removal

  Seems I’ve made it on the short-list at the SEMMY 2009 Awards in the Search Tech category. Great ego food. I’m honored. Thanks for nominating me that often! And thanks to John Andrews and Todd Mintz for the kind judgement! Now that you...

• Crawling vs. Indexing

  Posted on Monday October 20th, 2008 at 12:08 in SEO, robots.txt, crawler directives, robots meta tags, x-robots-tag

  Sigh. I just have to throw in my 2 cents. Crawling means sucking content without processing the results. Crawlers are rather dumb processes that fetch content supplied by Web servers answering (HTTP) requests of requested URIs, delivering those cont...

• @ALL: Give Google your feedback on NOINDEX, but read this pamphlet beforehand!

  Posted on Monday February 25th, 2008 at 05:08 in robots.txt, search quality, crawler directives, robots meta tags, x-robots-tag, url removal

  Matt Cutts asks us How should Google handle NOINDEX? That’s a tough question worth thinking twice before you submit a comment to Matt’s post. Here is Matt’s question, all the background information you need, and my opinion. What is ...

• Get a grip on the Robots Exclusion Protocol (REP)

  Posted on Thursday January 17th, 2008 at 08:26 in web development, microformats, robots.txt, crawler directives, robots meta tags, x-robots-tag, xml-sitemaps, url removal

  Thanks to the very nice folks over at SEOmoz I was able to prevent this site from becoming a kind of REP/robots.txt blog. Please consider reading this REP round up: Robots Exclusion Protocol 101 My REP 101  links to the various standards (robots...

• Getting URLs outta Google - the good, the popular, and the definitive way

  Posted on Monday January 14th, 2008 at 15:01 in web development, duplicate content, webmaster central, robots.txt, crawler directives, robots meta tags, x-robots-tag, url removal

  There’s more and more robots.txt talk in the SEOsphere lately. That’s a good thing in my opinion, because the good old robots.txt’s power is underestimated. Unfortunately it’s quite often misused or even abused too, usually be...

• My plea to Google - Please sanitize your REP revamps

  Posted on Thursday January 3rd, 2008 at 18:02 in web development, msn, microformats, nofollow, robots.txt, crawler directives, robots meta tags, x-robots-tag, xml-sitemaps

  Standardization of REP tags as robots.txt directives This draft is kinda request for comments for search engine staff and uber search geeks interested in the progress of Robots Exclusion Protocol (REP) standardization (actually, every search engine m...

• Google to change the Robots Exclusion Protocol again

  Posted on Friday November 30th, 2007 at 12:33 in microformats, nofollow, robots.txt, crawler directives, robots meta tags, x-robots-tag, xml-sitemaps

  Web crawler directives, partly standardized in the Robots Exclusion Protocol (REP), evolved since 1994. Nowadays we’ve to deal with a conglomerate of not binding de facto standards and microformats, all of them extended by various organizations...

• Act out your sophisticated affiliate link paranoia

  Posted on Tuesday November 13th, 2007 at 01:09 in web development, e-commerce, nofollow, cloaking, robots.txt, redirects, paid links, search quality, crawler directives, x-robots-tag, risky linkage

  My recent posts on managing affiliate links and nofollow cloaking paid links led to so many reactions from my readers that I thought explaining possible protection levels could make sense. Google’s request to condomize affiliate links is a bit,...