So, you’re not happy with search engines (Google in particular) for exposing your content to an audience who aren’t paying for it. You don’t like the way they include snippets in their search results of your stories and you definitely don’t like them referring visitors to your websites (unless they are paying customers). You seem aggravated with aggregation and less than happy about linking. So here’s a suggestion for you.

Google themselves (yep those folks you keep moaning about) have kindly provided a really easy way for you to get your content out of their search index, and you can just block the news search crawler if you want now too! It couldn’t be easier. Let’s take a fictional website In the very first directory where the website files are stored (the root) you’ll find a text file called robots.txt (so that file lives at All you need to do is find that file, open it up in Notepad (or a text editor of your choosing), and add the two lines of text below to it. Save it and your Google problems are over (if this all proves a bit tricky, get one of your lovely web developers to help you (while you’re at it why not ask them if they think it’s a good idea too?).

User-agent: Googlebot

Disallow: /

Job done, give it a couple of weeks and none of your pages will be in Googles index anymore. That troublesome traffic will be no more. Satisfied?

Now come on, thats not the answer is it? Here’s an idea for you. Why not devote your time, energies, finances and skilled personnel into coming up with a new model to make all this free traffic and advertising work for you? Rather than moan about it, find a way to make it work for you. It’s about time someone made some advances in the world of online display advertising and I’d have thought that with all your web properties you’d be just the man/organisation to do so. The possibilities are endless, start to tap into the rich data you can glean from the tracks your web visitors leave each time they visit. Learn from it, find ways to encourage repeat visitors and new channels to monetise them through. Of course you may want to reconsider my earlier recommendation first, otherwise you won’t have enough traffic to benefit from any improvement to your advertising and other revenue streams.

If you can design experiences that encourage visitors to become loyal users of your content maybe you could even sell them something? Maybe (just maybe) if you make your sites engaging enough some might even subscribe! There are so many ways you could make more revenue from so much traffic. I’ll be available in July 2010 to help if you haven’t worked it out by that time…

What are you going to do? Just block the traffic, alienate potentially loyal users and try and get people to pay for your content? Or move forwards proactively, embrace the fact you get so much traffic to your sites (it’s a good thing, honest) and work out a really viable model to monetise it properly.

Personally, I’d go with the latter (with so many pages, so much content and so much traffic you have endless possibilities). The former just strikes me as the reactionary moves of an industry with so much promise in the digital world that gives off an impression of being on it’s last legs.


Lot’s of Yahoo coverage in the blogosphere at the moment. Here’s a few key pieces:

Yahoo are still launching new products and innovating. Examples include the News Globe mashup (via Mashable) and the exciting new mobile product oneConnect (an app allowing email IM and social networking to all come alive on your mobile phone, very promising).

The layoffs have started (again via Mashable).

Key players are leaving, Bradley Horowitz a key figure responsible for the area of Yahoo who come up with new and innovative products is apparently jumping ship to Google (via Techcrunch). This is a big loss, I’ve seen him at conferences and he’s a very sharp guy.

Microsoft are still pushing for the merger to happen and rumours are that they may take their offer straight to the shareholders (again via Mashable).

And to top it all, Yahoo are said (via Techcrunch) to be in continuing talks with News Corp about a deal which could see them become a massive powerhouse with the properties to compete with Google (although not quite on eyeballs).

Where will this all lead? Who knows, but Yahoo need to resolve the merger issues and rebuild confidence in their employees and shareholders, losing key hires and all this talk of deals is going to begin to hurt them if they don’t settle on a direction to follow soon.