Now the May Day panic has calmed down, I've done some investigation behind it.
Please bare in mind that this is speculation backed up by what other people have said and the trends I've been seeing.
There is nothing official until Google makes an announcement about how they screwed the internet -and they've had 18 days to say something about May Day so far.
...this is probably a transition period where the new updates are being implemented which means the “trust factor” for authority sites have been temporarily turned off and naturally the long tail pages lose their place to smaller micro-niche websites during this period since the latter is well optimized. So once the update is complete and trust factor is turned back on, these sites will get back to their original position.
I agree with this, but I don't believe the trust factor and authority will return.
That's just wishful thinking. Google is not a merciful God!
I believe that it comes down to a couple of things:
1) Google Doesn't Crawl Sites As Deeply
It skims the top level and what the top level links to. Focus on new and fresh content. It stores info from an early crawl and doesn't recrawl pages that get buried in a hurry.
2) Niche Sites Rewarded
Nothing has been said about keyword domain name urls.
Much like domain name urls, niche sites got a boost or stayed the same.
4) Exact Match Long Tail Rewarded
In long tail search, exact match seems to take priority over domain strength now.
5) Last May Was Poor for Traffic and Sales
I've attached a screenshot of our Google Analytics with some photoshopped additions.
Last May was appauling for both traffic and sales. So until we're at the end of June we can't be sure what kind of impact the May Day update has really had.
* Big sites have lost a lot of long tail traffic
* Smaller niche sites (including blogs) have stayed the same or increased in traffic
* Sites with lots of exact match long tail terms in their content got rewarded.
We've seen what I call "first come, first serve"
Like FCFS, I believe the May Day update is more Google favouritism of what's new and fresh by focussing their engine on what's hip and happening in the online social network.
This is why google bot doesn't crawl as deeply. If it's not on or linked to from the top of your site, then it's old hat and out of date.
As a result, Google regularly crawls site front pages (I'm thinking of blogs in this instance), which host their latest content there while the more intelligent designs link from the homepage direct to important articles like 'walking boot guide' for example, which will then be crammed full of long tail search terms.
Niche sites write about anything and everything. So it's expected that they'll mop of a lot of the information seeking terms and a lot of long tail with it. And when it's a niche within a niche, it's even stronger.
However, the less savvy sites wil write something like 'walking boot guides' or 'guide to walking boots' when the big search term is 'walking boots guide'. Through exact match long tail these get dredged up.
Also, google still cobbles together search terms from sites. So spammy keyword density is also a factor. Hence why a lot of spammy affiliate sites have apparently done well from the May Day update.
More on site content
More blog content
Site wide optimisation of pages and unique content with various combinations of keywords.
Optimise the homepage meta in the same way that Blacks.co.uk has for multiple rankings.
TROUBLE IN PARADISE
If our site was already optimised to the eyeballs and our traffic dropped, what then?
To mop up all the other long tail terms massed blog posts would be called for.
Better integration between website and blog could be called for to convert sales from blog posts.