What has the Ministry of Magic Quadrants got against me?
I went into a restaurant
Lookin’ for the cook
I told them I was the editor
Of a famous etiquette book
- BOB DYLAN’S 115TH DREAM
So, Web Content Management has progressed from a Gartner MarketScope in 2008 to a Magic Quadrant in 2009. I’m normally quite a fan of Gartner, and was fortunate enough to hear Mick MacComascaigh (the lead WCM Analyst) give a great presentation at a recent event. We even had a nice chat about WCM Maturity Models afterward. However, I’ve got to say that it’s quite difficult not to treat this research as a giant advert for Oracle. To start, here it is:
Gartner Magic Quadrant for WCM Aug 2009
Who Is It For?
I think the introduction to the research is interesting. It starts by listing who would benefit from it:
This Magic Quadrant will help CIOs, and business and IT leaders that are analyzing their Web strategies to assess whether they have the right WCM offering to support them.
This seems to list everyone except the people that are actually going to use the thing and, as a result, seems to place very little weight on the things I care most about: usability for the editors and a warm fuzzy feeling for the developers. The report is far more concerned about market share, geographical penetration and long term company prospects. History shows us that a typical WCM implementation have an average lifespan of only three years. Is this because the buyers aren’t thinking about long term CIO/Business/IT issues? Or is it because the world changes fast and we shouldn’t worry our pretty little heads too much about things too far in the future?
CMS Watch had something to say about the report, mainly around the fact that it is too high level and strategic, and tends to ignore the “nitty gritty” details that can be so important. CMS Wire also talk about it here but they seem more happy with the whole thing.
As an aside, I also don’t necessarily think that a large stable company always leads to a large stable product. Just look at all the recent acquisition activity. The road map for some of the big boys is far from clear and some well established products might start to fester due to lack of R&D investment.
Gartner vs Forrester
So, how much to the two big analyst firms agree? Let’s have a look at the Forrester Wave from a couple of months ago:
The Forrester Wave™: Web Content Management For External Sites Q2 2009
So they do agree on a most things. Both have Autonomy/Interwoven and SDL Tridion in the Leader area. Fatwire, Open Text (pre-Vignette acquisition) and Day are all up there. Microsoft is struggling on both, although Forrester prefer their strategy while Gartner prefer their current offering. IBM, Vignette and EMC are also put into the same ballpark.
Gartner covers a few more vendors. The three smaller .NET vendors (Ektron, SiteCore and EPiServer) all make the grade. I’d have expected to see the EPiServer dot in almost the same place as SiteCore as, in my experience, the two always come extremely close in all evaluations I’ve seen. I’d have put them both ahead of Ektron, but maybe that’s just me. I guess EPiServer are only just starting their US invasion which might have penalised them a bit.
The two reports also agree on not including Open Source vendors for various reasons. Quite a few people in the blogosphere are upset about this. The cynical amongst you might think that this is because Open Source vendors don’t pay analysts as much to include them on reports, but this couldn’t possibly be anything to do with it.
But What About Oracle?
Aaah, yes. The anomaly. Oracle UCM nee Stellent comes first in the Gartner report while a distant tie sixth in the Forrester one. Oracle is going mental about this on Twitter and any other advertising channels it can find. The research cites Oracle’s strengths as the ability to integrate with other Oracle products, including their CRM system. I’m not a fan of these so-called “tighter integrations” which are diametrically opposed to my view of loosely-coupled separation of concerns. Sounds like Gartner want Oracle UCM to become a monolith which reaches far beyond the boundaries of what I’d define as Web Content Management. Interestingly, Gartner’s three leaders are all vendors more traditionally associated with ECM. Price doesn’t seem to be a factor at all in the quadrant.
For the very observant among you, note that all the URLs to Gartner’s public “sponsored” research contain /oracle/, not just this one. So I wouldn’t get hung up on that – I presume it’s the platform they use? [UPDATE: See the comment about this below]
Niche Players: Good News and Bad News
Here’s the good news. Gartner advises that maybe a Niche Player is good for you:
Gartner advises organizations against simply selecting vendors that appear in the Leaders quadrant. All selections should be buyer-specific, and vendors from the Challengers, Niche Players or Visionaries quadrants could be better matches for your business goals and solution requirements.
Here’s the bad news. I can’t for the life of me figure out why anyone would want to look at a Niche player according to Gartner’s metrics. They have no ability to execute, and no complete vision. Sound like a bunch of losers. In fact, in my view it’s probably better not to be on this MQ at all than to be a niche player. You’ll save yourself a bunch of money, and a bunch of bad publicity. Nstein tweeted happily about being included on the quadrant for the first time. On the other hand, I’d be furious if I was an established vendor like Alterian and got stuck in the bottom right. So, a question to all you vendors – would you rather be on this as a Niche Player, or not on it at all?
Note that in the MarketScope from 2008, both IBM and MediaSurface (Alterian) were “Cautions”. IBM have progressed safely into the Challenger zone while I’m not sure exactly what they’ve done in the last 12 months to get there. And poor Alterian seem like the victims here, being penalised primarily, it seems, for having more than one WCMS product. Now while this can be confusing, they certainly aren’t the only vendor in this position.
I’ve decided I certainly prefer the research that doesn’t rank the vendors and serve as self-fulfilling marketing for the vendors that do well. The strengths/weaknesses listed are really high level and vague, so you need to take a leap of faith. This kind of advertorial isn’t for me, I’m afraid. Maybe if Gartner published the complex calculations (and they are complex) that go on behind the scenes I’d trust it more.
So Why Trust Me?
Well, I’m Head of Development for the company that came first in the most recent Forrester Wave™: European Interactive Agencies — Web Design Capabilities. What more proof do you need?
Arthur Weasley: This is very, very peculiar. It seems as if your hearing is to be in front of the entire Wizengamot.
Harry Potter: I don’t understand. What has the Ministry of Magic got against me?
RSS Feed
Follow Me
Can we not come up with some cool 3d magic CMS hexagon to rule them all?
Also, every buyer should have some chance to define if he wants rabbits or pigeons flying out of his system before the quadrant is generated…
To be honest, – I think the Gartner report is quite useful, but still lack some kind of dimension (Finnur Jokes about a 3D hexagon, but…:)
I think Gartners report should also illustrate which kind of companies the software is designed for. For example, Ektron and some of the other companies is best suitable for small/medium organizations, while vendors such as Open Text/SDL/Sitecore is for medium/large organizations and Oracle only for large organizations.
If I was about to select software for my large corporation and would be using the Gartner Quardrant to support my decision, – I would be very unhappy to shortlist Ektron.
In the other hand, was I a smaller company, shortlisting Oracle would be a waste of time as well.
Yeah, for me the whole thing lacks the detail you need. If only they would “show their workings” it might reveal a whole lot more. As it stands, it seems like one generic answer to a whole lot of different questions. Your Ektron example is a good one.
The link has oracle because they are paying for everyone to read it. They are happy and want everyone to see it.
Great post. Formulating some thoughts to add. Need more coffee first.
-Pie
You’re right about the link, they don’t all have Oracle. For example:
Microsoft came 5th in this but still sponsored it:
http://mediaproducts.gartner.com/reprints/microsoft/vol6/article3/article3.html
Oracle came tie 3rd in this:
http://mediaproducts.gartner.com/reprints/oracle/article31/article31.html
Can anyone shed any more light on how the process works? I know vendors participate without knowing where they’ll end up on the curve. Do they fight over prime sponsorship after the results come out?
SiteCore seem to be the one shouting about their position the loudest in the Twittersphere.
Sometimes when I’ve read “reports” like these, I just get the sense that it’s some unimportant person trying to mention all the guys he want to feel important with. And naturally bash the ones he’s tried but failed to feel important with.
But then again, I’m not very corporate so I might be missing something.
Follow up to my observation above would be “Does these reports actually matter and do they have an impact?”. Can you “on the floor” feel the wind changing when a report like this is publicised?
Yep, they matter. We’re going to see them in many a PowerPoint deck from all the guys that did well. And the customers often (wrongly, IMVHO) use them to produce a vendor long/shortlist. They’ve got a lot of influence, so become self-fulfilling sometimes. Which is why vendors cough up the dough to be included.
These reports are largely filled with vague information that does not necessarily contribute to selecting a software product. Even using reports for generating a shortlist can be dangerous as previously mentioned. The takeaway from market research is just that – it can be used to justify the growth or contraction of an industry, and the bottom line is that the WCM market is thriving.
Each web project’s needs should be carefully evaluated and prioritized to aid in a CMS selection process. In Ektron and Oracle’s defense, a website’s core functional requirements carry more weight than lumping an organization’s needs into those of a large corporation vs. those of a SMB. In some cases Ektron may be a fit for large org’s, but less so for Oracle strictly from a cost perspective.
Full Disclosure: I am an employee of Ektron, but my thoughts are of an objective nature and do not represent statements of the firm.
Now SiteCore are also sponsoring a download of this thing. But you’ve got to register first:
http://www.sitecore.net/en/Products/Resources/whitepapers/Gartner-Magic-Quadrant-2009.aspx
I’m the Gartner Ombudsman and I can answer Jon’s question about the Reprints process. The documents you find through various vendor’s websites are research reprints purchased by the vendor. They are not sponsored research. The vendor decides to purchase a reprint once they see the final version of the document which is why it’s typically the vendors who like their position who make the document available on the web. Here is more information about reprints on gartner.com http://www.gartner.com/it/products/research/media_products/reprints.jsp
Gartner does not accept sponsorship of research or produce vendor-sponsored white papers. The analysts determine which vendors are included in these reports by setting clearly defined inclusion criteria explained in the document.
I hope this clears up any confusion.
Larry,
I can’t tell you how much I appreciate the clarification straight from the horses mouth. My skepticism stems from ignorance of a process which, to the uninitiated, seems rather opaque. Combined with the fact that the “sponsorships” seem to be in place before the report is launched to the public causes doubt.
That said, the process you describe seems completely legit to me. And, as Darren from SiteCore pointed out, the company for which I work proudly cites our rankings on various waves and quadrants. A case of “Physician, heal thyself” and all that.
So, in summary, is the process something like this: Vendor pays Gartner X to be included in the research, but have no idea how they will be ranked. Then, once the evaluation is complete, there is some way they can pay Y to freely distribute the research? Finally, do you get any backlash from the vendors that pay to be included, and don’t do as well as they’d hoped?
Thanks again for shedding light on a process that someone like me didn’t understand.
I love how we are all contributing to this thread on a open source platform and that the inclusion to such Quadrants mean minimum $8m of license revenue you cannot make the Magic Q !! My commercially supported open source organisation did nearly $20m last year and we are still growing globally unfortunately missed out which begs for some ongoing debate…..
Thanks for starting this thread Jon boy.. twitter.com/squizme
Hey Mermaid Boy!
I agree with you that it’s a pity the Open Source vendors don’t qualify for inclusion. I understand the reasoning but I still wish they’d be included.
That said, I also agree 100% with Eric Barroca’s discussion of Commercial Open Source which, I think, might not gel with your business model. I’d love your thoughts on his recent post:
http://blogs.nuxeo.com/ebarroca/2009/08/commercial-open-source-or-just-a-free-demo.html
Jon – actually, vendors don’t pay Gartner to be included in the report. Gartner selects a list of vendors that it considers to be the top vendors in the field. They then invite those vendors to participate in the report. Regardless of how they rank, they fact that Gartner asked them to participate *at all* is a sign that they are amongst the top in their field. Gartner then evaluates in a multi-month process that takes in account more than just the vendors responses to a detailed set of questions. I’ll leave it to Gartner to speak to any details, but having done many of these types of evals from different vendors, I can say that they analysis is not vendor-driven. Far from it.
There is always the risk of being ranked low. Luckily at Day, we are quite pleased
You take the risk because even if you fare poorly in the overall quadrant, the fact that you are included at all is an important statement. It is better to be in the quadrant than not.
Now, some have blogged about some of the criteria Gartner uses. And that’s why we luckily all look at different types of sources and perspectives. But as a specific source, Gartner’s – like CMS Watch and others – is not based on who paid to be included.
That’s just a vendors perspective. You fret about whether or not you’ll be included, then you fret about where you place. It’s only fun after the fact. And so for that reason, hats off to everyone in the report. Of course, do take a close look at that Day dot …
Oh, and final note on open source being excluded (aside from the obvious fact that at Day we see ourselves as a commercial vendor based on an open source core) … read the full report where Gartner specifically mentions the criteria for inclusion and the list of open source vendors they evaled against those criteria. I wouldn’t be surprised to see more open source vendors in next year, especially with vendors like Nuxeo and Jahia now entering the North American market (geographic reach is one criterion for inclusion). Good discussion.
Hi Jon,
As I have already said on Twitter, nice post and thank you for the mention.
You’ve asked how we feel about being on the report, in the position we find ourselves – would it be better not to be included?
This is consistent with feedback I have had, but when contrasted with the comments in the report like “This product remains a top-class WCM product and its impressive…etc etc”- that appear inconsistent with the bottom left position – we have to say that engagement with Gartner is valuable – providing folks take the time to read the report and glance beyond the headlines.
In the interest of balance, the poor showing in the quadrant is apparently down to what they consider to be poor US visibility and the negative comments about strategy. These comments owe much to the lag to get the report published from the briefing and how much we were able to share at the time. Incidentally subsequent conversations where we have eased open the kimono (I believe that’s the custom – otherwise I looked pretty foolish) have been received very positively.
On the MarketScope comment that was made last year, I feel the need to point out that ‘cautious’ came around the time of the Alterian acquisition and reflected what they considered to be risk to us as a standalone vendor.
Anyway, super post and great debate – love the fact you attracted the attention of the “Gartner Ombudsman” – (isn’t it great there jobs out there called that?)- and hope I have gone some way to answer your question!
Cheers,
Ian
Ah.. just been pointed out to me that I’ve failed to type the URL correctly in the website link on my last comment. It’s http://www.persuasivecontent.com and I am the VP, WCM Product Strategy at Alterian.
Hi Jon, Kevin’s explanation is exactly right and I hope it puts to rest any perception of “sponsorships”. I’m always happy to answer any questions … larry.perlstein@gartner.com or 203-316-6024.
Ok, so the vendors may not be paying for the evaluation, but they do pay fees to these firms. According to Theresa Regli, Gartner spends c. 70% of time advising vendors and investors, 30% of time advising buyers. Should clients have confidence in that?
But my issue is that assessments are made based on a vanilla basis with little practical project or support experience. How many of these assessments are actually made by going to systems integrators who’ve had to work with these products, or to clients who’ve had a bad experience with them?
Moreover, how much evaluation is done with the tools on the ground, rather than with the latest sandbox of the newest version that actually represents only 1% of the client base?
When you’re picking a CMS just try to find the one that’s the right fit for your organisation, not someone else’s.
By the way, that applies even to digital agencies who find themselves on the crest of the wave!
Even better if it is a long wave
Forrester haven’t done another Interactive Agencies wave in Europe since. So we’re still on top. They did one this year in the US where our office their did pretty well. They’re listed as Icon Nicholson on there.
I do quite like the way Forrester rate sites. But the fact that the wave is based entirely on 2 *self-volunteered* projects is a bit odd.
EMC has finally given up on their current “WCM” offering and partnered with FatWire – info here.
So Forrester had it right.
I’ve become an enthusiastic supporter of your site for some time and not really offered anything at all back, I really hope to alter that later on with more debate.Thanks for another new addition to your web site.
Some truly nice and useful info on this site, likewise I believe the design has got excellent features.
[...] way that Jon discusses how both Forrester and Gartner don’t go into enough details in their Content Management comparison reports, I feel they do the same when discussing and comparing agencies. Good digital agencies do provide [...]
Excellent confident analytical eye designed for fine detail and may foresee difficulties before they occur.
I have a notable analytical vision with regard to detail and can foresee troubles just before they will occur.
NHL Jerseys are between the smartest matters on the North American marketplace now.
Kravitz also notes that travelers should always purchase helicopter tours online.
Replacing the 300W potential deliver adds an alternative
$50-$150 to your total price.
Feel free to surf to my homepage league of legends reddit wallpaper – Eloboosting.livejournal.com -
I’m not sure where you’re getting your info, but great topic.
I needs to spend some time learning more or understanding more.
Thanks for wonderful information I was looking for this info for
my mission.
Oh my goodness! Awesome article dude! Thanks, However I am going
through problems with your RSS. I don’t know
why I can’t join it. Is there anybody getting the same RSS problems?
Anyone that knows the solution can you kindly respond?
Thanks!!
It’s genuinely very complex in this busy life to listen news on Television, therefore
I simply use the web for that reason, and take the latest information.
This is very interesting, You are a very skilled blogger.
I’ve joined your rss feed and look forward to seeking more oof your excellent post.
Also, I have shared your web site in my social networks!
my weblog herbal products
It’s hard to find your blog in google. I found it on 18 spot,
you should build quality backlinks , it will help you
to increase traffic. I know how to help you, just search in google – k2 seo tips and tricks
I got this website from my friend who informed me concerning this site and now this
time I am browsing this site and reading very informative posts here.
Great post. I was checking continuously this blog and I am impressed!
Veery helpful info particulrly the last part
I care foor such information a lot.
I was seeking this certain information foor
a long time. Thank you and good luck.
Here is mmy web-site; Communication plan – Karla -
Nice blog right here! Additionally your site
so much up fast! What web host are you the use of? Can I
get your affiliate link for your host? I desire my website loaded up as quickly as yours lol
With new discoveries and advancement in life
saving equipments Air Ambulance has came under limelight.
excellent Blog I’m a big bet365.com fan from Holland
excellent blog I’m a huge Las Vegas travellerfan from Thailand
Hey, on occasion I get a 503 website error when I arrive at your page. Just a heads up, regards
Hey, sometimes I get a 500 website error when I browse your website. Just a heads up, cheers
Hey, on occasion I get a 503 site error when I browse your site. Just a heads up, regards
You actually make it appear so easy together with your presentation however I in finding this matter to be actually one thing that I feel I’d never understand. It kind of feels too complex and very huge for me. I am taking a look forward to your next put up, I’ll attempt to get the hang of it!
We absolutely love your blog and find the majority of your post’s to be exactly what I’m looking for.
Do you offer guest writers to write content for you?
I wouldn’t mind publishing a post or elaborating presentation on zakat in urdu many of the subjects you write regarding here.
Again, awesome website!
You can definitely see your expertise in the work you write. The world hopes for even more passionate writers like you who aren’t afraid to mention how they believe. All the time follow your heart.|
It is best to participate in a contest for among the finest blogs on the web. I’ll advocate this website!
nike kyrie 5 http://www.kyrie5s.us