How to Keep a CMS Vendor on their Toes

You say my eyes are pretty and my smile is nice
Well, I’ll sell it to ya at a reduced price
You don’t understand it – my feelings for you
You’d be honest with me, if only you knew
- HONEST WITH ME

We’ve all read plenty of CMS Evaluation RFP response documents. And we’ve all sat through many a long CMS Vendor demo. Maybe it’s just me, but they’re all starting to look pretty similar. Most vendors say yes to nearly everything, and the top products have interfaces and patterns that are converging. Many experts are agreeing that having long lists of requirements doesn’t help anyone anyway. So, how do you differentiate between these things?

When picking a CMS, I think the actual product is only about half the story. The other half revolves around the intangibles including the vendor themselves, their support, their roadmap and priorities and, very importantly, how much you trust them. So, to mix things up a bit, why don’t you ask the vendors a few of these questions during your next RFP or Q&A session. When you interview a candidate for a job, you always throw in a few odd questions to keep them on their toes. Let’s do the same to the vendors.

Who was the last vendor to beat you in the last round of a selection exercise? Why do you think they won? – This doesn’t have a right answer, but every vendor has lost a few. It would be interesting to see how they answer this.
If, in a few years time, we decided to move away from your product, how would I go about migrating all my content into a new system? – I want a system with a nice content export and a vendor willing to admit it is a possibility.
How many active developers do you have on your developer forums? – This is something you can check. If you ask in the demo and you think they’ve made up a number, say “Show me”.
All of these are important, but please rate these in order of your priority: a) Product Features b) Performance and Stability c) Usability d) Security – Again, no correct answer. I’d expect most to say c) as customers give this as the biggest selection criteria. But still interesting to hear what they say after they’ve repeatedly said they’re all important.
How much would I expect to pay a contractor developer that is skilled with your CMS, and are they easy to find? – If a vendor says they don’t know, they’re probably talking horseshit.
Assuming your CMS license cost 10 groats, how many more groats would you guesstimate we need for our entire implementation? – I’d expect them to say somewhere between 10 and 40, depending on the project. But fun to hear what they say. Implementing costs more than buying.
Why are the URLs on your demo site/corporate site so ugly? – Most of the demo sites have crap, non-SEO friendly URLs. If the implementation of Friendly URLs is as easy as the vendor claims, what haven’t they done it on their own sites.
If we selected your CMS, how would you recommend we went about selecting an implementation partner? – Three kinds of answers here: a) we’ll build it for you b) we have an extensive partner network or c) we recommend Company XYZ. I’m skeptical of a), but even if you like that option it’s a good discussion to have with the vendor.
How important is accessibility to you, and why doesn’t your site (which is presumably built with your CMS) validate? – They almost never do. I did a W3C validation test here not so long ago. Most will blame the implementation. Say will say it wasn’t a requirement, which I don’t buy. They fact that they don’t validate 100% isn’t a disaster, but it has got to look like they’ve tried.
Your product also includes modules for analytics/search/community/collaboration/outbound email. When should I use these instead of the existing products I own for these, and what benefits do your products give? - Many vendors seem to use features that I wouldn’t consider core to Content Management in order to differentiate. I like my CMS to be focussed, but I’ll concede that sometimes getting other components as part of the same product might be interesting. But ask things like “So, how does your analytics compare with Omniture” and see what they say.

Even better, during the meeting you could have a screen running a Twitter search for the vendor’s name. You’ll see a fair bit of “Looking at XYZ” and “XYZ Industry News”. But you’ll also see a lot of “A Pox on XYZ and all of their houses” from angry developers.

Assuming it isn’t illegal to ask this kind of thing, I’d love it if people gave it a try. Or do you already have similar questions you ask that I could add to the list? One day I might be sitting as the same meeting as you and we can both enjoy some questions we haven’t heard twenty times before.

Jon On Tech

Archives