Today the process is much more rigorous, even onerous, for the analysts. Key criteria are published for inclusion, usually based on some lower limit cut-off for revenue. An analyst has to defend any decision to exclude a vendor if they meet those published criteria.
In addition to frequent briefings from each vendor the analyst tracks the number of inquiries received about each product from the Gartner client base of 11,000 mostly large, enterprises. A questionnaire is put together with as many as 150 questions in spreadsheet form that address everything from feeds and speeds to product revenue by region. When all the results are tabulated, rankings of 1-5 and weights of low-standard-high are applied and the positions of the dots are determined.
Of course, the rankings and weights are subjective but at least a common set of questions and weights are applied and the results retain some objectivity. And, the analyst can indeed use the Magic Quadrant to push an agenda. Analysts are not just researchers and reporters. Their job is to see where the industry is going and help it along a path that addresses the needs of the Gartner client base -- large enterprises.
Here are three factors to incorporate in your own product selection methodology that can help you leverage the Gartner magic Quadrant.
First, you only have limited resources to devote to doing the research on the myriad products in each sector. Consider the expertise of the Gartner analysts that compile the MQ. They usually have hands-on experience with at least one of the products. While that may have been years ago they are truly experts in evaluating companies and are fully immersed in determining which vendors are best at addressing your problems.
Your in-house experts may be better technologists than any analyst but they cannot help playing favorites. They resist change and may have a long relationship with a particular vendor. They may be best friends with the incumbent sales person. They often lack objectivity. The Gartner analyst knows more than you do. Leverage that knowledge.
Use the MQ as a starting point. Do not short list the Leaders and ignore the other quadrants. Look at each vendor and apply your own criteria. A vendor in the Niche quadrant may be there because they serve your industry sector exclusively. Or they may serve only your region. Your own IT infrastructure may qualify a vendor for the short list. If you are an exclusive IBM shop you are going to over-weight the product from IBM. So, take advantage of the leg work the Gartner Analyst has done but apply your own criteria to develop a short list for evaluation and proof of concept. If you are a Gartner client do the industry a favor and keep your analyst in the loop throughout the process. You will be informing future Magic Quadrants.
Sign up for CIO Asia eNewsletters.