You Got KLAS, Kid!

When I buy a car, or a computer, or a digital camera, I want to tap into the experience of others who have purchased a particular model. Hence, I subscribe to Consumer Reports, and peruse Internet sites such as CNET.com. For PACS systems and other healthcare needs, the KLAS report from KLAS Enterprises at HealthComputing.com is perhaps the most publicized of the few available "consumer" reviews. KLAS surveys those who have purchased or are in the market for various medical systems, and compiles reports regularly. I have actually filled out a survey on some of the equipment I use, although physicians comprise only about 5% of their respondents. KLAS does make it very clear that:

'This report is a compilation of data gathered from interviews with healthcare provider executives and managers. Data gathered from these sources includes strong opinions reflecting the emotion of exceptional success and, at times, failure. The information is intended solely as a catalyst for a more meaningful and effective investigation on your organization's part and is not intended nor should it be used to replace your organization's due diligence."

Responding gives one access to their data, although

"...(t)his report, and its contents, are provided under copyright by KLAS Enterprises, LLC and are intended solely for your organization. Any other organization, consultant, investment company or vendor gaining unauthorized access to this report will be liable to compensate KLAS for the full retail price of this report...."

Therefore, I won't be cutting and pasting anything here from KLAS. In brief, KLAS asks several dozen questions, and compiles individual scores of "Primary" and "Detail" indicators, and also yearly Business indicator trends. These are then distilled into a final grade.

Vendors who have paid the full retail price of the report are allowed to publish snippets and they do so if it suits their needs. The latest Interim, Midyear, Early 21st Century results from the KLAS surveys are in, and Stentor iSite wins the coveted Number One spot. (This was e-mailed by Stentor to the zillions of folks on their lists. Amicas was number two, although they are touting more the areas in which they were number one rather than their overall standing.) For the past two or three years, Stentor, Amicas, and DR Systems have jockeyed for first, second, and third, leaving some of the "Big Iron" vendors far behind.

I looked at the data from the following vendors: Agfa, Amicas, DR Systems, Fuji, GE, ScImage, and Stentor. Finding a true trend is difficult, and discussing it here is certainly impaired by the fact that I'm too chicken to quote the actual numbers. Still, one thing seems clear: the web-based products are prevailing over everything else. The ease of deployment and cost-savings with this approach cannot be ignored. There is another interesting trend: those who have done well in the KLAS survey over the past few years (Amicas, DR, and Stentor) are showing a minor down-tick in their indicators beginning in mid-to-late 2004. Amicas and DR show reversal of this trend as of early 2005, but Stentor's Primary/Detail indicators continue to fall, with plateauing of their Business indicators. I'm going to hazard a guess that these three at the top of the list have had an explosion of business generated in part by the KLAS results themselves, and are having to scramble to support the new demand. AGFA and GE actually show up-ticks on average in the same time frame, perhaps due to having a well-established sales and service force. Fuji had a bump-up around late Summer 2004, but has been down ever since. ScImage didn't make the survey until Fall 2004, and has one up-tick in May; KLAS gives the caveat that ScI's numbers are statistically below their confidence level, and one or two responses could skew the curve.

The individual rating components give a little more insight into a PACS purchase, but really not so much as I would like. There are many questions about support and contract negotiations, and keeping of promises. As far a promise-keeping, the web-based products (including Fuji) seem to do far better job than "big iron". DR, ScImage and Stentor do the best with items such as complete and fair contracts. DR gets the best rating in on-time implementation. And so on and so on....

To me as the end-user, the most important question is only partially covered by the question of "product quality". Every product on my short list is average or above, with Stentor winning by a very, very small margin. This still doesn't address how the bloody things actually work in practice. There are a few comments here and there, generally on the order of "our docs love it", but no real review of the details themselves. Maybe that is for the best. Selection of a PACS, for me the GUI, is a personal thing. I would rather test out the program under working conditions than rely on someone else's opinion anyway. To that end, would the vendors consider easier availability of on-line demos? But you all have to promise not to peek at each other's wares....