Monday, September 01, 2008

Desperately Seeking Updates

I hope everyone has had a happy and enjoyable Labor Day weekend. Sadly, it's now time to think about going back to the regular grind, you know, work. There is indeed a reason why work is a four-letter word.
I really don't mind going back after a long weekend, although when I've been off for one week, or more rarely two, it's a bit harder. And part of the joy is my realization that depending upon the site to which I'm assigned on that black Monday, I will have to wrestle with one balky PACS or another to accomplish my tasks for the day.
As I'm not interested in generating trouble for anyone in the form of letters from acronymical government agencies, I will avoid the pleasure of naming names and repetitively repeating lists of problems. But it seems some of the problems we are having with one of our systems are fixed by an update that came out a long time ago. However, on many PACS, adding updates can be a real pain that has the potential to shut down the enterprise. Thus, in the interest of avoiding yet another trip to the servers, we decided to wait for the update that was supposed to be available this January. Ah, but alas, here it is September 1st, and this wonderful update that fixed even more stuff is nowhere to be seen. And why is this? Well, the only word that has filtered down to my peon level is that the company has been adding more features to this update to satisfy various customers.
How should we feel about this turn of events? In my bowel-unrest over this situation, I have to conclude that the update process for many if not most companies is broken. Waiting this period of time for critical fixes is unacceptable, but there doesn't seem to be a way to get anything but an absolutely critical, emergency, life-saving patch out the door in any reasonable amount of time. This might have something to do with the way the code is written or managed. Perhaps there is a more modular approach that would allow the repair of one section without tearing apart another? That's beyond my expertise, but maybe I'm somewhere near the right track?
And, there needs to be a much better method of deployment. Updates should not be major events that are life-threatening to the PACS staff (who are in danger from the radiologists if they can't get the system back up in 5 minutes). Why is it that even horrible bloatware like Windows can automatically update in the background, but PACS server software can't? I know that some companies use a three server configuration, including the main production server, a test server, and a fail-over server. While it is still a pain to update with this system, the whole thing doesn't crumble during the process, we don't have to point to different servers during the update, etc. There has got to be a better way.
Finally, in our particular situation, the blame for the rather long delay in the update is laid at the feet of the customers (might be us, but I don't think it is) who are demanding new and different functionality. I'm assuming these requests are coming from existing customers; if the additions are for potential customers, well, let's just say Hell hath no fury like a Dalai scorned, and someone better be ready to take one for the team.
Lemme give every company out there a big hint: FIX what's wrong with what's out there BEFORE you start adding NEW thing that will break. I don't think that's a radical concept.
So, I'm begging PACS vendors to consider this request: Put some work into your update process. And your updates. And the rest of your software, as needed. It would be most appreciated.
I'll be sure to, ummmm, update everyone when I see some progress. Enjoy the rest of Labor Day!


Anonymous said...

Amen to that. Our last major patch burned our regional system so badly that we aren't even looking to accept new patches until they have been deployed out in the field a good 4 to 6 months. Of course once everyone starts doing this updates will never be released. It's like "Well that sure is a nice list of new features but the pain of the last update is still fresh in all our minds so I think we'll pass for a while." Actually my exact words to our vendor was "There is nothing here worth risking a repeat of the last patches disaster."

Anonymous said...

Is 'acronymical' a real word? If not, should be.

The whole concept of software 'product' is fundamentally broken. It worked through the 80's and 90's because the low hanging fruit was so plentiful. But now the idea of generating revenue by lock-in to a single vendor is becoming increasingly discredited (although Healthcare generally - not just Radiology - is one of the last to see this - largely IMO due to the infighting/turf wars). As long as the primary source of revenue is the license fee, of course there is no incentive to vendors to provide adequate post-sales service and it is naive to think that relying on a vendor's concern for its reputation will fill the hole. Is there an answer? Personally I believe Open Source and Free software is at least one of the answers although I recognise there are folks who remain sceptical and certainly in Healthcare, the options available are yet to mature. But in many industries the monolithic vendors are being squeezed by a combination of Open Source and Cloud Computing (software-as-a-service) and that is a trend that is likely to grow.

Even 3-way implementations don't always help - I've certainly encountered update/upgrades that do not communicate with the previous version, so it is virtually impossible to maintain parallel operation.

But talking virtually - virtualisation is a fantastic way to mitigate against this kind of risk. Now virtualisation is very much mainstream, ALL new contracts for ANY system in ANY environment (sorry for shouting) should include an option to deploy as many virtual instances as needed to test properly, as long as such instances are only used for testing. Will the current vendor's licensing schemes allow that? Almost never, but that's their problem (or should be).

Anonymous said...

What is worse to me is is that the only way you can get the updates is to PAY for them by having a service agreement that equals roughly 20% of the price you paid for the system. After 5 years you have essentially bought a second system yet have nothing to show for it. And do the updates work? As you have found out, they fix some problems yet create many others. And for this you are paying through the nose. Upgrades almost everyone pays for- with or without a contract.

If you look closely at the companies who break out their revenues 60-80% of revenues come from service (which includes software updates) so there is no incentive to change the system. I bought something that doesn't work right yet have to pay to get it fixed? Somehow that's just not right.

Next month I'm taking on the acronym wonders directly- you can stand behind me as the bullets fly- because this nonsense has to stop...