Sunday, May 03, 2009

PACS Updates, The Best of Times, The Worst of Times

Well, OK, maybe it wasn't that bad, but it was close. Our largest hospital system just installed the latest Impax 6.3.x service update. This operation gives us 71 improvements over the last iteration, some of which include:

Cycle list redesign
When the Cycle list contains many patients, each with a large number of studies, the list can get quite long; finding the desired patient becomes difficult. To solve this issue, the Cycle list has been redesigned. Each patient is now listed with their studies hidden. To view the studies of a patient, click the + next to the patient. To open all of the studies for a patient in the Image area, in the Cycle list, click the desired patient. To open a specific study, expand the studies and select the desired one.

Wildcard for Keyword search in Advanced Search
Keywords are descriptive tags that you can add to a study. Later, you can search for the study using these keywords. To enable the ability to search on keywords without using an exact match, the following conditions have been added to the Keywords criterion in Advanced Search:
1. Contains
2. Does not Contain

Fixed issues:

Slow performance of IMPAX Client when selection of studies is opened in the Image area. When a selection of studies is cycled into the Image area, the amount of time it takes to display them significantly exceeds the amount of time to display a single study.

Series logic broken after combining series in Available Series palette and spanning from one to two monitors. Combined series jumps back to the first series after spanning from view one monitor to view two monitors.

Removing column from Text area Study List causes IMPAX Client to crash. Removing a column from the Text area Study List causes the IMPAX Client to crash.

Study level prompt on dictation interlock is not working correctly. Study level dictation interlock does not behave as desired. Active worklist does not function in this case. As well active worklist is using study based notification when it should not.

Unable to view priors when they are partially cached. Relevant priors cannot be viewed if the study is partially cached; these priors can be viewed after the study is fully cached.

Prior is selected instead of current study when Image area is opened. A current study has a prior study attached to it, the current study is in the local cache, and the prior study is not in the local cache. When attempting to load the current study into the Image area, the prior study is displayed instead.

Active Worklist does not work when cycled studies are also priors of newest study. Active worklist does not always perform correctly when cycling several new studies with relevance on.

User profile is corrupted, not able to see scout images. The user profile in ADAM is being corrupted after logging into the Client a few times. The Actual Width in ADAM for IMAGE_AREA_SCOUT is replaced with zero so the user is not able to see the scout images.

There are numerous fixes for Mammography as well. Obviously, this was a very, very necessary update, and so far this morning, it is functioning nicely.

So, from whence comes my acerbic attitude? I guess you had to be here, but for the six hours of the upgrade, my life was not as much fun as it could have been. I was stationed at our trauma center, the busiest hospital in our enterprise, where the normal workflow is hectic at best. There's nothing like the joy of clicking off the last study on your list only to find ten more have appeared. Obviously, any minor annoyance or disruption becomes major at a place like that.

I'm not going to go into the specific mechanics of the update, mainly because I don't have a firm grasp of them, and moreover, I'm not sure I care that much. I do believe the process involved our Oracle database, the heart of the operation, as it were. What I know for certain, however, is that for the six hours of the update, we were running off of the axillary test server (which had already been updated), and functionality was quite limited.

Because the central database, the Oracle, was out of commission, image data was limited to the new studies, and a preload of the last few week's worth of priors. Could I get a study from four weeks ago? NO. Were there any reports available? NO. Well, yes, but not on PACS. To get a prior report, AND to get an accession number, AND to get a history (if the techs didn't provide such, which they didn't), we had to log into our RIS, which happens to be Cerner RadNet. There is a reason we avoid doing so routinely, which has to do with the fact that it likes to crash, and it is rather unintuitive. Can you guess, even with tool tips, what these buttons do?

The fourth button from the left is the "Radiology Order Viewer", and that is where old reports live, in case you were wondering.

So, for six hours, we were functioning, but in a rather hobbled fashion. Every one of the roughly one hundred studies that came through during this time had to be augmented by this manual drudge though the RIS. You can just imaging how much fun that wasn't.

Now, I don't mean to cast aspersions on our PACS team, nor upon the people from Agfa handling this update. They worked very hard, and did a bang-up job getting us through this (although one of our people had a giant flyswatter ready to whack me if I got too whiny.) No, the problem is in the philosophy of how updates and upgrades work.

I will have you refer back to Dalai's Laws of PACS, specifically the First Law: PACS IS the Radiology Department, and the Third Law: Once PACS, never back. Alter the function of PACS, and you destroy your established workflow, and ultimately impede patient care. Taking PACS away, or even hindering it in the slightest is a really bad idea. The busiest radiology departments cannot have this level of disruption, even for an hour, let alone six hours, or longer.

Updates are necessary and even desirable. They fix problems and deficiencies, and at least sometimes enhance the product. BUT there has to be a better way to handle the upgrade. Frankly, I'm not aware of any particular company that has totally painless updates/upgrades as yet, but perhaps they will listen and learn.

There are multiple architectural issues that necessitate a partial shut-down as we experienced. Probably the most important factor is the existence of the single central database. Beyond this, the rather complex connections between PACS and RIS denied us the ability to hook up the test server to the RIS, even temporarily. Speaking from an only semi-educated standpoint, it seems that we need some additional redundancy, such as a back-up database for these situations. We should already have one, for disaster recovery, yes? So, why not press it into service for this particular non-emergent emergency? Reconciling the changes made to the backup during the downtime should be fairly straight forward. Now, this does assume that the backup is on spinning disk rather than tape. Perhaps what I'm proposing is the establishment of an intermediate database, that holds, say, the last year's worth of data. I don't think this would be all that expensive, folks. And, while I'm no HL-7 guru, I should think that an alternate connection to the test-server (or a fail-over server for that matter) ought to be possible as well.

Bottom line: PACS updates should be a lot less painful than they are now. I don't think it would be that difficult or expensive to make that happen. Like many other problems in this business, this probably hasn't been addressed because no one has yelled loud enough until now. OK, I'm yelling at the top of my lungs! Do you vendors think you could have this fixed by the time of the next update/upgrade? I would really, really, appreciate it. Otherwise, I might have to have a bigger tantrum, and that wouldn't be dignified. Or pretty.


Mike, Jacqui and Ieuan Mackrill said...

I am the PACS Admin for a medium sized hospital in NZ, we have a server that holds about 3 months worth of data, which we can get our HIS to point at when we upgrade (we have a web based clinician reports & images distribution system, which is accessed via an icon on our HIS) Even 1TB+ servers are cheap now, and this works for us (though it didn't once, long story).

Great blog BTW! Getting real user feedback is hard

Gabriel said...

Glad to see you had good things to say about the Agfa people working on this upgrade. I work with those guys, and I know they're good! :-)