Friday, October 15, 2010

Dalai's Laws of PACS, Revised...The RANZCR Address

Note:  Video of the presentation from Perth is now available online at:
http://webcast.ranzcr.edu.au/Mediasite/Play/b6626adc21b24e0abbee6d85a0de17f21d?catalog=1e393b2b-7491-49f7-9d67-23cdbe177cbe

G'day!

When I spoke on behalf of healthinc in Brisbane a year ago, I never thought I would be asked to return for a repeat performance. But here I am and I'm really thrilled to speak with you today, even though I've been looking over my shoulder since I arrived in Australia. And by the way, thanks for having me over when your dollar reached parity with ours! Anyway, I hope you will consider these few moments time well spent, an amusing break from your real educational activities. And remember, if you don't take any notes, you'll have plausible deniability of ever having heard me!


You may have seen my blog, DoctorDalai.com. If so, you might ask how this average private-practice radiologist from Columbia, South Carolina, a small city in the Deep South of the United States, become the Dalai Lama of PACS? Here's the story in brief: In 1992, when I was just a few years out of residency, one of my senior partners introduced me to this thing called PACS, (although we really didn’t use the term so much back then.) We slowly implemented a piece here, and a piece there, and by the mid 1990's our main hospital was one of the first to be totally (almost) filmless, and ultimately paperless. My partner aimed for the idea of “Sit here, read there”. Among us, anyway, he was a visionary, although he always had some problems getting his home computer to work properly. Sadly, he’s since gone on to a better place (no, he's not dead, he's in Florida!), but I’ve been able to oversee the implementation of his idea. My group now covers five hospitals, two outpatient centers, and numerous doctors’ offices, all interconnected either with the hospitals' PACS or a system we own. Just about every examination can be accessed from anywhere, the dream fulfilled. Or is it a nightmare? That depends on the day.

I won’t exaggerate my role in this achieving this dream, but as a former Electrical Engineer, it did seem natural that I would get involved in the process. In large part, I’ve had to be the spokesman, and occasionally the champion for my group of radiologists, since many of my colleagues had little idea of how this PACS thing worked, or even how it should work, and by and large didn't care much as long as it did work. I began to post on AuntMinnie.com, initially asking what to do about a system that failed constantly (but that my senior partner loved.) To hide from him, and from the little company involved (ScImage), I adopted a nickname I thought no one would connect to me. Totally out of the blue, I came up with “Dalai Lama”. It didn’t fool anyone, but an Internet presence was born. My immodest assumption that some of my AuntMinnie posts were worthy of a wider audience led to the creation of Dalai's PACS blog, and the rest is history. Obviously, the blog postings, as well as my thoughts today, are my own personal experiences, opinions and observations and I just want everyone to take them for what they are worth. No divine knowledge or inspiration is implied. Certainly no offense is intended toward His Holiness, the XIVth Dalai Lama.


In fact, His Holiness is speaking at Emory University next week if you should happen to be in Atlanta.

The sense that medical imaging products were not all they could be, and the willingness to make honest statements online about my observations, has made me what I am today, the premier radiologist PACS blogger. Actually, I’m still the ONLY radiologist PACS blogger, but that provides job security, I guess. In a sense, I have appointed myself the spokesman for radiologists concerning PACS functionality as a whole. That rather lofty opinion of my role comes more from the vendors than from my inflated ego. As one of the loudest and least-civilized voices on such things, I seem to have reached the threshold of attention of at least some of the PACS vendors. Indeed, I have it on good authority that one of the main targets of my bilious postings thinks I've been "unfair" to the industry, meaning that particular company. I'll let you be the judge of that as we progress.

Whether in the Deep South of the US, or the Southern Hemisphere, we radiologists are all experiencing many of the same problems with PACS and the associated infrastructure, and dare I say it, the politics surrounding both. I’m one of a very few speaking publicly about these observations, probably because the audience is pretty small, we perceive no one cares, most of us aren’t crass enough to lay things out as blatantly as I do, and many are hesitant to take on the vendors, and others who don't get it. But after a day slogging against a malfunctioning system, I think most of us do care, and want some changes.

When I left residency in 1990, film was ubiquitous, and there was no such thing as PACS out in the field, at least not in a place like ours. My group had purchased a teleradiology system the year before I joined, however, consisting of video capture of the digital modalities, and a video camera on a stick suspended over a view-box for transmission of plain radiographs.


This conncected via analog telephone line to the Image Data Corporation Multiview Photophone, which we affectionately called the "humpback". Now those were the days. A 9 inch grayscale screen with a Touch-Tone numerical keypad serving as the only control. Things have of course become somewhat more complicated since then.

What has happened to the PACS market over the years is a story of Biblical intricacy, substituting corporate mergers and acquisitions for the "begats." This very Photophone device is a good example. You see, Image Data Corporation was purchased by eMed, which was bought by ACCESS, which then changed its name back to eMed, which in turn was purchased by Merge. In the meantime, AMICAS, my favorite PACS, was purchased by Vital Works, which changed its name back to AMICAS, then purchaed Emageon, and just this year, the whole package was purchased by none other than Merge. Thus, at least my PACS world has come full-circle within 20 years and Merge triumphs. Literally hundreds of PACS vendors have come and gone since I first started dabbling in this venue. There are well over 100 companies out there today in PACS, although how many are truly viable is up for debate.

Of the dozens, and even hundreds of PACS products, there is one positive comment to be made about each and every one: they all do show the images. Sort of. Some don’t do much more than that, and may in fact make it quite difficult to see the images, which is the whole point of their existence! A very few are obviously written with the radiologist in mind, with input from a number of rads. Others are clearly authored by computer geeks who had little idea how to spell PACS, let alone how to handle X-rays and CTs and their associated workflow. The common thread with most of these is the utter lack of understanding of what we do and how we do it. Approaches range from the ridiculous to the sublime.

One interface was even made to look like a spaceship control panel (I'm not exaggerating).



Fortunately, I was prepared for that one.

If you can't beat them, you might find yourself assimilated into the collective...

One small company even sold us one of the earliest implementations of an online real-time MPR viewer, wrapped in one of the worst GUI's I have ever seen.



Getting the images into the thing was an exercise in agony. It just didn't occur to this vendor that there was more to the software than the core viewer.

Another larger and more familiar vendor still insists upon pursuing the concept of toggling tools on and off with no obvious rhyme or reason. This company, among others thinks that the more buttons there are, the better the deal. (Which is something that some of their IT-based customers also believe.) Clearly, no practicing radiologist actually touched these programs before their release.

In addition to the joys of the disparate, sometimes poorly written software, I’ve had to deal with our IT departments and their lack of understanding of our workflow, our needs, or often anything at all about what we do.


Sometimes it really does seem like a horror movie.

Overall, it has always seemed to me that in a life and death business like ours, things could be done better.

Over the years, certain trends and patterns in PACS and our relations with PACS vendors as well as Information Technology became clear. I’ve distilled these into the LAWS of PACS. I have revised this list over the years, as the landscape keeps changing. Here they are, without further ado. . . try to picture me as Moses coming down off Mount Sinai:



I. PACS IS the radiology department.



This one is as obvious as it gets, but many still can't grasp the concept. Once PACS is in place, film is essentially gone forever, and in a very real sense, the mass of wires and computers is the entire department. Yes, there are still modalities, CR cassettes, barium, and so on, but for all intents and purposes, PACS is the department's face to the world. It should go without saying that the system actually has to work. Every single time it's used. You see, patients' very lives depend on this technology. Let me repeat that. PATIENTS' LIVES DEPEND ON PACS. IT HAS TO WORK.

I believe there has been some local experience with PACS malfunctions, and not long ago, I personally endured a 3.5 hour outage of a central PACS system that covered three hospitals, including a Level I Trauma Center. You can very well imagine the impact this had on our hospital and on our patients.

II. PACS exists to improve patient care. Its users are the radiologists and radiologic technologists. The entire goal of the PACS team is to optimize PACS function for its users.

This should be a no-brainer. In theory at least, everything and everyone in a hospital or clinic exists to improve patient care, from the guy who mops the floor to the neurosurgeon, to the Chief Executive Officer (yes, the CEO thinks he is above the surgeons, and we'll let him hold onto his fantasy). In essence, we all work for the patients, not the CIO, not the CEO (and certainly not the vendors).

I admit to some degree of bias, but I have to believe that PACS exists for us, the radiologists, to use for patient care. Quite often, though, IT doesn't accept this very important reality. The IT version of this law might read, "We provide PACS because it is made up of computers, which we own throughout the enterprise. We know far better than you do how our computers work, and what software will be easiest for us to maintain. We would be much happier if you would refrain from actually touching the mouse or the keyboard." Fine, let them push the barium too!

III. PACS should be the shared responsibility of the radiology and IT departments.

Notice the word "should." PACS is one of those little projects that requires the help and expertise of a lot of folks. As in Dalai's Second Law, PACS is indeed at its core a collection of computers and wires, an IT project if there ever was one, right? But as per Dalai's First Law, PACS is the radiology department, governing everything from its workflow to its profit margin -- things understood best by radiology, and it is a critical component of PATIENT CARE!

Therefore, I make the bold statement that management of this very important system should be shared between IT and Radiology. It only makes sense to let the various departmental expertise apply to where it can do the most good, again for Patient Care.

Sadly, PACS, being a rather high priced item, sometimes becomes a hostage, yanked back and forth to the department that wields the greatest power. Thus, territorial squabbling comes into play. Sometimes, the participants, whether from IT, or the government, or a vendor (this wouldn't happen to Radiology) tend to lose their orientation, which should of course be directed toward...THE PATIENT!

IV. PACS should not get in your way.

A corollary of Dalai's Second Law: PACS exists to let me, the radiologist, look at my patients' images. Anything that gets between me and that image is a distraction, and gets in my way. Obviously, some of this will be necessary, but if I have to click excessively, or take 39 trips to the menu bar before I'm done with the exam, something is wrong. Let me illustrate with an automotive analogy:

The newer BMW’s have something called iDrive, which is a big mouse that controls 700 functions. Most people end up accessing about 20 of these. Disgust with the vast excess is called “feature fatigue." The multiplicity of stuff available distracts one from the road, creating a potential hazard. Lexus vehicles on the other hand have simple, intuitive controls that make you feel more a part of the road. (Sadly, Lexus now has a new model with its own version of iDrive. Apparently my influence is limited.)

V. Workflow is inversely proportional to the number of buttons on the PACS desktop.

This is a corollary of the Fourth Law. I cannot, for love, money, or excessive ranting on my blog, get some of the big PACS companies to understand this. I was told by the head of a major PACS project from a major PACS company that its solution to providing a feature when you ask for A and I ask for B is to add a button that does both A and B. Here is what happens when the buttons are allowed to multiply like rabbits:

Many modern systems have in this way become hyperconfigurable and suffer from something I call the Lego-PACS syndrome -- that is, one can organize the buttons on the interface in so many ways that the permutations would take a century to exhaust. I assume the building toy Lego is as popular here as it is back home, and it was indeed my favorite growing up. But the sad fact is, I don't want to be spending my time searching through a sea of buttons and menu items; I want to read my studies. There is a minimum feature set necessary to accomplish this simple goal, and beyond that, every extra function has the potential to slow me down. That’s not to say that I don’t like advanced functionality. I do love power! But there are ways to simplify and organize these controls so they are unobtrusive, but available.

VI. PACS is not film.


In the early days of PACS, displays were designed to mimic a film view box. This seems rather foolish to us today. The versatility of a soft-copy display is so much greater than that of a piece of film it just isn't funny. Can you window and level a piece of film? Can you cine through images on a filmed CT? Well, I suppose you could stack them up like a deck of playing cards and riffle through them, but come on! Take away film and your workflow improves 10-fold.

VII. The degree of understanding of Radiologist workflow is inversely proportional to the size of the PACS company.



While not a hard and fast rule, it does seem that smaller companies can be more innovative and responsive, at least to a point. With some of the huge monolithic companies, having things "my way" is simply not in the cards. The products of the larger companies seem to reflect the mentality of largess, and even "bloatware". With this comes the anathema of the Lego PACS and obstructive designs I have bemoaned above. Sometimes, a large company (in this case large is spelled l-a-r-Capital G-Capital E) tries to emulate the flexibility of a smaller company by simply purchasing said smaller companies. Perhaps they are trying in this manner to become all things to all rads, or maybe they are just trying to find something that actually works. So far, their latest assimilation into the collective hasn’t worked very well.

VIII. Vendors work for us, not the other way around.

I suppose it's easier for the manufacturers to attempt to adapt the user to the program rather than vice versa, and our first mistake is allowing this to happen. I was shown a demo recently of the latest version of our PACS, one which I haven't been terribly pleased with to this point. After five years of whining (on my part), they had finally added on a simple function of spine labelling. The new toy was well-executed, but far more complex than what we had requested. Something simple could have made it out the door as a service release. Some other features that we had despised stayed on. When I protested, I was told, "But that's what makes us different!" I refrained from responding, "that's what makes you inferior!"

IX. A true PACS guru is worth his/her weight in gold.

There is absolutely no way in the known universe to successfully implement PACS without a guru. What is a guru? Someone who knows the PACS in and out, knows the radiologists and technologists better than his or her own family, and can make the system work for the end users, as per Dalai's Second Law.

The vendors are usually present for the initial installation, although we all know they may not follow through quite as well as we might like. Ultimately, there must be someone there on the ground to keep everything running smoothly.

X. All software errors, including those within PACS code, can be repaired if the vendor is sufficiently motivated to do so.

I've had some of the larger PACS companies tell me they just can't fix some bug, even one that crashes a system. What they really mean is, they won't fix the problem, or at least they feel the resources required would be more profitably deployed somewhere else, say on the next version of vaporware. As Windows users, we are all participating in the world's largest and longest beta test. At least Microsoft eventually fixes most of the things we discover. Why should we expect any less from our PACS vendors?

XI. If IT doesn’t like something, it will be termed a security risk.

I recently found a way to thwart IT, or so I thought, by using a macro program to automatically refresh a RIS window that would otherwise close every 30 minutes. Signing in to a Citrix window gets boring after the tenth time that day, you know. The macro worked, but IT removed it from all computers, saying that someone could create a macro that would bypass password entry. Sure they could. They could also use a wax pencil to inscribe their password on the monitor, as many of my colleagues have over the years. I’m not certain if we should ascribe IT's behavior to paranoia, concern, laziness or simple meanness. Probably all four. IT actually did decide to let me use my macro in the end, after I made my case to the head of IT security. In the meantime, however, they've taken away the right-click from our computers because I was nasty enough to use it to change the vendor's logo background on the desktop. All I did was shuffle the letters A, G, F, and A around a little bit...

XII. The PACS needs to be operable by the least technically-savvy radiologist on staff.



This is inspired by one of my partners, one of the best interventional radiologists I have ever met. Ironically, he has great difficulties handling anything run by computer. I often tell the story about the time he called me from an airplane about to take off to ask how to turn up the volume on his laptop so his children could watch a DVD. While the PACS has to work for me, it has to work for my less-computer-savvy partner as well, or it might as well not work at all.

XIII. Drive before you buy.



It is impossible to get the feel for a PACS graphical user interface (GUI) in a 10-minute demonstration, no matter how well it is presented. In fact, the better the presentation, the more chance that you are being deceived.

Ask, nay, demand, direct access to the program under consideration, something you can pound on without the salesman in attendance. The joys and pitfalls of various systems do not become apparent until you have actually tried to use them in a production environment. You wouldn't buy a Mercedes after watching the salesman drive it around the lot, and PACS tends to be significantly more expensive than a car. Keep in mind as well that the system that works wel for your mate down the road might not fit your particular workflow.

And as Moses (or was it Pharoh?) said, "So let it be written, so let it be done!"

These lessons have been painfully learned. Ultimately, my experiences and observations have been punctuated by fights, I mean discussions, with our IT folks, administrators, and of course, our vendors. On many issues, I have had to use my blog to garner attention to significant problems with PACS systems, when there was no other way to get anyone to listen. In fact, that is probably why a vendor we all know well labels my blog "unfair". This is such a relatively small niche market that problems might well go unnoticed and thus sales not be affected, were it not for a loudmouth such as myself. Since I'm the only Radiologist blogging about PACS, my "unfair" observations do seem to reach their intended audience. Have I impacted a sale or two? That's not my intention, but were problems dealt with properly, I would have no influence at all. If complaints were addressed, and patient care not impacted, I wouldn't be making any "unfair" observations whatsoever. Sadly, whistle-blowers are generally not appreciated, especially by those who were tattled upon.

I've taken a number of approaches in reporting the joys of our various systems on my blog, from direct opinion-pieces to little cartoons (I'm the one with the stethescope and the nice tan):


I’ve even parodied some famous songs. My favorite example of the latter is my Belgian Rhapsody, which begins (and I'll do you all the favor of reading it and not singing it):

Is PACS online today-

Is that just fantasy?

Still caught in downtime

With no functionality-

Open your files

Look up from the dials and see

I’m just a poor rad, trying to get through the day

‘Cause PACS is easy come easy go

Why it glitches, I don’t know

Any way the circuits blow doesn’t really matter to me

To me…..

--------------------------------------------------

Mama, just read a scan

Had a CT of the head

Clicked it off, now PACS is dead

Mama, it had just come up

But now I’ve gone and crashed it all again

Mama, Oooo, ooo ee ooo

Didn’t mean to make it die

If it’s not online again this time tomorrow

Back to film, Back to film, as if PACS doesn’t matter..

Apologies to Freddie Mercury, wherever he is…


 Dalai's Laws cover the importance of PACS and how it should work. We in this room understand that criticality quite well, but our IT and other politically-minded colleagues often do not. Once the infrastructure is in place to view our patients' images with electronic, soft-copy reading, there is no turning back. If PACS doesn't work, the hospital might as well shut down. I believe you call this situation a "Code Yellow," an internal emergency disaster.


My main trauma hospital, and its two sister hospitals which are all connected via the same PACS, (one you know well) recently endured a complete PACS shutdown lasting 3.5 hours. To this day, no one knows what happened, or at least they aren't telling me. We were dead in the water with NO PACS functionality. All we could do was to look at a few of the studies on the modality consoles. It seems our downtime plan involved printing film, but wouldn't you know it, someone disposed of the printers! IT's response?

Firstly, I would like to apologize for the unexpected downtime that occurred this past weekend. Impax is a very important system in our enterprise, and it definitely prohibits optimal workflow and efficiency when any high priority system goes down.


I have met with the parties involved in responding to the downtime, and the procedure for reporting issues to AGFA was followed appropriately. That being said, the field engineer who responded from AGFA should have escalated the outage to a senior product specialist who is intimately knowledgeable of our system, and he did not. This has been brought up to the service manager for follow-up and resolution.


As far as a failover system, Impax is configured to run in its test environment (on a test server). Since this affects how the modalities send the studies, the time frame for decision-making to initiate this process is 4 hours. This allows the vendor to respond, make assessments, and hopefully resolve any problems. During this time, studies do have to be viewed at the modalities.


I appreciate your feedback and your dedication to (our hospital) Radiology.



Excuse me? FOUR HOURS is acceptable? Not to me, it isn't.

Clearly, those upon whom we all depend for support may not understand the situation. Of course, this is the very same PACS team that when called upon to fix a malfunctioning PACS computer during a tumor conference declared, "Sorry, I'm still in bed. I'll be there in an hour and a half." This truly happened to us a few months ago.

Those vendors who land in my crosshairs got there by a similar combination of deafness, ignorance, arrogance, and hubris, and a significant amount of bad luck. Again and again and again, we are delivered products that the engineers (keep in mind, I am an engineer by training) think we should like, but never asked us if we actually do. Waiting five years for something simple like spine labeling because the vendor didn't want to listen to me when I told them to make it simple is ridiculous. Hearing that some poorly-conceived core functionality will be kept as it is because "that's what makes us special" is enough to drive me to drink. More than I already do, that is.

With bad designs, bad installations, and bad service, the vendors have great potential to wreak havoc upon us, and some do this quite well. Honestly, I don't think the vendors deliberately set out to create problems, but somehow they manage to do so anyway. A fellow in Canada responded to one of my rants against a certain PACS as follows:

Frankly, I have given up on reporting bugs. After one week Level 1 Support says you are doing something wrong...after 1 month Level 2 Support says they will look into it, and have never seen this problem before...after several months Level 3 support says 'Oh yes, that is a known issue, but it has not been given priority'. Is this the reputation any vendor would really want?

No, it's not. And if I were a vendor, I would really hate to bear the liability of a malfunctioning PACS.

What is the answer? Simply put, we need better communication, and that is a two-way street. Somehow, there needs to be dialogue between the end-users of these things and the folks that create them. I've attempted to open the discussion with my blog, but that's just the first step, and a fairly feeble and occasionally misunderstood one at that. But there is one other factor to consider. As long as we keep buying the suboptimal products they are selling, there won't be much incentive for improvement. The disconnect is inherent in the mechanism of that very purchase. We, the end-users, very rarely get much of a say in the selection of the PACS. Rather, it may be driven by some political motives, sometimes even by IT's desire to select a vendor that will do a bit of their work for them. We have an old saying in the States: "No one ever got fired by choosing GE." Clearly, those people don't work for me!

We have to make known our needs, and the manufacturers, the software writers, the vendors, IT, and all levels of governance over us need to listen. And of course we need to hear and understand their restrictions and limitations as well. But if we say nothing, if we are content with suboptimal products, suboptimal service, and indifferent, condescending oversight, well then, we deserve what we get. But our patients deserve better than that, don’t they?

In the end, that's what Dalai's Laws are all about. PACS is a tool for improving patient care, and patients' lives depend on it.

It is very encouraging to realize that all of us in this field face and surmount similar challenges. We must keep fighting the good fight on behalf of our patients, in spite of what some larGE companies might, well, imagine.

My second trip to Australia confirms what I learned on the first: Your country is absolutely beautiful, and your people incredibly friendly. Perhaps you'll make me an honorary Australian after all this, assuming I'm allowed to leave the country, that is.

In closing, I would like to once again thank the convenors of RANZCR for granting me the opportunity of speaking to you this morning. Now, back to the more educational part of the meeting. Thank you for your attention.

3 comments :

PACSFerret said...

Doc.. Good stuff. Although I think you're still being a little hard on us IT folk. We shouldn't all be tarred by the same brush (although I'd be happy to tar some of my 'collegues' myself).

Your note about 4 hour downtime being unacceptable is dead right. Unfortunately though vendors have tended to price high availability in a space that is difficult to justify in 'economic' terms (note thats a 'business' call not IT or clinical). I'm in favour of having dual (vendor independent) pathways to deal with stuff like that but that is still expensive unless you have expertise to handle the excellent open source offerings.

"X. All software errors, including those within PACS code, can be repaired if the vendor is sufficiently motivated to do so."

Yes, but in reality some defects require fundamental rewrites of major system components (I'm facing the consequences of one of those as we speak) . BUT.... the best way to ensure this is to have someone who can keep the vendors honest by knowing enough about their own gear to know when they are nonsense. That, I'm afraid, is likely to be one of those "IT geeks"

The vendors know this.. which is why they are happy to add fuel to the Radiology/IT schism (BTW not only in radiology).

Anonymous said...

From a PACS administrator's perspective, this is well said. Keep chewing on the vendors, sooner or later they may begin to pay attention.

Hussein Alsayiegh said...

Please check this url.

http://q8pacs.blogspot.com/2010/11/pacs-laws-by-dr-dalai.html

this is a summary of your article.

i think many of the blog readers liked it ..