Saved By Cancer

There is an old Arab saying that goes something like this: "The enemy of my enemy is my friend." Here is a medical illustration of that sentiment.

Below are three slices from an abdominal CT of a patient with known lymphoma. The most recent examination showed much less disease, and a completely normal aorta. Today, however, the periaortic adenopathy has increased markedly. But....look at the aorta!





At this point, I'm not certain if there is a small dissection, a small leak forming a pseudoaneurysm, an ulceration, or even an infectious process involving the aorta. What I do know is that the huge node mass appears to be tamponading, or sealing, the abnormality in the aorta, and quite ironically is keeping the patient from bleeding out.

The patient was to go for surgical consult; I would expect he will undergo endografting rather than any sort of open procedure.

Wonders never cease in this business.

GE PACS Target of FDA Warning Letter

Fair is fair. I reported on the warning letter Siemens received about its PET/CT manufacturing process. Our local GE salesperson did, after all, send out a press release publicising the embarrassing news, and it had already been published on about 100 websites by the time I got around to posting it.

While no other PACS companies have called me to make me aware of the situation, it would appear to be GE's turn on the FDA's carpet, and to be fair and balanced, this gets equal time. I first learned of this courtesy of AuntMinnie.com, with additional coverage on HealthImaging.com. The original text of the FDA's letter can be found here.

So what's the problem? Actually, there are quite a few.

The letter was sent to Mr. Vishal K. Wanchoo, President and CEO of GE Healthcare Integrated IT Solutions in Barrington, Illinois. I believe this is the site of GE's Intergalactic IT and PACS headquarters, which I visited years ago. The main product produced here is software, specifically Centricity PACS and related items. The IT shop was inspected from April 15 through May 13 of this year. Here again, as with Siemens, the "medical devices" created in Barrington were found to be

adulterated within the meaning of Section 501(h) of the Act [21 U.S.C. 351(h)], in that the methods used in, or the facilities or controls used for, their manufacture, packing, storage, or installation are not in conformity with the current Good Manufacturing Practice (cGMP) requirements of the Quality System (QS) regulation found at Title 21, Code of Federal Regulations (CFR), Part 820.

There were 12 points of failure, although the letter rather explicitly states that the list is "not all-inclusive." Most were addressed by GE, with most of the responses deemed "inadequate." The letter contains chapter and verse of the regulations that were not satisfied, with examples thereof. There were significant problems with analysis of processes, records, etc. The procedure for implementing design changes was flawed. An MDR report was not submitted with 30 days of receiving information that a device had malfunctioned and could "cause or contribute to a death or serious injury if the malfunction were to recur." Apparently, this refers to a a misdiagnosis which occurred after an incorrect patient jacket opened on a Centricity PACS workstation.

The letter concludes:

You should take prompt action to correct these deviations and to establish procedures to prevent their recurrence. Failure to promptly correct these deviations may result in FDA initiating regulatory action without further notice, including but not limited to, seizure, injunction, and/or civil money penalties. Also, federal agencies are advised of the issuance of all Warning Letters about devices so that they may take this information into account when considering the award of contracts. Additionally, premarket approval applications for Class III devices to which the Quality System Regulation deviations are reasonably related will not be approved until the violations have been corrected. Requests for Certificates to Foreign Governments will not be granted until the violations related to the subject devices have been corrected.

Please notify this office, in writing, within 15 working days of receipt of this letter of the specific steps you have taken to correct the violations, including:

• An explanation of your plan to prevent these violations from recurring.

• Any documentation of the corrective actions you have taken.

• An explanation of each step being taken to identify and make corrections to any underlying systems problems necessary to assure that similar violations will not recur.

• A timetable for implementation of corrective actions. If they will not be completed within 15 working days, state the reason for the delay and the timetable for completing the corrective actions.

Finally, you should know that this letter is not intended to be an all-inclusive list of the violations at your facility. It is your responsibility to ensure compliance with applicable laws and regulations administered by FDA. The specific violations noted in this letter and in the Inspectional Observations, Form FDA-483, issued at the closeout of the inspection may be symptomatic of serious problems in your firm's manufacturing and quality assurance systems. You should investigate and determine the causes of the violations, and take prompt actions to correct the violations and to bring your products into compliance.

I think they mean business. AuntMinnie reported GE's response:

"GE Healthcare takes the warning letter very seriously, and is actively working on providing the information requested by FDA in the warning letter and on identifying further enhancements to the site's quality systems that may be appropriate," the statement said. "GE Healthcare is committed to providing its customers and their patients with products that are safe and effective for their intended uses, and to following FDA requirements and guidance on how this can be best achieved."

I'm sure they will take care of these little problems.

This whole episode teaches us some important lessons. First off, no one and no company is perfect, even GE. Second, complaints are a critical part of the process, and must be handled rapidly and seriously. The FDA knows this, and I think every manufacturer of a medical device knows it too. There is no way to test every possible permutation of code in the lab, and the end-users will almost certainly find something unforseen. Ultimately, we all have to listen to each other. Medical device manufacturers have an even more specific duty to understand the complaints of their users. A glitch in this realm has the potential to kill, and that is not an exaggeration. Granted, there are some of us users that just like to whine, but others really are trying to help improve the product, and make it safer and and better for everyone. After all, every one of us, IT CEO's, software engineers, and even doctors, will be patients one day.

Sorry about that, Mr. Wanchoo.

So, if there happens to be a list of problems floating around on the Internet somewhere, do have a look. It could perhaps help limit the joys of a Letter of Warning.

Emily--She's Unreal!

From the UK's Times Online comes a fascinating discussion of a new era in computer animation. Have a look at the video below:




This amazing clip shows just how far we have come in this realm. The girl in the movie is the "output" of a computer system at Image Metrics, the folks that animated Grand Theft Auto. What you see in the movie is a completely digital rendering of actress Emily O'Brien. I can't tell the difference between the real thing and her digital doppleganger.

The data is acquired as follows:

One of the attributes that sets Image Metrics apart is their unencumbered video-capture process. Using standard or HD cameras, or a head-mounted camera, an actor's performance is perfectly captured with all of its nuances and subtleties. There are no time-consuming markers or makeup to apply. The director is free to capture the performance he or she needs, while the actor has the freedom to move unhindered about the stage. The result is spectacular performance-driven animation every time.

An actor reads the script and performs the scene as they would on stage.

A director controls the actor's performance as cameras capture it.

The actor can perform freely, without markers or makeup.

The absence of those same markers and makeup allow directors to see the subtleties of an actor's performance.

In addition to taking less time to set up, the director can get the performance he or she wants for their film.

The improvement of Image Metrics' current methods over earlier attempts is the introduction of much finer control, essentially down to the individual pixel level. Beyond this,

"Ninety per cent of the work is convincing people that the eyes are real," Mike Starkenburg, chief operating officer of Image Metrics, said.

"The subtlety of the timing of eye movements is a big one. People also have a natural asymmetry - for instance, in the muscles in the side of their face. Those types of imperfections aren't that significant but they are what makes people look real."

Technology had to catch up for this to be possible:

For many years now, animators have come up against a barrier known as "uncanny valley", which refers to how, as a computer-generated face approaches human likeness, it begins take on a corpse-like appearance similar to that in some horror films.

As a result, computer game animators have purposely simplified their creations so that the players realise immediately that the figures are not real.

"There came a point where animators were trying to create a face and there was a theory of diminishing returns," said Raja Koduri, chief technlology officer in graphics at AMD, the chip-maker.

AMD last week released a new chip with a billion transistors that will be able to show off creations such as Emily by allowing a much greater number of computations per second. "If you're trying to process the graphics in a photo-realistic animation, in real-time, there's a lot of computation involved," said Mr Koduri.

By 2020, the line between photo and photoreality may well be completely blurred. I'll bet it comes sooner than that, based on Emily.

Think of the possibilities. With the proper apparatus, you could have yourself scanned and placed in whatever movie you want....

Brave New World, ain't it?

Virtual Servers and PACS

The thought of multiple operating systems running on one computer is nothing new. My very own Dell Outlet-Site 8Gb RAM extravaganza now can be booted into Windows XP x64, to take full advantage of those 8G's, or into XP Pro, to use the peripherals that x64 doesn't support (pretty much all of them.) One of the things computers do quite well is to run lots of different things simultaneously, although I can't with my present set-up run XP Pro and x64 at the same time.

It turns out that with the proper layer of software, such as the products from VMWare, one can simultaneously run two (or more) virtual machines on one piece of hardware. This lends itself well to servers, but it can work with desktops too.

Why would you do this? From VMWare:

• VMware Server enables users to quickly provision new server capacity by partitioning a physical server into multiple virtual machines. With VMware Server, users can:


• Provision a new server in minutes without investing in new hardware


• Run Windows and Linux operating systems and applications on the same physical server


• Increase the utilization of a physical server


• Move virtual machines from one physical host to another without re-configuration


• Easily migrate virtual machines to VMware Infrastructure


• Access enterprise-class support organization


• Efficiently provision, monitor and manage infrastructure with VirtualCenter add-on


• Capture state of an entire virtual machine and roll back to that state with the click of a button

So what can you do with this?

• Streamline software development and testing by allowing developers to create multiple environments with different operating systems on the same server


• Simplify IT testing of patches, new applications and operating systems by allowing systems administrators to test in secure virtual machines and be able to roll back to a clean state by leveraging snapshots.


• Simplify server provisioning by building a virtual machine once and deploying it multiple times


• Evaluate software in ready-to-run virtual machines without installation and configuration


• Re-host legacy operating systems such as Windows NT Server 4.0 and Windows 2000 Server in a virtual machine running on new hardware and operating system


• Leverage pre-built, ready-to-run virtual appliances that include virtual hardware, operating system and application environments. Virtual appliances for Web, email, proxy and other infrastructure services are available for download from VMTN.

And how does this work?

VMware Server installs and runs as an application on top of a host Windows or Linux operating system. A thin virtualization layer partitions the physical server so multiple virtual machines can run simultaneously on a single server. Computing resources of the physical server are treated as a uniform pool of resources that can be allocated to virtual machines in a controlled manner. VMware Server isolates each virtual machine from its host and other virtual machines, leaving them unaffected if an application in a virtual machine crashes. Data does not leak across virtual machines and applications can only communicate over pre-configured network connections. VMware Server encapsulates a virtual machine environment as a file, which is easy to back-up, move and copy.

The banner at the top of the post suggests that the main impetus for this software might be green, in otherwords running fewer computers, and using less energy. If your energy bills are outrageous, you might consider this rationale. Reuters reported that a hospital in Munich, Germany, achieved a 50% energy savings in this manner. However, reducing the number of hardware servers altogether is probably a better reason. Our PACS implementation uses about 10 servers, with production, test and fail-over devices for PACS and RIS, and others for additional applications. Several of these might not require the full resources of an individual server, and the combination of virtual servers would suffice.

A whitepaper from Stratus Technologies dated June, 2008 further discusses the use of server virtualization:

Server virtualization is getting attention as hospitals and other healthcare delivery organizations increase IT usage while seeking ways to accommodate its cost. Among many other advantages, virtualization allows healthcare delivery organizations to save money by consolidating a number of applications on the same physical server. Server virtualization can be defined as the practice of using a software layer to let one physical computing server run multiple virtual machines that support multiple applications.

Why Virtualize?

Server Consolidation: Many applications in the enterprise require as little as 5-10 percent of a server’s capacity. Business enterprises that have implemented virtualization are opting to consolidate a number of these applications on a single physical server, while planning capacity for growth in the total workload.

Application Lifecycle Management: Compared with typical enterprise applications, healthcare IT applications are distinguished by the need for long lifecycles-possibly a decade or more. After the IT solution is in production, healthcare delivery organizations want to ensure stability and reduce risk by avoiding changes to the software application itself, as well as the operating system and the server hardware.

• Speed and ease of provisioning. Server virtualization allows you to create a standard virtual machine - consisting of software files that include the application and an operating system - that can be copied onto a server in a matter of minutes when additional capacity is required. The virtual machine can be qualified and tested in advance to ensure it will work as expected.


• Hardware and capacity upgrades. When more processing power or storage capacity is needed, virtualization can similarly let you move the virtual machine to newer hardware with no change to the application or operating system. You are not locked into older hardware.


• Failover and disaster recovery (DR). A virtual machine’s image - including configuration state, disk state and so on - residing on one physical server can also be periodically replicated to another physical server for backup or fast restart. Healthcare delivery organizations can use this capability to support DR. Not only is it prudent to provide for recovering electronically stored health data, but also The Health Insurance Portability and Accountability Act (HIPAA) requires a DR plan to do so. Some virtualization software also allows for point-in-time rollbacks. Useful when data corruption has occurred, rollback lets an administrator revert the virtual machine to an earlier known good state.


• Upgrades without downtime. A capability known as live migration allows for planned hardware and operating system upgrades (in cases where the operating system is not visible to the application) with virtually no interruption to the application and little perceived impact by users. Note that the operating system that can be upgraded is at the host OS/hypervisor layer; guest operating systems cannot be upgraded online.

So, it's an interesting idea, but Stratus identifies some potential problems, particularly worrisome in the health-care realm:

While consolidating applications on fewer computer servers has significant benefits, risks exist when the underlying platform (including hardware, virtualization software layer and drivers) is not sufficiently robust. A problem that affects the platform could cause downtime or performance issues for all the applications on that platform. The incident could then be followed by a long recovery time.

Stratus projects that this process will evolve into "True Utility Computing":

People want pools of servers that not only run multiple applications, but also enable dynamic and fluid use of resources. In the future, automated management capabilities - such as application-sensitive monitoring and intelligent policy making - will be necessary to bring this flexibility to the mainstream.

Until we reach this point, we'll have to be satisfied with VMWare, or other (nonfree) options such as Microsoft Virtual Server.

I haven't found many references to the use of virtual servers with PACS. Some posts on the Fuji Synapse message board suggest some hesitation:

I would not dream to install it on anything but a real server running real Windows OS. Some things like your core database you should not compromise on in the CIO / CTO'seffort to save expenses and virtualized the data center. You need a robust cluster to avoid downtimes. This I would insist and draw a line in the sand on if I were you...Same goes with your IIS ( DICOM ) Servers. Unless you can get 100 MB right up to the blade where the blade running Vmware / windows is, your performance in image transfer is going to be poor. I set up a test of this and the Vmware instance of a Fuji DICOM server would only allow users to download image to workstations at ~ 9 IPS versus a real servers speeds of 20+ IPS.

So, virtual servers might not yet be ready for PACS primetime, but certainly some of the less-demanding services, RIS, corporate email, and so on, could work well in this environment. Maybe cutting the purchase of one server won't make that much difference in your enterprise, but then again, it might, and this technique allows that possibility.

Could radiologists ever go virtual? They already have! Well, not really, but you get the idea.

90,000 Hits . . .While Dalai Does Acapulco

A lucky reader from Mount Clemens General Hospital, in Mount Clemens, Michigan, scored the 90,000th hit on DoctorDalai.com. I'm going to have to create a contest or offer a prize for #100,000.

We've had several hundred hits since then. You see, I do need to get away from the world of PACS every now and then, so Mrs. Dalai, my daughter "Dolly", and I spent a few days in Acapulco. You can get some good last minute deals, and in particular, Orbitz.com was able to deliver the hotel and the flights I wanted at a very good price.

Acapulco occupies a lovely bay off of the Pacific coast of Mexico. At night, the lights go on up into the hills, and they create one of the most beautiful vistas in the world.

We returned to a place called Las Brisas, where we had stayed 18 years ago. The motto of this pink and whitewashed hillside resort is that children are "seldom seen, but often made" there. My daughter is 20, but when we told people of our prior visit, we could see the wheels turning. Not quite, folks. (There were actually a bunch of kids staying at the resort this trip, reversing the philosophy.) Each room, or "cassita," has its own pool into which the staff throws hibuscus blossoms (some rooms share a pool), and the effect is very striking:

Las Brisas used to provide Jeeps to the guests for their use about the resort and in town. Sadly, traffic in Acapulco is totally unmanagable for a gringo like me, and apparently there were some incidents with the local police stopping the well-labelled tourists. Today, if you want a car, you have to rent your own. I could have had a SmartCar (not so smart in Acapulco traffic) or a non-pink Jeep, but I'm not good enough or crazy enough to drive there today.

We saw the obligatory sights, including the cliff divers (no doubt there are some out there who would have encouraged me to give it a go...)

I did get conned into trusting my life to this lovely horse for a ride on the beach. I won't be doing that again anytime soon...

The worst part of any vacation, of course, is the return to reality once it's over. I hope you'll forgive my brief hiatus in posting, but Las Brisas is only now acquiring WiFi. So much for paradise...

Amicas V6. . .It's Almost Here

I'm just back from Boston, having spent the past several days holed up with the developers of Amicas' latest and greatest, Version 6, or V6 for short. This was a lot more fun than you might think, as these are not only brilliant folks, but they are also some of the nicest and most down-to-earth people I know. They welcomed me and Mrs. Dalai (who was not amused by her title) to Amicas' world-wide headquarters (which is in the New Balance building by chance) and made us feel right at home.

I had the chance to simulate my workday sitting in front of a V6 workstation. As an added bonus, we were using one of the new Barco 6MP color monitors. I want one, although the $17K price tag will keep it out of my reach for a while.

HQ is a maze of cubbies and computers, much like any of the dot.coms in their heyday. There is no impending sale of the company, thank you, and work proceeds nicely on several fronts.

I cannot post any specifics on V6 at the moment, but your patience will be rewarded before too long. I will go out on a limb and tell you that its code name is "Phoenix", which is a bit of hyperbole based on the fact that this is one of a very few PACS products that has been completely rewritten from the ground up. Since its predecessors are still functioning well, the imagery of "rising from the ashes" is a little overblown. When I first saw the initial stub-laden pre-alpha version, I commented that it should be called "Freedom PACS" as in freedom from the way other big companies do things. That hasn't caught on as yet, but I keep hoping for a change of heart, and a chance at residuals for coining the name.

Since I can't tell you much about V6, I am left with telling you what it doesn't do (obviously stuff that can be found on some competing products):

1. V6 doesn't have an unstable worklist
2. It doesn't allow two people to be dictating the same study at the same time.
3. It won't show the demographics for the last patient you were looking at for 20 seconds or so while displaying the images for the next.
4. It will not show you the wrong report associated with the prior study you are viewing.
5. It will not randomly choose a prior to show you even after you have clicked the one you do want.
6. It doesn't have a hanging protocol setup so badly done that you don't bother using it.
7. V6 doesn't make you wait for everything in the worklist to be cached before it lets you start viewing an image.
8. It will not mistake "relevance" for "relevants"
9. It will not limit me to one viewport for one series, and float the series selectors over the viewing real-estate.
10. It will not utilize "toggling".
11. It will not treat examinations of the same patient from different campuses of the same hospital system as coming from entirely different patients, thus impairing comparison.
12. It will not hide functions in unexpected places.
13. It will not be confused as to how a three-dimensional cursor should operate.
14. It will not assume that changes (like windowing) to one slice of a CT or MR should not propagate to the rest of the series.
15. It will not erase the search-term you last used when you click in the text-window.
16. It does not make you open a study to see if someone else is reading it.
17. It will not display a study that was on your worklist when you first started reading, but has since been dictated.
18. It will not restrict you to six measurements per slice or image.
19. It does not make you buy another client and separate program to run via the web.
20. V6 will not be integrating someone else's software into an older version of itself to form any sort of hybrid product.
21. It will not require additional (extra cost) software to display MPR's or VR's.
22. It will not automatically change the parameters of a search from "all times" to "Last 7 Days".
23. It will not restrict you to placing a series in one and only one viewport.
24. It will not force you to use clone windows.
25. It will not make extensive use of toggling.
26. It will not promise individual functions (pre-cans, worklists, etc.) that somehow become system-wide and not individual at all.
27. It will not randomly change column-widths and sorting parameters and such.
28. It will not carry through old ideas that didn't work for anyone except for a select few members of a very focused focus group.
29. It was not created by letting the inmates run the asylum (thank you Alan Cooper!)

V6 will show the industry how PACS should work. As with Amicas' previous offerings, it will not be perfect, but it is pretty darn good. Trust me on that.

I'll let you know more shortly.