Saturday, July 24, 2010

Dalai's First Law

PACS IS the Radiology department. 

Agfa IMPAX 6.3 has been completely dead for the last three plus hours.  This takes down a Level One trauma center, and two other smaller hospitals.  So far, I have no explanation, and no end in sight.  The trauma center might have to consider a Code Yellow, as we have seen in Western Australia. 

I'm thinking we need to reconsider the concept of distributed architecture, which was discarded for the central archive and production processor model which is currently betraying us.  Even with three redundant application servers, we are down and dead. 

I have no further comments at this time.  I will let everyone know what happens when we get back up and running, although there will be a very large number of studies to be read when that happens.

I will invite Agfa to submit their narrative of these events once the dust settles.  This will, I'm sure, prove interesting and informative.

ADDENDUM:

We seem to be back up, after three and one half hours of downtime.  I've got some work to do, if you'll excuse me...

Friday, July 23, 2010

Agfa Fixes Problem!
One Down, 1,000,000 To Go...



In a previous post, I bemoaned the fact that Agfa IMPAX 6.X has about a million buttons with about a zillion permutations as to how they could be arranged. 

This turns out to be more of a problem than I thought.

You see, for the past year or so, we've been experiencing significant lags in PACS response, especially on the workstations with four high-res Barco monitors.  New workstation computers and new back-end software helped to some degree, but the lag was still there. Dr. Killer, our finest and most aggressive power-reader, was fit to be tied.  Even more than usual, that is. 

Now comes word from Waterloo that the etiology of this problem has been discovered.  It seems that years ago, when IMPAX was first installed (it only seems like decades), a customized Radiologist Role was created, adding most of the available tools (see image above) to the toolbars.  That was found to be unwieldy, and unnecessary tools were removed from the rad's accounts.  However, IMPAX, being the really intelligent program that it is, remembers that all of those lovely buttons were once there, and goes through the entire list of buttons each and every time we change a study.  To fix this little problem, all we have to do is delete the Radiologist role (not the radiologist, as some in IT and at Agfa might wish), reset our Application Servers, and then log back in.  Of course, each doc will have to customize the refreshed account all over again.  What fun that will be!   Since mine isn't all that slow, and I appreciate a little break in between studies, I'll probably stick with what I have.

So, tip of the hat to Agfa, and especially the good people in Waterloo.  Many thanks for taking care of this.  Anyone want to fess up as to who wrote this into the software?

Friday, July 09, 2010

Meeting in Chicago

As usual, it has taken me far too long to get this report to you, but I only have one laptop up here in the North Woods, where I'm playing Camp Quack (pediatrician) for the week.  Naturally, Mrs. Dalai's Farmville crops will suffer if they aren't tended to constantly, and so I have to wait until the plowing and harvesting is complete before I can do something so mundane as post to my blog.  Beaucoup thanks, Facebook.  Love ya, mean it.

I did finally get to Chicago, accompanied by our intrepid salesman, and former PACS administrator, Garn.  As an aside, Garn's family has been very close to my group over the years, his dad and sister having sold us our very first Advanced Video Products/eMed/Access teleradiology system in the ancient, pre-Dalai era.  Garn earned his keep on this trip, literally running through three terminals to hold our USAir flight from Charlotte to Chicago that departed about 3 minutes after we arrived from South Carolina.  (I got stuck behind some slow-pokes on the moving walkway.)  That's what I call service.

I love Chicago, but I wouldn't want to live there.  Traffic from O'Hare downtown was murderous, even at 8 P.M.  For better or worse, my daughter loves Chicago and does want to live there, and will shortly begin a four-year stint at Northwestern Medical School.  (You don't want to know what that will cost, but we're talking yet another delay in my retirement.)  I had the chance to scope out her apartment building, and I was glad to see that security wouldn't let me in.  You have to find some peace wherever you can when you send a small (4'10") child to live in the big city.  It's going to be a long four years.

Merge Headquarters is in a very nice, new building just south of the Chicago River, with a great view of Grant Park.  We were fortunate enough to catch the Taste of Chicago in progress down in the park, and had a great lunch there, instead of sandwiches in the airy conference room.

Comparing Merge Central to the old AMICAS headquarters in Boston's New Balance Building is an apples vs. oranges exercise.  The new Merge office has miles of glass, with all executive offices completely visible to the hallways.  At AMICAS, the offices were, well, offices, and the place had a more informal dot.com-era feel with more free-form workspaces.  Yes, everyone at Merge wears a dark suit, a white shirt (although I saw a few renegades with pinstriped shirts), and a requisite orange tie, or scarf for the ladies.  The Merge logo is everywhere, and I quickly gave up on my grand idea to suggest a corporate name change.

You see, this Merge is not your daddy's Merge.  When the Click/Merrick folks (Messrs. Ferro, Dearborn, et. al.) took over, they remade Merge into a completely different company than its predecesor.  No more financial shenanigans, and no more anything but a drive to turn the old framework into a billion-dollar company.  I think they have a good chance to do it, too.  But when us old folks (like me and Mike Cannavo) think of Merge, the spectres of the past and visions of eFilm on every monitor loom close to the surface.  I'll say it here: Merge might be better off renaming itself AMICAS, but there seems to be a significant investment in things orange and blue, so I'll drop that line of thought.

I had the opportunity to meet with some old friends from AMICAS, Paul Merrild, now Senior Vice President - Marketing Corporate Strategy, and Kurt Hammond, VP Sales.  Justin Dearborn, Merge CEO was present, as was Nancy Koenig, now Executive VP for Sales.  Alas, Mr. Ferro could not be there, but I was told he had recently returned from Washington and various White House meetings.  More on that shortly.

When you've questioned a big company publicly, as I've done here, it creates an uncertainty, a degree of apprehension, that can only be defused by a face-to-face meeting.  I'm sure the Merge folks (maybe even the old AMICAS guys) were expecting this version of Dr. Dalai:


Of course some will say after reading this piece that this is what Dr. Dalai had to drink in Chicago:


Which is true?  That is in the mind of the reader.  To be enigmatic, I'm reminded of the pilot episode of "Fantasy Island".  Mr. Roarke's assistant, Tattoo, was asked what he thought of his employer.  Tattoo said, "Some call him..." (Tattoo points to Heaven), "and some call him..."(Tattoo points down).  "And what do you call him, Tattoo?"  "I call him Mr. Roarke."

The truth is, as usual, somewhere in between.  We had a good, frank discussion, I came away impressed, and the Merge folks probably were slightly relieved.

On the wall of the Merge conference room are 60 framed patents from the Cedara division in Toronto.  About 45 of these are for various imaging processes or devices, and 15 are for master data management.  I believe Merge considers the Toronto office, and the 120 engineers (including 12 DICOM experts) located there, one of the most important parts of its operation.  (There are 50 more engineers out in the field.)  Cedara provides a number of OEM'ed products sold to the likes of GE and other big vendors, distributing Merge products much more widely than I had imagined.  But in the realm of RIS/PACS, I was told that Merge is now the Number Four vendor, behind GE, Philips, and McKesson.  They claim a rather large footprint of 1500 hospitals, and 2200 Imaging centers, for almost 4000 "points of presence".  I don't think this includes the ubiquitous eFilm users, either. It does represent a coming together of several cultures, Merge, AMICAS, Emageon, etc., with all that entails for the company.

I'm hoping this high figure doesn't include eFilm, as I'm not a big fan.  However, Merge loves eFilm, which it acquired in 2002:
In 2002, Merge also acquired a Canadian software developer eFilm that developed medical imaging and information workflow products and services. eFilm provided Merge with a software-only image viewer that could be downloaded over the Internet. That acquisition allowed Merge to attain its goal of becoming a global leader in diagnostic imaging software tools by achieving a record 20,000 downloads of its eFilm Workstation software in 2002. eFilm Workstation displays diagnostic images, using a standard PC and monitor, and provides exceptional navigation and viewing tools for optimal radiological interpretation. 
Someone at the table jokingly called eFilm the "Duct Tape of PACS" and then instantly regretted providing me with a pithy quote for the blog.  But most of us, especially males, appreciate the comparison and find it complementary, so here it is in print.  The analogy is apt:  a duct tape repair will hold, but it might not be pretty, and neither is the eFilm interface.  But it does work, and it brings in significant (and easy) revenue for Merge.  Apparently it is quite popular in other countries such as Brazil where there is only 5% penetration of digital imaging.

Merge has a far-reaching long term outlook, which will utilize their know-how in various fields to provide solutions to other vendors, such as Meditech and Allscripts, as well as for the various 'ologies, Cardiology, Anesthesiology (perioperative monitoring software), Pathology (storage for the rather large images of tissue slides, etc.), and Gastroenterology (in the form of storage for endoscopic images).
While Merge doesn't want to get into the HIS and similar ends of the EHR business, they wish to be complementary to those who are in that space.

The patient experience will be an important part of Merge's future, and they have a very nice patient kiosk up and running.  It can do everything from check you in to dispense a CD-ROM of your recent imaging studies.  In fact, it was this technology that Michael Ferro was trying to get the White House to understand as one aspect of the Meaningful Use of EHR technology.  Merge is also trying to get PACS wedged in there as well.

I had a brief demo as well of some of the CAD products from Cedara, including a very nice system for breast MRI and liver CT and MRI.  I gave Merge the Million Dollar Idea after seeing these programs:  rework the engine for PET/CT.  The autodetection and so forth would work perfectly in that venue.  I'll gladly accept $500K for this really great suggestion.  Small bills, please.

The ultimate plans for PACS involve Enterprise Content Management (ECM), a vendor-neutral archive developed by Emageon that came with AMICAS, as the main repository.  For the clinicians at least, a Web Access platform will overlay this, providing a zero-client viewer via AJAX (and not Flash) technology, with fast server-side rendering.  Note that this is not an appended web server (ala the bad old Web 1000) but an integrated viewer.

A related product promises to help with the portable patient problem I've been bemoaning for years.  The "Outside Study Gateway" is useful for trauma and other transferred patients with outside exams, which encompasses just about every patient in my experience.  This is to be accomplished as part of PACS, since the PACS is already in place, you know.  It should be capable of operating without a VPN, which to me, having people that can do VPN's at my beck and call, isn't a big deal.  I believe module actually requires an add-on server, which will cost extra, but if it works as advertised, I would strongly recommend that you buy it.

But what I really came to Chicago to discover was the future of AMICAS PACS, and I think the answer here is satisfactory.  The Merge people feel that Merge and AMICAS were more complementary than competitive, except in the realm of RIS/PACS.  Everyone agrees that AMICAS had a great PACS, and Merge had a good RIS.  These two programs, respectively, will be the go-forward products.  Everyone at the table wanted to be certain I deliver this message clearly, so I will quote verbatim:  "The approach to RIS/PACS will be the consolidation of the best practices of all of the applications into a single platform workflow and viewing solution."  Well, there you have it.  All Merge RIS/PACS customers, including those using AMICAS PACS, Fusion PACS, or RadStream (from Emageon) WILL BE SUPPORTED, and will have an upgrade path.  How much the upgrades will be wasn't mentioned.  What I will call AMICAS 7 will ultimately include the best of all worlds, blending the best of the legacy products, and using the ECM as the back-end.  (In discussing this with my PACS administrator today, we wondered how this will affect the way AMICAS currently stores images, with DICOM going to backup and JPEG2000 on spinning RAID.  Perhaps everything will now be DICOM, which would require a larger online storage facility.  Fortunately, storage is cheap these days.)

Feeling I had to be at least a little ornery, I questioned whether Merge would be able to graft the AMICAS front-end onto the Emageon ECM.  After all, our friends at another larGE company have yet to successfully do something similar with the new GUI they bought a few years back and their shiny new LINUX back-end.  I got a few laughs with that one.  Basically, Merge feels that the ECM is a very different animal than the LINUX thingie (technical term mine), more DICOM-friendly, and more amenable to having other interfaces plugged into it.  (I refrained from asking if I could get IMPAX 6.x to plug into it, too.)  The ECM should have better standards, and an interface has already been created for the Camtronics/AMICAS Vericis cardiac program.

At this point, in the meeting, I felt cautiously optimistic.  I was impressed by way my old AMICAS friends have transitioned into this new reality, as well as Ms. Koenig's and Mr. Dearborn's enthusiasm for where the company is and where it's going.  Merge will not only make a go of all this, but will actually succeed mightily.  But now I'm now going to share with you the best part of the show, and the person I met that completely and totally "gets it".  I'm referring to Luc, the Director of Patient Experience.  After the talking points and the white-boarding, and so forth, Luc walked me down a corridor, and opened an unmarked door.  Inside the long, narrow room, were....Video Games!  There were various driving games, a machine that played multiple legacy games, and a coin-operated version of Guitar Hero.  Luc asked me which I thought was the most popular, and flailing a bit, I pointed to the multiple game console.  80,000 games in one box is not to be sneezed at, right?  Well, Luc laughed a bit and shook his head.  No, the most popular game is the machine with the simplest, easiest-to-use interface, which would be Guitar Hero.  So what was the point of this lesson?  Simply this:  To create an interface, especially for patients, but quite applicable to physicians, one must simplify, but still have a complete understanding of how the thing had to work in the hands of the users.  It was not enough, Luc said, to bring a radiologist or two into the company and see how they like things.  Rather, it is necessary to go observe how the radiologist uses the GUI in the hospital, in his environment.  Only then will it be apparent how things should work.

Looking back, I'm not sure if Luc was thinking more about the patient-kiosk interface we were about to see than the AMICAS PACS interface that is already here.  But I still think he gets it, and by extension, I'm hoping Merge does as well.

This is a different path than I hoped to be following six months ago, but I'm more and more hopeful that it is a good one.  I know that my favorite AMICAS developers have been working to impart their knowledge about Version 6 to the Cedara folks in Toronto, and for that I will be forever grateful.  Certainly this was above and beyond the norm for a departing employee, but that's the sort of people they are.  With some luck, their colleagues in Toronto will prove to be worthy successors.

So, did I drink the Kool Aid?  Nah, I had a Diet Pepsi at the Taste of Chicago...

Monday, July 05, 2010

Inca Trails and Tribulations

Dalai's Note: As I posted earlier, we made it to Machu Picchu...for only two hours. Here is the story as told to the tour company, whose name is being withheld...for now. Ah, the joys of traveling in Third-World nations.

To:  Tour Company
 
From: Dr. Dalai and Friends
 
RE:  Recent trip to Peru
 
We recently traveled to Peru with an itinerary and services provided by your company.  With the advent of the strike in Cuzco and subsequent travel restrictions, we experienced the following alterations in our schedule:
 
1.  Time at Machu Picchu limited to two hours.
2.  We were not able to stay in the Sanctuary Lodge, but instead spent two nights in the three-star Casa Andina Classic Hotel in Cuzco. 
3.  We did not receive meals that were to have been provided at the Sanctuary 
4.  We did not experience the Sacred Valley Tour.
5.  We traveled a day earlier to Lima (Friday 6/18) and spent that night in the Casa Andina Private Collection Hotel. 
 
Additional payments and refunds must be evaluated in light of these changes.
 
We need to discuss the way in which your agency dealt with the strike and subsequent events.  It should be clarified that the strike was not by Peru Rail per se, but rather by the General Workers’ Confederation (Cgtp), Peru's largest union.  It was a precautionary measure for the Ministry of Tourism to suspend the rail service linking Cuzco to Machu Picchu.  The Minister stated that this was done “in order not to expose travelers to potential acts of violence”. 
 
Your agents in Peru were aware of the strike, but initially told us it would not extend into Friday, June 18.  It was not until we were actually touring Machu Picchu that our guide began hinting that the strike would be in effect through Friday, and when I spoke with the agent in Lima by cell-phone, she was certain it would go into Friday.  She did offer some hope that the Sanctuary, owned by the same company as Peru Rail, could arrange for our evacuation on Friday.  As we held unrefundable airfares, we felt we had no choice but to leave Machu Picchu emergently on Wednesday.
 
We took what we were told was the last train from Aguas Callientes to the hydroelectric station at 1:15 PM that day, with assurances that a van would meet us at the station no later than 4PM to take us back to Cuzco.  (As an aside, the guide's phone ran out of charge.  We tried to use mine (at international rates) but I could not get a consistent signal.  The guide also asked us to pay her $8 train fare.)  By 4:30, she informed us that she had to take the last train back to Aguas Calientes because she had a group coming in on Saturday June 19.  She left us with the number of her discharged cell phone, as well as that of the driver who was supposedly en route to pick us up from the station.  She asked Jose, a railroad employee manning the station, to watch out for us while we waited.
 
The hydroelectric station stop has very few amenities, although it did have a restroom that our guide didn't know about; instead, she sent us 400 meters up the tracks to use the facilities at a tiny restaurant.  Ultimately, The driver appeared at 7PM, 5 hours after we had arrived.  I had attempted to call your Lima representative with no luck, and finally called our Galapagos outfitter, who had referred us to you.  He was able to reach one of your offices, and someone, after several attempts, contacted me via cell, informing us that "due to the strike" (which was not to start until midnight), the driver had to take alternate roads and was delayed, but he was on his way.  We learned that much of the delay was simply due to the very poor condition of the roads between Santa Theresa and the station.  We were tired, dirty, and most of us had been stung or bitten multiple times by tiny black insects, which left painful lesions that we still have today, a week later. 
 
The ride to Santa Theresa was one of the most harrowing of our lives, over narrow paths with no barrier to adjacent drop-offs down to the river, and "bridges" over rushing water that could barely accommodate the van.  We did arrive in Santa Theresa, transferred to another van driven by another driver, bought a snack and used the restroom with the urging that we had to get moving quickly to avoid the strikers.  We then had a SEVEN hour ride over similarly treacherous "roads" with no stops for bathrooms or food.  At one point, not far from Santa Theresa, we were stopped behind a stalled car, and had to allow a truck to pass the other way, with literally an inch to spare.  As we came out of that particular pass, we were stopped by a two men and a woman who knocked on the window.  Our driver opened the door to these strangers, who claimed (and ultimately proved to be, we think) Americans stranded and needing to get back to Cuzco.  We allowed them to join us.  We were stopped at least four more times by police, and at each stop our driver got out and showed his papers to them.  At one of the stops, the police shined flashlights into the van, scaring the youngest child in our group.  Just outside Cuzco, Leo stopped and got out to talk with some men who flagged us down but were not wearing uniforms.  We finally reached our hotel in Cuzco at 3:30 AM.
 
Your company and its agents did not handle this situation well.  We realize that the strike itself was beyond your control, but there are several additional factors that caused us great distress.  There was an inexcusable lack of disclosure, communication, and information, which becomes more and more apparent as we review the facts of the situation:
 
1.  Strikes happen often in Peru.  From the US State Department:

Political demonstrations and labor-related strikes and marches regularly occur in urban and some rural areas and sometimes affect major highways. They can also cause serious disruptions to road, air, and rail transportation. Demonstrations are often – but not always – announced in advance. While these activities are usually peaceful, they can escalate into violent confrontations. As a general rule, it is best to avoid large crowds and demonstrations. Visitors are encouraged to keep informed by following the local news and consulting hotel personnel and tour guides.

The date and duration of the strike was known to other tour agencies, who adjusted the schedules of their clients so they would be in and out of Machu Picchu without difficulty.  Why did your company not have these details? 
 
2.  You placed our party of eight, including children, in unnecessary danger with the treacherous van ride back to Cuzco.  We were not informed of the degree of danger we were to experience by traveling at night, on back "roads" barely worthy of a foot-path, let alone the direct danger of being confronted by strikers.  In addition, the State Department notes:

Crime also occurs on roads, particularly at night and outside urban areas. Clandestine, impromptu roadblocks can appear on even major highways, where bus and automobile passengers are robbed. The risk is even greater on rural roads after dark.
 
Road travel at night is extremely dangerous due to poor road markings and frequent unmarked road hazards. Drivers should not travel alone on rural roads, even in daylight. Convoy travel is preferable. Spare tires, parts, and fuel are needed when traveling in remote areas, where distances between service areas are great. Fog is common on coastal and mountain highways, and the resulting poor visibility frequently causes accidents. Inter-city bus travel is dangerous. Armed robbers, who force passengers off buses and steal their belongings, sometimes hold up inter-city buses at night. Bus accidents resulting in multiple deaths and injuries are common, and they are frequently attributed to excessive speed, poor bus maintenance, and driver fatigue.

3.  The second driver placed us in further danger by opening the door of the van to non-uniformed people he didn't know.
 
4.  Neither of the drivers that night spoke English.  Had my daughter not been capable of understanding Spanish, we could very well have been in even deeper trouble. 
 
We placed our trust in you, anticipating a safe adventure and visit to one of the new Wonders of the World.  Your agents betrayed that trust, and disregarded our safety, literally placing our lives in jeopardy, not to mention depriving us of all but a glimpse of our destination that was something I had waited forty years to see.  We do not believe their intent was malicious, but the situation was handled in a manner that can only be termed incompetent. 
 
I believe proper compensation would be a refund of ALL monies paid to you, as well as a letter of apology from your owners, for the complete and total, and nearly tragic failure of your agents to deal with a situation you should have anticipated.  
 
We await your response.  
 
Sincerely,
 
The Dalai Family

Saturday, July 03, 2010

Please Stand By...

I'm currently at 30,000 feet, using Delta's GoGo inflight WiFi service, en route to the North Woods where I will reprise my roll as Camp Doc, or Quack as the kids there put it. I guess my reputation precedes me. Look for the writeup of my visit with Merge shortly.