Something I Have + Something I Know = Two Factor Authentication.

Math is awesome. Especially when it is simple.

The big idea now in IT security is multi-factor authentication. The goal, of course is to prove beyond any doubt that the individual trying to access your system is who they say they are.

People can share passwords.

People can share credit card sized devices.

People can lose credit card sized devices.

People can leave a sticky note with a username AND password stuck to the very monitor of the system that is trying to be protected with said username and password.

So how can we be sure that the person logging in is who they say they are? As this is the third and final part of our series on authentication devices for healthcare, we will try to wrap all of this up into nice little bow.

If only it were that easy.

I keep forgetting about smart cards. But there is a good reason. I swear… the reason for which I will divulge after we discuss them more thoroughly.

The idea of smart cards is to give a very strong method of authentication by incorporating essentially a mini computer to help the big computer verify that the user is not a spy, thief, secret agent guinea pig or the ex-boyfriend of a patient trying to find out if his ex and her new husband are pregnant.

The smart card is really a miniaturized computer. Many people really don’t know that there is a micro processor and RAM built-in to these nifty little devices. Some of them are capable of quite a bit, but for this discussion we’ll just focus on a normal enterprise environment (typical hospitals).

The smart card will usually contain an encrypted unique identifier that can only be released to the big (host) computer by the correct password being entered by the user. This provides our two factors of authentication. The card as something you have and the password as something the user knows. Utilizing the internal storage of the card there could be additional levels of ‘something you know’ stored on the card like “Mother’s maiden name” or “first pet”.

From a user side: The user slips the card into the smart card reader, enters a password and gets to work. When the user is done, they simply remove the card and go on their way, which will automatically terminate or lock the user’s desktop until a smart card is inserted and the process starts over again.

As an added bonus, quite a few computer manufacturers are incorporating smart card slots into the computer hardware as standard equipment.

So we do indeed have a much stronger method of authentication by using smart cards. But here are the issues (and why I often disregard them as a viable authentication method for healthcare):

1. Cost: Smart cards (at least the brands you want to use) run from $3.00 to $20.00 a piece and that’s buying in bulk. That can get expensive for an entire user-base that is likely to forget those cards at a rate of 20-30% per year (see #3 below). The smart card readers are also much more likely to break or wear out and need replacement or more regular maintenance than prox readers.
2. Breakage: The microchips are fragile and can wear out, making the card unreadable. Since the card has to be inserted into a reader, the plastic and laminate wears out much faster than a proximity card (which is zero touch/rub).
3. Forgotten cards: Since the user has to leave the card in the reader for the duration of the session, the card is frequently forgotten at the end of the session and left in the reader… often with the session still open for someone walking by to use.

The difference between a true smart card (as described above) and a prox card as described in Part 1 of this series, is the fact that the smart card is unique and encrypted where a prox card has an easily readable number that is just passed to the prox reader. In theory, this number can be forged into a ‘dummy prox card’- but the user of the dummy card still has to have the password.

The ultimate in security is a biometric identifier stored on a smartcard that also requires a password. This goes beyond two factor authentication to something else entirely and even if you have the Russians (or Ethan Hunt of the Impossible Mission Force) trying to steal state secrets, your Chief Security Officer will feel pretty good about him/herself. Of course, it will make the user’s lives a nightmare for access… but that’s progress!

Authentication Device Wrap-Up

From a Healthcare perspective, although all methods discussed in this series can work: there are currently two technologies that make the most sense from a total security, TCO and usability/speed of access standpoint. What I take into account most (after security) is the effect on the users and the fact that doctors and nurses are already being slowed down by new EHR and CPOE systems and so we are looking for the simplest and fastest way to get them in and out with minimal frustration.

1. Fingerprint biometrics. Until the cost comes down on palm-vein scans, fingerprints offer the best security the 2^nd best usability, 2^nd best cost and allow for the possibility of the user not having to remember ANY passwords (huge plus for the users and the help desk). The main downside is implementation and getting the users trained. It may sound simple but making sure the users put their finger in the right place on the sensor is a big deal and causes user frustration when the “device doesn’t work”.
2. Passive Proximity cards. Prox cards are less expensive than smart cards, wear out at a much slower rate and are less likely to be left at the last computer used. They currently offer the fastest possible method of access and users usually don’t screw up waving a card in front of the device (though I have seen users trying to wave the card at the monitor).

Coupled with the right Single Sign-On or ease of access solution, either of these methods will make for a happy CSO and happy doctors and nurses (which ain’t easy).

While speaking with hospital personnel about what type of authentication technology is most appropriate, people’s minds seem to wander to the movies.

When we talk about fingerprint biometrics I’ve had two people actually ask about faking the fingerprint or someone “cutting off a finger to gain access”. To which my reply both times was:

“If you have an intruder willing to cut off someone’s finger to gain access to patient’s records, you have much bigger problems than patient privacy or HIPAA.”

But biometrics are so bleeding edge and have been around in movies for so long there is always a little ‘Mission Impossible’ to it.

"Must... get... Tom Brady's knee surgery records. Only 6 seconds left."

Biometrics can eliminate the need for multi-factor authentication because it solves the much better “who are you” question instead of the “what you have” or “what you know” questions, which can be shared. Some state laws may still require the user to enter a pin or password for drugs (especially narcotics access).

You can’t steal a hand or eye (leaving aside the murderous evil-doers mentioned above) and you can’t borrow it. You know for certain who accessed that system.

For ‘warm and fuzzies’ in the security department, you have your technology.

More importantly, you can’t forget your finger/eye/hand at home or at the last workstation as you can with various card technologies.

But what is really useful? Practical? Cost Effective? PROVEN???

What makes sense for a hospital? First, a little background data. Keep in mind that a large part behind these technologies is not necessarily acquiring the data (fingerprint, vein scan, etc.) but more the algorithms and methods of storing and comparing the captured data to the existing database.

Fingerprint

Fingerprint has been a known biometric identifier since 1788 when a German by the name of Mayer made the discovery that fingerprints are actually unique to each individual. The process to identify the ridges on the fingers and match them for identification purposes was very cumbersome until computers helped the process. As technology developed, other methods of identifying the prints made the process quicker. For instance, many technologies don’t focus on the lines themselves but on the minutiae points on any given finger. See illustration.

Different types of minutiae points.

Any one of those items will be a minutiae point. The computer then maps them, which makes the comparison easier as the computer now has a much smaller sample set of points to compare as opposed to trying to compare the location and length of lines.

Some fingerprint technologies make it even easier by using the location and reference of those points and turning it into a unique number using complex algorithms, which makes it even easier to search. This is the method that Symantec Workspace uses with the SagemMorpho biometric. Some call this a ‘one to many’ match technology. Searching for a number is much faster than ‘one to one’ matching of prints. Funny enough, as you look across the web; there is some contradicting information on the specific terminology of ‘one to one’ or ‘one to many’. As long as you get the “fastest”, you are all set.

It is also important to mention the two major types of fingerprint scanning technology.

Capacitive: Uses the electrical current transferred by the pressing of two  plates to generate the image of the ridges in the fingerprint.

These are rather cheap, compact and a pain in the butt .

Capacitive sensor common on many modern laptops.

Many laptop manufacturers integrate capacitance scanners into their laptops these days. The problem with capacitance is the error rates (both false positives and false negatives). Capacitance scanners are also the easiest to trick or fake. You can even find youtube videos of people fooling them using gummi bears.

We often have customers ask if we can use the scanners already built-in to their computers. Our answer: NO. Even if the algorithms were the same, we wouldn’t want to for the sheer annoyance the users would suffer and then there’s the sheer volume of false positives.

About 5 years ago when we first started down the Single Sign-On path and before we got into complete “Access Management“, I was doing a demonstration at the national  HIMSS conference (I think it was in San Diego) with capacitive fingerprint sensors. I would put my finger down and be into the system… great!  But then a nurse asked what would happen if she put her finger down? “Nothing” was my reply. So she did. And got in… as me! Not only was it embarrassing. It was a wake-up call. No matter what the manufacturer told us about accuracy rates, that was unacceptable. We vowed to not mess with those things again. We don’t have those issues at all with optical.

Optical: Optical fingerprint scanners are simply using camera based technology (Charged Coupled Device or CCD) to gain the image and then compare it. Optical scanners are usually a bit larger than capacitive scanners and the good ones are more expensive. Optical scanners are far more accurate (depending on the manufacturer) and a bit harder to trick than capacitance.

If you choose to use optical scanners for authentication, be aware that the bigger the platen (area for image capture), the more likely the user’s finger is to be captured correctly the first time and user experience is a huge factor in the acceptance of these technologies. In this case, bigger is better.

Palm (Vein) Scans

By far the coolest is the vein scanning technology that companies like Fujitsu are pioneering. The user holds the hand above a near Infrared signal that is bounced back to the device…except where the veins are actively transporting deoxygenated blood which will absorb the signal, giving a vein pattern that is unique to the individual.

Palm Vein Scan

This is awesome technology for Healthcare (almost). It is zero-touch and secure. It has been used in Japan ATM machines for the last 3-5 years. There is some question as to the speed and it needs to mature a little bit as far as how the devices are going to integrate with a desktop, but it’s pretty darn close.

Unlike fingerprints, the veins are inside the hand and will not wear off. This means that every user that has a live hand can be enrolled.

The real gotcha or ‘almost’ is cost. Right now at $1000 a device, it is just not cost-effective yet to put on all the devices a clinician might access.

Added Coolness: Even identical twins will have different vein scans.

Another side benefit is that if someone cuts off the hand, there is no blood flow and thus no vein signature. Whew…  now I have a really good answer to those murderous villains trying to access your Aunt Betty’s  heart history.

We’ll have to wait to see how the Mission Impossible team fakes palm vein scans.

Iris/Retinal

Another movie favorite is Iris Scanning. A digital picture is taken of the iris, converted to a digital template and is matched against others. The algorithms currently developed make it the fastest authentication method for finding a ‘one to many’ match.

Retinal Scan

For computer authentication in healthcare, Iris scanning is problematic. Mounting the scanner is important and fraught with peril. Simply mounting to the monitor is not usually good enough since most monitors now have thin bezels and are often way to far from the user for the scanners that make sense from a cost standpoint.  Since image quality is key to the process, cheap scanners are not acceptable.

There is also the tricky part of keeping the users still for the brief duration of the scan and I don’t see many doctors and nurses standing in one spot for long, do you?

Due to these limitations, there have been very few manufacturers doing the software development to integrate iris scanning into their products.

It makes for a good movie scene, such as in Angels & Demons when the good Doctor’s eye is cut out to get access to the deadly anti-matter, but retinal scanners are only looking for a picture and cannot detect between fake eyes and real ones.

Healthcare Use

At the moment: based on cost, usability, availability and track record my opinion is that Optical Fingerprints are the best biometric method of authentication.

As I mentioned above, the practicality of iris scanning in the healthcare setting just isn’t there.

I’ve seen optical fingerprints used for authentication in hospitals for more than 5 years now with the only major concerns being cost (of big optical scanners), enrollment and infection control.

Palm Vein Scans might be there soon enough, but the cost needs to come down.  Palm Vein scans look to be positioned to eliminate the complaints we have now with fingerprints mentioned above and give the added benefit of being able to enroll everyone (except amputees).

In Star Wars IV: A New Hope there is the scene where Luke, Leia, Han & Chewie are trying to escape from the Death Star, there are two quick sequences:
One where Luke actually uses his blaster to close the door as Stormtroopers follow closely behind, only to realize a second later that the controls he blasted were the only way to extend the bridge to bring them to safety.

Password not Verified. Please try again.

The other sequence is where the Stormtroopers are chasing Han & Chewie and the Stormtroopers are yelling “Close the blast doors, Close the blast doors”. Then the doors get closed with them on the wrong side and they yell, “Open the blast doors, Open the blast doors”.  Who are they talking to? How does the entity on the other end of the communication know which of the 10,000 plus doors on the Death Star they mean?

The scene with Luke sticks out for me because I also have a vivid memory of a Doctor in a local hospital literally hitting his keyboard as hard as he could and (nearly) yelling “That was the right password!” (I left out an expletive.)

The keyboard did not survive and the doctor had to go find another computer to abuse.

Arthur's Holy Grail was a little different. A little cartoonish, if you ask me.

The holy grail for most clinicians is not having to ever remember one of the 10-20 passwords their systems require for access. Now, that isn’t always realistic (especially with some state’s eRX policies) but Single Sign-On solutions like Symantec Workspace Corporate or Imprivata OneSign can bring that down to one or two.

But if the hospital really wants to get close to that Holy Grail, they need some sort of hardware device to help. (For my idea of the complete Holy Grail, see previous blog post: SSO vs User Experience.) Right now those options include:

1. Active Proximity Device
2. Passive Proximity Card (‘Pop Cards’)
3. Biometric (finger, palm, iris, facial)
4. SmartCard

In a three part blog series, I’m going to cover all of them. With Part I, we’ll start with both types of Proximity, since they all too often get confused as the same thing.

Active Proximity

Active proximity devices usually require two pieces of hardware to work. A device attached to the computer and a device that the user wears (usually attached to a lanyard around the neck) which will have a built-in battery that extends the range of the proximity device to as much as 3-10 feet.  The device is always sending a signal and when the user comes within the radius, the device attached to the computer detects the user device and credentials can be automatically entered or a login screen can be presented with username already filled in.

The concept is great: the device the user wears sends out a constant signal that gets picked up as the user walks near a computer. The computer can then be programmed to automatically log the user in with minimal to zero interaction. This is soooo close to the Holy Grail but it can get rather problematic in close quarters (like an ER) where multiple proximity devices are close enough to the reader to cause unintended log-ins and sometimes login the wrong user. In those intances it gets to be extremely frustrating.

Active Proximity Pros

• Easy to use
• Zero Touch (no infection control issues)
• Speeds up log-in process
• addresses the “something you have” part of authentication

Active Proximity Cons

• Bulky
• Battery maintenance concerns
• Multiple user conflicts
• Unintended log-ins
• Does not address “what you know” part of authentication
• Cost

Passive Proximity

Passive Proximity devices require a USB hardware device attached to the computer and the user to have a prox card (or ‘pop’ card).  The card is held a few inches from the card reader and the reader passes card number to the SSO system. The system can then automatically log the user in or at least present the login screen with the username already filled in.

A very large number of facilities already use this technology for physical door access. Extending this technology to computer access will make a lot of sense because users are already accustomed to swiping the card to open a door and they don’t have to get used to another device. Adoption by physicians and nurses has been very high compared to other forms of technology.  RFIdeas is the vendor we use due to their ability to read all types of existing proximity cards from various vendors.

Passive Proximity Pros

• Easy to use
• Leverage existing physical access system
• Users probably already carry a prox card
• Zero Touch (no infection control issues)
• Speeds up log-in process
• Addresses the “something you have” part of authentication
• Least expensive method of hardware authentication

Passive Proximity Cons

• does not address “what you know” part of authentication
• lost cards

Both forms of proximity have their use cases. Passive proximity, due to low cost and the fact that most hospitals already use prox-cards for door access, tends to be the preferred choice. For active proximity we see the problems with unintended log-ins will often negate the benefits in the real world.

We are seeing more facilities leverage passive proximity in a scenario where the user will be able to tap their card, enter a password and get into the system. In some facilities we will configure the system so that the users will not have to enter a password again for the next 2-6 hours.  This gives a level of security in that if the card gets stolen, it is only good for a short amount of time. For states and hospitals requiring dual authentication, some SSOs like Workspace corporate will allow the hospital to use dual-authentication only for the pharmacy application. This is pretty darn close to the Holy Grail and nearly eliminates the doctor getting getting irritated because the machine can’t remember his password correctly. I’ll do a more comprehensive side-by-side once after all 3 parts of the authentication series.

Part II is next: Biometric Authentication for Healthcare.

Wandering through the local Best Buy and seeing the pre-pre-black friday mayhem: pallets upon pallets of brand new technology, just waiting to be discounted to nothing, a particular line hit me that was advertising their geek squad:

“We’ve read the manual”

There had better be a free 100" Plasma and an all you can eat buffet at the end of that line.

Now if you read my last post: Christmas Eve for Healthcare IT, you may have guessed that when it comes to kids toys and Trade Show Booths, I don’t always read the manual. (In my defense, there was no manual for the trade show booth).

But I always read the manual at work.

Does it do any good? Depends on the author, but in the real world, it is important to make sure you know how things fit together beforehand.

The sad thing is that right now, there are a lot of HIT departments that want a manual. They’d give anything for one – A definitive answer for how to pick the right solution(s) and then how to integrate them right the first time. Although IT is built on ‘trial and error’, there is a lot of trial going on and way too much error going on in Healthcare right now.

And that’s because there isn’t a manual.

I can go out and find a rather definitive blueprint on how to setup Windows 2008 Infrastructure, using VMware vSphere in  a 20,000 user+ environment and probably integrate some Cisco networking infrastructure. And that ain’t easy.

If I want to know the best way to setup a complete HIT environment, from the EHR to the backend Windows infrastructure to an actual access management solution for a 600 bed hospital… it just doesn’t exist. You can’t even find one for a 100 bed hospital.

I got into a short theoretical discussion on twitter yesterday with Paul Roemer (author of http://healthcareitstrategy.com/) regarding how we could use the internet to be the backbone to push out EHR. For me, this discussion quickly got down to the fact that everyone is too far behind and every hospital seems to have their own ideas of how they want to do things.

You can even look at two hospitals on the same version of McKesson Horizon and they will have a completely different PACS system and a different pharmacy (eRx) program, a different Scheduling system with a different back-end infrastructure and a different way to deploy those programs to the end-user.

WHY?  This is crazy, isn’t it?

It starts with a complete lack of leadership from the industry after Bush’s 2004 State of the Union where he called for Electronic records by 2014. While an admirable goal, no vision that big will ever succeed without cohesive leadership and a real plan. Money isn’t even the issue anymore, it’s the figuring it out part.

There are so many stakeholders at each individual facilty and so much money at stake that the decision making process is gets bogged down in committees, just like the Obama’s big Healthcare proposal now in congress. Politics.

Without the proper leadership and vision, politics is becoming a stumbling block. And after we get over that stumbling block, we get to the “How the heck do we do that?”.

5 years after Bush’s EHR goal, I am not the first one to say we won’t make it. I know firsthand of hospitals that aren’t even scheduled to begin Phase 1 until 2012. The individual physician practices are way behind that.

Look familiar? If not, it will soon.

First we need the leadership (define meaningful use). Then we need consultants that can be trusted.

Now there are a few third party consultants out there who have been-there, done that. They should be able to help the next group avoid the potholes someone else has stepped in.

But there are always more potholes.

And there aren’t enough of them (consultants)… mostly because there are so few complete and successful implementations so far.

But we need that soon.

We need someone to write the manual(s). If only it were as easy as installing a Plasma TV.

I just returned from the regional Midwest  HIMSS conference where we had a booth to demonstrate my company’s immense breadth and depth of knowledge in the hospital marketplace. It was a great show. We spoke to a lot of great people.

The day before it started, I arrived at the booth, dumped out all the parts needed to assemble the booth and assorted equipment and then had a sudden flashback to Christmas.

As any good parent knows, Santa does all of his work on December 24^th in a frenzied 12-24 hour period of realization that whoever decided for him that all this should be done in one night was not a good project manager. As kids get older, Santa has more and more complex toys to assemble.

As in the trade show booth, I usually just open the boxes, spread everything out onto the floor and then figure out how to make it work, hoping I have the right tools or at least that my son didn’t  take the drill and leave it discharged in a pile of dinosaurs.

Last year I made the mistake of opening a kitchen and a racetrack at the same time. It didn’t seem like it would be an issue: the parts look nothing alike but, in retrospect, the total scope of the mess I hadn’t really appreciated beforehand.

Actual stock photo of the Costco kitchen that ruined my Christmas Eve. My wife was thrilled.

This image of a father sitting down to put together some incredibly complex toy with hundreds of parts, screws, glue, stickers and a hard deadline translates well to what Hospital IT departments now have to deal with as they are currently in the throes of implementing many complex software/hardware packages that may or may not play well together.

What to do first? Do I have the tools? Do I have enough people to do the heavy lifting? How much did that cost? My deadline is when?

Now for most HIT departments, this mess isn’t their doing.  Someone from up on high, with impetus from the government, board members, physician community has pretty much decided to take every possible major and most expensive project possible and throw it at them with only a few short years till Go-Live.  Politics, Lack of Standards and Money also had a very large factor in the delays that brought us to where we are.

But the reality is that we are now standing over a pile of parts at 1am on Christmas Eve, with the likelihood that the kids will wake up even earlier than they usually do and won’t be very happy when they unwrap it and it doesn’t work.

Here’s a generic list of what they are looking at. Somewhat in order of what someone else might think their priority should be:

• HIS implementation
• CPOE (eRx)implementation
• EHR implementation
• Point of Care Charting
  • Hardware
    • What endpoint (tablet, laptop, desktop, mobile device)
    • How to get endpoint to bedside (WOW, tablet, wall mount)
    • How to secure endpoint
  • Software
  • Ancillary Devices (scanners, glucometer, spirometer, Dictaphone…..)
• Single Sign-On
• Endpoint Virtualization
• Server Virtualization
• Next Generation Desktop Hardware
• Next Generation desktop OS
  • XPe, CE,  etc.
  • Windows 7 migration
• Point of Care Med Distribution
  • 5 Rights
  • Security
  • Workflow
• Data Aggregation
• PACS storage projects
• General Storage Projects. SAN
• Wireless Implementation
• DR (everywhere else in the world, this means disaster recovery. For HIT, it usually means physician)
• HIE (Health Information Exchange) Planning

And this is a assuming you have other projects already complete like PACS, Billing, Claims Processing, Scheduling, etc. That may be a huge assumption.

The difference between HIT and this hamster is that HIT has a Go-Live (or several).

The sad part is these deadlines are set by people (Presidents, Government, board of directors, CEOs) who have no idea what is involved in making these things work together and are somehow neglecting the fact that everything in the hospital is a mission critical application. It has to work or people could die.

So, how do we organize these? What to do first? How do I avoid mixing race track parts with toy kitchen parts? That is a topic for another day.

But it’s a good thing Santa has beer in the refrigerator.

I had the opportunity to talk to one of VMware’s lead View engineers. Great guy. Energetic and full of information.

During his talk he specifically said that View deployments right now are taking 18 months if the customer does it themselves and 6 months if the customer uses a partner.

This is a great advertisement for VMware partners and consultants.

And this isn’t a big surprise to me. See my previous post for why:  http://rx4it.wordpress.com/2009/09/24/virtualization-is-sexy/

What is a surprise is that he admitted it in front of all those potential customers (150+ attendees in that break-out session). He is just being honest, which is awesome. And it will also apply to XenDesktop.

I cringed a bit. Here’s why: He is talking about XP deployments. (At least I have to assume he is. Even preferred Microsoft customers haven’t had Windows7 for 18 months).

If you start a full desktop refresh (thick clients) of a major enterprise from XP to Windows7,  it would take a lot of testing, torture testing, time and more testing.  There are so many things to think about.  Time is not on your side.

I know a few hospital systems that took a full 18 months to migrate to XP from 2000.  One I can think of isn’t even done yet.

Now, let’s assume you are going to do VDI. Everything is different. Now that Windows7 is official (as of today), would you start a new VDI (View4 or XD4) implementation and put all that time into making XP work?  Some will. But what an opportunity you’ve lost.

Windows7 will be the Virtual Desktop of the future. Sooner rather than later. And it makes sense. I’ve been playing with the Release Candidate for months and I just installed the production version on my daily use machine. It’s great.

But back to the point: So now, an enterprise must take all the time to test Windows7 on a large scale and then add a minimum of 6 months to 18 months to test and deploy VDI? *cringe*

Or the alternative… implement VDI with XP. It’s safer and will save some application compatibility testing but it will waste time in the long run.

Healthcare IT (HIT) departments, currently already at a feverish pace to push out eHR, eRx and anything else that is going to get them to meaningful use, don’t have that kind of time right now.

2012 is coming fast.

And it is. It solves most of the problems and annoyances desktop admins have been having for years…and creates some new ones.

But it was mentioned to me that maybe it’s just the term virtualization that’s sexy. “Isn’t server virtualization still the hot topic?”

With what I’m seeing, it is and it isn’t.

It is a hot topic because it just makes sense. Saves money. Makes DR easier….

But in healthcare there are other priorities, and server virtualization seems to be taking a backseat. Here’s why:  HIPAA and “Meaningful Use”.

Hospital IT departments are working faster than ever to try to implement CPOE(Computerized Physician Order Entry), HIS (Hospital Information Systems) and EHR (Electronic Health Records) so they can get all the Federal funds  as soon as possible that will help pay for them. The sooner it gets done the better.  No matter how meaningful use is eventually defined, it will mean that the doctors and nurses will need to have access to the system. In most hospitals, that means adding thousands of computers to clinical floors, Operating Rooms, the ED, etc.

And on those computers?… How do I present these new CPOE and patient charting records to the user?

In the old days and in most non-healthcare environments that meant putting xxx,xxx thick clients out in the clinical areas. But with HIPAA, anything that resembles patient information being brought down and left on a local computer that can get stolen is a no-no. So that leaves us with Server Based Computing or Endpoint Virtualization or thin-client computing or however it will be phrased this week.

Even if the IT departments ignore the HIPAA and meaningful use, there is the fact that you are completely changing the physicians’ rounding time by adding a login process and waiting for applications to load (see previous blog post: SSO vs. User Experience) that can take away up to two full hours out of a physician or nurse’s day. Doctors are not accepting this. This means endpoint virtualization has become a priority.

This (endpoint virtualization) has to be done. There is money at stake, there are legal issues at stake and there is the happiness of the doctors (physician satisfaction) at stake. All are at the forefront of the hospital’s mind right now.

“It slows them down!” Thus, the problem is better defined: it is not so much that the software is too expensive, but that doctors can’t afford it.

See this blog post for complete article: http://www.healthcareguy.com/?p=663

That is why Server virtualization may be happening, but it isn’t at the same level of sexiness as the Endpoint Virtualization right now.

Six months to a year ago when speaking with hospitals, we found that only (yes, I’m generalizing) the ‘big’ facilities had virtualized servers and they were all (generalizing again) on VMware ESX x.x.

With the current budget numbers and time-restrictions (see future posts) facing a Hospital IT department, I’m starting to see that when hospitals are choosing to virtualize their servers they are turning more and more to the free versions of Hyper-V, ESXi and XenServer.  Granted most of this is because it was about six months ago that they all became free, but a lot of the good features still need to be paid for.

So, in the absence of cash they are starting to virtualize using the free versions.

With free, the ROI models go even further through the roof!

At least, they are if they have a guy they can free up to get trained on it or who happens to play with some in his basement… which is a big ‘if’. Especially in the smaller community hospitals that may only have 4 IT guys.

Though I can certainly hold my own, I’ll freely admit that I’m no expert in server virtualization but I know a couple things:

1. Free Hyper-V is clunky to use. I personally can’t stand managing VMs in it. But it’s HCL (Hardware Compatibility List) is 10X what VMware’s is and 5X what XenServer’s is (so I hear).
2. Free ESXi is my favorite to use. Mostly because the normal vSphere client manages them and though the vSphere client has a few short comings (like taking a million clicks to get into the datastore), it is much smoother to use and manage guest OS than hyper-V.
3. My experience with XenServer is limited but I’ve run into only one or two hospitals that have told me they are using it. I need to bust it out in my test environment soon.

The most interesting thing to me is the more and more hospitals I am running into saying they are using (or going to use) free Hyper-V. Not the majority, mind you. But the percentage is picking up.

And I never would have guessed that six months ago. Based on my own preference, I don’t understand why.

I plan on asking the ‘why’ from now on. I’ll report back later.

This is one of my favorite lines of all time. If you have kids though, I would posit that french fries have solved some major problems on long road trips. Temporarily, at least.

Sometimes though, we see IT departments missing the real point of an ‘ease of use’ conversation and maybe trying to solve their problem with the equivalent of french fries- they satisfy an immediate problem but it often doesn’t address the long term considerations.

As more hospitals bring their PACS, CPOE, HIS, and EHR systems on-line we are seeing a less-than-enthusiastic response from clinicians that now have to change the whole way they operate.

Although the push to electronic records is a very good thing in the long run, we are finding that a lot of this progress can actually hinder the doctors and nurses who already have limited time to spend the quality time with patients they need to deliver effective care.

Of course, the transition to the electronic hospital means computers… and adding computers to the mix has been, shall we say, problematic.

Instead of grabbing a chart, scribbling some notes and/or dictating into a recorder as they walk to the next patient, a doctor is forced to:

• find an available computer
• log-in to the computer using their domain credentials
• wait for the computer to log in (30 seconds to 3 minutes in some cases)
• launch the CPOE application
• wait (sometimes another 30 seconds or more)
• log in to the CPOE application using that application’s different username & password (if they remember it)
• wait
• find the appropriate patient
• weed through the some windows and select the appropriate drug & dose
• hopefully log-off
• go to next patient and start all over again

In the real world, this adds quite a bit of time, as much as 5-10 minutes per interaction. There have been documented cases of this process adding as much as two hours to a typical physician’s day. That isn’t acceptable to anyone.

The typical response of an organization to this workflow issue is to look for a Single Sign-On.  And this is reasonable. The complaint is that it takes a lot of time to enter a user and password for the domain and then every other application the clinician uses. It’s also next to impossible to get normal human beings to remember 12 sets of credentials that change on a ‘seemingly’ random basis every other month.

A good SSO will indeed make life a bit better. It will reduce frustration, reduce calls to the helpdesk to change passwords and even speed up the process a little bit. If we looked my unscientific list above, you’ll see that we have really on resolved a small portion of the total problem with the SSO.

The problem is SPEED. How long is it taking for the doctor to get to the point where she can actually start entering data? If they have to walk away, what does it take to get back to where they were?

This is why we often tell our clients that SSO is just a small piece of the overall piece of the puzzle that is clinical access to electronic data. Other problems regarding computer access also need to be solved in order to avoid being a hindrance to patient care and improve end-user experience:

1. Log-in times. Some facilities are still dealing with 45 second to as much as 3 minute log-in times. Multiply this by the dozen or so time the user has to log-in and you add quite a bit of time to the user’s day.
2. Roaming. A typical clinician’s work-flow is such that they rarely are able to stay in one place for a length of time and they often get interrupted. When they can get back to a computer, it often isn’t the same one they started at and they might be in a completely different part of the hospital.  An ER nurse I spoke to the other day summed it up nicely: “I want to be right where I left off.”
3. Consistent experience. We’ll see over and over again that a computer in the ICU will have different applications and different look and feel than a computer in MEDSURG. This is extremely frustrating to users that just want everything to be the same. Hunting for applications is no way to spend a day.
4. Kiosk Capability. In places like the ER you have a limited amount of space for computers and what seems like an unlimited amount of users. You need to be able to get users in and out of their own session in rapid succession.
5. Printing. This is one of the consistently bad experiences users have. Where does it print? Which one I need to select? Why did that 50 page report print on a label printer?
6. HIPAA. I often see patient records left open because a clinician had to attend to another patient and didn’t want to walk away and start all over again. “It would just take too long to get back in.” Patient confidentiality is paramount in most hospitals but it is sometimes being neglected in favor of time. This is not a compromise a hospital should have to make.

If you don’t have these problems now, you will as you adopt more and more components and become truly electronic.

Solving them will take more than potatoes (or french fries).  It will take a comprehensive solution that includes one or all types of virtualization (terminal server, vdi, application, pc blades) and can be specifically tailored for hospital use cases.

Some of them are waiting to ‘try it on’, just hoping that they can find the size that fits in a style they can afford.

Your CEO has asked about it.

There are conferences, webcasts, seminars, blogs, websites, email blasts and brunch-and-learn events.

The new styles from VMware, Microsoft, Symantec and Citrix are gaining the attention of talk shows everywhere… and there are plenty of duplicators as companies everywhere try to ride the new wave of virtualization.

Do I want to be more specific? Probably a good idea.

We’ll use the term ‘virtualization’ in two ways: endpoint and server. If we keep with the fashion metaphor, we break it down into lines for men and lines for women.

We’ll say server virtualization is like men’s clothing lines. It’s serious, less risqué and easy to coordinate. You can figure out ahead of time what your savings will be.

Endpoint virtualization isn’t quite as easy to figure out… But the marketing machines are in overdrive. ‘Replace your thick client environment with a Virtual Desktop implementation that is just as robust but easier to manage’. ‘Save money!’ (how? – don’t worry about that) ‘Dynamically provision desktops.’ ‘Live Migration.’ That’s why it’s sexy. Cool marketing terms.

However, the ROI isn’t as obvious as it seems. It takes a lot more work.

The potential is there. And there is so much. If we use the Brian Madden (www.brianmadden.com) definition of Server Based Computing (SBC), we can further break endpoint virtualization into two further types: terminal server based and desktop virtualization (formerly Virtual Desktop Infrastructure). But terminal server based virtualization has been around too long to be trendy now.

So now we really get down to it. It is Desktop Virtualization that is sexy. It’s young. It’s hip. It’s the talk of the IT world. VMware View, Citrix XenDesktop, Microsoft VDI Suite, Red Hat Virtualization Manger for Desktops, SUN Virtual Desktop Infrastructure are all players with VMware and Citrix currently leading the chase.

But does it make real sense? Especially in the context of this blog that focuses on healthcare use cases?

Not yet, it’s not. Now there is a lot of potential, especially with all the competition driving both innovation and costs. This (competition) is the one thing the Terminal Server world has been missing by being dominated (really owned) by Citrix for the past 10-15 years.

But still, right now it has a couple things that make it lose its luster once you get a closer look.

1. It’s just too expensive. It requires a Storage Area Network (not a cheap investment) and the user density is just not where it needs to be to compete with a Terminal Server environment that now has application virtualization solutions like Symantec’s Workspace Virtualization or Microsoft’s App-V to help alleviate the application conflicts that Terminal Server has always been plagued with.
2. All the bugs haven’t been worked out. Getting VDI login times down to an acceptable level for healthcare just takes a lot of work. Especially since if you do it now, you are one of the trailblazers figuring it all out on your own instead of having the benefit of forums and white papers from your predecessors to draw from.
3. Right now it will probably take a bunch of 3^rd party solutions to stand it up. How to manage profiles, storage, logon scripts?
4. The strange thing is, and Brian Madden calls it “Madden’s Paradox”, is that by the time you get done securing a VDI environment, you lose most of the functionality you were looking for when you made the leap to Virtual Desktops in the first place. Terminal Server would have been just fine.

With all this and the fact that Terminal Server now has application virtualization help alleviate the application conflicts that Terminal Server has always been plagued with, and it is difficult to choose VDI over Terminal  Server for a healthcare use case in the short term.

Long Term for VDI? User densities need to go up, management tools need to get better, storage requirements need to decrease and 3^rd party vendor tools like AppSense will need to be integrated into VDI solutions.

And we’ll let someone else work out the bugs in their production environment….