Using Google Glass in Surgery

In June of this year Dr. Rafael Grossmann of the Eastern Maine Medical Center performed surgery wearing Google Glass. Then in August, Dr. Christopher Keading of Ohio State University’s Wexner Medical Center wore Google Glass while performing surgery on a 47-year-old patient’s knee ligament.

As far as we know, dozens of other surgeons have experimented with Google Glass in surgery since it first became available to developers in 2012.

Dr. Grossmann wrote about his experience in PC Magazine; “I wanted to show that this device and its platform are certainly intuitive tools that have great potential in healthcare and specifically for surgery, could allow better intra-operative consultations, surgical mentoring and potential remote medical education, in a very simple way.”

Grossmann set up a Google + Hangout site where invitees could connect and watch a live-stream of what he was seeing through his Google Glass headset.

“I was able to show not just the patient’s abdomen, but also the endoscopic view, in a very clever, simple and inexpensive way. I think that there should be ways to directly stream the endoscopic view thru Google Glass” continues Glassman.

What are the implications of a face mounted and wearable computer that will allow surgeons to stream a real-time point-of-view video of the surgery to anyone logged onto the Google + Hangout site?

Since Google Glass is online, it can call up X-rays, MRI images and reference any other reports, materials or even hold a video consult, real time, with colleagues. And it is all controlled with natural language audio commands

Live streaming, consulting, surfing the internet—all while looking into the patient and performing surgery – using a heads up screen on the eye glasses themselves.

Is this cool or what?

Google Glass

Glass (styled “GLΛSS”) is a wearable computer with an optical head-mounted display (OHMD). Developed in Google’s secret Lab X, the device displays information in a smartphone-like hands-free format that can communicate with the Internet via natural language voice commands.

Surgeons, or frankly anyone who wears Google Glass, can allow other people to see everything they see. From any computer (assuming they are logged into the right web site) other people can follow the surgeon throughout the surgery and—indeed, until the Google Glass is removed, around all day.

First Google Glass Surgery

Google Glass is lighter than the average pair of sunglasses and in the future, new designs will allow integration of the display into people’s normal eyewear. In early 2013 Google delivered an Explorer Edition to testers and Google I/O developers in the U.S. for $1, 500. A consumer version is on the way and should be available in 2014 for “significantly less” than the Explorer Edition.

Google Glass began testing in April 2012. In May 2012, Google demonstrated the 720p HD first-person video recording capabilities by giving the Glass to skydivers, abseilers (folks who rappel), and mountain bikers who then live-streamed their point of view to a Google+ Hangout.

On June 21, 2013, Spanish doctor Pedro Guillen, chief of trauma service of Clínica CEMTRO of Madrid, became the first physician in the world to broadcast a surgery through the use of Google Glass. Thanks to the Spanish company Droiders, rights holder of this system in Spain, a chondrocyte implantation in the knee of a 49-year-old patient, could be streamed worldwide over the Internet, while allowing another physician, Dr. Homero Rivas (director of innovative surgery, School of Medicine, Stanford University, California), an expert in telemedicine, to participate in the surgery.

One Writer’s First Person Experience

Gary Shteyngart, writing in the August 5 edition of The New Yorker, described his experience with Google Glass as follows:

“A pink rectangle above his field of vision, which looks like a twenty-five-inch television screen floating some eight feet away from him, is replaced by another message: ‘SVO Hav Su flight 150 225pm delayed.’ The man has been Googling the N.S.A. leaker Edward Snowden on his computer, and now his glasses, which are synched to his Google Plus account, are informing him of a delay on the next Aeroflot (Su) flight to Havana out of SVO (Moscow’s Sheremetyevo airport). Another flick of the index finger and a different screen clicks into place. Now it would appear that someone named Chris Brown is defending himself on Twitter and that a water bed for cows has been developed. The man has subscribed to all the news sources currently available for his spectacles: the New York Times, CNN, and Elle (hence news of the Forever 21 world traveler denim shorts, available at the flick of a finger). The man feels a tingle at the back of his ear, and a voice tells him his friend Christine Lee is ready to do a video call, also called a ‘hangout.’ The image of Christine at her desk beams above the man’s right eye. He can see her and she, in turn, can see everything he sees through his glasses, the quiet green streetscape of East Eighty-Eighth Street streaming on her computer screen. He’s going to go to the Guggenheim later and promises her that she will be able to watch the new James Turrell exhibition through his eyes.”

Specifications: “OK Glass”

As of early 2013, Google is working on models that can be used with prescription lenses, which will be available to Explorers before the end of 2013.

The Explorer Edition receives data through Wi-Fi, or it can tether via Bluetooth to an Android device or iPhone, and use its 3G or 4G data; the Glass also has a GPS chip. Users issue voice commands by first saying “ok glass, ” followed by the command, or they can scroll through the options using a finger along the side of the device.

On April 15, 2013, Google released the Mirror API, allowing developers to start making apps for Glass. The terms of service, state that developers may not put ads in their apps or charge fees. Many developers and companies have built applications for Glass, including news apps, facial recognition, photo manipulation, and sharing to social networks, such as Facebook and Twitter.

On April 16, 2013, Google announced that production was complete for the initial Glass Explorer Edition units and the corporation would begin shipping immediately. On the same day, Google also released a web-based setup page for Glass as well as the MyGlass companion app. Developers were also given first access to the Mirror API for Glass.

As of August 27, 2013, 10, 000 people were selected as part of a contest to test the early version of the device.

Google Glass has the ability to take photos and record 720p HD video. While video is recording, the screen stays on.

A touchpad is located on the side of Google Glass, allowing users to control the device by swiping through a timeline-like interface displayed on the screen. Sliding backward shows current events, such as weather, and sliding forward shows past events, such as phone calls, photos, circle updates, etc.

Google Glass applications (Glassware) are free applications built by third-party developers. Glass also uses many existing Google applications, such as Google Now, Google Maps, Google+, and Gmail.

Medical Apps

One application in development is the MedRef for Glass app which is designed for hospital employees and uses facial-recognition technology to pull patient information. Also, we just saw an announcement from Palomar Health which is entering into a partnership with Qualcomm Life (a subsidiary of Qualcomm)) to launch Glassomics.

Glassomics is supposed to be an incubator and innovation lab aimed at exploring the applications of wearable technology in health care. By “wearable technology” Glassomics is referring to, of course, Google Glass but also smart watches and fitness bands.

Using Google Glass as a launching point, several developers, such as Augmedix, are working on a new generation of augmented reality Google Glass apps that promise to turn the device into a full-fledged next generation medical tool.

“Augmented reality” sounds a bit gamey or gamer—but it is effectively a cloud-based data engine that projects over (or overlays on top of) a real-time image. For example, the app would overlay a patient’s medical record over the image of the surgery—or perhaps other data or images over the actual view of the surgery. In other words, “augment” a user’s view of reality.

Non-Medical Apps

On May 16, 2013, Google announced the release of seven new apps, including reminders from Evernote, fashion news from Elle, and news alerts from CNN. Also noted was a “Glass Boutique, ” a store that will allow synchronization to Glass of Glassware and APKs.

Version XE8 made a debut for Google Glass on August 12, 2013. It brings an integrated video player with playback controls, the ability to post an update to Path, and lets users save notes to Evernote. Several other minute improvements include volume controls, improved voice recognition, and several new Google Now cards.

Other than the touchpad, Google Glass can be controlled using “voice actions.” To activate Glass, wearers tilt their heads 30° upward (which can be altered for preference) or tap the touchpad, and say “OK Glass.” Once Glass is activated, wearers can say an action, such as “take a picture, ” “record a video, ” “hangout with [person/Google+ circle], ” “Google ‘What year was Wikipedia founded?’, ” “Give me directions to the Eiffel Tower”, and “Send a message to John” (many of these commands can be seen in a product video released in February 2013).

For search results that are read back to the user, the voice response is relayed using bone conduction through a transducer that sits beside the ear, thereby rendering the sound almost inaudible to other people.

Google is also considering partnerships with sunglass retailers such as Ray-Ban or Warby Parker, and may also open retail stores to allow customers to try on the device.

If you have any experience with Google Glass, send us your story, to robin@ryortho.com.