If you believe the tech blogs 2015 is the year of virtual reality. Industry experts believe this will be due to the potential commercial release of the poster boy of this new revolution, the Oculus Rift. This is a headset that gives a stereoscopic 3D view with excellent depth, scale and parallax which it does by presenting different images to each eye.
My personal 2015 started with a very real first world problem. I had managed to get hold of one of these headsets, as a developer, and waited anxiously over the Christmas holiday for its arrival. Sadly an email alerted me to the fact that our friendly delivery man had dropped it in next door as we were out. On receipt of the email I ran next door to ring on the bell to pick up the desired gadget. I could see a delivery box marked fragile and my heart soared; alas they were out on that day, the following day and as it transpired the subsequent five. Who were these people who go on holiday during the holiday season?
After finally getting my hands on it I fired it up, opened a (real) bottle of red wine and took a virtual walk around Tuscany. So as ophthalmologists and doctors why should we care about something that clearly belongs to the video gaming community, as well as middle-aged men who drink in front of their computer alone while pretending to be in Italy? The answer is because for the first time using the very same tools that game programmers use we can create high fidelity virtual reality simulations of clinical signs. A new industry has been born called serious gaming and there is an arms rush of independent as well as the usual larger companies clamouring to see who can use this new technology to create the most innovative simulators in the work place.
The Oculus Rift is not the only choice. There are a number of alternative models including some that load your phone into a headset with adapted optics. Zeiss, which has been supplying ophthalmologists with equipment since time immemorial, has brought out the Zeiss vitreoretinal (VR). This is a cheaper option as the user can insert their phone into the headset. Adapted optics in the headset, supposedly using the phone’s screen, are purported to give the wearer access to that immersible world. A cheaper option still is Google cardboard. As the name suggests it is a pair of goggles made of cardboard that you insert your phone into. The high tech velcro holds the android phone in place and voila a virtual world is created (Figure 1).
Figure 1: Google cardboard.
Simulators are not new to ophthalmology and there are probably very few ophthalmologists that have not had a go on one of the virtual phacoemulsification machines, sutured an artificial rectus muscle with simulated ocular surgery or practiced a capsulorhexis on a grape. These techniques and methods are great as they teach proprioception. There is a growing body of evidence that surgical simulation works [1]. Apart from practicing on a grape, one of the great drawbacks to simulation has been cost. As the VR and gaming communities expand their knowledge, skill and hardware the cost to consumers, and by proxy ophthalmic trainees, will hopefully drop.
The case for virtual reality training
The 2008 report ‘Safer Medical Practice’ by Donaldson outlined the reasons why we should be pushing simulation in training [2].
The expectation from the public. Many of my friends and family are horrified at the old adage “see one, do one, teach one.” Patients expect quite rightly that when they are treated by someone they are not practicing their clinical / surgical skills for the very first time.
There have been changes in working patterns. The environment in which we work has changed dramatically in the previous two decades. We are more sub-specialised and there are expanding roles for associated healthcare professionals. This more complex scenario means that there is a role for simulation in the development and assessment of these skills.
Technological development and opportunities. This is where my interest lies. I should point out at this juncture that whilst the main focus of this article is tech heavy and concentrating on high fidelity, we should also not forget the low fidelity options that have been around for years as part of a structured educational framework and have an important role to play. Many grapes were sacrificed in my personal mission to conquer the capsulorhexis.
Reduced training time. The general surgeons I bump into in the doctors’ mess claim the European Working Time Directive has compromised their exposure to on the job training. As ophthalmologists our training may be cut from seven to five years. The use of simulators means that we can revise and prepare effectively when not at work seeing our fifth marginal keratitis in a row.
This approach is mirrored in other industries such as aviation, nuclear and various military industries. I know very little about the nuclear industry but I assume it would be beneficial to practice a melt down on a simulation rather than a Homer Simpson style effort in real life. The military are also leading the way. According to research firm Visiogain global, spending on military simulation and virtual training alone has reached a staggering $8.4 billion [3].
Over a decade ago during the storm of Medical Training Application Service (MTAS) I flirted with the idea of becoming a pilot and sat in full size simulators and played around with PC flight simulator games. I remember at the time thinking why we didn’t have these PC games in medicine. As doctors we also need controlled practice and exposure to rare conditions, critical incidents and crisis management. I envisaged a world where we can practice diagnosing rare conditions; life-threatening ones that we may only see once in a career, ones we do not want to miss and incorrectly manage.
As a medic I have done many undergraduate and postgraduate clinical exams. I tell my mother I passed most of them through hard work and experience. Sadly, there is a large element of luck for the vast majority of us (sorry medical educators). The quarterbacks at medical school who just seemed to be more knowledgeable than everyone else always flew through and those that struggled always struggled. The rest of us in the middle worked hard and prayed for a condition that we knew, an examiner who wasn’t an arse and some clinical signs that were reliable. Even now and again though if all those things went against them, a quarterback was occasionally caught in the pocket and sacked.
Multiple choice and freestyle written papers never have questions of variable levels of difficulty between candidates. Potentially using these new environments all candidates can get the same clinical case. The actual movement through space can be tracked. Did they accurately chart out the visual field defect? Readouts of where their hands were could calculate a level of skill. The days of candidates whining how unfair the whole exam was will never be over; a more robust standardised exam would, however, decrease their numbers.
“There are probably very few ophthalmologists that have not had a go on one of the virtual phacoemulsification machines, sutured an artificial rectus muscle with simulated ocular surgery or practiced a capsulorhexis on a grape.”
I set out to make a clinical simulator two years ago and found out how hard it is to program a biological system. Flight simulators work on Newtonian physics. The flaps are put down which changes the air flow over the wing and the computer via an algorithm creates a change in the airplane’s flight path. A biological system such as ocular motility as a series of pulleys and levers doesn’t always obey the rules given and neuronal innervation is as different between individuals as the size of their noses! There were many panicked phone calls to the orthoptists who helped me with this project. As an ophthalmologist with an interest in glaucoma this project exposed quite a few things I did not appreciate about ocular motility.
Figure 2: The NeuroOphthal app logo.
Inventing a simulator
NeuroOphthal was produced in partnership between my company suprAnuclear and the very clever people at Centre of Excellence of Mobile Applications Services (CEMAS) operating out of the University of South Wales. The funding came through a European business funded development programme. The goal of the project was to as accurately portray as closely as possible the clinical signs found in orthoptics and neuro-ophthalmology. The game was built in a 3D games engine on an iPad; there is, however, the possibility to export it to the Oculus Rift.
Figure 3: Our virtual reality man showing off his partially accommodative
esotropia in one of the nine positions of gaze.
The NeuroOphthal clinical simulator uses a virtual patient to accurately portray over 100 ocular motility, lid and pupil conditions. This clinical reference / educational app accurately recreates both ocular motility and neurology signs on an interactive interface, enabling the clinician through the app to make accurate diagnoses.
Our goal was to accurately as possible replicate the following groups of conditions:
- pupil responses
- esotropia and exotropia
- cranial nerves pathologies
- neurogenic conditions
- myogenic conditions
- supranuclear conditions
- internuclear conditions
- nystagmus
- alphabet patterns
- heterophorias
Clinical skills we hoped the doctor would learn through the application included:
- assessment of saccades, smooth pursuit, convergence and the oculocephalic reflex
- cover test with accommodative elements at distance and near
- distinguishing between incomitant and concomitant strabismus
- assessment of nystagmus amplitude, frequency and direction
- fatigability in myasthenia
- bielschowsky head tilt test
- ptosis assessment
Whilst we were programming the application we tried to make the process as close to reality as possible with as few buttons as possible. Therefore, most of the clinical exam is done by touching the screen as opposed to a button that starts an animation. For example, touching the screen once generates a saccade under normal conditions at a speed of 400 degrees a second. Dragging your finger across the screen gives a slower smooth pursuit. There are lots of clinical signs thrown in such as lid retraction, fatigability and inferior oblique overactions that are not accessed by buttons but on the user undertaking the correct clinical exam to illicit them.
The process started out by myself writing the code as pseudo code, which is a halfway house between writing a functioning program and everyday normal English language. It was then translated into the proper thing by the programmer Priyanka Angotra, which was a real mean feat given that she was presented with this mountain of orthoptic jargon explained in my amateur coding. Should programming ever become obsolete, through her hard work on this project, a role in the orthoptic clinic awaits.
The graphics where done by Dan whose face can be seen peering at you with most orthoptic conditions known to afflict mankind. One of the unstated goals was to make the app successful enough so that Dan could gain rock star status should he ever find himself wandering around an orthoptic conference.
The future
NeuroOphthal has now gone live and can be accessed on an iPad at the appstore. However, as alluded to before, this is not the end of the story. We are pressing on to play our part in this new virtual world and are currently planning to update and improve the current iPad application by converting NeuroOphthal in its current format to a completely immersed virtual reality application using the Oculus Rift.
By the end of 2015 we will have a fully immersible virtual reality orthoptic simulator. The course of new technology, however, rarely runs smoothly. Last year was supposed to be the year of wearable technology. Google glass has just been pulled leaving in the lurch lots of development companies who had invested in the hardware and software. I have seen very few people investing in smart watches. So the 2014 tech blogs heralding the new era of wearables have been wrong before and may be wrong again.
The first dawn of virtual reality occurred in the 90s when Sega inadvertently discovered the barfogenic zone. The problem with Sega’s then prototype, the Sega VR project, was that the screens in front of the eye did not refresh in time as the user’s head moved. It therefore created an awful motion sickness, which even the most ardent gamer could not overcome. It is an occasional problem with Oculus Rift but it appears to be dependent on the program / game being run.
A new phenomenon which we have certainly come across has been coined by Paul Sizio in Boston. He describes the ‘uncanny valley’. As humans when we are confronted with interactive graphical representations of humans it gives a great sense of unease. Certainly, Dan’s face, which we have rendered in a 3D model never gets a “he looks odd” in real life. However, his graphical representation on the iPad app with a see saw nystagmus has been described as freaky.
The reason why this new virtual world may fail is the same reason why the grape and capsulorhexis is so successful. Making that perfect circle on the grape relies so much on proprioception that the interaction is more than just visual. Whilst we can walk round a virtual Tuscany, interacting with the environment using the other senses such as touch is not at this stage possible. This interaction is definitely still in the starter phase. Whilst the NeuroOphthal iPad version has a touch screen, the gloves with pressure pads that will enable us to grab the occluder or test obiciularis function are still not in play.
My predictions for 2015
- I think we will start to see the crossover into the consumer sector of virtual reality games. Teenagers and middle-aged ophthalmologists may start to run around on treadmills shooting things whilst wearing silly headsets in their living rooms
- I don’t think the college will have initiated virtual reality postgraduate exams.
- More ST1s will practice on phaco simulators
- Hopefully the serious games industry will grow and more doctors will practice their skills using these games
- Finally, the humble grape will still be the best way to train that propriocpetive part of the brain that takes charge during the capsulorhexis
References
1. Sikder S1, Tuwairqi K, Al-Kahtani E, et al. Surgical simulators in cataract surgery training.Br J Ophthalmol 2014;98(2):154-8.
2. Donaldson L. The 2008 Annual report of the Chief Medical Officer, Safer Medical Practice.
3. The Military Simulation and Virtual Training Market 2010-2020. Visiongain; August 2010.
https://www.visiongain.com/Report/514/
The-Military-Simulation-and-Virtual
-Training-Market-2010-2020
Last accessed May 2015.
Declaration of competing interests: The author is the founder and director of suprAnuclear.
COMMENTS ARE WELCOME
