Features

A View of Mixed-Reality Tech’s Future in Orthopedics

With mixed-reality headsets, orthopedic surgeons can improve procedural precision, streamline workflow, and democratize care.

March 19, 2024

By: H. John Cooper

Columbia University Irving Medical Center

In total knee arthroplasty, once an orthopedic surgeon has performed resections to the patient’s bone, turning back is not a viable option.

While resections can be modified by resecting additional bone, they cannot be undone. That sense of finality is always with me in the OR, and it has often made me wish I could peer at the patient’s individual anatomy prior to picking up the saw to confirm the accuracy of my planned resections.

Now, with the help of mixed-reality (MR) technology, I can experience that level of vision and precision. Worn by surgeons during procedures, MR headsets work by projecting personalized anatomical data onto a patient’s body, mixing virtual information with the real-world environment to provide a data-driven 3D view of the surgical site.¹

For the first time in surgical settings, MR is being utilized differently from augmented reality (AR) devices. While AR devices typically overlay a preoperative image onto the anatomy as a reference, the new generation of MR devices are digital measurement tools that scan the landscape, analyze the information, and provide real-time precision accuracy. The result is clinically meaningful data about real-world anatomy.

A new approach in orthopedics involves the STELLAR Knee by Polaris,^AR a customizable digital-assistant technology that employs optical tracking algorithms and spatial computing software to improve functional outcomes by allowing the surgeon to intra-operatively map individualized patient data and create a precise surgical plan. MR technology projects the data holographically from the doctor’s portable headset onto the patient’s anatomy, readjusting as the surgeon moves to preserve accuracy. The surgeon can then interact with the data to refine the surgical plan and ultimately use the MR technology to perform accurate resections.

These visual data overlays offer the kind of precision that previously was only associated with large, complex robotic systems. Yet, compared with surgical robots, MR headsets are much smaller and more affordable, lending themselves to efficient ergonomics and workflows. And, in the case of STELLAR Knee, the technology is widely applicable across orthopedics because it can be used with any total knee implant system.

Exploring the Evidence

While orthopedic surgeons have embraced both robots and fiber-optic arthroscopy cameras in recent decades, these innovations have not streamlined their ability to digest and apply anatomical information during procedures. Short of memorizing the data, surgeons must periodically glance away from their patients to read notes, screens, and light boards on display throughout the OR.

MR is changing that by literally putting the information right at our fingertips.

There’s no question the novelty of MR devices will require a change in our thinking—a “rewiring” that prepares us to absorb multiple data streams at once. It’s a bit like playing a video game, but even for those of us with little experience in that arena, MR is intuitive, and with a short learning curve, its benefits are well worth our investment.

The options for this kind of surgical guidance are growing, as the FDA has cleared numerous first-generation MR and AR devices for medical use over the past couple of years² and early adopters have begun to demonstrate what these technologies can accomplish. In 2021, Mayo Clinic performed its first knee replacement using an AR headset to guide the procedure,³ and neurosurgeons at Johns Hopkins successfully fused one patient’s vertebrae and removed another’s cancerous spinal tumor with guidance from similar technology.⁴

Thought-provoking news also came from a 2022 literature review that described good clinical outcomes across 15 studies with cohorts totaling 241 patients whose spine surgeries had involved AR, VR, or MR.⁵ All patients experienced improvement in the clinical symptoms with which they had originally presented, with complication rates of 6.1% or lower (mainly encompassing suboptimal pedicle screw placement). There were no complications that led to clinical sequelae.

In my own experience pilot testing an MR headset, I’ve come to trust the technology can precisely measure, plan, and guide surgeries at a level equivalent to that of a surgical robot.

That makes the emergence of this technology a milestone moment for orthopedic surgery, one with the potential to shift the standard of care for experts across our field.

Putting AR Into Practice

Like some of my fellow orthopedic surgeons at Columbia University, I’ve investigated the capabilities of AR by becoming an early adopter of STELLAR Knee, which received FDA 510(k) clearance in November 2023.⁶

I ventured into this digital environment by performing simulated-use procedures in cadavers. Across this experience, the MR device informed my clinical decisions by empowering me to make quick assessments and real-time micro-adjustments to my surgical plans.
I particularly appreciated several aspects of this technology:

It provides surgeons with in-the-moment access to sub-millimetric intraoperative data about hard and soft tissue, offering accurate and precise resection guidance.
It digitizes anatomical information into holographic displays that open a real-time data exchange between the surgeon and the technology, informing clinical decisions.
It has intuitive control mechanisms, giving surgeons full interactive capabilities so they can make decisive assessments and micro-adjustments to the surgical plan.
Its software offers a voice-control feature that allows surgeons to call up specific data for instantaneous computing and overlay without losing their focus on the patient.
It includes a “Fin,” a novel instrument used to ensure adherence with the plan a surgeon has defined, so all bone resections are performed precisely at the intended angles and depths.

While surgeons using conventional equipment and images can typically achieve resections that fall within varying degrees of the desired angle and resection depth, MR technology can help narrow that variation to under 2.0 mm and 2.0 degrees of planned depth and alignment, consistent with the precision and accuracy offered by current robotic technology.

MR also promises some broader advantages for our field. A key benefit is it brings world-class precision capabilities into the OR in a small package, without the need for the large capital investment or physical footprint that come with traditional surgical guidance systems. Part of MR’s cost-efficiency lies in its minimal physical setup and a future option for multiple members of surgical teams to wear headsets so they can track the progress of a surgery.

Due to these features, MR can help democratize healthcare by bringing precision procedures into centers that do not have a budget for large robotic systems—for instance, ambulatory surgery centers dedicated to cost-efficiency and small hospitals in rural America or other underserved communities or countries.

This technology can also help equalize care by boosting opportunities for medical education, and with the industry focused on access to information and excellence, livestreaming and other capabilities are in the immediate future. Although minimally invasive procedures can be challenging to observe because they’re conducted on such a small scale, livestreams that share the surgeon’s unobstructed view through an MR headset promise a new level of clarity for students and peers, whether they are across the operating suite or across the world.

As an instructor, it would be hugely beneficial to know my surgical residents and fellows are seeing exactly what I see, which would help me to direct their focus at crucial moments. Ultimately, I expect this exchange to raise the level of orthopedic surgical training.

Creating a New Standard

The time is right for technology experts to forge this progress, and we can expect to see a surge of MR innovation across orthopedic surgery in 2024.^7,8 For doctors who embrace technology, the introduction of MR products is exciting, as is the ongoing enhancement of these devices.

For instance, because MR runs on complex software that is consistently being tweaked and upgraded, the software device developers at Polaris^AR have worked to ensure digital displays are simple and meaningful, and dialing in precise resection angles and depths can be accomplished efficiently in the OR.

While these advancements may be off the radar of patients who are focused strictly on their procedural outcomes and recovery, more savvy surgical candidates may soon start requesting the use of MR headsets because they want to take advantage of the field’s latest innovations.

As a highly accessible emerging technology, there’s little to stand in the way of our uptake of MR. As with all enabling technologies, there will be a potential lag in surgeon adoption due to concern over the learning curve. However, I’m confident with proper surgical training and the momentum sparked by early adopters and educators, MR may soon sweep into the majority of our orthopedic ORs.

That evolution could be practice-changing, especially for facilities that, until now, have faced cost- or space-related barriers to obtaining surgical guidance systems. At those centers and others, MR has the potential to streamline workflow while supporting the highest levels of precision and accuracy in orthopedic procedures. Ultimately, the technology will improve patient outcomes by providing surgeons with the insights and vision they need to eliminate unforeseeable variances during surgery.

Not long ago, this kind of futuristic technology would have seemed unattainable. Today, that future is here, and I’m excited to see where it leads our field.

References

[Disclosures: Dr. Cooper is a consultant for Polaris^AR]

H. John Cooper, M.D., is an orthopedic surgeon and an associate professor of orthopedic surgery at Columbia University Irving Medical Center. He specializes in adult reconstructive surgery and has considerable experience in direct anterior hip arthroplasty, robotic knee arthroplasty, and complex primary and revision joint replacement. A well-respected clinician, educator, and researcher, he has published over 150 peer-reviewed articles and book chapters on clinical outcomes and complications of hip and knee replacements and has been an invited and awarded speaker on these topics at national and international orthopedic meetings.