Making the case for hyperspectral imaging in surgical practices

Hyperspectral cameras truly “make the invisible visible” by illuminating an object and then capturing its reflected light in many narrow spectral bands.

The technology has already found use in applications ranging from precision agriculture (think weed control and pest surveillance) to industrial machine vision for corrosion detection or quality inspection and environmental monitoring (e.g., spotting oil spills or plastic waste).

Hyperspectral cameras are now poised to revolutionize the medical realm. Imagine surgeons being able to make decisions based on real-time information about tissues’ chemical composition at a molecular level, seeing the oxygenation of blood flowing through arteries and vessels, or distinguishing in vivo between healthy and anomalous tissue, such as tumor cells.

This glimpse into the future is much closer than you might expect. But while hyperspectral sensors and cameras have evolved tremendously during the past few years, several limitations continue to prevent the technology from straightforwardly being used in surgical practice. For one, devices are typically quite bulky and incompatible with an operating room’s already-crammed sterile field. And integrating them into hospitals’ stringent clinical workflows is no easy feat, either.

A recent research project took these challenges to heart. It explored whether hyperspectral imaging technology can support the in vivo detection of low-grade gliomas (a diverse group of brain tumors), which is an arduous task, even for well-trained and highly experienced surgeons.

Exploring in vivo detection of low-grade gliomas

Low-grade gliomas arise directly within the brain. They often develop in young, otherwise healthy patients. While typically considered benign in origin, studies have shown that these cancer cells can expand at a rate of 4 to 5 mm annually and come with the risk of malignant transformation. Their early surgical resection has become a much-favored treatment option, despite in vivo detection of low-grade gliomas and retrieving their exact demarcations being notoriously difficult, even with the aid of surgical microscopes.

This is partly because low-grade gliomas do not lend themselves to detection through commonly used fluorescence methods. Instead, a magnetic resonance imaging (MRI) or computed tomography (CT) scan needs to be performed before the tumor’s resection; this merely allows surgeons to identify a general region of interest (instead of the tumor’s exact boundaries).

Giving surgeons the proper tools to detect intrinsic brain tumors in vivo would be a gigantic step forward. And hyperspectral imaging technology shows great potential to do this. It is a breakthrough achieved by mounting imec’s snapscan VNIR 150 hyperspectral camera onto a standard surgical microscope. This proved a compact setup generating hyperspectral data has the potential to help detect low-grade gliomas in vivo. As a next step, these data can be fed into a deep-learning neural network, providing valuable insights on healthy vs. anomalous tissue.

Small form factor meets high accuracy

Imec researchers focused on acquiring relevant data within a (sterile) intraoperative environment with minimal adaptations to both the system and data pre-processing methods. To do so, they had to rethink the camera’s existing spatial and spectral calibration methods and interface the device directly with the surgical microscope’s optics and light source.

Lighting was one of the key parameters explored. To avoid excessive noise, the researchers adapted the spectral range of the snapscan camera to match that of the microscope’s internal light source. They also looked at the system’s integration time modalities to find the right balance between saturation and noise—while optimizing the system’s imaging parameters.

Video-rate hyperspectral imaging for real-time tumor classification

This project allowed imec researchers to specify which hyperspectral bands are critical when interfacing with high-end surgical microscopes for an in vivo classification of low-grade gliomas. The underlying algorithms can detect the presence of low-grade gliomas on unseen data with 80% accuracy, which is a major step forward.

Admittedly, the setup’s intraoperative use is premature. So far, the approach has been successfully tested using a clinical dataset of six patients at Belgium’s Leuven University Hospital. These promising results lay the groundwork for moving to video-rate hyperspectral imaging, which will allow researchers to explore the real-time detection of low-grade gliomas.

This study was conducted in collaboration with Carl Zeiss Meditec AG and Belgium’s Leuven University Hospital, and the results were presented at SPIE BiOS 2023 in the following papers:

• Integrating hyperspectral imaging in an existing intra-operative environment for detection of intrinsic brain tumors, R. Vandebriel (imec)
• Intra-operative brain tumor detection with deep learning-optimized hyperspectral imaging, X. Zhang (Carl Zeiss Meditec AG)