Scientists from the National Institutes of Health (NIH; Bethesda, MD) and collaborators from the University of Chicago (Illinois) and Zhejiang University (China) have developed new image processing techniques for microscopes that can reduce post-processing time up to several thousand-fold.
In a paper that describes the work, Hari Shroff, Ph.D., chief of laboratory on High Resolution Optical Imaging at the National Institute of Biomedical Imaging and Bioengineering (NIBIB), demonstrates new techniques that can significantly reduce the time needed to process the highly complex images that are created by the most cutting-edge microscopes. Such microscopes are often used to capture blood and brain cells moving through fish, visualize the neural development of worm embryos, and pinpoint individual organelles within entire organs.
As microscopes continue to get better, creating higher-resolution images faster, researchers are finding they have more data than time to process it. While the videos themselves can be captured in minutes, the images could be terabytes in size and require weeks or, in some cases, months of processing time to be useable.
One reason it takes so long is that the videos often capture tiny objects that are blurred by the microscope. Such blurring can be reduced by a procedure called deconvolution, but this procedure requires a lot of computing power and time.
A second issue is that many of the microscopes in use today take multiple views of the same organism or cell. Those images need to be positioned correctly and then combined to make 3D images and video. Creating high-resolution images from the raw data takes a significant amount of computer processing. And so, while the microscopes have developed to provide researchers with increasingly complex, high-resolution images, computing power has limited what techniques are practical for researchers to use—since they know that the majority of the data they collect will go unused.
The first thing Shroff’s lab and his collaborators attempted to do was modify the deconvolution algorithm that is used by many researchers, so it would run faster. This approach was originally proposed for other areas of medical imaging such as computed tomography (CT); however, this is said to be the first time it was successfully adapted for use with fluorescence microscopy. Fluorescence microscopy uses dyes to improve contrast in the specimen, allowing researchers to focus on specific parts of a sample and see how different elements interact with each other.
Second, they reduced the time needed to position and stitch together multiple views of a sample. A key part of this advance relied on a process called parallelization. It is an approach that is sometimes used in supercomputing where instead of processing each individual function one after another, the job is broken up into smaller tasks that can be analyzed concurrently. It is like asking thousands of people to each solve one math problem simultaneously instead of asking one person to solve thousands of problems.
Finally, the researchers showed that they could further reduce the time it takes to process the data by using a neural network, a kind of artificial intelligence (AI). AI is increasingly being used to assist imaging processing and diagnoses. In this case, Shroff and his team trained the neural network to produce cleaner and higher-resolution images much more quickly than would be possible otherwise.
“Acquiring modern imaging data is a bit like drinking from a firehose,” said Shroff. “These methods help us obtain valuable biological information faster, which is essential, given the massive amount of data that can be produced by these microscopes.”
These advances expand the use of existing technology, including allowing for imaging of thick samples that produce huge amounts of image data when examined with fluorescence microscopes. The advances are also essential for the use of a growing number of ‘computational microscopes’ in which the post-processing of unintelligible raw data is an essential step in producing the final high-resolution image.
Shroff and his collaborators hope that they will help researchers with approaches they would not have thought to try, due to how labor-intensive it would otherwise be to create meaningful images.
Full details of the work appear in the journal Nature Biotechnology.