High-speed gigapixel microscope images, records in high-res 3D

What started as a proof-of-principle device created by graduate students at Duke University (Durham, NC) has become a sought-after gigapixel 3D microscope.

The students’ original imaging device comprised 24 smartphone cameras into a single platform. The images and videos captured with it were stitched together, after which the team discovered they could also see the heights of objects. It ultimately became a single camera that takes gigapixel images. With this prototype system, researchers in Duke’s Pratt School of Engineering developed a high-speed multi-camera array microscope (MCAM) capable of imaging and video recording in 3D (see video).

The MCAM consists of dozens of miniature microscopes packed tightly together within an array. It can image across an extended area simultaneously without sacrificing resolution. The researchers, including Roarke Horstmeyer, an assistant professor of biomedical engineering at Duke, say the 3D imaging capabilities of the new microscope are the result of using partially overlapped fields of view between adjacent imagers within the array.

“Essentially, neighboring cameras view overlapped regions from different perspectives, giving us access to 3D height information in a manner analogous to how humans perceive depth with two eyes,” Horstmeyer says. “While our brain processes the visual information collected by our eyes into 3D depth perception, we use newly developed computational algorithms involving neural networks to reconstruct quantitative 3D information for the MCAM.”

Overcoming obstacles

Their new algorithms overcome challenges the team discovered in relation to data processing. Just a few minutes of video from the microscope produced more than a terabyte of data in some cases. The algorithms are more efficient at handling larger video datasets. Specifically, machine learning is combined with physics to merge the video streams from all of the system’s cameras—this allows the team to recover 3D behavioral data across space and time.

MCAM also hurdles limitations presented by conventional imaging systems. “As microscopists, we are often frustrated by the limited amount of information that a standard microscope can capture. We can either get high resolution over a small area or low resolution over a large area,” Horstmeyer says. “We are often limited to capturing 2D information at high frame rates and therefore can mainly acquire videos primarily of flat specimens.”

Using multiple microscopes in parallel circumvents these tradeoffs and limitations, he explains, “enabling us to perform 3D imaging at high resolution over very wide areas, and at high speeds.”

MCAM at work

The Duke researchers are now enhancing the new microscope, building on work by colleagues at the school’s Naumann Lab and others in the Department of Neurobiology. The latest MCAM advancement features 54 lenses that tout higher speeds and resolution than the original prototype. And newly developed software allows it to take 3D measurements and offer unprecedented detail at smaller scales. This also results in smoother, clearer videos.

“Our goal is to apply our system to solve new problems in a wide range of fields,” Horstmeyer says, adding that the team is collaborating with different biologists and behavioral scientists to understand their needs and potentially tailor the MCAM system accordingly and “unlock new scientific discoveries.”

The new technology is proving helpful for “high-content” imaging experiments, he says, in which many unique specimens need to be observed or measured. The Duke team is also actively working to incorporate fluorescence capabilities into the system.

As the MCAM overcomes the tradeoff between field of view and resolution, it’s unlocking what biomedical researchers can study, particularly at the systems biology level, where changes across time and space need to be observed at high resolution. For example, Horstmeyer says, the new microscope can simultaneously image the microscopic physiological features and dynamics across multiple model organisms. It can also capture their macroscopic interactions occurring over large areas during their unconstrained behavior.

“The MCAM can advance our understanding of how genetics correlates with behavior, and even social interaction, which has relevance in understanding various neurological disorders,” he says. “Also, given the high throughput of our system, we can accelerate drug discovery by monitoring multiple experiments in parallel.”

The team is now working to commercialize the MCAM via their startup company, Ramona Optics.