AI opens a new era in scientific imaging

CONSTANTIN KAPPEL

Throughout the history of microscopy, innovation has generally manifested in steady improvement rather than seismic change. While it’s less common to see technology developments fundamentally alter how life scientists work, this is exactly the kind of shift underway in scientific imaging. Advances now coming to the fore not only portend breakthrough discoveries, they will forever change the way we think about and conduct scientific research. The catalyst for this transformation is artificial intelligence (AI)—in particular, the ability to use deep learning to process huge volumes of scientific imagery comprehensively and with super-human accuracy in a fraction of the time required by teams of human scientists (see Fig. 1).

FIGURE 1. In just the past two years, machine learning has made significant progress in application for life sciences. This confusion matrix, a tool used in AI development, demonstrates this progress by reporting prediction accuracy for an algorithm trained to identify 19 cell organelles in biological specimens. With one exception, all objects are predicted here at or near 100% accuracy.

Coupled with smart imaging technologies that can connect and autonomously collaborate, AI is ushering in a new era that will remove some of research’s biggest bottlenecks and exponentially accelerate progress in areas like developmental biology, cancer research, neuroscience, and immunology.

The next frontier

Just a few years ago, even the most sophisticated microscope was an isolated tool able to produce remarkable images, but in itself unable to provide answers. Only the scientist using it could add that dimension. Now, however, as in countless other sectors, AI has enabled such equipment to facilitate solutions previously impossible—with less and less human intervention.

Such functionality starts with the ability of advanced software to improve imaging quality. For example, Leica’s recently launched Thunder imager series uses a method called Computational Clearing to eliminate out-of-focus blur when using camera-based fluorescence microscopes. Such approaches enable quick, high-quality imaging of thick 3D samples.

Deep learning techniques enable intelligent systems to self-improve as they go. A human still needs to provide “training” data for the neural network to learn correct mapping, but once trained, an algorithm can operate independently (see Fig. 2). In the foreseeable future, human supervision may be made easier as machines learn how to cluster information automatically or navigate complex environments on their own.

FIGURE 2. Predictions generated by a trained algorithm, such as the one shown in Figure 1, will ideally form clearly defined clusters, as evident in this plot, which was verified using the dimensionality reduction technique t-Distributed Stochastic Neighbor Embedding (TSNE).

As the technology matures to the point where tasks can be automated, it will increasingly liberate researchers who currently find themselves mired in tedious, time-consuming tasks. The world’s first Personal AUtomated Lab Assistant, Leica’s PAULA, is being developed as a device able to analyze imagery captured by continuous monitoring of a specimen and automate a response when there are changes.

Opening doors in science

Importantly, AI generates insights by drawing learning from large bodies of data, so instead of being anecdotal, analysis becomes objective.

In addition to quickly analyzing large data pools from a single source (for example, an individual microscope or a series of imaging systems from a single laboratory), Internet of Things (IoT) connectivity means that AI will be able to access information from networked microscopes and other laboratory devices that integrate publicly available data. Microscopes will be connected to the cloud, and via the cloud to other devices or to open-source databases such as academic publications. Cross-referencing data from the host device with data from millions of online sources will unlock a type of collaboration never before possible in life sciences, at a velocity heretofore unseen.

Historically, one might need to look at 1000 different proteins to investigate a particular change in a cancerous cell. This could mean 1000 different PhD theses by 1000 different students, each viewing thousands of images. AI could complete all these analyses in less than a day, meanwhile identifying interaction patterns inaccessible to students looking at proteins in isolation.

Prerequisites

Before we can achieve automated connectivity, however, we must overcome challenges around digital infrastructure. A key hurdle is the array of file formats currently used by different makes of microscopes, which threatens to trip up machine learning algorithms.

For AI and machine learning technologies to bring their full potential to the life sciences, we need investment in key areas, including skills development. We’re now at a defining period where traditional scientific disciplines are sitting alongside very different disciplines: computer science, mathematics, statistical modeling, and even robotics. Some traditionalists have been reluctant to embrace this collision of approaches, but we must embrace it if we are to take science to the next level.

We’re already seeing some remarkable examples of such investment, including Cell Atlas, headed by Associate Professor Emma Lundberg of KTH Royal Institute of Technology (Stockholm, Sweden). Encompassing a decade’s worth of work, this project has specifically stained virtually every protein of vertebrate cells and used microscopy to image where they go.

Now, we have a colossal bank of images that we can analyze with deep learning techniques to determine patterns in subcellular spatiotemporal distribution of proteins (see image at top of page).

Leica Microsystems recently sponsored a competition through Kaggle to create an algorithm to sort these images into 28 classes, each showing different organelles of the cell. Nearly 2200 teams of data scientists worldwide took on the challenge of predicting where a protein has gone in the cell with only an image as a clue. Such a task is particularly tricky, as some proteins go to multiple places at the same time and some of the patterns are very rare. To succeed, the scientists had to search for independent data to complement the competition data, use models pre-trained on millions of photographs, and fine-tune them to recognize microscopy images or to exploit techniques borrowed from facial recognition. Endeavors like this will help us to understand the role of proteins in health and disease on the level of individual cells.

We need more AI innovation of the sort Cell Atlas has pioneered. As the Kaggle competition attests, such projects inspire others to believe in the potential of AI and invest in it. Those who fear that this spells the end of traditional disciplines need to realize such progress doesn’t replace human endeavor and intelligence, but rather supplements it with powerful tools that will enable humans to achieve more and do so far more efficiently.

Because a single research project can involve hundreds of thousands of microscopy images or more—too many for a human to evaluate—microscopy has traditionally been a rather-phenomenological discipline. Now, with the age of AI upon us, we look forward to a new era of innovation.

Dr. Constantin Kappel is Product Manager, Confocal Microscopy at Leica Microsystems, Wetzlar, Germany; e-mail: [email protected]; www.leica-microsystems.com.