When Cutting-Edge Microscopes Meet Deep Learning Algorithms
Lei Tian’s Computational Imaging Systems group is calculating new ways to see molecules, cells, and life in action
By Kat J. McAlpine
Research images provided by Lei Tian | Photos by Kelly Peña
Lei Tian – a faculty member at BU’s Photonics Center and a BU College of Engineering (ENG) assistant professor of electrical and computer engineering – has spent his career harnessing physics, electronics, and optics to create new imaging systems capable of applications ranging from the enormous (illuminating the flow of massive oil spills within oceanwater) to the minute (detecting precise cellular activity within biological tissues).
He credits holography – which manipulates laser light to display 3D images – as his entry point into the field of computational imaging at a time when it was still a nascent idea, while he was earning his PhD at Massachusetts Institute of Technology (MIT). There, he was focused on digital holography, employing sensors to capture real 3D information and then reconstructing objects computationally by modeling the way light diffracts through space.
Using that approach to develop holography systems for submersible vehicles, Tian recalls that the 2010 Deepwater Horizon oil spill added a new layer of interest and urgency to his work. “We wanted to design better 3D cameras for large-scale applications” like detecting where the oil had spread throughout ocean columns and currents, he says.
Working on the macroscale, however, led to a new inspiration. “We also realized these techniques would be good for looking at very, very small things on the microscopic scale.”
After completing his PhD, he pursued postdoctoral work at University of California, Berkeley, where he developed computational microscopy techniques for imaging cells, leveraging tiny distortions of light that happen as it travels through cells.
Since then, Tian’s passion for inventing novel computational microscopy tools has been ablaze. Upon arriving at BU in 2016 to start his own lab, “all my projects, both large and small, focused on microscopy.” Now, leading BU’s Computational Imaging Systems Lab (CISL), Tian is making incredibly powerful and extremely small new types of microscopes possible – with the goal of visualizing how cells and the brain’s neurons communicate and behave with higher resolution and clarity.
“Through computational imaging, we are augmenting imaging hardware with advanced algorithms to increase what microscopes are capable of capturing,” Tian says. “Harnessing deep learning to understand the way light travels through space, the way light travels through tissue… that’s the special sauce of what we’re doing.”
And it’s taking a village of research partners. “It’s a nexus of research that’s bringing together many different people – to solve big problems like this, understand and construct the hardware, and computationally extract and analyze gigantic amounts of information, we need people from all these different backgrounds.”
Tian received a Dean’s Catalyst Award from ENG, which promotes such cross-disciplinary research, that helped him launch a hub of collaborations. CISL has stretched its tendrils into a network of collaborations across BU’s Charles River and Medical campuses. Tian is affiliated with BU’s Department of Biomedical Engineering, the Neurophotonics Center, the Center for Information and Systems Engineering (CISE), the Hariri Institute for Computing and Computational Science and Engineering, and the Nanotechnology Innovation Center.
The lab’s goals have evolved to be “about 80 percent focused on neuroscience applications,” Tian says, crediting the support of David A. Boas, director of BU’s Neurophotonics Center, and research partnerships with BU Neurophotonics faculty members Jerry L. Chen and Ian G. Davison as essential to his team gaining momentum in brain science applications, which Tian had no prior experience in.
These days, CISL’s work is getting noticed near and far.
In January 2023, Tian’s Computational Imaging Systems Lab was awarded more than $1.3 million over two and a half years in funding from the Chan Zuckerberg Initiative (CZI), part of a grant program that backs advancements in monitoring biological processes in motion and across time and space. With the CZI support, the team is designing miniaturized microscopes that can image whole mouse brains at single-cell resolution, seeking to visualize molecular and cellular activity like never before.
Their petite, lightweight microscope designs are empowered by special miniaturized lenses to enable powerful imaging despite their compact size. Aboard each ‘scope, the team typically employs an array of several microlenses to peer into samples from multiple angles, combining those readouts computationally to build 3D images of the brain.
They call their devices mesoscopes because they can achieve both single-cell imaging of neurons and a large field of view across brain tissue; the prefix “meso-” originates from the Greek word for middle, and used here it describes a mesoscope’s ability to image both individual cells and whole tissues.
“We’re building these as wearable devices for fundamental research in mice, with the goal that we can mount one of our mesoscopes on a mouse’s head, allowing us to image neural activity across its entire brain as the mouse moves around and performs various behaviors,” Tian says.
Within his lab, teammates are working on several approaches to creating their miniaturized wearable microscopes. PhD student Joseph Greene is experimenting with different architectural designs of non-conventional optics and also investigating which algorithms best enhance their imaging capabilities. In May 2023, Tian and Greene were authors of a paper published in Neurophotonics describing a new low-cost, head-mounted miniscope that can capture images from deeper within mouse brain tissue using miniature diffractive optics and an algorithm that compensates for the effect of light scattering through tissue.
Meanwhile, Yujia Xue, a former Tian lab member (who earned his PhD and is now developing new cameras at Apple, Inc.), and Qianwan Yang, a current PhD student, have been dedicated to the algorithm and deep-learning aspect of microlens arrays, seeking to computationally extract the most 3D information from each sample as accurately as possible. In August 2023, Xue, Yang and Tian published a study in Optica describing a computational miniature mesoscope design that utilizes deep learning image reconstruction to achieve high resolution, wide-field microscopy.
In October 2023, Tian’s team reported in arXiv an ultrafast imaging technique using single-shot, wide-field 3D imaging augmented with a sensor that detects changes in neural activity, or “events”. It gives its readout in response to these events, rather than frame by frame like more conventional ‘scopes. They further enhanced the information of the event-based readout by designing an algorithm to accurately interpret imaging data both spatially and in the context of time. In the lab, they demonstrated how this technique can capture clear and contextually rich images of freely moving C. elegans worms. Tian and his co-authors say the method could have wide-ranging applications across biological and biomedical research.
Beyond fundamental research, Tian’s lab is also closely collaborating with researchers at BU’s Chobanian & Avedisian School of Medicine (MED), including faculty members Ann McKee, B. Russell Huber, and Jonathan Cherry, leaders at BU’s dedicated research centers focused on Chronic traumatic encephalopathy (CTE), Alzheimer’s, and other neurodegenerative diseases. Together, they’re working to advance computational imaging systems and deep learning algorithms to image and interpret human brain data.
“We’re building multi-scale datasets and looking to cover the entire human brain so that we can gain more insights into disease development and progression in people,” Tian says. The collaborators envision that computational imaging could help pinpoint the exact location of molecules like tau proteins, the spread of which is a key catalyst in both Alzheimer’s and CTE.
Undergraduate researcher Sunni Lin, who was mentored by former Tian lab member Shiyi Cheng, is using deep learning to extract new information from human brain images, essentially looking to map out structural data using digital – rather than traditional chemical – staining. (Cheng defended his thesis this year and is now at Apple, prototyping camera features and video algorithms.) Digital staining could simplify sample preparation, making neural imaging more broadly accessible to researchers with diverse training. It could also create more bandwidth within tissue samples for prioritizing other types of labels and markers attached to molecules of interest.
“The focus on deep learning algorithms continues to grow within my group,” Tian says. “We’re not just using off-the-shelf datasets or general-purpose neural networks for training our [microscopy] algorithms. We’re really trying to integrate as much physical knowledge as we can through experimental data – and that’s the core philosophy of computational imaging, to synergistically combine algorithms with imaging methods.”
Within the last year, Tian’s team has also received funding support from the National Institute of Neurological Disease and Stroke at the National Institutes of Health, National Institute of Biomedical Imaging and Bioengineering, and Samsung.