Skip to main content

Image Fusion

This bio-computational development integrates two distinct technologies: imaging mass spectrometry, which is rich on chemical information but coarse in spatial resolution, and microscopy, which has low chemical specificity but high spatial information. The resulting images combine the advantages of both technologies, enabling prediction of the distribution of an ionized molecule concurrently at high spatial resolution – by ten times or more relative to the actual IMS spatial resolution (Figure 1).  We have developed a proof of concept version of an image fusion algorithm [1] that can capture integrative connections between IMS and microscopy, implemented as a multivariate regression problem that models variables in one technology using variables from the other technology and enrich biological signals and attenuate instrumental artifacts, revealing insights that cannot be extracted from either microscopy or IMS separately. We are at the beginning of this exciting new image correlation process that facilitates a new multi-modality paradigm for tissue exploration whereby mining relationships between different imaging techniques yields a novel imaging modality that combines and surpasses what can be gleaned from the source technologies alone.


Figure 1. Sharpening of ion spatial distributions through image fusion of IMS and microscopy data. This example in mouse brain fuses a measured ion image for m/z 762.5 at 100 μm spatial resolution (a) with a measured H&E-stained microscopy image at 10 μm resolution (b). The resulting image (c) predicts the ion distribution of m/z 762.5 at 10 μm spatial resolution with an overall reconstruction score of 82%. A neighboring section imaged at 10 μm spatial resolution (d) compares a physical measurement to the fusion approach for m/z 762.5. Modified from: Nat Methods 12 (4):366-72.
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),

ga(‘create’, ‘UA-61343597-1’, ‘auto’);
ga(‘send’, ‘pageview’);

A complete package to enable image fusion capabilities is available for download. Free licensing is available for academic and non-profit groups.


NIH/NIGMS 5R01 GM058008-16


1.  Van de Plas R, Yang J, Spraggins J, Caprioli RM (2015) Image fusion of mass spectrometry and microscopy: a multimodality paradigm for molecular tissue mapping. Nat Methods 12: 366-372. PMCID: Not listed. Return to text.