Holographic deconvolution microscopy for high-resolution particle tracking
Abstract.
Rayleigh-Sommerfeld back-propagation can be used to reconstruct
the three-dimensional light field responsible for the recorded
intensity in an in-line hologram.
Deconvolving the volumetric reconstruction with an optimal
kernel derived from the Rayleigh-Sommerfeld propagator itself
emphasizes the objects responsible for the scattering pattern
while suppressing both the propagating light and also such
artifacts as the twin image.
Bright features in the deconvolved volume may be identified
with such objects as colloidal spheres and nanorods.
Tracking their thermally-driven Brownian motion through
multiple holographic video images provides estimates
of the tracking resolution, which approaches 1 in all three dimensions.
In-line holographic video microscopy offers time-resolved information regarding the three-dimensional distribution of matter that scatters a beam of laser light (1); (2). The scattered light interferes with the unscattered portion of the beam in the focal plane of an otherwise conventional microscope. A video camera then records the intensity of the magnified interference pattern with time resolution set by the exposure time and frame rate of the camera. Each holographic snapshot in the resulting video stream contains, in principle, comprehensive information regarding the three-dimensional distribution of matter.
Variants of this technique differ principally in how the holographic snapshots are analyzed. Fitting recorded holograms to results based on the exact theory of light scattering (3); (4) has been used to track colloidal spheres with nanometer resolution (3); (4); (5), to characterize individual spheres' sizes, refractive indexes (3); (4); (6); (7), and porosities (8), and to detect molecular-scale coatings on micrometer-scale colloidal substrates (4); (6). This wealth of information comes at a price. The nonlinear least-squares fits are computationally intensive, and must be targeted specifically for the kinds of samples being analyzed.
In-line holograms also may be analyzed more generally and
more efficiently by numerically reconstructing
the three-dimensional light field (1); (2); (9)
with scalar diffraction integrals (10).
Such volumetric reconstructions
do not commonly account for
the physical properties of the scattering medium.
Rather, the medium is treated as an isotropic and homogeneous
dielectric.
Bright features in the reconstruction then
are identified with discrete objects in the sample
(1); (2); (5), particularly those
that are smaller than the wavelength of light (5).
Dynamical measurements based on tracking such features
through holographic video sequences have
demonstrated 10 resolution for three-dimensional location
of colloidal spheres (5) and nanorods (11),
albeit with large systematic axial offsets in some cases (5).
Features in holographic reconstructions
can be emphasized by deconvolving the reconstructed
light distribution with the volumetric point-spread function for
the reconstruction process (12); (13).
Here, we demonstrate and quantitatively assess the tracking
resolution of holographic deconvolution microscopy using the
Rayleigh-Sommerfeld diffraction integral both to reconstruct
the volumetric intensity distribution and also to deconvolve it.
For micrometer-scale colloidal spheres, the results suggest
typical in-plane resolution approaching 1 and
axial resolution of 10
.
Three-dimensional tracking of metal-oxide nanorods yields
comparably good results for locating the center of mass, and
orientation resolution in three dimensions.
Our in-line holographic microscope has been described previously
(2).
It consists of a commercial inverted microscope stand
(Nikon TE2000U) outfitted with a high-numerical-aperture
oil-immersion objective (Nikon Plan-Apo, , NA 1.4).
The conventional illuminator is replaced with the collimated
beam from a fiber-coupled diode laser (Coherent Cube)
operating at a vacuum wavelength of 445
.
The linearly polarized beam has a total power of
15
spread uniformly over
.
Holographic images are captured by a low-noise gray-scale video camera
(NEC TI 324A-II) and are recorded as an uncompressed digital
video stream at 30 frames per second with
a total system magnification of 135
pixel.
The camera's 0.5
exposure time is fast enough
to avoid measurable effects of motion blurring
(4); (14); (15); (16).

The incident plane wave,
![]() |
(1) |
is assumed to propagate along with
an real-valued
amplitude
that may depend on position
in the transverse plane,
and uniform polarization
.
The scattered wave,
![]() |
(2) |
propagates in three dimensions with complex amplitude
and spatially varying polarization
.
Their superposition in the focal plane
yields the interference pattern
![]() |
![]() |
(3) | ||
![]() |
(4) |
Deliberately placing the scatterer well above the focal plane ensures
both that polarization rotations are small,
, and also that the scattered wave is substantially
less intense than the illumination.
Normalizing by the illumination's intensity distribution
then yields
![]() |
(5) |
where the reduced scattered field is
.
Dropping
from the definition of
simplifies the analysis that follows
at the cost of ignoring interference due to multiple scatterers.
It therefore limits the complexity of the samples to which this
formalism may be applied.
In practice, the background image,
,
can be obtained either by
moving the sample out of the field of view, or by taking a
running median filter of a time-evolving sample.
If the complex scattered field were completely specified in the
focal plane, it could be reconstructed at height
as the convolution
![]() |
(6) |
of the scattered amplitude in the focal plane with the Rayleigh-Sommerfeld propagator (10)
![]() |
(7) |
where .
The sign convention for
accounts for the object's position
upstream of the focal plane.
Equation (6) may be rewritten with the
Fourier convolution theorem as
![]() |
(8) |
where
![]() |
(9) |
is the in-plane Fourier transform of and
where
![]() |
(10) |
is the Fourier transform of
(10); (17); (18).
To use this formalism to reconstruct the scattered field,
we note that
the Fourier transform of
is
![]() |
(11) |
From this,
![]() |
(12) |
may be recognized as the superposition of the scattered field
at height above the focal plane and a spurious field due to
the object's mirror image in the focal plane, which is known
as the twin image.
The twin image's influence on the reconstructed field
may be reduced by moving the sample away from the focal plane.
In the additional approximation that the illumination is uniform, the reconstructed field is
![]() |
(13) |
The associated intensity,
, is
an estimate for the scattered light's intensity at height
above the focal plane.
This reconstruction differs from a numerically refocused image (1)
because it comprises only the scattered field and not its superposition
with the incident field.
The reconstruction also is not simply a representation of the object responsible for the scattered field, or even of the scattered field itself. The projection operation in Eq. (6) is formulated for light propagating through a homogeneous medium, and so does not account for the optical properties of any objects in the medium. The reconstructed field thus includes artifacts upstream of each scattering center in the sample. These artifacts and the twin image of the reconstructed field are both clearly visible in the volumetric reconstructions of colloidal spheres and a copper-oxide nanorod (11); (19) in Figs. 1(a) and (b), respectively. These artifacts, together with out-of-focus images tend to obscure features of interest in the reconstructed volume. Neither the nature nor the number of the scattering objects is immediately evident from the reconstructions in Fig. 1(a) or Fig. 1(b). These sources of blurring therefore hinder efforts to identify and track colloidal particles.
Most of these distractions can be removed
by deconvolving the reconstructed data with the
point spread function used for the reconstruction.
Previous reports (12); (13) have used point-spread functions for particular objects
in an effort to optimize detection resolution.
Our deconvolution scheme instead is
motivated by the observation that the Rayleigh-Sommerfeld
formalism transforms an ideal point-like
focal caustic (20)
at height above the focal plane
into the three-dimensional intensity
distribution
.
The caustic's twin image, similarly, is projected into
.
Deconvolving with
therefore should
reduce the full three-dimensional scattering pattern reconstructed
from the hologram
to the set of focal caustics created by the sample.
Twin images also will be reduced to focal caustics,
but on the other side of the effective focal plane, which
will not be reconstructed, and so will not be seen.
Using the Rayleigh-Sommerfeld point-spread function in this way
also eliminates the need for calibration measurements
(12); (13).
Deconvolving with
is
most easily implemented with the Fourier convolution theorem:
![]() |
(14) |
where is the three-dimensional
Fourier transform of
,
is the Fourier transform of
,
and
is a small factor chosen for numerical stability.
The deconvolved intensity,
is
obtained as the inverse Fourier transform of
.
Figures 1(c) and (d) (Media 1) were obtained
in this way from the data in Figs. 1(a) and
(b), respectively, and are rendered with the same color and
transparency tables.
Blurring from artifacts and out-of-focus images is
strongly suppressed in the deconvolved volumes,
leaving distinct and well-resolved
bright features that we associate with the focal caustics
created by the samples.
For example, the sample in Fig. 1(c) now
can be recognized to consist of 9 colloidal spheres arranged
in a body-centered cubic lattice, while that in
Fig. 1(d) is quite clearly an inclined rod.
These results were obtained with
,
and comparably good contrast and resolution
were obtained for
.
The three-dimensional positions of the focal caustics can be recovered by analyzing moments of the three-dimensional intensity distribution (21). For objects with dimensions comparable to the wavelength of light, furthermore, focal caustics should lie substantially on the objects themselves. This suggests that bright features in the deconvolved reconstruction also track the shape, size, position and orientation of such objects (5); (11). A similar analysis is used to track objects in a volume deconvolved with a targeted point-spread function (12). This approach requires the point-spread function to measured for the target object, or else to be computed based on a priori knowledge.







Tracking Brownian objects as they diffuse enables
us to measure the accuracy and resolution attainable with
holographic deconvolution microscopy
(5); (11); (14); (15); (21).
The apparent mean-square displacement of a colloidal sphere
freely diffusing along ,
![]() |
(15) |
increases linearly with time according to the familiar
Einstein-Smoluchowski result, but is offset by the measurement
error for centroid location along that axis
(21).
In this, we assume that the camera's shutter is fast enough
that motion blurring may be ignored (14); (15); (16).
The data in Fig. 2(a) were obtained
for a
diameter
colloidal polystyrene sphere (Bangs Laboratories, lot number 912)
diffusing in water at room temperature,
.
The three single-axis
diffusion coefficients,
,
(
) are
consistent with each other and also are consistent with the
Stokes-Einstein result
given
and using the radius
obtained from Lorenz-Mie characterization (3).
Consistency among the three diffusion coefficients confirms that
the sphere was far enough from the walls of the sample container
that hydrodynamic coupling may be ignored.
The fit values for the centroid tracking errors,
,
and
, are a factor of two
smaller than the corresponding errors obtained with Rayleigh-Sommerfeld
reconstruction without deconvolution (5),
and are a factor of two larger than those obtained with
Lorenz-Mie fits to the same holograms (5).
Although both the Rayleigh-Sommerfeld reconstruction and the
subsequent deconvolution are numerically intensive, they can be
less costly than Lorenz-Mie fitting, particularly when
multiple particles are in the field of view.
Our implementation
in the IDL programming language requires 30 seconds to locate
the 9 spheres in Fig. 1(c) through
fits to the Lorenz-Mie theory (5) on a multi-core
processor running at 3 , but only 20 seconds for
the present approach.
Unlike fitting to Lorenz-Mie theory, holographic deconvolution
does not yield high-resolution measurements of sphere radius
or refractive index.
The estimated axial position, furthermore, includes
a systematic offset that depends on
the size and composition of the object (5).
This complicates measurements of three-dimensional separations.
Any such concerns are offset, however, by the generality
of holographic deconvolution microscopy, which requires no
foreknowledge of an object's shape or composition.
For instance, the nanorod's deconvolved reconstruction
in Fig. 1(d) (Media 1)
features a roughly cylindrical volume of length
, whose length remains unchanged
through a sequence of 5,000 snapshots
as the nanorod rotates in three-dimensions.
It is tempting to identify the center of this cylinder
with the nanorod's position, and the orientation of its
axis with the nanorod's orientation.
Figure 2(b) shows the mean-squared
displacement
![]() |
(16) |
of the orientational unit vector
obtained in this way.
The nanorod's apparent orientation and center-of-mass position
are extracted from the deconvolved reconstruction using a
peak-tracking skeletonization algorithm originally
developed (11) to analyze
volumetric reconstructions of the type
presented in Fig. 1(b).
The solid curve in Fig. 2(b)
is a fit to
![]() |
(17) |
for the rod's rotational diffusion coefficient
and
the measurement error
in the nanorod's three-dimensional orientation.
The latter figure is consistent with an error of
in
the rod's orientation, which improves on previous results by
more than a factor of two.
The nanorod's translational diffusion is difficult to analyze in the laboratory frame because its viscous drag coefficient depends on its orientation, which is always changing (22). In the co-rotating frame, however, its mean-squared displacement should evolve linearly in time,
![]() |
![]() |
(18) | ||
![]() |
![]() |
(19) |
when projected along and transverse to the measured orientations,
respectively.
The results plotted in
Fig. 2(c) are consistent with this
interpretation, and yield
and
.
The three diffusion coefficients for a freely diffusing rod,
![]() |
![]() |
(20) | ||
![]() |
![]() |
(21) | ||
![]() |
![]() |
(22) |
depend on the rod's length and diameter
,
and also on a geometric factor
(23).
From these, we obtain
, which
is consistent with the optical measurement,
, and
.
The associated tracking error for the rod's centroid
is 10
in the plane and 50
along the optical
axis, averaged over orientations.
The success of holographic deconvolution microscopy for tracking colloidal spheres and nanorods with nanometer resolution does not necessarily extend to more general objects, particularly those with dimensions larger than the wavelength of light. Equation (14) ignores interference due to light emanating from multiple sources. Provided the scattering centers are separated by multiple wavelengths of light, however, interference effects manifest themselves principally in the dimmer regions of the reconstructed intensity distribution, well away from interesting intensity maxima. When these conditions are not met, however, Eq. (14) can introduce artifacts of its own. The data in Fig. 1(d), for example, show the reconstruction of a lithographically defined colloidal particle in the form of the letter “X” (24). Crafted from polymethylmethacrylate, this dielectric particle has features roughly one micrometer across, and thus scatters the incident plane wave illumination into a pattern that depends strongly on the particle's orientation. The deconvolved volumetric reconstruction emphasizes caustics in that scattering pattern, rather than the shape itself, and so its true shape is not readily apparent. The same will be true of more highly structured samples such as biological cells. The neglect of the scattered intensity in the derivation of Eq. (5) also may have contributed to a loss of definition.
When applied with appropriate care, deconvolving the Rayleigh-Sommerfeld reconstruction of an in-line hologram with the Rayleigh-Sommerfeld point-spread function yields precise and accurate information for three-dimensional particle tracking. This technique excels for highly symmetric colloids such as spheres whose caustics precisely track the particle's position, albeit with an axial offset (5). Objects such as nanorods that have dimensions smaller that the wavelength of light similarly can be tracked with nanometer-scale resolution. Because the projected caustic coincides with the physical position of the object in such cases, the measurement's accuracy can approach its precision. For all such systems, deconvolution with the Rayleigh-Sommerfeld point-spread function greatly simplifies three-dimensional tracking by de-emphasizing extraneous features of the three-dimensional scattering pattern in favor of its singularities. The measurements on model samples that we have presented confirm that deconvolution can substantially enhance tracking resolution compared with reconstruction alone (5).
We are grateful to Tom Mason for providing a sample of LithoParticles. This work was supported in part by the QORS Program of DARPA and in part by the National Science Foundation through grant DMR-0922680. The software used in this study is available for download under the terms of the GNU Public License at http://physics.nyu.edu/grierlab/software.html
References
-
(1)
J. Sheng, E. Malkiel, and J. Katz, “Digital holographic microscope for measuring three-dimensional particle distributions and motions,” Appl. Opt. 45, 3893–3901 (2006).
-
(2)
S.-H. Lee and D. G. Grier, “Holographic microscopy of holographically trapped three-dimensional structures,” Opt. Express 15, 1505–1512 (2007).
-
(3)
S.-H. Lee, Y. Roichman, G.-R. Yi, S.-H. Kim, S.-M. Yang, A. van Blaaderen, P. van Oostrum, and D. G. Grier, “Characterizing and tracking single colloidal particles with video holographic microscopy,” Opt. Express 15, 18,275–18,282 (2007).
-
(4)
F. C. Cheong, B. Sun, R. Dreyfus, J. Amato-Grill, K. Xiao, L. Dixon, and D. G. Grier, “Flow visualization and flow cytometry with holographic video microscopy,” Opt. Express 17, 13,071–13,079 (2009).
-
(5)
F. C. Cheong, B. J. Krishnatreya, and D. G. Grier, “Strategies for three-dimensional particle tracking with holographic video microscopy,” Opt. Express 18, 13,563–13,573 (2010).
-
(6)
F. C. Cheong, K. Xiao, and D. G. Grier, “Characterization of individual milk fat globules with holographic video microscopy,” J. Dairy Sci. 92, 95–99 (2009).
-
(7)
K. Xiao and D. G. Grier, “Multidimensional optical fractionation with holographic verification,” Phys. Rev. Lett. 104, 028302 (2010).
-
(8)
F. C. Cheong, K. Xiao, D. J. Pine, and D. G. Grier, “Holographic characterization of individual colloidal spheres' porosities,” Soft Matter in press (2011). DOI: 10.1039/c1sm05577a
-
(9)
B. Rappaz, P. Marquet, E. Cuche, Y. Emery, C. Depeursinge, and P. J. Magistretti, “Measurement of the integral refractive index and dynamic cell morphometry of living cells with digital holographic microscopy,” Opt. Express 13, 9361–9373 (2005).
-
(10)
J. W. Goodman, Introduction to Fourier Optics, 3rd ed. (McGraw-Hill, New York, 2005).
-
(11)
F. C. Cheong and D. G. Grier, “Rotational and translational diffusion of copper oxide nanorods measured with holographic video microscopy,” Opt. Express 18, 6555–6562 (2010).
-
(12)
T. Latychevskaia, F. Gehri, and H.-W. Fink, “Depth-resolved holographic reconstructions by three-dimensional deconvolution,” Opt. Express 21, 22,527–22,544 (2010).
-
(13)
Y. Cotte, M. F. Toy, N. Pavillon, and C. Depeursinge, “Microscopy image resolution improvement by deconvolution of complex fields,” Opt. Express 18, 19,462–19,478 (2010).
-
(14)
T. Savin and P. S. Doyle, “Role of finite exposure time on measuring an elastic modulus using microrheology,” Phys. Rev. E 71, 041106 (2005).
-
(15)
T. Savin and P. S. Doyle, “Static and dynamic errors in particle tracking microrheology,” Biophys. J. 88, 623–638 (2005).
-
(16)
L. Dixon, F. C. Cheong, and D. G. Grier, “Holographic particle-streak velocimetry,” Opt. Express 19, 4393–4398 (2011).
-
(17)
G. C. Sherman, “Application of the convolution theorem to Rayleigh's integral formulas,” J. Opt. Soc. Am. 57, 546–547 (1967).
-
(18)
U. Schnars and W. P. O. Jüptner, “Digital recording and reconstruction of holograms,” Meas. Sci. Tech. 13, R85–R101 (2002).
-
(19)
T. Yu, C. H. Sow, A. Gantimahapatruni, F. C. Cheong, Y. W. Zhu, K. C. Chin, X. J. Xu, C. T. Lim, Z. X. Shen, J. T. L. Thong, and A. T. S. Wee, “Patterning and fusion of CuO nanorods with a focused laser beam,” Nanotechnology 16, 1238–1244 (2005).
-
(20)
J. F. Nye, Natural Focusing and Fine Structure of Light (Institute of Physics Publishing, Bristol, UK, 1999).
-
(21)
J. C. Crocker and D. G. Grier, “Methods of digital video microscopy for colloidal studies,” J. Colloid Interface Sci. 179, 298–310 (1996).
-
(22)
Y. Han, A. M. Alsayed, M. Nobili, J. Zhang, T. C. Lubensky, and A. G. Yodh, “Brownian motion of an ellipsoid,” Science 314, 626–630 (2009).
-
(23)
M. Doi and S. F. Edwards, The Theory of Polymer Dynamics (Clarendon Press, Oxford, 1986).
-
(24)
C. J. Hernandez and T. G. Mason, “Colloidal alphabet soup: Monodisperse dispersions of shape-designed LithoParticles,” J. Phys. Chem. 111, 4477–4480 (2007).