next up previous
Next: Results Up: Flow visualization and flow Previous: Introduction


Holographic video microscopy

Our in-line holographic video microscope (1,3), shown schematically in Fig. 1, is based on an inverted microscope (Zeiss Axiovert 100 STV) outfitted with a $ 100\times$ numerical aperture 1.4 oil immersion objective (Zeiss S Plan Apo). The conventional incandescent illumination is replaced with the collimated coherent beam from a solid-state laser (Coherent Verdi 5W) operating at a vacuum wavelength of $ \lambda = 532~\unit{nm}$ . Individual colloidal spheres scatter a small proportion of the incident beam, and the scattered light interferes with the unscattered portion in the focal plane of the microscope's objective lens. The microscope magnifies the interference pattern and projects it onto the face of a low-noise gray-scale video camera (NEC TI 324 IIA), with a total system magnification of 101 nm/pixel. The video stream is recorded as uncompressed digital video with a digital video recorder (Pioneer H520S) for subsequent analysis. The illuminating beam's fluence is on the order of $ 1~\unit{nW/\ensuremath{\unit{\mu m}}\xspace ^2}$ , comparable to that of conventional microscope illumination.

The measured intensity at point $ \vec{r}$ in the focal plane,

$\displaystyle I(\vec{r}) = \left\vert \vec{E}_0(z - z_p) + \vec{E}_S(\vec{r}-\vec{r}_p) \right\vert ^2,$ (1)

is the superposition of the incident plane wave, $ \vec{E}_0(z)$ , propagating along $ \vec{z}$ and the scattered wave, $ \vec{E}_S(\vec{r}-\vec{r}_p)$ , that propagates from the particle at $ \vec{r}_p(t)$ to the point of observation, $ \vec{r}$ . This scattered field is described by Lorenz-Mie theory (6) and depends not only on the particle's position, but also on its radius, $ a_p$ , and its refractive index, $ n_p$ relative to the refractive index, $ n_m$ , of the surrounding medium. Consequently, measured images such as the example in Fig. 1 can be fit to Eq. (1) with $ \vec{r}_p(t)$ , $ a_p$ and $ n_p$ as adjustable parameters (1). The computed uncertainties in the fit parameters are found to accurately assess each such measurement's precision (4,1,7,5). This procedure routinely yields the three-dimensional position and radius of a micrometer-scale dielectric sphere to within a nanometer and its refractive index to within one part in a thousand (1).

Figure 1: In-line holographic video microscope. A collimated laser beam illuminates the sample. Light scattered by the sample interferes with the unscattered portion of the beam in the focal plane of the objective lens. The interference pattern is magnified, recorded and then fit to predictions of Lorenz-Mie theory to obtain measurements of the particle's position, its size, and its refractive index.

GPU acceleration

Fitting image data to the results of Lorenz-Mie theory is computationally intensive, and requires initial estimates for the adjustable parameters. Previous implementations were not suitable for high-speed automated analysis because each sphere had to be identified in each image by hand and each fit then required several seconds of computation (8,1,9,10). Fortunately, holographic fitting lends itself to parallel processing on the graphical processing unit (GPU) of a computer graphics card (11). A sphere's scattering pattern typically subtends tens of thousands of pixels and must be computed dozens of times in the course of each fit. Each scattering pattern, furthermore, is expressed as an expansion in special functions, each of which must be separately computed. Whereas conventional CPU-based implementations operate on each pixel in sequence, a GPU-enabled algorithm operates on multiple pixels simultaneously, and so is substantially faster.

We analyze holographic images with software developed in the IDL programming language (ITT Visual Information Solutions, Boulder, CO), taking advantage of the MPFIT suite of Levenberg-Marquardt nonlinear least-squares fitting routines (12). We implemented a GPU-enabled computation of $ \vec{E}_s(\vec{r})$ using the GPUlib (13) (Tech-X Corp., Boulder, CO) extensions to IDL on an nVidia GTX 280 graphics card (nVidia Corp., Santa Clara, CA) installed in the host computer. GPUlib provides access to the underlying CUDA framework ( for mathematical computation on GPUs. With these enhancements, our implementation computes a $ 200 \times 200$ pixel trial image in 3 ms, a factor of 20 faster than the equivalent calculation performed on the central processing unit (CPU) alone. Substantial further acceleration could be attained by performing the fits on the GPU and implementing the entire software suite in an optimized compiled programming language. When supported by a multi-core CPU, the GPU can process multiple computational threads simultaneously, yielding a proportional increase in processing speed. Our implementation uses three independent computational threads on a quad-core CPU to attain the equivalent of 8 frames/s. Adding a second graphics card nearly doubles the processing speed to 15 frames/s.

Unattended feature identification

Figure 2: Detecting particle images in a video hologram. Original (a) and transformed (b) holographic images of three colloidal spheres. Superimposed line segments in (a) indicate the votes cast by three representative pixels. Intensity in (b) is scaled by the number of votes, with black representing 0 and white representing 800 votes. Superimposed surface plots illustrate the middle sphere's transformation. Scale bar indicates $ 10~\unit{\mu m}$ .

We automate GPU-accelerated fitting by first identifying particle images in the field of view and then estimating the fitting parameters at low precision through a combination of standard methods. Estimates for each identified particle then are refined to high precision with GPU-accelerated fits to Eq. (1).

Each sphere appears in a snapshot, such as the example in Fig. 2(a), as concentric bright and dark rings. The gradient of the intensity at each pixel therefore defines a line segment in the imaging plane along which a sphere's center may lie. The intersection of such lines defines an estimate for the sphere's centroid in the focal plane. We identify such intersections with a simplified variant of the circular Hough transform (14) in which each pixel in the original image casts ``votes'' for the pixels in the transformed image that might be centroids. The three line segments superimposed on Fig. 2(a) indicate votes cast by three representative pixels in the original image. Figure 2(b) shows how the image in Fig. 2(a) is transformed by accumulating all possible single-pixel votes. The inset surface plots demonstrate how the extended interference pattern due to a single sphere is transformed into a sharply defined peak, even if two or more spheres' holographic images overlap.

Those pixels in the transformed image with the most votes are taken to indicate the positions of spheres. Initial estimates for their in-plane coordinates then are computed as the brightness-weighted center of brightness for each peak. This procedure typically identifies particles' centroids to within a few tenths of a pixel (7), or a few dozen nanometers. We find in practice that this is sufficiently precise to ensure convergence of the subsequent Lorenz-Mie fit.

Given a sphere's in-plane centroid, we estimate its axial position, radius and refractive index with low-resolution Monte Carlo fits over the anticipated range of parameters. This process can be slow for large, high-index particles whose error function has many local minima. For micrometer-scale latex spheres, on the other hand, it is both fast and robust. Furthermore, in cases where the refractive index and particle size are known to within a few percent, this bootstrapping process can be very fast. For the data presented here, automated preprocessing took up no more than ten percent of the processing time for each fit.

Because each sphere's image extends over a large number of pixels, reliable and accurate results may be obtained even when particles' images overlap or intersect the edge of the field of view. Systematic studies of the limits of single-particle fitting based on sphere concentration, image truncation, and camera noise will be reported elsewhere. The present studies focus on colloidal dispersions whose concentrations are sufficiently low to avoid errors due to overlapping images.

The combination of hardware acceleration and rapid initialization transform quantitative holographic video microscopy into a suitable tool for large-scale automated colloidal tracking and characterization. We next demonstrate its performance on a model colloidal dispersion and then apply it to label-free detection of avidin binding to functionalized latex beads.

next up previous
Next: Results Up: Flow visualization and flow Previous: Introduction
David Grier 2009-07-17