Holographic microscopy of holographically trapped three-dimensional structures

Sang-Hyuk Lee

David G. Grier

Department of Physics and Center for Soft Matter Research, New York University, New York, NY 10003

Abstract.

Holographic optical trapping uses forces exerted by computer-generated holograms to organize microscopic materials into three-dimensional structures. In a complementary manner, holographic video microscopy uses real-time recordings of in-line holograms to create time-resolved volumetric images of three-dimensional microstructures. The combination is exceptionally effective for organizing, inspecting and analyzing soft-matter systems.

pacs: 090.1760, 180.6900, 140.7010

Holographic optical trapping (1); (2) uses computer-generated holograms to trap and organize micrometer-scale objects into arbitrary three-dimensional configurations. No complementary method has been available for examining optically trapped structures except conventional two-dimensional microscopy. Three-dimensional imaging would be useful for verifying the structure of holographically organized systems before fixing them in place (1); (3); (4); (5). It also would be useful for interactively manipulating and inspecting three-dimensionally structured objects such as biological specimens. Integrating three-dimensional imaging with holographic trapping might seem straightforward because both techniques can make use of the same objective lens to collect and project laser light, respectively. The problem is that most three-dimensional imaging methods, such as confocal microscopy, involve mechanically translating the focal plane through the sample. Holographic traps, however, are positioned relative to the focal plane, and would move as well. Although the trapping pattern could be translated to compensate for the microscope's mechanical motion, the added complexity, reduced imaging speed, and potential for disrupting the sample are clear drawbacks.

Digital holographic microscopy addresses all of these technical concerns, providing real-time three-dimensional imaging data without requiring mechanical motion (7); (8); (9); (10); (6). A particularly compatible variant of in-line holographic microscopy simply replaces the conventional illuminator in a bright-field microscope with a collimated laser (6). Light scattered out of the laser beam by the object interferes with the remainder of the incident illumination to produce a heterodyne scattering pattern that is magnified by the objective lens and recorded with a video camera. Provided that this interference pattern is not obscured by multiple light scattering, it contains comprehensive information on the scatterers' three-dimensional configuration. Each two-dimensional snapshot in the resulting video stream encodes time-resolved volumetric information that can be analyzed directly, or decoded numerically into three-dimensional representations. This Article demonstrates digital holographic microscopy in a holographic optical manipulation system, and uses the combined capabilities to directly assess both techniques' accuracy and limitations.

Figure 1. Combined dynamic holographic optical trapping and holographic video microscopy system.

Figure 1 shows a schematic representation of the integrated system, which is based on an inverted optical microscope (Zeiss Axiovert S100-TV) outfitted with a $100\times$ NA 1.4 oil immersion objective lens. This lens is used both to project holographic optical traps, and also to collect in-line holographic images of trapped objects. Holographic traps are powered by a frequency-doubled diode-pumped solid state laser (Coherent Verdi) operating at a wavelength of $532~\mathrm{n}\mathrm{m}$ . A liquid crystal spatial light modulator (Hamamatsu PAL-SLM X7550) imprints the beam's wavefronts with phase-only holograms encoding the desired trapping pattern (11). The modified beam then is relayed to the input pupil of the objective lens and is focused into optical traps.

The trapping beam is relayed to the objective lens with a dichroic mirror tuned to the trapping laser's wavelength. Other wavelengths pass through the dichroic mirror and form images on a CCD camera (NEC TI-324AII). We replaced the standard combination of incandescent illuminator and condenser lens with a helium-neon laser providing 5 mW collimated beam of coherent light at a wavelength of $\lambda=632~\mathrm{n}\mathrm{m}$ in air.

Figure 2. Holographic imaging of a three-dimensional configuration of holographically trapped colloidal spheres. (a) Conventional bright-field image of five colloidal spheres trapped in the $xy$ plane. Scale bar indicates 5 $\mathrm{\upmu}\mathrm{m}$ . (b) The pattern is rotated around the $y$ axis by $45^{\circ}$ . (c) Bright-field image of the rotated pattern, as seen in the $xy$ plane. (d) Coherent image of the same structure, as seen in the $xy$ plane. (e) Holographic reconstruction of an $xz$ slice through the tilted pattern. Circles denote the intended particle coordinates.

Figure 2 demonstrates holographic imaging of colloidal spheres holographically trapped in a three-dimensional pattern. These 1.53 $\mathrm{\upmu}\mathrm{m}$ diameter silica spheres (Bangs Labs Lot No. L011031B) are dispersed in a 50 $\mathrm{\upmu}\mathrm{m}$ thick layer of water confined within a slit pore formed by sealing the edges of a #1 cover slip to the surface of a clean glass microscope slide. Each sphere is trapped in a separate point-like optical tweezer (12), and the individual optical traps are positioned independently in three dimensions (1); (13); (14); (15); (11). Figure 2(a) shows a conventional bright-field image of the particles arranged in the focal plane. Projecting a sequence of holograms with the trapping positions slightly displaced enables us to rotate the entire pattern in three dimensions, as shown in Fig. 2(b). As particles move away from the focal plane, their images blur, as can be seen in Fig. 1(c). Indeed, it is difficult to determine from this image whether the most distant particles are present at all.

Figure 2(d) shows the same field of view, but with laser illumination. Each particle appears in this image as the coherent superposition of the laser light it scatters with the undiffracted portion of the laser beam. Other features in the image result from reflections, refraction and scattering by surfaces in the optical train. These have been minimized by subtracting off a reference image obtained with no particles in the field of view.

Enough information is encoded in two-dimensional real-valued images such as Fig. 2 to at least approximately reconstruct the three-dimensional complex-valued light field. The image in Fig. 2(e) is an example showing a numerically reconstructed vertical cross-section through the array of particles. This demonstrates the feasibility of combining holographic microscopy with holographic optical trapping. The reconstruction is consistent with the anticipated 45 ${}^{\circ}$ inclination of the array, and with the calibrated 5.4 $\mathrm{\upmu}\mathrm{m}$ separation between the particles. Intended particle coordinates are shown as circles superimposed on the image. This quantitative comparison demonstrates the utility of holographic microscopy for verifying holographic assemblies. Because holographic images such as Fig. 2(d) can be obtained at the full frame rate of the video camera, holographic microscopy offers the benefit of real-time data acquisition over confocal and deconvolution microscopies.

Figure 3. Axial structure of the light field scattered by a colloidal sphere. (a) Hologram recorded in the $xy$ plane of a single sphere trapped at $z=17~\mathrm{\upmu}\mathrm{m}$ above the focal plane. (b) Real part of the scattered field reconstructed from (a). (c) Hologram recorded with sphere at $z=0$ . (d) Axial section of the scattering field obtained by translating the particle past the focal plane in $\Delta z=0.122~\mathrm{\upmu}\mathrm{m}$ steps. (e) Equivalent reconstruction using conventional illumination. Scale bar indicates 5 $\mathrm{\upmu}\mathrm{m}$ . (f) Axial intensity profiles from (b) and (d) demonstrating accuracy of the axial reconstruction.

Previous implementations of in-line holographic digital video microscopy (6) have at least implicitly invoked the Fresnel far-field approximation to analyze digital holograms. Holograms such as Fig. 2(d), however, form at ranges comparable to the wavelength of light. More accurate results can be expected, therefore, from the Rayleigh-Sommerfeld formalism (16).

The field $u({\boldsymbol{r}},z)$ scattered by an object at height $z$ above the microscope's focal plane propagates to the focal plane, where it interferes with the reference field, $a({\boldsymbol{r}})$ , comprised of the undiffracted portion of the laser illumination. Here, ${\boldsymbol{r}}=(x,y)$ is the position in the focal plane. The Rayleigh-Sommerfeld propagator describing the object field's propagation along the optical axis is (16)

$h_{z}({\boldsymbol{r}})=-\frac{1}{2\pi}\,\frac{\partial}{\partial z}\frac{e^{{ikR}}}{R},$

(1)

where $R^{2}=r^{2}+z^{2}$ and $k=2\pi\, n/\lambda$ is the light's wavenumber in a medium of refractive index $n$ . The field in the focal plane is the convolution $u({\boldsymbol{r}},0)\otimes h_{z}({\boldsymbol{r}})$ . The observed interference pattern, therefore, is

$I({\boldsymbol{r}})=\left|a({\boldsymbol{r}})\right|^{2}+2\Re\left\{ a^{\ast}\,(u\otimes h_{z})\right\}+\left|u\otimes h_{z}\right|^{2}.$

(2)

The first term in Eq. (2) can be approximated by measuring the intensity when no objects are in the field of view. Figure 2(d) was obtained by subtracting such a reference image from the measured interference pattern. If we further assume that the scattered field is much dimmer than the reference field, the second term in Eq. (2) dominates the third. In that case,

	$\displaystyle b({\boldsymbol{r}})$	$\displaystyle=\frac{I({\boldsymbol{r}})-\left\|a({\boldsymbol{r}})\right\|^{2}}{\left\|a({\boldsymbol{r}})\right\|}\approx 2\,\frac{\Re\left\{ a^{\ast}\,(u\otimes h_{z})\right\}}{\left\|a({\boldsymbol{r}})\right\|}$		(3)
		$\displaystyle\approx 2\,\Re\{ u\otimes h_{z}\}$		(4)

provides a reasonable basis for reconstructing $u({\boldsymbol{r}})$ . The final approximation in Eq. (3) requires gradients in the illuminating field's phase to be more gradual than any phase gradients of interest.

The three-dimensional intensity field is most easily reconstructed from $b({\boldsymbol{r}})$ using the Fourier convolution theorem, according to which

	$\displaystyle B({\boldsymbol{q}})$	$\displaystyle\equiv\int _{{-\infty}}^{\infty}b({\boldsymbol{r}})\,\exp(-i{\boldsymbol{q}}\cdot{\boldsymbol{r}})\, d^{2}r$		(5)
		$\displaystyle\approx U({\boldsymbol{q}})\, H_{z}({\boldsymbol{q}})+U^{\ast}({\boldsymbol{q}})\, H_{z}^{\ast}({\boldsymbol{q}}),$		(6)

where $U({\boldsymbol{q}})$ is the Fourier transform of $u({\boldsymbol{r}},0)$ and

$H_{z}({\boldsymbol{q}})=\exp\left(ikz\,\left[1-\left(\frac{\lambda q}{2\pi\, n}\right)^{2}\right]^{\frac{1}{2}}\right)$

(7)

is the Fourier transform of the Rayleigh-Sommerfeld propagator (17); (9); (16).

The estimate for the Fourier transform of the object field at height $z^{\prime}$ above the focal plane is obtained by applying the appropriate Rayleigh-Sommerfeld propagator to translate the effective focal plane:

$B({\boldsymbol{q}})\, H_{{-z^{\prime}}}({\boldsymbol{q}})\approx U({\boldsymbol{q}})\, H_{{z-z^{\prime}}}({\boldsymbol{q}})+U^{\ast}({\boldsymbol{q}})\, H_{{-z-z^{\prime}}}({\boldsymbol{q}}).$

(8)

The first term in Eq. (8) is the reconstructed field, which comes into best focus when $z^{\prime}=z$ . The second is an artifact that is increasingly blurred as $z^{\prime}$ increases. Unfortunately, this term creates a mirror image around the plane $z=0$ with the result that objects below the focal plane cannot be distinguished from objects above. This ghosting is apparent in Fig. 2(e).

Our final estimate for the complex light field at height $z$ above the focal plane is

	$\displaystyle v({\boldsymbol{r}},z)$	$\displaystyle\equiv\left\|v({\boldsymbol{r}},z)\right\|\,\exp(i\phi({\boldsymbol{r}},z))$		(9)
		$\displaystyle=\frac{1}{4\pi^{2}}\,\int _{{-\infty}}^{\infty}B({\boldsymbol{q}})\, H_{{-z}}({\boldsymbol{q}})\,\exp(i{\boldsymbol{q}}\cdot{\boldsymbol{r}})\, d^{2}q.$		(10)

Equation (9) can reconstruct a volumetric representation of the instantaneous light field in the sample from a single holographic snapshot, $I({\boldsymbol{r}})$ . The image in Fig. 2(e) is a cross-section through the reconstructed intensity distribution, $\left|v({\boldsymbol{r}},z)\right|^{2}$ .

Figure 4. Resolution limits for occluded objects. (a) Hologram of the holographically organized structure rotated to $90^{\circ}$ , with 4 spheres arranged along the optical axis. Scale bar indicates 5 $\mathrm{\upmu}\mathrm{m}$ . (b) Holographic reconstruction of $\left|v({\boldsymbol{r}},z)\right|$ in the $yz$ plane. (c) The same section through $\left|\Im\left\{ v({\boldsymbol{r}},z)\right\}\right|^{2}$ . (d) Axial traces through $\left|\Im\left\{ v({\boldsymbol{r}},z)\right\}\right|^{2}$ showing positions of axially stacked spheres compared with individual sphere (filled red trace).

Each sphere in Fig. 2(e) appears as a bright axial streak centered on the object's three-dimensional position. Circles superimposed on Fig. 2(e) indicate the coordinates used to compute (11); (18) the trap-forming hologram that arranged the spheres. The very good agreement between the optical traps' design and features in the resulting reconstructed field attests to the accuracy of both the projection and imaging methods.

Contrary to previous reports (6), images such as those in Fig. 3 suggest that the axial resolution of our holographic reconstruction approaches the diffraction-limited in-plane resolution. Figure 3(a) shows a hologram obtained for a sphere held by an optical tweezer at height $z=17~\mathrm{\upmu}\mathrm{m}$ above the focal plane. Figure 3(b) is an axial section through the real part of field reconstructed from (a), $\Re\{ v({\boldsymbol{r}},z)\}=\left|v({\boldsymbol{r}},z)\right|\,\cos(\phi({\boldsymbol{r}},z))$ . This representation has the benefit of most closely resembling the scattering field observed in conventional three-dimensional bright-field microscopy (19). The sphere, in this case, is centered at the crossover between bright and dark regions.

The effective axial resolution can be assessed by scanning the sphere past the focal plane and stacking the resulting images to create a volumetric data set. Figure 3(c) is a hologram of the same sphere from Fig. 3(a) at $z=0$ . Compiling a sequence of such images in axial steps of $\Delta z=0.122~\mathrm{\upmu}\mathrm{m}$ yields the axial section in Fig. 3(d). The $2^{\circ}$ tilt in the scanned image reflects the inclination of the trapping system's axis relative to the imaging train. Figure 3(e) was obtained by scanning the sphere with conventional incoherent illumination. It features the same tilt seen in Fig. 3(d), but has a much shallower depth of focus.

Figure 3(f) shows axial intensity profiles obtained from the images in Figs. 3(b) and (d). The very close agreement between these two traces demonstrates that the holographic reconstruction approaches diffraction-limited resolution. The zero crossing in either case can be resolved to within 20 $\mathrm{n}$ $\mathrm{m}$ , which is comparable to the instrumentally limited in-plane tracking resolution (20).

Structure in the spheres' images along the axial direction can be analyzed to track the spheres in $z$ , as well as in $x$ and $y$ . For the micrometer-scale particles studied here, for example, the centroid is located in the null plane between the downstream intensity maximum and the upstream intensity minimum along the scattering pattern's axis. Holographic microscopy of colloidal particles therefore can be used to extract three-dimensional trajectories more accurately than is possible with conventional two-dimensional imaging (20); (21) and far more rapidly than with scanned three-dimensional imaging techniques (22). In particular, in-plane tracking can make use of conventional techniques (20), and tracking in depth requires additional computation but no additional calibration.

Analyzing images becomes far more challenging when objects occlude each other along the optical axis, as Fig. 4 demonstrates. Here, the same pattern of spheres from Fig. 2 has been rotated by $90^{\circ}$ , so that four of the spheres are aligned along the optical axis. Figure 4(a) is a detail from the resulting hologram and Figs. 4(b) and (c) are vertical sections through the amplitude and imaginary part of the reconstructed field, respectively. The latter has been squared to mimic the contrast of a conventional intensity representation. Each spheres in Fig. 4 is centered on a local maximum in $\left|\Im\left\{ v({\boldsymbol{r}},z)\right\}\right|^{2}$ . These maxima, in turn, correspond to the points of inflection in $\Re\left\{ v({\boldsymbol{r}},z)\right\}$ that were used to establish resolution limits in Fig. 3.

The central observation from Fig. 4 is that all four spheres are resolved, even though they directly occlude each other. An axial trace through $\left|\Im\left\{ v({\boldsymbol{r}},z)\right\}\right|^{2}$ along the spheres' centers, plotted in Fig. 4(d), clearly shows the three downstream spheres. The conventionally in-focus sphere at $z=0$ is suppressed in this representation, but can be seen in the amplitude representation in Fig. 4(b). A fifth sphere, not directly occluded by the others was included as a reference, and is visible to the right of the others in Figs. 4(a), (b) and (c).

The lower spheres in Fig. 4 appear progressively brighter than the spheres they occlude because they act as lenses, gathering light scattered from above and focusing it onto the optical axis. Equation (9) does not take such multiple light scattering into account when reconstructing the light field. The resulting uncertainty in interpreting such results can be mitigated by acquiring images from multiple focal planes, or by illuminating the sample from multiple angles, rather than directly in-line (23). Results also would be improved by more accurate recordings. Each pixel in our holographic images contains roughly 6 bits of usable information, and no effort was made to linearize the camera's response. The camera was set to 1/2000 s shutter speed, which nonetheless allows for some particle motion during each exposure. A wider dynamic range, calibrated intensity response and faster shutter all would provide sharper, more accurate holograms, and thus clearer three-dimensional reconstructions.

With these caveats, the traces in Fig. 4(d) highlight the potential importance of holographic imaging for three-dimensional holographic manipulation. The most distant particle appears to be very slightly displaced along the optical axis relative to the reference particle even though both were localized in optical tweezers projected to the same height. Three-dimensional visualizations confirm the structure of the projected trapping field (24). The apparent axial displacement was not evident for inclinations less than roughly $80^{\circ}$ . It therefore reflects either a three-dimensional imaging artifact or, more likely, a real displacement of the particles from their designed configuration. This is reasonable because light from the traps projected closer to the focal plane exerts forces on particles trapped deeper into the sample. Inter-trap interactions are exacerbated by particles trapped closer to the focal plane, which deflect light onto more distant particles, altering their effective potential energy wells. This effect has been exploited for in-line optical binding of particles trapped along thread-like Bessel beams (25); (26). Holographic imaging provides a means for measuring such distortions, and thus a basis for correcting them. Adaptive structural optimization can be critically important for processes such as the holographic assembly of photonic heterostructures, which rely on accurate placement of microscopic-scale objects (4); (5).

This work was supported by the National Science Foundation through Grant Number DBI-0629584 and Grant Number DMR-0606415. SHL acknowledges support of a Kessler Family Foundation Fellowship.

References

(1)
E. R. Dufresne and D. G. Grier. “Optical tweezer arrays and optical substrates created with diffractive optical elements.” Rev. Sci. Instrum. 69, 1974–1977 (1998).
(2)
D. G. Grier. “A revolution in optical manipulation.” Nature 424, 810–816 (2003).
(3)
P. Jordan, H. Clare, L. Flendrig, J. Leach, J. Cooper and M. Padgett. “Permanent 3D microstructures in a polymeric host created using holographic optical tweezers.” J. Mod. Opt. 51, 627–632 (2004).
(4)
Y. Roichman and D. G. Grier. “Holographic assembly of quasicrystalline photonic heterostructures.” Opt. Express 13, 5434–5439 (2005).
(5)
R. Agarwal, K. Ladavac, Y. Roichman, G. Yu, C. M. Lieber and D. G. Grier. “Manipulation and assembly of nanowires with holographic optical traps.” Opt. Express 13, 8906–8912 (2005).
(6)
J. Sheng, E. Malkiel and J. Katz. “Digital holographic microscope for measuring three-dimensional particle distributions and motions.” Appl. Opt. 45, 3893–3901 (2006).
(7)
U. Schnars and W. Jüptner. “Direct recording of holograms by a CCD target and numerical reconstruction.” Appl. Opt. 33, 179–181 (1994).
(8)
E. Cuche, P. Marquet and C. Depeursinge. “Simultaneous amplitude-contrast and quantitative phase-contrast microscopy by numerical reconstruction of Fresnel off-axis holograms.” Appl. Opt. 38, 6994–7001 (1999).
(9)
U. Schnars and W. P. O. Jüptner. “Digital recording and reconstruction of holograms.” Meas. Sci. Tech. 13, R85–R101 (2002).
(10)
W. Xu, M. H. Jerico, I. A. Meinertzhagen and H. J. Kreuzer. “Digital in-line holography of microspheres.” Appl. Opt. 41, 5367–5375 (2002).
(11)
M. Polin, K. Ladavac, S.-H. Lee, Y. Roichman and D. G. Grier. “Optimized holographic optical traps.” Opt. Express 13, 5831–5845 (2005).
(12)
A. Ashkin, J. M. Dziedzic, J. E. Bjorkholm and S. Chu. ``Observation of a single-beam gradient force optical trap for dielectric particles.” Opt. Lett. 11, 288–290 (1986).
(13)
M. Reicherter, T. Haist, E. U. Wagemann and H. J. Tiziani. “Optical particle trapping with computer-generated holograms written on a liquid-crystal display.” Opt. Lett. 24, 608–610 (1999).
(14)
E. R. Dufresne, G. C. Spalding, M. T. Dearing, S. A. Sheets and D. G. Grier. “Computer-generated holographic optical tweezer arrays.” Rev. Sci. Instrum. 72, 1810–1816 (2001).
(15)
J. E. Curtis, B. A. Koss and D. G. Grier. “Dynamic holographic optical tweezers.” Opt. Comm. 207, 169–175 (2002).
(16)
J. W. Goodman. Introduction to Fourier Optics (McGraw-Hill, New York, 2005), 3rd ed.
(17)
G. C. Sherman. “Application of the convolution theorem to Rayleigh's integral formulas.” J. Opt. Soc. Am. 57, 546–547 (1967).
(18)
Y. Roichman, A. S. Waldron, E. Gardel and D. G. Grier. “Performance of optical traps with geometric aberrations.” Appl. Opt. 45, 3425–3429 (2005).
(19)
M. S. Elliot and W. C. K. Poon. “Conventional optical microscopy of colloidal suspensions.” Adv. Colloid Interface Sci. 92, 133–194 (2001).
(20)
J. C. Crocker and D. G. Grier. “Methods of digital video microscopy for colloidal studies.” J. Colloid Interface Sci. 179, 298–310 (1996).
(21)
M. Wu, J. W. Roberts and M. Buckley. “Three-dimensional fluorescent particle tracking at micron-scale using a single camera.” Expt. Fluids 38, 461–465 (2005).
(22)
A. D. Dinsmore, E. R. Weeks, V. Prasad, A. C. Levitt and D. A. Weitz. “Three-dimensional confocal microscopy of colloids.” Appl. Opt. 40, 4152–4159 (2001).
(23)
S. A. Alexandrov, T. R. Hillman, T. Gutzler and D. D. Sampson. “Synthetic aperture Fourier holographic optical microscopy.” Phys. Rev. Lett. 97, 68102 (2006).
(24)
Y. Roichman, I. Cholis and D. G. Grier. “Volumetric imaging of holographic optical traps.” Opt. Express 14, 10907–10912 (2006).
(25)
K. Dholakia, H. Little, C. T. A. Brown, B. Agate, D. McGloin, L. Paterson and W. Sibbett. “Imaging in optical micromanipulation using two-photon excitation.” New J. Phys. 6, 136 (2004).
(26)
T. Cizmar, E. Kollarova, Z. Bouchal and P. Zemanek. “Sub-micron particle organization by self-imaging of non-diffracting beams.” New J. Phys. 8, 43 (2006).