All Publications

  • Full-colour 3D holographic augmented-reality displays with metasurface waveguides. Nature Gopakumar, M., Lee, G. Y., Choi, S., Chao, B., Peng, Y., Kim, J., Wetzstein, G. 2024


    Emerging spatial computing systems seamlessly superimpose digital information on the physical environment observed by a user, enabling transformative experiences across various domains, such as entertainment, education, communication and training1-3. However, the widespread adoption of augmented-reality (AR) displays has been limited due to the bulky projection optics of their light engines and their inability to accurately portray three-dimensional (3D) depth cues for virtual content, among other factors4,5. Here we introduce a holographic AR system that overcomes these challenges using a unique combination of inverse-designed full-colour metasurface gratings, a compact dispersion-compensating waveguide geometry and artificial-intelligence-driven holography algorithms. These elements are co-designed to eliminate the need for bulky collimation optics between the spatial light modulator and the waveguide and to present vibrant, full-colour, 3D AR content in a compact device form factor. To deliver unprecedented visual quality with our prototype, we develop an innovative image formation model that combines a physically accurate waveguide model with learned components that are automatically calibrated using camera feedback. Our unique co-design of a nanophotonic metasurface waveguide and artificial-intelligence-driven holographic algorithms represents a significant advancement in creating visually compelling 3D AR experiences in a compact wearable device.

    View details for DOI 10.1038/s41586-024-07386-0

    View details for PubMedID 38720077

    View details for PubMedCentralID 8208705

  • High-brightness holographic projection. Optics letters Chao, B., Gopakumar, M., Choi, S., Wetzstein, G. 2023; 48 (15): 4041-4044


    We propose a holographic projection system that achieves high image quality, brightness, and light efficiency. Using a novel, to the best of our knowledge, light-efficiency loss function, we are able to concentrate more light on the projection region and improve display brightness compared with conventional projectors. Leveraging emerging artificial intelligence-driven computer-generated holography and camera-in-the-loop calibration techniques, we learn a holographic wave propagation model using experimentally captured holographic images and demonstrate state-of-the-art light reallocation performance with high image quality.

    View details for DOI 10.1364/OL.489617

    View details for PubMedID 37527113

  • Partially-Coherent Neural Holography with Fast Spatial Light Modulators Choi, S., Gopakumar, M., Peng, Y., Kim, J., O'Toole, M., Wetzstein, G., Ehmke, J., Lee, B. L. SPIE-INT SOC OPTICAL ENGINEERING. 2023

    View details for DOI 10.1117/12.2655404

    View details for Web of Science ID 001012190800001

  • Neural 3D Holography: Learning Accurate Wave Propagation Models for 3D Holographic Virtual and Augmented Reality Displays ACM TRANSACTIONS ON GRAPHICS Choi, S., Gopakumar, M., Peng, Y., Kim, J., Wetzstein, G. 2021; 40 (6)
  • Unfiltered holography: optimizing high diffraction orders without optical filtering for compact holographic displays OPTICS LETTERS Gopakumar, M., Kim, J., Choi, S., Peng, Y., Wetzstein, G. 2021; 46 (23): 5822-5825


    Computer-generated holography suffers from high diffraction orders (HDOs) created from pixelated spatial light modulators, which must be optically filtered using bulky optics. Here, we develop an algorithmic framework for optimizing HDOs without optical filtering to enable compact holographic displays. We devise a wave propagation model of HDOs and use it to optimize phase patterns, which allows HDOs to contribute to forming the image instead of creating artifacts. The proposed method significantly outperforms previous algorithms in an unfiltered holographic display prototype.

    View details for DOI 10.1364/OL.442851

    View details for Web of Science ID 000722896900011

    View details for PubMedID 34851899

  • Speckle-free holography with partially coherent light sources and camera-in-the-loop calibration. Science advances Peng, Y., Choi, S., Kim, J., Wetzstein, G. 2021; 7 (46): eabg5040


    [Figure: see text].

    View details for DOI 10.1126/sciadv.abg5040

    View details for PubMedID 34767449

  • Optimizing image quality for holographic near-eye displays with Michelson Holography OPTICA Choi, S., Kim, J., Peng, Y., Wetzstein, G. 2021; 8 (2): 143–46
  • High-quality holographic displays using double SLMs and camera-in-the-loop optimization Choi, S., Peng, Y., Kim, J., Wetzstein, G., Kress, B. C., Peroz, C. SPIE-INT SOC OPTICAL ENGINEERING. 2021

    View details for DOI 10.1117/12.2584044

    View details for Web of Science ID 000704228200017

  • Neural Holography with Camera-in-the-loop Training ACM TRANSACTIONS ON GRAPHICS Peng, Y., Choi, S., Padmanaban, N., Wetzstein, G. 2020; 39 (6)
  • Volumetric Head-Mounted Display with Locally Adaptive Focal Blocks. IEEE transactions on visualization and computer graphics Yoo, D. n., Lee, S. n., Jo, Y. n., Cho, J. n., Choi, S. n., Lee, B. n. 2020; PP


    A commercial head-mounted display (HMD) for virtual reality (VR) presents three-dimensional imagery with a fixed focal distance. The VR HMD with a fixed focus can cause visual discomfort to an observer. In this work, we propose a novel design of a compact VR HMD supporting near-correct focus cues over a wide depth of field (from 18 cm to optical infinity). The proposed HMD consists of a low-resolution binary backlight, a liquid crystal display panel, and focus-tunable lenses. In the proposed system, the backlight locally illuminates the display panel that is floated by the focus-tunable lens at a specific distance. The illumination moment and the focus-tunable lens' focal power are synchronized to generate focal blocks at the desired distances. The distance of each focal block is determined by depth information of three-dimensional imagery to provide near-correct focus cues. We evaluate the focus cue fidelity of the proposed system considering the fill factor and resolution of the backlight. Finally, we verify the display performance with experimental results.

    View details for DOI 10.1109/TVCG.2020.3011468

    View details for PubMedID 32746283

  • Neural Holography Peng, Y., Choi, S., Padmanaban, N., Kim, J., Wetzstein, G., ACM ASSOC COMPUTING MACHINERY. 2020
  • Tomographic Projector: Large Scale Volumetric Display with Uniform Viewing Experiences ACM TRANSACTIONS ON GRAPHICS Jo, Y., Lee, S., Yoo, D., Choi, S., Kim, D., Lee, B. 2019; 38 (6)
  • Optimal binary representation via non-convex optimization on tomographic displays OPTICS EXPRESS Choi, S., Lee, S., Jo, Y., Yoo, D., Kim, D., Lee, B. 2019; 27 (17): 24362–81


    There have been many recent developments in 3D display technology to provide correct accommodation cues over an extended focus range. To this end, those displays rely on scene decomposition algorithms to reproduce accurate occlusion boundaries as well asretinal defocus blur. Recently, tomographic displays have been proposed with improved trade-offs of focus range, spatial resolution, and exit-pupil. The advantage of the system partly stems from a high-speed backlight modulation system based on a digital micromirror device, which only supports 1-bit images. However, its inherent binary constraint hinders achieving the optimal scene decomposition, thus leaving boundary artifacts. In this work, we present a technique for synthesizing optimal imagery of general 3D scenes with occlusion on tomographic displays. Requiring no prior knowledge of the scene geometry, our technique addresses the blending issue via non-convex optimization, inspired by recent studies in discrete tomography. Also, we present a general framework for this rendering algorithm and demonstrate the utility of the technique for volumetric display systems with binary representation.

    View details for DOI 10.1364/OE.27.024362

    View details for Web of Science ID 000482098300060

    View details for PubMedID 31510326