Shangzhe Wu
Postdoctoral Scholar, Computer Science
All Publications
-
DOVE: Learning Deformable 3D Objects by Watching Videos
INTERNATIONAL JOURNAL OF COMPUTER VISION
2023
View details for DOI 10.1007/s11263-023-01819-5
View details for Web of Science ID 001013157000001
-
Unsupervised Learning of Probably Symmetric Deformable 3D Objects From Images in the Wild (Invited Paper)
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
2023; 45 (4): 5268-5281
Abstract
We propose a method to learn 3D deformable object categories from raw single-view images, without external supervision. The method is based on an autoencoder that factors each input image into depth, albedo, viewpoint and illumination. In order to disentangle these components without supervision, we use the fact that many object categories have, at least approximately, a symmetric structure. We show that reasoning about illumination allows us to exploit the underlying object symmetry even if the appearance is not symmetric due to shading. Furthermore, we model objects that are probably, but not certainly, symmetric by predicting a symmetry probability map, learned end-to-end with the other components of the model. Our experiments show that this method can recover very accurately the 3D shape of human faces, cat faces and cars from single-view images, without any supervision or a prior shape model. On benchmarks, we demonstrate superior accuracy compared to another method that uses supervision at the level of 2D image correspondences.
View details for DOI 10.1109/TPAMI.2021.3076536
View details for Web of Science ID 000947840300082
View details for PubMedID 33914682
-
Self-supervised learning for using overhead imagery as maps in outdoor range sensor localization
INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH
2021; 40 (12-14): 1488-1509
Abstract
Traditional approaches to outdoor vehicle localization assume a reliable, prior map is available, typically built using the same sensor suite as the on-board sensors used during localization. This work makes a different assumption. It assumes that an overhead image of the workspace is available and utilizes that as a map for use for range-based sensor localization by a vehicle. Here, range-based sensors are radars and lidars. Our motivation is simple, off-the-shelf, publicly available overhead imagery such as Google satellite images can be a ubiquitous, cheap, and powerful tool for vehicle localization when a usable prior sensor map is unavailable, inconvenient, or expensive. The challenge to be addressed is that overhead images are clearly not directly comparable to data from ground range sensors because of their starkly different modalities. We present a learned metric localization method that not only handles the modality difference, but is also cheap to train, learning in a self-supervised fashion without requiring metrically accurate ground truth. By evaluating across multiple real-world datasets, we demonstrate the robustness and versatility of our method for various sensor configurations in cross-modality localization, achieving localization errors on-par with a prior supervised approach while requiring no pixel-wise aligned ground truth for supervision at training. We pay particular attention to the use of millimeter-wave radar, which, owing to its complex interaction with the scene and its immunity to weather and lighting conditions, makes for a compelling and valuable use case.
View details for DOI 10.1177/02783649211045736
View details for Web of Science ID 000703251000001
View details for PubMedID 34992328
View details for PubMedCentralID PMC8721700
-
De-rendering the World's Revolutionary Artefacts
IEEE COMPUTER SOC. 2021: 6334-6343
View details for DOI 10.1109/CVPR46437.2021.00627
View details for Web of Science ID 000739917306054
-
Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild
IEEE. 2020: 1-10
View details for DOI 10.1109/CVPR42600.2020.00008
View details for Web of Science ID 000620679500001
-
Deep High Dynamic Range Imaging with Large Foreground Motions
SPRINGER INTERNATIONAL PUBLISHING AG. 2018: 120-135
View details for DOI 10.1007/978-3-030-01216-8_8
View details for Web of Science ID 000594207400008
-
Image Generation from Sketch Constraint Using Contextual GAN
SPRINGER INTERNATIONAL PUBLISHING AG. 2018: 213-228
View details for DOI 10.1007/978-3-030-01270-0_13
View details for Web of Science ID 000603403700013