Zhengyi (Zen) Luo

Hi there! My name is Zhengyi Luo (罗正宜) and I am a second year PhD student at Carnegie Mellon University’s Robotics Institute, School of Computer Science, advised by Prof. Kris Kitani. I earned my bachelor’s degree from University of Pennsylvania in 2019, where I worked with Prof. Kostas Daniilidis.

My research interest lies at the intersection of vision, learning, and robotics. I am working on topcis including human pose estimation, human-object interaction, human motion modelling etc. Through my research, I want to create methods that effectively interpret spatial-temporal sensory input and build a compositional representation of the 3D world to reason about the interactions between agents and the physical environment. On the application side, I am excited about assistive robots, autonomous vehicles, and AR/VR.

Pittsburgh PA, 15213

news

Sep 14, 2022 One paper on embodied human pose estimation accpeted to NeurIPS 2022!
Sep 1, 2022 Joining Meta as a Visiting Research at Meta Reality Labs, Pittsburgh for a year!
Aug 30, 2022 Awarded Qualcomm Innovation Fellowship 2022!
May 15, 2022 Joining Nvidia as a Research Scientist Intern at Toronto AI Lab for Summer 2022!
Jan 28, 2022 One paper on simulated agent design accpeted to ICLR 2022!
Sep 28, 2021 One paper on egocentric human pose estimation accpeted to NeurIPS 2021!
Show more

Affiliations

Ph.D. in Robotics
School of Computer Science
Aug 2021-Present
Visting Researcher
Meta Reality Labs
PI: Weipeng Xu
Aug 2022-Aug 2023

Research Scientist Intern, Toronto AI Lab
PI: Sanja Fidler
May 2022-Aug 2022

M.S in Robotics
School of Computer Science
Aug 2019-Aug 2021

3D Software Engineer Intern, AR/VR
PI: Novaira Masood
May 2018/2019-August 2018/2019

B.S.E in CS
School of Engineering and Applied Science
Aug 2015-May 2019

Publications

  • Perpetual Humanoid Control for Real-time Simulated Avatars
  • Zhengyi Luo, Jinkun Cao, Alexander Winkler, Kris Kitani, Weipeng Xu
  • In submission
  • [Project]
  • Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusion
  • Davis Rempe*, Zhengyi Luo*, Xue Bin Peng, Ye Yuan, Kris Kitani, Karsten Kreis, Sanja Fidler, Or Litany
  • CVPR 2023
  • * Equal contribution
  • [Paper] [Project]
  • Embodied Scene-aware Human Pose Estimation
  • Zhengyi Luo*, Shun Iwase*, Ye Yuan, Kris M. Kitani
  • NeurIPS 2022
  • * Equal contribution
  • [Video] [Paper] [Project] [Code]
  • From Universal Humanoid Control to Automatic Physically Valid Character Creation
  • Zhengyi Luo, Ye Yuan, Kris M. Kitani
  • ArXiv
  • [Paper] [Project] [Code]
  • Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design
  • Ye Yuan, Yuda Song, Zhengyi Luo, Wen Sun, Kris M. Kitani
  • ICLR 2022 (Oral Presentation — Top 1.6%)
  • [Paper] [Project] [Code]
  • Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimation
  • Zhengyi Luo, Ryo hachiuma, Ye Yuan, Kris M. Kitani
  • ICCVW (CV4ARVR) 2021, NeurIPS 2021
  • [Video] [Paper] [Project] [Code]
  • 3D Human Motion Estimation via Motion Compression and Refinement
  • Zhengyi Luo, S. Alireza Golestaneh, Kris M. Kitani
  • ACCV 2020
  • Oral Presentation
  • [Video] [Paper] [Project] [Code]
  • Learning Shape Representations for Clothing Variations in Person Re-Identification
  • Yu-Jhe Li, Zhengyi Luo, Xinshuo Weng, Kris M. Kitani
  • arXiv, 2020
  • [Paper]
  • Cross-Domain 3D Equivariant Image Embeddings
  • Carlos Esteves, Avneesh Sud, Zhengyi Luo, Kostas Daniilidis, Ameesh Makadia
  • ICML 2019
  • [Paper]
  • Cloud Chaser: Real Time Deep Learning Computer Vision on Low Computing Power Devices
  • Zhengyi Luo, Austin Small, Liam Dugan, Stephen Lane
  • ICMV 2018
  • [Paper] [Project] [Code]
  • The rural–urban stress divide: Obtaining geographical insights through Twitter
  • Kokil Jaidka, Sharath Chandra Guntuku, Jane H Lee, Zhengyi Luo, Anneke Buffone, Lyle H Ungar
  • Computers in Human Behavior, 2020: 106544
  • [Paper] [Project]
  • Visual Analytics Approach to Vessel Behaviour Analysis
  • Liang Jin*, Zhengyi Luo*, Shu Gao
  • Journal of Navigation, 2018,71(5): 1195-1209
  • * indicates equal contribution

---

Professional Services

Conference Reviewer: ICML (2020, 2021, 2022, 2023), ICLR (2022, 2023), NeurIPS (2021, 2022), CVPR (2022, 2023), ICCV (2021), ECCV (2022)

Journal Reviewer: IJCV, TMM