LL4MA lab

Email: tucker.hermans@utah.edu
Bluesky: thermans.bsky.social
Office: MEB 2164 (Map)

[ Curriculum Vitae ]

I'm a professor in the Kahlert School of Computing at the University of Utah and direct the Utah Learning Lab for Manipulation Autonomy. I am also a core member of the University of Utah Robotics Center and the Utah Center for Data Science. I additionally consult as a Senior Research Scientist at NVIDIA in the Seattle Robotics Lab.

My research focuses on autonomous learning, planning, and perception for robot manipulation. I am particularly interested in enabling robots to autonomously discover and manipulate objects with which they have no previous knowledge or experience. Much of my lab's recent work examines this problem of novel object manipulation in the context of multi-fingered hands, deformable objects, and object collections.

Since joining Utah I have received several awards including the NSF CAREER Award in 2019, the 3M Non-Tenured Faculty Award in 2019, and a Sloan Fellowship in 2021. With my students and collaborators we have won the CoRL 2019 Best Systems Paper Award, the ICRA 2017 Best Medical Robotics Paper Award, and were finalist for the ICRA 2019 Best Manipulation Paper Award and for the Best Paper and Best Student Paper awards at ISMR 2021.

Previously I was a postdoctoral researcher in the Intelligent Autonomous Systems lab at TU Darmstadt in Darmstadt, Germany. There I worked with Jan Peters on tactile manipulation and robot learning, while serving as the team leader at TUDa for the European Commission project TACMAN.

I was at Georgia Tech from 2009 to 2014 in the School of Interactive Computing. There I earned my PhD in Robotics under the supervision of Aaron Bobick and Jim Rehg in the Computational Perception Laboratory. My dissertation research dealt with robots learning to discover and manipulate previously unknown objects. The learning was performed on a Willow Garage PR2 robot, which performed pushing tasks using visual feedback control. At Georgia Tech I also earned an MSc in Computer Science with specialization in Computational Perception and Robotics.

I earned my bachelors from Bowdoin College, where I double-majored in Computer Science and German. At Bowdoin I was a member and team captain of the Northern Bites SPL RoboCup team. We were world champions in 2007, placed third in 2008, and were runner-up in 2009. I additionally developed computer forensics software for the Maine State Police.

[ Google Scholar | BibTex | Full List of Publications | arXiv ]

• "First Plan Then Evaluate: Use a Vectorized Motion Planner for Grasping"; Martin Matak, Mohanraj Devendran Shanthi, Karl Van Wyk, Tucker Hermans; IEEE Robotics & Automation Letters (RA-L), 2026.

• "Align Before You Act: Runtime Steering for Reasoning-Action Alignment in Vision-Language-Action Models"; Yilin Wu, Anqi Li, Tucker Hermans, Fabio Ramos, Andrea Bajcsy, Claudia Pérez-D'Arpino; IEEE International Conference on Robotics and Automation (ICRA), 2026.

• “DiffDef: a Diffusion Model for Generating Multimodal Goal Shapes From Demonstrations for Deformable Object Manipulation”; Bao Thach, Tanner Watts, Siyeon Kim, Britton Jordan, Mohanraj Devendran Shanthi, Shing-Hei Ho, James M Ferguson, Tucker Hermans, Alan Kuntz; IEEE International Conference on Robotics and Automation (ICRA), 2026.

• "Continuum Robot State Estimation with Actuation Uncertainty"; James M Ferguson, Alan Kuntz, Tucker Hermans; IEEE Robotics & Automation Letters (RA-L), 2026.

• "The Reality Gap in Robotics: Challenges, Solutions, and Best Practices"; Elie Aljalbout, Jiaxu Xing, Angel Romero, Iretiayo Akinola, Caelan Reed Garrett, Eric Heiden, Abhishek Gupta, Tucker Hermans, Yashraj Narang, Dieter Fox, Davide Scaramuzza, Fabio Ramos; Annual Review of Control, Robotics, and Autonomous Systems, 2026.

• "Constrained Stein Variational Gradient Descent for Robot Perception, Planning, and Identification"; Griffin Tabor, Tucker Hermans; Arxiv Preprint, 2025.

• "Integrating Learned Retraction and Model-Based Resection Planning for Automated Central Airway Obstruction Removal"; Tanner Watts, Mariana Smith, James Ferguson, Nural Yilmaz, Paul Maria Scheik, Bao Thach, Brendan Burkhart, Tucker Hermans, Anton Deguet, Axel Krieger, Alan Kuntz; IEEE Transactions on Medical Robotics and Bionics, 2025.

• "Points2Plans: From Point Clouds to Long-Horizon Plans with Composable Relational Dynamics"; Yixuan Huang, Christopher Agia, Jimmy Wu, Tucker Hermans, Jeannette Bohg; IEEE International Conference on Robotics and Automation (ICRA), 2025.

• "scene_synthesizer: A Python Library for Procedural Scene Generation in Robot Manipulation"; Clemens Eppner, Adithyavairavan Murali, Caelan Garrett, Rowland O'Flaherty, Tucker Hermans, Wei Yang, Dieter Fox; Journal of Open Source Software, 2025.

• "Differentiable GPU-Parallelized Task and Motion Planning"; William Shen, Caelan Reed Garrett, Nishanth Kumar, Ankit Goyal, Tucker Hermans, Leslie Pack Kaelbling, Tomás Lozano-Pérez, Fabio Ramos; Robotics Science and Systems (RSS), 2025.

• "Fail2Progress: Learning from Failures with Stein Variational Inference for Robot Manipulation"; Yixuan Huang, Novella Alvina, Mohanraj Devendran Shanthi, Tucker Hermans; RSS Workshop on Out-of-Distribution Generalization in Robotics, 2025.

• "Fail2Progress: Learning from Real-World Robot Failures with Stein Variational Inference for Robot Manipulation"; Yixuan Huang, Novella Alvina, Mohanraj Devendran Shanthi, Tucker Hermans; Conference on Robot learning (CoRL), 2025.

• "Learning Multimodal Probabilistic Models of Manipulation Skill Effects"; Siyeon Kim, Mohanraj Devendran Shanthi, Yixuan Huang, Tucker Hermans; CoRL Workshop on Learning to Simulate Robot Worlds, 2025.

• "Robust Bayesian Scene Reconstruction by Leveraging Retrieval-Augmented Priors"; Herbert Wright, Weiming Zhi, Martin Matak, Matthew Johnson-Roberson, Tucker Hermans; IEEE Robotics & Automation Letters (RA-L), 2025.

• "Differentiable GPU-Parallelized Task and Motion Planning"; William Shen, Caelan Garrett, Ankit Goyal, Tucker Hermans, Fabio Ramos; CoRL Workshop on Differentiable Optimization Everywhere: Simulation, Estimation, Learning, and Control, 2024.

• "Points2Plans: From Point Clouds to Long-Horizon Plans with Composable Relational Dynamics"; Yixuan Huang, Christopher Agia, Jimmy Wu, Tucker Hermans, Jeannette Bohg; CoRL Workshop on Learning Effective Abstractions for Planning (LEAP), 2024.

• "Magnetic Manipulation of Unknown and Complex Conductive Nonmagnetic Objects with Application in the Remediation of Space Debris"; Griffin F Tabor, Lan N Pham, Jake J Abbott, Tucker Hermans; The International Journal of Robotics Research (IJRR), 2024.

• "DextrAH-G: Pixels-to-Action Dexterous Arm-Hand Grasping with Geometric Fabrics"; Tyler Ga Wei Lum, Martin Matak, Viktor Makoviychuk, Ankur Handa, Arthur Allshire, Tucker Hermans, Nathan D. Ratliff, Karl Van Wyk; Conference on Robot Learning (CoRL), 2024.

• "V-PRISM: Probabilistic Mapping of Unknown Tabletop Scenes"; Herbert Wright, Weiming Zhi, Matthew Johnson-Roberson, Tucker Hermans; IEEE International Conference on Intelligent Robots and Systems (IROS), 2024.

• "L4KDE: Learning for Kinodynamic Tree Expansion"; Tin Lai, Weiming Zhi, Tucker Hermans, Fabio Ramos; IEEE International Conference on Intelligent Robots and Systems (IROS), 2024.

• "Point Cloud Models Improve Visual Robustness in Robotic Learners"; Skand Peri, Iain Lee, Chanho Kim, Li Fuxin, Tucker Hermans, Stefan Lee; IEEE International Conference on Robotics and Automation (ICRA), 2024.

• "Latent Space Planning for Multi-Object Manipulation with Environment-Aware Relational Classifiers"; Yixuan Huang, Nichols Crawford Taylor, Adam Conkey, Weiyu Liu, Tucker Hermans; IEEE Transactions on Robotics (T-RO), 2024.

• "Pick and Place Planning is Better than Pick Planning then Place Planning"; Mohanraj Devendran Shanthi, Tucker Hermans; IEEE Robotics & Automation Letters (RA-L) (Presented at ICRA@40), 2024.

• "Position Regulation of a Conductive Nonmagnetic Object with Two Stationary Field Sources"; Devin Dalton, Griffin Tabor, Tucker Hermans, Jake J. Abbott; IEEE Transaction on Robotics (TRO), 2024.

• "Out of Sight, Still in Mind: Reasoning and Planning about Unobserved Objects with Video Tracking Enabled Memory Models"; Yixuan Huang, Jialin Yuan, Chanho Kim, Pupul Pradhan, Bryan Chen, Li Fuxin, Tucker Hermans; IEEE International Conference on Robotics and Automation (ICRA), 2024.

• "DefGoalNet: Contextual Goal Learning from Demonstrations For Deformable Object Manipulation"; Bao Thach, Tanner Watts, Shing-Hei Ho, Tucker Hermans, Alan Kuntz; IEEE International Conference on Robotics and Automation (ICRA), 2024.

• "Hospitals not Homes, Applied not Speculative"; Jason Wiese, Tucker Hermans; RoboCare Design Workshop @ DIS '24, 2024.

• "Ready, Set, Plan! Planning to Goal Sets using Generalized Bayesian Inference"; Jana Pavlasek, Stanley Lewis, Balakumar Sundaralingam, Fabio Ramos, Tucker Hermans; Conference on Robot Learning (CoRL), 2023.

• "Composable Part-Based Manipulation"; Weiyu Liu, Jiayuan Mao, Joy Hsu, Tucker Hermans, Animesh Garg, Jiajun Wu; Conference on Robot Learning (CoRL), 2023.

• "Probabilistic Pick and Place Planning instead of Pick then Place Planning"; Mohanraj Devendran Shanthi, Tucker Hermans; IROS Workshop on Differentiable Probabilistic Robotics: Emerging Perspectives on Robot Learning, 2023.

• "Adaptive Magnetic Control using Stein Variational Gradient Descent computed Distribution of Object Parameters"; Griffin Tabor, Tucker Hermans; IROS Workshop on Differentiable Probabilistic Robotics: Emerging Perspectives on Robot Learning, 2023.

• "Latent Space Planning for Unobserved Objects with Environment-Aware Relational Classifiers"; Yixuan Huang, Jialin Yuan, Weiyu Liu, Chanho Kim, Fuxin Li, Tucker Hermans; IROS Workshop on Causality for Robotics: Answering the Question of Why , 2023.

• "StructDiffusion: Language-Guided Creation of Physically-Valid Structures using Unseen Objects"; Weiyu Liu, Yilun Du, Tucker Hermans, Sonia Chernova, Chris Paxton; Robotics: Science and Systems, 2023.

• "23 DoF Grasping Policies from a Raw Point Cloud"; Martin Matak, Karl Van Wyk, Tucker Hermans; ICRA Workshop on Geometric Representations, 2023.

• "Planning for Multi-Object Manipulation with Graph Neural Network Relational Classifiers"; Yixuan Huang, Adam Conkey, Tucker Hermans; IEEE International Conference on Robotics and Automation (ICRA), 2023.

• "DefGraspNets: Grasp Planning on 3D Fields with Graph Neural Nets"; Isabella Huang, Yashraj Narang, Ruzena Bajcsy, Fabio Ramos, Tucker Hermans, Dieter Fox; IEEE International Conference on Robotics and Automation (ICRA), 2023.

• "Planning Visual-Tactile Precision Grasps via Complementary Use of Vision and Touch"; Martin Matak, Tucker Hermans; IEEE Robotics and Automation Letters (RA-L) (Presented at IROS 2023), 2023.

• "First Plan Then Evaluate: Use a Vectorized Motion Planner for Grasping"; Martin Matak, Mohanraj Devendran Shanthi, Karl Van Wyk, Tucker Hermans; IEEE Robotics & Automation Letters (RA-L), 2026.

• "DextrAH-G: Pixels-to-Action Dexterous Arm-Hand Grasping with Geometric Fabrics"; Tyler Ga Wei Lum, Martin Matak, Viktor Makoviychuk, Ankur Handa, Arthur Allshire, Tucker Hermans, Nathan D. Ratliff, Karl Van Wyk; Conference on Robot Learning (CoRL), 2024.

• "23 DoF Grasping Policies from a Raw Point Cloud"; Martin Matak, Karl Van Wyk, Tucker Hermans; ICRA Workshop on Geometric Representations, 2023.

• "DefGraspNets: Grasp Planning on 3D Fields with Graph Neural Nets"; Isabella Huang, Yashraj Narang, Ruzena Bajcsy, Fabio Ramos, Tucker Hermans, Dieter Fox; IEEE International Conference on Robotics and Automation (ICRA), 2023.

• "Planning Visual-Tactile Precision Grasps via Complementary Use of Vision and Touch"; Martin Matak, Tucker Hermans; IEEE Robotics and Automation Letters (RA-L) (Presented at IROS 2023), 2023.

• "DefGraspSim: Physics-based Simulation of Grasp Outcomes for 3D Deformable Objects"; Isabella Huang, Yashraj Narang, Clemens Eppner, Balakumar Sundaralingam, Miles Macklin, Ruzena Bajcsy, Tucker Hermans, Dieter Fox; IEEE Robotics and Automation Letters (RA-L) (Special Issue on Robotic Handling of Deformable Objects), 2022.

• "DefGraspSim: Simulation-based grasping of 3D deformable objects"; Isabella Huang, Yashraj Narang, Clemens Eppner, Balakumar Sundaralingam, Miles Macklin, Tucker Hermans, Dieter Fox; RSS Workshop on Deformable Object Simulation in Robotics (DO-Sim), 2021.

• "Contextual Reinforcement Learning of Visuo-tactile Multi-fingered Grasping Policies"; Visak C. V. Kumar, Tucker Hermans, Dieter Fox, Stan Birchfield, Jonathan Tremblay; NeurIPs Workshop on Robot Learning, 2020.

• "Multi-Fingered Active Grasp Learning"; Qingkai Lu, Mark Van der Merwe, Tucker Hermans; IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020.

• "Multi-Fingered Grasp Planning via Inference in Deep Neural Networks"; Qingkai Lu, Mark Van der Merwe, Balakumar Sundaralingam, Tucker Hermans; IEEE Robotics & Automation Magazine (Special Issue on Deep Learning and Machine Learning in Robotics), 2020.

• "Learning Continuous 3D Reconstructions for Geometrically Aware Grasping"; Mark Van der Merwe, Qingkai Lu, Balakumar Sundaralingam, Martin Matak, Tucker Hermans; IEEE International Conference on Robotics and Automation (ICRA), 2020.

• "Modeling Grasp Type Improves Learning-Based Grasp Planning"; Qingkai Lu, Tucker Hermans; IEEE Robotics and Automation Letters (RA-L) (Presented at ICRA 2019), 2019.

• "Relaxed-Rigidity Constraints: Kinematic Trajectory Optimization and Collision Avoidance for In-Grasp Manipulation"; Balakumar Sundaralingam, Tucker Hermans; Autonomous Robots, 2019.

• "Geometric In-Hand Regrasp Planning: Alternating Optimization of Finger Gaits and In-Grasp Manipulation"; Balakumar Sundaralingam, Tucker Hermans; IEEE International Conference on Robotics and Automation (ICRA), 2018.

• "Planning Multi-Fingered Grasps as Probabilistic Inference in a Learned Deep Network"; Qingkai Lu, Kautilya Chenna, Balakumar Sundaralingam, Tucker Hermans; International Symposium on Robotics Research (ISRR), 2017.

• "Relaxed-Rigidity Constraints: In-Grasp Manipulation using Purely Kinematic Trajectory Optimization"; Balakumar Sundaralingam, Tucker Hermans; Robotics: Science and Systems (RSS), 2017.

• "Learning Contact Locations for Pushing and Orienting Unknown Objects"; Tucker Hermans, Fuxin Li, James M. Rehg, Aaron F. Bobick; IEEE-RAS International Conference on Humanoid Robotics (Humanoids), 2013.

• "Guided Pushing for Object Singulation"; Tucker Hermans, James M. Rehg, Aaron F. Bobick; IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2012.

• "Push Planning for Object Placement on Cluttered Table Surfaces"; Akansel Cosgun, Tucker Hermans, Victor Emeli, Mike Stilman; IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2011.