LL4MA lab

Email: tucker.hermans@utah.edu
Tel: 801-581-8122
Office: MEB 2164 (Map)

[ Curriculum Vitae ]

I'm an associate professor in the Kahlert School of Computing at the University of Utah, where I am affiliated with the University of Utah Robotics Center and direct the Utah Learning Lab for Manipulation Autonomy. I am also a Senior Research Scientist at NVIDIA in the Seattle Robotics Lab.

My research focuses on autonomous learning, planning, and perception for robot manipulation. I am particularly interested in enabling robots to autonomously discover and manipulate objects with which they have no previous knowledge or experience. Much of my lab's recent work examines this problem of novel object manipulation in the context of multi-fingered hands, deformable objects, and object collections.

Since joining Utah I have received several awards including the NSF CAREER Award in 2019, the 3M Non-Tenured Faculty Award in 2019, and a Sloan Fellowship in 2021. With my students and collaborators we have won the CoRL 2019 Best Systems Paper Award, the ICRA 2017 Best Medical Robotics Paper Award, and were finalist for the ICRA 2019 Best Manipulation Paper Award and for the Best Paper and Best Student Paper awards at ISMR 2021.

Previously I was a postdoctoral researcher in the Intelligent Autonomous Systems lab at TU Darmstadt in Darmstadt, Germany. There I worked with Jan Peters on tactile manipulation and robot learning, while serving as the team leader at TUDa for the European Commission project TACMAN.

I was at Georgia Tech from 2009 to 2014 in the School of Interactive Computing. There I earned my PhD in Robotics under the supervision of Aaron Bobick and Jim Rehg in the Computational Perception Laboratory. My dissertation research dealt with robots learning to discover and manipulate previously unknown objects. The learning was performed on a Willow Garage PR2 robot, which performed pushing tasks using visual feedback control. At Georgia Tech I also earned an MSc in Computer Science with specialization in Computational Perception and Robotics.

I earned my bachelors from Bowdoin College, where I double-majored in Computer Science and German. At Bowdoin I was a member and team captain of the Northern Bites SPL RoboCup team. We were world champions in 2007, placed third in 2008, and were runner-up in 2009. I additionally developed computer forensics software for the Maine State Police.

[ Google Scholar | BibTex | Full List of Publications | arXiv ]

• "Probabilistic Pick and Place Planning instead of Pick then Place Planning"; Mohanraj Devendran Shanthi, Tucker Hermans; IROS Workshop on Differentiable Probabilistic Robotics: Emerging Perspectives on Robot Learning, 2023.

• "Ready, Set, Plan! Planning to Goal Sets using Generalized Bayesian Inference"; Jana Pavlasek, Stanley Lewis, Balakumar Sundaralingam, Fabio Ramos, Tucker Hermans; Conference on Robot Learning (CoRL), 2023.

• "Composable Part-Based Manipulation"; Weiyu Liu, Jiayuan Mao, Joy Hsu, Tucker Hermans, Animesh Garg, Jiajun Wu; Conference on Robot Learning (CoRL), 2023.

• "Latent Space Planning for Unobserved Objects with Environment-Aware Relational Classifiers"; Yixuan Huang, Jialin Yuan, Weiyu Liu, Chanho Kim, Fuxin Li, Tucker Hermans; IROS Workshop on Causality for Robotics: Answering the Question of Why , 2023.

• "23 DoF Grasping Policies from a Raw Point Cloud"; Martin Matak, Karl Van Wyk, Tucker Hermans; ICRA Workshop on Geometric Representations, 2023.

• "StructDiffusion: Language-Guided Creation of Physically-Valid Structures using Unseen Objects"; Weiyu Liu, Yilun Du, Tucker Hermans, Sonia Chernova, Chris Paxton; Robotics: Science and Systems, 2023.

• "Planning for Multi-Object Manipulation with Graph Neural Network Relational Classifiers"; Yixuan Huang, Adam Conkey, Tucker Hermans; IEEE International Conference on Robotics and Automation (ICRA), 2023.

• "DefGraspNets: Grasp Planning on 3D Fields with Graph Neural Nets"; Isabella Huang, Yashraj Narang, Ruzena Bajcsy, Fabio Ramos, Tucker Hermans, Dieter Fox; IEEE International Conference on Robotics and Automation (ICRA), 2023.

• "Planning Visual-Tactile Precision Grasps via Complementary Use of Vision and Touch"; Martin Matak, Tucker Hermans; IEEE Robotics and Automation Letters (RA-L) (Presented at IROS 2023), 2023.

• "Adaptive Magnetic Control using Stein Variational Gradient Descent computed Distribution of Object Parameters"; Griffin Tabor, Tucker Hermans; IROS Workshop on Differentiable Probabilistic Robotics: Emerging Perspectives on Robot Learning, 2023.

• "DefGraspSim: Physics-based Simulation of Grasp Outcomes for 3D Deformable Objects"; Isabella Huang, Yashraj Narang, Clemens Eppner, Balakumar Sundaralingam, Miles Macklin, Ruzena Bajcsy, Tucker Hermans, Dieter Fox; IEEE Robotics and Automation Letters (RA-L) (Special Issue on Robotic Handling of Deformable Objects), 2022.

• "In Vitro Simulation of Shoulder Motion Driven by 3D Scapular and Humeral Kinematics"; Hema J Sulkar, Tyler W. Knighton, Linda Amoafo, Klevis Aliaj, Christopher W. Kolz, Yue Zhang, Tucker Hermans, Heath B. Henninger; Journal of Biomechanical Engineering, 2022.

• "StructFormer: Learning Spatial Structure for Language-Guided Semantic Rearrangement of Novel Objects"; Weiyu Liu, Chris Paxton, Tucker Hermans, Dieter Fox; IEEE International Conference on Robotics and Automation (ICRA), 2022.

• "Learning Visual Shape Control of Novel 3D Deformable Objects from Partial-View Point Clouds"; Bao Thach, Brian Y Cho, Alan Kuntz, Tucker Hermans; IEEE International Conference on Robotics and Automation (ICRA), 2022.

• "Correcting Robot Plans with Natural Language Feedback"; Pratyusha Sharma, Balakumar Sundaralingam, Valts Blukis, Chris Paxton, Tucker Hermans, Antonio Torralba, Jacob Andreas, Dieter Fox; Robotics: Science and Systems (RSS), 2022.

• "DULA and DEBA: Differentiable Ergonomic Risk Models for Postural Assessment and Optimization in Ergonomically Intelligent pHRI"; Amir Yazdani, Roya Sabbagh Novin, Andrew Merryweather, Tucker Hermans; IEEE International Conference on Intelligent Robots and Systems (IROS), 2022.

• "Attracting Conductive Nonmagnetic Objects With Rotating Magnetic Dipole Fields"; Devin K Dalton, Griffin F Tabor, Tucker Hermans, Jake J Abbott; IEEE Robotics and Automation Letters (RA-L) , 2022.

• "Adaptive Manipulation of Conductive, Nonmagnetic Objects via a Continuous Model of Magnetically Induced Force and Torque"; Griffin F Tabor, Lan N Pham, Jake J Abbott, Tucker Hermans; Robotics: Science and Systems (RSS), 2022.

• "StructDiffusion: Object-Centric Diffusion for Semantic Rearrangement of Novel Objects"; Weiyu Liu, Tucker Hermans, Sonia Chernova, Chris Paxton; CoRL Workshop on Language and Robot Learning, 2022.

• "Ergonomically Intelligent Physical Human-Robot Interaction: Postural Estimation, Assessment, and Optimization"; Amir Yazdani, Roya Sabbagh Novin, Andrew Merryweather, Tucker Hermans; AAAI Artificial Intelligence for Human-Robot Interaction Symposium (AI-HRI), 2021.

• "Toward Learning Context-Dependent Tasks from Demonstration for Tendon-Driven Surgical Robots"; Yixuan Huang, Michael Bentley, Tucker Hermans, Alan Kuntz; International Symposium on Medical Robotics (ISMR), 2021.

• "Planning Sensing Sequences for Subsurface 3D Tumor Mapping"; Brian Y. Cho, Tucker Hermans, Alan Kuntz; International Symposium on Medical Robotics (ISMR), 2021.

• "Parallelised Diffeomorphic Sampling-based Motion Planning"; Tin Lai, Weiming Zhi, Tucker Hermans, Fabio Ramos; Conference on Robot Learning (CoRL), 2021.

• "Predicting Stable Configurations for Semantic Placement of Novel Objects"; Chris Paxton, Chris Xie, Tucker Hermans, Dieter Fox; Conference on Robot Learning (CoRL), 2021.

• "Dexterous magnetic manipulation of conductive non-magnetic objects"; Lan N. Pham, Griffin F. Tabor, Ashkan Pourkand, Jacob L. B. Aman, Tucker Hermans, Jake J. Abbott; Nature, 2021.

• "DULA: A Differentiable Ergonomics Model for Postural Optimization in pHRI"; Amir Yazdani, Roya Sabbagh Novin, Andrew Merryweather, Tucker Hermans; RSS Workshop on Robotics for People: Perspectives on Interaction, Learning and Safety (R4P), 2021.

• "DefGraspSim: Simulation-based grasping of 3D deformable objects"; Isabella Huang, Yashraj Narang, Clemens Eppner, Balakumar Sundaralingam, Miles Macklin, Tucker Hermans, Dieter Fox; RSS Workshop on Deformable Object Simulation in Robotics (DO-Sim), 2021.

• "DeformerNet: A Deep Learning Approach to 3D Deformable Object Manipulation"; Bao Thach, Alan Kuntz, Tucker Hermans; RSS Workshop on Deformable Object Simulation in Robotics (DO-Sim), 2021.

• "Is The Leader Robot an Adequate Sensor for Posture Estimation and Ergonomic Assessment of A Human Teleoperator?"; Amir Yazdani, Roya Sabbagh Novin, Andrew Merryweather, Tucker Hermans; IEEE International Conference on Automation Science and Engineering (CASE), 2021.

• "Risk-Aware Decision Making for Service Robots to Minimize Risk of Patient Falls in Hospitals"; Roya Sabbagh Novin, Amir Yazdani, Andrew Merryweather, Tucker Hermans; IEEE International Conference on Robotics & Automation (ICRA), 2021.

• "A Model Predictive Approach for Online Mobile Manipulation of Nonholonomic Objects using Learned Dynamics"; Roya Sabbagh Novin, Amir Yazdani, Andrew Merryweather, Tucker Hermans; International Journal of Robotics Research, 2021.

• "Optimizing Hospital Room Layout to Reduce the Risk of Patient Falls"; Sarvenaz Chaeibakhsh, Roya Sabbagh Novin, Tucker Hermans, Andrew Merryweather, Alan Kuntz; International Conference on Operations Research and Enterprise Systems (ICORES), 2021.

• "In-Hand Object-Dynamics Inference using Tactile Fingertips"; Balakumar Sundaralingam, Tucker Hermans; IEEE Transactions on Robotics, 2021.

• "Near-Optimal Area-Coverage Path Planning of Energy Constrained Aerial Robots with Application in Autonomous Environmental Monitoring"; Katharin Jensen-Nau, Tucker Hermans, Kam K. Leang; IEEE Transactions on Automation Science and Engineering, 2021.

• "DextrAH-G: Pixels-to-Action Dexterous Arm-Hand Grasping with Geometric Fabrics"; Tyler Ga Wei Lum*, Martin Matak*, Viktor Makoviychuk, Ankur Handa, Arthur Allshire, Tucker Hermans, Nathan D. Ratliff, Karl Van Wyk; Conference on Robot Learning (CoRL), 2024.

• "23 DoF Grasping Policies from a Raw Point Cloud"; Martin Matak, Karl Van Wyk, Tucker Hermans; ICRA Workshop on Geometric Representations, 2023.

• "DefGraspNets: Grasp Planning on 3D Fields with Graph Neural Nets"; Isabella Huang, Yashraj Narang, Ruzena Bajcsy, Fabio Ramos, Tucker Hermans, Dieter Fox; IEEE International Conference on Robotics and Automation (ICRA), 2023.

• "Planning Visual-Tactile Precision Grasps via Complementary Use of Vision and Touch"; Martin Matak, Tucker Hermans; IEEE Robotics and Automation Letters (RA-L) (Presented at IROS 2023), 2023.

• "DefGraspSim: Physics-based Simulation of Grasp Outcomes for 3D Deformable Objects"; Isabella Huang, Yashraj Narang, Clemens Eppner, Balakumar Sundaralingam, Miles Macklin, Ruzena Bajcsy, Tucker Hermans, Dieter Fox; IEEE Robotics and Automation Letters (RA-L) (Special Issue on Robotic Handling of Deformable Objects), 2022.

• "DefGraspSim: Simulation-based grasping of 3D deformable objects"; Isabella Huang, Yashraj Narang, Clemens Eppner, Balakumar Sundaralingam, Miles Macklin, Tucker Hermans, Dieter Fox; RSS Workshop on Deformable Object Simulation in Robotics (DO-Sim), 2021.

• "Contextual Reinforcement Learning of Visuo-tactile Multi-fingered Grasping Policies"; Visak C. V. Kumar, Tucker Hermans, Dieter Fox, Stan Birchfield, Jonathan Tremblay; NeurIPs Workshop on Robot Learning, 2020.

• "Multi-Fingered Active Grasp Learning"; Qingkai Lu, Mark Van der Merwe, Tucker Hermans; IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020.

• "Multi-Fingered Grasp Planning via Inference in Deep Neural Networks"; Qingkai Lu, Mark Van der Merwe, Balakumar Sundaralingam, Tucker Hermans; IEEE Robotics & Automation Magazine (Special Issue on Deep Learning and Machine Learning in Robotics), 2020.

• "Learning Continuous 3D Reconstructions for Geometrically Aware Grasping"; Mark Van der Merwe, Qingkai Lu, Balakumar Sundaralingam, Martin Matak, Tucker Hermans; IEEE International Conference on Robotics and Automation (ICRA), 2020.

• "Modeling Grasp Type Improves Learning-Based Grasp Planning"; Qingkai Lu, Tucker Hermans; IEEE Robotics and Automation Letters (RA-L) (Presented at ICRA 2019), 2019.

• "Relaxed-Rigidity Constraints: Kinematic Trajectory Optimization and Collision Avoidance for In-Grasp Manipulation"; Balakumar Sundaralingam, Tucker Hermans; Autonomous Robots, 2019.

• "Geometric In-Hand Regrasp Planning: Alternating Optimization of Finger Gaits and In-Grasp Manipulation"; Balakumar Sundaralingam, Tucker Hermans; IEEE International Conference on Robotics and Automation (ICRA), 2018.

• "Planning Multi-Fingered Grasps as Probabilistic Inference in a Learned Deep Network"; Qingkai Lu, Kautilya Chenna, Balakumar Sundaralingam, Tucker Hermans; International Symposium on Robotics Research (ISRR), 2017.

• "Relaxed-Rigidity Constraints: In-Grasp Manipulation using Purely Kinematic Trajectory Optimization"; Balakumar Sundaralingam, Tucker Hermans; Robotics: Science and Systems (RSS), 2017.

• "Learning Contact Locations for Pushing and Orienting Unknown Objects"; Tucker Hermans, Fuxin Li, James M. Rehg, Aaron F. Bobick; IEEE-RAS International Conference on Humanoid Robotics (Humanoids), 2013.

• "Guided Pushing for Object Singulation"; Tucker Hermans, James M. Rehg, Aaron F. Bobick; IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2012.

• "Push Planning for Object Placement on Cluttered Table Surfaces"; Akansel Cosgun, Tucker Hermans, Victor Emeli, Mike Stilman; IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2011.