====== Planning Multi-Fingered Grasps as Probabilistic Inference in a Learned Deep Network ======
===== Abstract =====
We propose a novel approach to multi-fingered grasp planning leveraging learned deep neural network models. We train a convolutional neural network to predict grasp success as a function of both visual information of an object and grasp configuration. We can then formulate grasp planning as inferring the grasp configuration which maximizes the probability of grasp success. We efficiently perform this inference using a gradient-ascent optimization inside the neural network using the backpropagation algorithm. Our work is the first to directly plan high quality multi-fingered grasps in configuration space using a deep neural network without the need of an external planner. We validate our inference method performing both multi-finger and two-finger grasps on real robots. Our experimental results show that our planning method outperforms existing planning methods for neural networks; while offering several other benefits including being data-efficient in learning and fast enough to be deployed in real robotic applications.

===== Publication =====
  - {{:project:grasps_detection.pdf | "Planning Multi-Fingered Grasps as Probabilistic Inference in a Learned Deep Network"}}, Qingkai Lu, Kautilya Chenna, Balakumar Sundaralingam, and Tucker Hermans, [[https://parasol.tamu.edu/isrr/isrr2017/ | International Symposium on Robotics Research]] (ISRR) 2017.
Here is the corresponding bibtex entry:
  @inproceedings{lu2017grasp,
  title={Planning Multi-Fingered Grasps as Probabilistic Inference in a Learned Deep Network},
  author={Lu, Qingkai and Chenna, Kautilya and Sundaralingam, Balakumar and Hermans, Tucker},
  booktitle={Int’l Symp. on Robotics Research},
  year={2017}
  }
===== Grasp Performance Video =====
{{ youtube>7Sg1uw_szlw?640x360&rel=0 }}
===== Experiments =====
We conduct multi-finger grasp experiments using the same four-fingered Allegro hand mounted on a Kuka LBR4 arm in simulation and on the real robot. We compare to a heuristic grasp procedure, as well as the two dominant approaches to deep-learning based grasp planning: sampling and regression. Finally, we show the applicability of our inference method to two-finger grippers by preforming real-world experiments on the Baxter robot.


==== Multi-finger Grasp Inference in Simulation ====
Multi-finger simulation grasp success rate:
{{ :project:multi_finger_sim_grasps.png?600 |}}
==== Real Robot Multi-finger Grasp Inference ====
Multi-finger real grasp success rate:
{{ :project:multi_finger_real_grasps.png?600 |}}

Multi-finger real grasp examples:
{{ :project:grasps_real_5.png?600 |}}
==== Real Robot Two-finger Gripper Grasp Inference ====
Two-finger grasp success rate:
{{ :project:two_finger_real_grasps.png?600 |}}
  
===== Source Code & Data =====
Source code:      
    
[[https://bitbucket.org/robot-learning/grasp_learning_inference_isrr_2017 |Multi-finger grasp learning and inference]]    

[[https://bitbucket.org/robot-learning/grasp_pipeline_isrr_2017 | Grasp pipeline]]   

[[https://bitbucket.org/robot-learning/blensor_ros_isrr_2017 | Blensor with ROS Indigo]] 
    
[[https://bitbucket.org/robot-learning/grasp_preshape_control_isrr_2017 | Grasp Preshape Controller ]]

[[https://bitbucket.org/robot-learning/object_segmentation_isrr_2017 | Object Segmentation]]
    
[[https://bitbucket.org/robot-learning/ll4ma_robots_description | ll4ma Robots Description]]:    
    
Git tag: isrr_2017 under branch grasp_sim_qingkai     
    
[[https://bitbucket.org/robot-learning/ll4ma_robots_gazebo | ll4ma Robots Gazebo]]:    
    
Git tag: isrr_2017 under branch qingkai_dev  

[[https://bitbucket.org/robot-learning/lwr_hardware | lwr_hardware KUKA Controller]]
    
Git tag: isrr_2017 under branch stiffness_controller 
    
[[https://bitbucket.org/robot-learning/grasp_learn_cornell_data_isrr_2017 | Two-fingered Learning and Inference]]
    
[[https://bitbucket.org/robot-learning/baxter_grasp_pipeline_isrr_2017 | Baxter Two-fingered Grasp Pipeline]]
    
Data:    
    
[[https://robot-learning.cs.utah.edu/data_archive/grasp_isrr_2017/grasp_data_isrr_2017.tar.gz | Training data, experiment data and trained network models]]