Reading group

We want to cover interesting papers with a focus on those recently published. Everyone is expected to read the papers on their own, so that the meeting can focus on discussion of the paper's merits, issues, and possible extensions.

While we'll have a nominal queue of papers to follow, we should be open to changing what we read and discuss the following week, based on our discussion of the current paper.

Each week one member of the reading group will be designated as the moderator. Your job should not be to present the entire paper to the group, but instead to encourage participation by summarizing important parts of the paper and having probing questions ready to excite discussion.

We would like to pick our papers from recent robotics and machine leaning conferences such as RSS, ICRA, IROS, ICML, NIPS, and Humanoids. Journal articles from IJRR, TRO, Autonomous Robots, JMLR, ML, and PAMI are also a good place to look. We have a more complete list of publication venues if you are interested.

We meet in the lab, MEB 2160, at 3:00PM-4:30PM on Wednesdays (Fall 2018).

We manage meetings through our slack channel: '#reading-group'

General Discussion Questions

List of general questions to consider in reading / leading the group:

  • What type of problem/task does this paper address?
  • Does the paper present an interesting machine learning method?
  • Which other tasks could this approach be applied to?
  • Where does this paper fit in the bigger picture of manipulation learning?
  • Which other papers are closely related to this one?
  • Which learned information can be directly transferred to different scenarios?
  • Would it be useful for our setup to implement this approach?
  • Is this approach equally applicable to grippers and dexterous hands?
  • What are the key weaknesses of the proposed approach?
Date Paper Leader
08/22/2018 Brainstorming and Semester Goals Tucker
08/29/2018 “Chimpanzee Intelligence in Nature and in Captivity: Isomorphism of Symbol Use and Tool Use”; Tetsuro Matsuzawa; Great Ape Societies, Chapter 15, 1996. Adam
09/05/2018 "Differentiable Physics and Stable Modes for Tool-Use and Manipulation Planning"; Marc Toussaint, Kelsey Allen, Kevin Smith, Joshua Tenenbaum; RSS 2018. Qingkai
09/12/2018 Cancelled for ICRA work
09/19/2018 A Survey of the Ontogeny of Tool Use: From Sensorimotor Experience to Planning; Frank Guerin, Norbert Krüger, and Dirk Kraft; IEEE TRANSACTIONS ON AUTONOMOUS MENTAL DEVELOPMENT 2013. Griffin
09/26/2018 Guided Search for Task and Motion Plans Using Learned Heuristics,Rohan Chitnis, Dylan Hadfield-Menell, Abhishek Gupta, Siddharth Srivastava, Edward Groshev, Christopher Lin, and Pieter Abbeel, ICRA 2016. Bala
10/03/2018 Towards Emergence of Tool Use in Robots: Automatic Tool Recognition and Use without Prior Tool Learning ; Keng Peng Tee ; Jun Li ; Lawrence Tai Pang Chen ; Kong Wah Wan ; Gowrishankar Ganesh; ICRA 2018. Amir
10/10/2018 No Meeting: Fall Break
10/17/2018 Autonomous Environment Manipulation to Assist Humanoid Locomotion ; Martin Levihn; Koichi Nishiwaki; Satoshi Kagami; Mike Stilman; ICRA 2014. Roya
10/24/2018 Simulation as an Engine of Physical Scene Understanding ; Peter W. Battaglia, Jessica B. Hamrick, and Joshua B. Tenenbaum; PNAS 2013. Matthew
10/31/2018 Understanding Tools: Task-Oriented Object Modeling, Learning and Recognition; Yixin Zhu, Yibiao Zhao, and Song Chun Zhu; CVPR 2015. Rebeka
11/07/2018 Object–object interaction affordance learning; Yu Sun, Shaogang Ren, Yun Lin; Robotics and Autonomous Systems 2014. Brian
11/14/2018 Learning Task-Oriented Grasping for Tool Manipulation from Simulated Self-Supervision; Kuan Fang, Yuke Zhu, Animesh Garg, et. al.; ArXiv preprint 2018 (for RSS 2019). Michael
11/21/2018 No Meeting: Thanksgiving Break
11/28/2018 What Can I Do With This Tool? Self-Supervised Learning of Tool Affordances From Their 3-D Geometry; Tanis Mar, Vadim Tikhanof, Lorenzo Natale; IEEE Transactions on Cognitive and Developmental Systems 2018. Mohanraj
12/05/2018 Tool Use Learning in Robots; Solly Brown and Claude Sammut; Advances in Cognitive Systems 2011. Qingkai
12/12/2018 Manipulating articulated objects with interactive perception; Dov Katz, Oliver Brock Bala
1/09/2019 TBD TBD
1/16/2019 TBD TBD
1/23/2019 TBD TBD
1/30/2019 TBD TBD
Date Paper
9/24/2015 PILCO: "Gaussian Processes for Data-Efficient Learning in Robotics and Control" PAMI, 2015.
Recent application: "Learning Legged Swimming Gaits from Experience", ICRA, 2015.
10/01/2015 "Probabilistic Differential Dynamic Programming", NIPS 2014.
10/08/2015 "A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems"; Emanuel Todorov and Weiwei Li; ACC 2005. - ILQG
"Adaptive Optimal Feedback Control with Learned Internal Dynamics Models"; Djordje Mitrovic, Stefan Klanke, and Sethu Vijayakumar; From Motor Learning to Interactive Learning in Robotics 2010
GP dynamics with iLQG
Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images
10/22/2015 Guided Policy Search (GPS)
Learning Complex Neural Network Policies with Trajectory Optimization
Variational Policy Search via Trajectory Optimization
10/29/2015 GPS with Unknown Dynamics
ICRA Paper
http://www.eecs.berkeley.edu/~svlevine/papers/mfcgps.pdf
http://rll.berkeley.edu/icra2015gps/index.htm
End-to-End Training of Deep Visuomotor Policies
Video
11/12/2015 DMPs
POWER for Robots
Policy Search in Robotics Survey
11/19/2015 PoWER and Policy Gradients
PoWER
Survey on RL in Robotics
12/03/2015 REPS
01/29/2016 Sequential REPS
02/05/2016 ProMPs
ProMPs
ProMPs on Robots
02/12/2016 Model-Free Probabilistic Movement Primitives for Physical Interaction, Paraschos, A.; Rueckert, E.; Peters, J; Neumann, G. IROS 2015
02/19/2016 "DATA AS DEMONSTRATOR with Applications to System Identification" Arun Venkatraman, Byron Boots, Martial Hebert, and J. Andrew Bagnell; NIPS Workshop on Autonomously Learning Robots 2014
02/26/2016 "A survey of reinforcement learning in Robotics"
03/04/2016 Continue with: "A survey of reinforcement learning in Robotics"
03/11/2016 "Incremental, Sensor-Based Motion Generation for Mobile Manipulators in Unknown, Dynamic Environments", Peter Lehner, Arne Sieverling, Oliver Brock, ICRA, 2015.
03/18/2016 "LQR-trees: Feedback Motion Planning via Sums-of-Squares Verification", Tedrake et al., IJRR 2014.
03/25/2016 "Generating Multi-Fingered Robotic Grasps via Deep Learning", Jacob Varley, Jonathan Weisz, Jared Weiss, Peter Allen, IROS 2015.
04/01/2016"Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours", Lerrel Pinto and Abhinav Gupta, ICRA 2016.
04/08/2016 "Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection", Sergey Levine, Peter Pastor, Alex Krizhevsky, Deirdre Quillen.
Blog Post
04/22/2016 Planning Grasp Strategies That Exploit Environmental Constraints,Clemens Eppner and Oliver Brock, ICRA 2015.
04/29/2016 "Integrated Task and Motion Planning in Belief Space,Leslie Pack Kaelbling, Tomas Lozano-Perez, International Journal of Robotics Research, 2013
05/06/2016 QLAP - Autonomous Learning of High-Level States and Actions in Continuous Environments, Jonathan Mugan and Benjamin Kuipers
05/13/2016 "Feedback Controller Parameterizations for Reinforcement Learning", IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), 2011
05/20/2016 "Methods for Collision-Free Arm Teleoperation in Clutter Using Constraints from 3D Sensor Data", Adam Leeper, Kaijen Hsiao, Matei Ciocarlie, Ioan Sucan, and Kenneth Salisbury, Humanoids 2013.
05/27/2016 "Tactile manipulation with biomimetic active touch";Luke Cramphorn, Benjamin Ward-Cherrier, and Nathan F. Lepora; ICRA 2016.
06/03/2016 "Robot Grasping in Clutter:Using a Hierarchy of Supervisors for Learning from Demonstrations";Michael Laskey, Jonathan Lee, Caleb Chuck, David Gealy, Wesley Hsieh, Florian T. Pokorny, Anca D. Dragan, and Ken Goldberg; CASE 2016.
06/10/2016 Data-Driven Online Decision Making for Autonomous Manipulation", Daniel Kappler, Peter Pastor, Mrinal Kalakrishnan, Manuel Wuthrich, Stefan Schaal, RSS 2015.
06/17/2016 "Deep Learning for Tactile Understanding From Visual and Haptic Data",Yang Gao, Lisa Anne Hendricks, Katherine J. Kuchenbecker,and Trevor Darrell, ICRA 2016.
06/24/2016 "Kinematically Constrained Workspace Control via Linear Optimization"; Zachary K. Kingston, Neil T. Dantam, and Lydia E. Kavraki; Humanoids 2015.
code
07/01/2016 "Learning Relevant Features for Manipulation Skills using Meta-level Priors", Oliver Kroemer, Gaurav S. Sukhatme.
07/08/2016 "Efficient Grounding of Abstract Spatial Concepts for Natural Language Interaction with Robot Manipulators"
07/15/2016 "Approximate Inference Optimal Control"; Marc Toussaint; ICML 2009.
07/22/2016 "Motion Planning as Probabilistic Inference using Gaussian Processes and Factor Graphs"; Jing Dong, Mustafa Mukadam, Frank Dellaert, Byron Boots; RSS 2016.
07/29/2016 "Optimal Control with Learned Local Models: Application to Dexterous Manipulation"; Vikash Kumar, Emanuel Todorov, and Sergey Levine; ICRA 2016.
08/05/2016 "A factor graph approach to estimation and model predictive control on Unmanned Aerial Vehicles"; Duy-Nguyen Ta, Marin Kobilarov, and Frank Dellaert, ICUAS, 2014.
08/12/2016 "Automatic LQR Tuning Based on Gaussian Process Optimization: Early Experimental Results"; Alonso Marco, Philipp Hennig, Jeannette Bohg, Stefan Schaal,and Sebastian Trimpe; ICRA 2016.
08/19/2016 "Robust Trajectory Optimization Under Frictional Contact with Iterative Learning" Jingru Luo and Kris Hauser, RSS 2015.
09/15/2016 Manipulation with Fluids; ppt
09/22/2016 Legged locomotion control
09/29/2016 "Coordinated multi-arm motion planning: Reaching for moving objects in the face of uncertainty"; RSS 2016.
10/20/2016 Gaze prediction using machine learning for dynamic stereo manipulation in games
11/03/2016 Assembly Sequence Planning for Constructing Planar Structures with Rectangular Modules
11/17/2016 Human-level control through deep reinforcement learning
12/01/2016 Deep Recurrent Q-Learning for Partially Observable MDPs
01/23/2017 "A reservoir computing approach for learning forward dynamics of industrial manipulators"; Athanasios S. Polydoros and Lazaros Nalpantidis; IROS 2016.
01/30/2017 "Extended and Unscented Kitchen Sinks"; Edwin V. Bonilla, Daniel Steinberg, and Alistair Reid; ICML 2016.
02/06/2017 "Verification and Synthesis of Admissible Heuristics for Kinodynamic Motion Planning"; Brian Paden, Valerio Varricchio, and Emilio Frazzoli; IEEE Robotics and Automation Letters 2017.
02/21/2017 "Value Iteration Networks"; Aviv Tamar, Yi Wu, Garrett Thomas, Sergey Levine, and Pieter Abbeel; NIPS 2016.
03/07/2017 "From Object Categories to Grasp Transfer Using Probabilistic Reasoning"; Marianna Madry, Dan Song and Danica Kragic; ICRA 2012.
03/14/2017 "Tell Me Dave: Context-Sensitive Grounding of Natural Language to Manipulation Instructions"; Dipendra K Misra, Jaeyong Sung, Kevin Lee and Ashutosh Saxena; IJRR 2016.
03/21/2017 "Trajectory learning from human demonstrations via manifold mapping"; Hiratsuka, Michihisa, Ndivhuwo Makondo, Benjamin Rosman, and Osamu Hasegawa; IROS 2016.
04/04/2017 "Edward: A library for probabilistic modeling, inference, and criticism"; Dustin Tran, Alp Kucukelbir, Adji B. Dieng, Maja Rudolph, Dawen Liang, and David M. Blei; Archiv 2016.
04/11/2017 "Auto-encoding variational Bayes"; D.P. Kingma, M. Welling; ICLR 2014.
04/18/2017 "Modeling Grasp Motor Imagery through Deep Conditional Generative Models"; Matthew Veres, Medhat Moussa, Graham W. Taylor; IEEE Robotics and Automation Letters 2(2): 757-764, 2017.
04/25/2017 "Generative Adversarial Networks"; Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio; Neural Information Processing Systems 2014. "Generative Adversarial Imitation Learning"; Jonathan Ho, Stefano Ermon; Advances in Neural Information Processing Systems 2016.
05/08/2017 "MOSAIC Model for Sensorimotor Learning and Control"; Haruno, Masahiko, Daniel M. Wolpert, and Mitsuo Kawato; Neural Computation 2001.
05/15/2017 "Computationally Efficient Rigid-Body Gaussian Process for Motion Dynamics"; Muriel Lang and Sandra Hirche; IEEE Robotics and Automation Letters July 2017.
05/22/2017 A Comparison of Autoregressive Hidden Markov Models for Multimodal Manipulations With Variable Masses; Kroemer, Oliver and Peters, Jan; IEEE Robotics and Automation Letters 207.
06/05/2017 "Coding and use of tactile signals from the fingertips in object manipulation tasks"; Nature, 2009.
06/12/2017 "Human-Inspired Robotic Grasp Control with Tactile Sensing"; IEEE Transaction on Robotics, 2011.
06/19/2017 "One-Shot Learning of Manipulation Skills with Online Dynamics Adaptation and Neural Network Priors"; IROS, 2016 .
06/26/2017 "Information Theoretic MPC for Model-Based Reinforcement Learning"; Grady Williams, Nolan Wagener, Brian Goldfain, Paul Drews, James M. Rehg, Byron Boots, and Evangelos A. Theodorou; ICRA, 2017.
07/03/2017 No meeting for observing Independance day
07/10/2017 "Gaussian process implicit surfaces for shape estimation and grasping"; Stanimir Dragiev, Marc Toussaint, and Michael Gienger; ICRA, 2011.
07/31/2017 "Simultaneous Trajectory Estimation and Planning via Probabilistic Inference"; Mustafa Mukadam, Jing Dong, Frank Dellaert, and Byron Boots; RSS 2017
08/07/2017 "A Probabilistic Planning Framework for Planar Grasping Under Uncertainty"
08/14/2017 "A kernel-based approach to learning contact distributions for robot manipulation tasks"; Oliver Kroemer, Simon Leischnig, Stefan Luettgen, Jan Peters; Autonomous Robots, 2017.
08/24/2017 "Probabilistic Data Association for Semantic SLAM"; Sean L. Bowman, Nikolay Atanasov, Kostas Daniilidis, George J. Pappas; ICRA, 2017.
08/31/2017 “Model-Predictive Motion Planning”, Thomas Howard, Mihail Pivtoraiko, Ross A. Knepper, and Alonzo Kelly.IEEE Robotics and Automation Magazine, 21(1):64-73, March 2014.
09/07/2017 "Using Geometry to Detect Grasps in 3D Point Clouds", Andreas ten Pas, Robert Platt. agile grasp code
09/21/2017 "Planning How to Learn" Haoyu Bai David Hsu Wee Sun Lee; ICRA 2013.
09/28/2017 "Combined Optimization and Reinforcement Learning for Manipulation Skills", Peter Englert, Marc Toussaint, RSS 2016.
10/05/2017 "The Intentional Unintentional Agent:Learning to Solve Many Continuous Control Tasks Simultaneously", Cabi et al. (DeepMind), CoRL 2017.
10/19/2017 "An Analysis of Monte Carlo Tree Search"; S. James, G.D. Konidaris, and B. Rosman; AAAI 2017.
10/26/2017 "Learning in POMDPs with Monte Carlo Tree Search", Katt, Sammie and Oliehoek, Frans A and Amato, Christopher, ICML 2017.
11/2/2017 "The Infinite Regionalized Policy Representation", Liu, Miao and Liao, Xuejun and Carin, Lawrence, ICML 2011
11/9/2017 "Efficient planning in non-Gaussian belief spaces and its application to robot grasping" Platt, Robert and Kaelbling, Leslie and Lozano-Perez, Tomas and Tedrake, Russ, ISRR 2011
11/16/2017 "Bayesian Eigenobjects: A Unified Framework for 3D Robot Perception" Benjamin Burchfiel and George Konidaris, RSS 2017
11/30/2017 "Goal Set Inverse Optimal Control and Iterative Re-planning for Predicting Human Reaching Motions in Shared Workspaces" Jim Mainprice, Rafi Hayne and Dmitry Berenson, IEEE Transaction on Robotics (TRO), 2016
12/07/2017 "Planning and control for dynamic, nonprehensile, and hybrid manipulation tasks" J. Zachary Woodruff, Kevin M. Lynch, ICRA 2017 - IEEE International Conference on Robotics and Automation, 2017
01/10/2018 Writing exercise for all participants; bring pen and paper; no reading
01/17/2018 "The optimal control of partially observable Markov processes over the infinite horizon: Discounted costs", Sondik, Edward J, Operations research 1978
01/24/2018 "Motion planning under uncertainty using iterative local optimization in belief space", Van Den Berg, Jur and Patil, Sachin and Alterovitz, Ron, The International Journal of Robotics Research (IJRR); 2012.
01/31/2018 "Integrated Perception and Planning in the Continuous Space: A POMDP Approach", Haoyu Bai, David Hsu, Wee Sun Lee; RSS 2013
02/07/2018 "Belief space planning simplified: Trajectory-optimized lqg (t-lqg)", Rafieisakhaei, Mohammadhussein and Chakravorty, Suman and Kumar, PR, arXiv preprint 2016
02/14/2018 "Dual Execution of Optimized Contact Interaction Trajectories", Marc Toussaint, Nathan Ratliff, Jeannette Bohg, Ludovic Righetti, Peter Englert, Stefan Schaal; IROS 2014
02/21/2018 "A Probabilistic Particle-Control Approximation of Chance-Constrained Stochastic Predictive Control", Lars Blackmore, Masahiro Ono, Askar Bektassov, and Brian C. Williams; TRO 2010.
02/28/2018 "Pose estimation for planar contact manipulation with manifold particle filters", Koval, Michael C and Pollard, Nancy S and Srinivasa, Siddhartha S, The International Journal of Robotics Research (IJRR) 2015
03/14/2018 "Pre-and post-contact policy decomposition for planar contact manipulation under uncertainty", Koval, Michael C and Pollard, Nancy S and Srinivasa, Siddhartha S, Robotics: Science and Systems 2014
03/21/2018 "Randomized Physics-based Motion Planning for Grasping in Cluttered and Uncertain Environments"; Muhayyuddin, Mark Moll, Lydia Kavraki, Jan Rosell; RA-L 2017.
03/28/2018 "Extending the Applicability of POMDP Solutions to Robotic Tasks"; Devin K. Grady, Mark Moll, Lydia E. Kavraki; IEEE Transactions on Robotics , 2015.
04/04/2018 "Learning Robot Objectives from Physical Human Interaction", Bajcsy, Andrea and Losey, Dylan P and O’Malley, Marcia K and Dragan, Anca D; Conference on Robot Learning ;2017.
04/11/2018 "Robust trajectory selection for rearrangement planning as a multi-armed bandit problem", Koval, Michael C and King, Jennifer E and Pollard, Nancy S and Srinivasa, Siddhartha S, Intelligent Robots and Systems (IROS) 2015
04/18/2018 "Model-based Bayesian reinforcement learning in partially observable domains", Poupart, Pascal and Vlassis, Nikos, Proc Int. Symp. on Artificial Intelligence and Mathematics 2008
04/25/2018 An Online and Approximate Solver for POMDPs with Continuous Action Space; K. Seiler and H. Kurniawati and S.P.N. Singh. ; Proc. IEEE Int. Conference on Robotics and Automation (ICRA). 2015. (Best Conference Paper Award Finalist)
05/02/2018 Deep Visual Foresight for Planning Robot Motion; Chelsea Finn and Sergey Levine; ICRA 2017.
05/09/2018 "POND-Hindsight: Applying Hindsight Optimization to POMDPs", Alan Olsen, Daniel Bryce, and "Shared autonomy via hindsight optimization", Javdani, Shervin and Srinivasa, Siddhartha S and Bagnell, J Andrew, arXiv preprint 2017
05/16/2018 Maximum Entropy Inverse Reinforcement Learning, Brian D. Ziebart, Andrew Maas, J.Andrew Bagnell, and Anind K. Dey, AAAI 2008
05/23/2018 Modeling Interaction via the Principle of Maximum Causal Entropy, Ziebart 2010
05/30/2018 Inverse Reinforcement Learning with PI^2 (short support paper) and Learning Objective Functions for Manipulation, Kalakrishnan, M., Pastor, P., Righetti, L. and Schaal, S., ICRA 2013
06/06/2018 Maximum Entropy Deep Inverse Reinforcement Learning, Wulfmeier, M., Ondruska, P. and Posner, I., arXiv preprint 2015
06/13/2018 Relative Entropy Inverse Reinforcement Learning, Boularias, A., Kober, J. and Peters, J., ICAIS 2011
06/20/2018 Compatible Reward Inverse Reinforcement Learning , Metelli, Alberto Maria, Matteo Pirotta, and Marcello Restelli, NIPS 2017
06/27/2018 Inverse Reinforcement Learning via Nonparametric Spatio-Temporal Subgoal Modeling, Šošić, A., Rueckert, E., Peters, J., Zoubir, A. M., & Koeppl, H., arXiv preprint 2018