US 12,304,072 B2
Reinforcement learning for contact-rich tasks in automation systems
Eugen Solowjow, Berkeley, CA (US); Juan L. Aparicio Ojea, Moraga, CA (US); Chengtao Wen, Redwood City, CA (US); and Jianlan Luo, Berkeley, CA (US)
Assigned to Siemens Aktiengesellschaft, Munich (DE); and The Regents of the University of California, Berkeley, CA (US)
Appl. No. 16/970,450
Filed by Siemens Aktiengesellschaft, Munich (DE); and THE REGENTS OF THE UNIVERSITY OF CALIFORNIA, Berkeley, CA (US)
PCT Filed Sep. 13, 2018, PCT No. PCT/US2018/050862
§ 371(c)(1), (2) Date Aug. 17, 2020,
PCT Pub. No. WO2019/168563, PCT Pub. Date Sep. 6, 2019.
Claims priority of provisional application 62/635,757, filed on Feb. 27, 2018.
Claims priority of provisional application 62/635,771, filed on Feb. 27, 2018.
Prior Publication US 2021/0107142 A1, Apr. 15, 2021
Int. Cl. B25J 9/16 (2006.01)
CPC B25J 9/163 (2013.01) [B25J 9/1633 (2013.01); G05B 2219/32335 (2013.01); G05B 2219/41387 (2013.01); G05B 2219/50391 (2013.01)] 8 Claims
OG exemplary drawing
 
1. A process performed by a robot control system, comprising:
executing a program to control a robot by the robot control system;
receiving robot state information by the robot control system;
receiving force torque feedback inputs from a sensor on the robot by the robot control system;
producing a robot control command for the robot using a guided policy search process, by the robot control system, based on the robot state information and the force torque feedback inputs and on a reference signal of a neural network, wherein the reference signal is based on the robot state information and the force torque feedback inputs, the force torque feedback inputs being added to a next-to-last layer of the neural network at force torque information nodes; and
controlling the robot using the robot control command, by the robot control system.