US 11,883,947 B2
Machine learning enabled visual servoing with dedicated hardware acceleration
Heiko Claussen, Wayland, MA (US); Martin Sehr, Kensington, CA (US); Eugen Solowjow, Berkeley, CA (US); Chengtao Wen, Redwood City, CA (US); and Juan L. Aparicio Ojea, Moraga, CA (US)
Assigned to SIEMENS AKTIENGESELLSCHAFT, Munich (DE)
Appl. No. 17/761,613
Filed by Siemens Aktiengesellschaft, Munich (DE)
PCT Filed Sep. 30, 2019, PCT No. PCT/US2019/053777
§ 371(c)(1), (2) Date Mar. 18, 2022,
PCT Pub. No. WO2021/066794, PCT Pub. Date Apr. 8, 2021.
Prior Publication US 2022/0347853 A1, Nov. 3, 2022
Int. Cl. B25J 9/16 (2006.01); B25J 13/08 (2006.01); G06F 30/10 (2020.01)
CPC B25J 9/1697 (2013.01) [B25J 13/08 (2013.01); G06F 30/10 (2020.01)] 14 Claims
OG exemplary drawing
 
1. A system controller for visual servoing, comprising:
a technology module comprising an accelerator processor for machine learning configured to:
retrieve a desired pose configuration of a workpiece object being manipulated in its pose by a robotic device;
iteratively receive from one or more sensors on or near the robotic device visual feedback information that includes a current pose configuration of the workpiece object; and
iteratively execute a machine learning model trained to process the visual feedback information and determine a low dimensional pose configuration error based on a difference between the current pose configuration of the workpiece object and the desired pose configuration of the workpiece object;
a pre-processing module configured to build a 3D CAD model of the workpiece object based on the visual feedback information on a condition that visual feedback information indicates that the object is unknown to the system controller, wherein the machine learning model is configured to use the 3D CAD model as training input to refine the machine learning model; and
a servo control module configured to iteratively adapt a servo control signal to the robotic device for manipulation of the workpiece object in response to the pose configuration error.