US 11,688,092 B2
Systems and methods for identifying package properties in an automated industrial robotics system
Shaun Edwards, San Antion, TX (US); Erik Nieves, San Antonio, TX (US); Paul Hvass, San Antonio, TX (US); and Joshua Justin Curtis, San Antonio, TX (US)
Assigned to Plus One Robotics, Inc., San Antonio, TX (US)
Filed by Plus One Robotics, Inc., San Antonio, TX (US)
Filed on Feb. 22, 2021, as Appl. No. 17/180,970.
Application 17/180,970 is a continuation of application No. 16/846,331, filed on Apr. 11, 2020, granted, now 10,949,635.
Claims priority of provisional application 62/832,587, filed on Apr. 11, 2019.
Prior Publication US 2021/0174039 A1, Jun. 10, 2021
Int. Cl. G06T 7/60 (2017.01); G05B 19/042 (2006.01); G06K 7/14 (2006.01); G06K 7/10 (2006.01); B65G 1/137 (2006.01); B65G 43/08 (2006.01)
CPC G06T 7/60 (2013.01) [B65G 1/137 (2013.01); B65G 43/08 (2013.01); G05B 19/042 (2013.01); G06K 7/10445 (2013.01); G06K 7/10861 (2013.01); G06K 7/1404 (2013.01); G06K 7/1443 (2013.01); G05B 2219/25174 (2013.01); G05B 2219/25186 (2013.01); G05B 2219/25252 (2013.01); G05B 2219/25257 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computing system, comprising:
a plurality of cameras configured to capture two-dimensional (2D) data in a package conveyance system, individual cameras being arranged at different positions with respect to markers within the package conveyance system;
a plurality of depth detection cameras configured to capture depth data in the package conveyance system, individual depth detection cameras being arranged at different positions with respect to the markers, the 2D data and the depth data captured substantially simultaneously;
at least one computing processor; and
memory including instructions that, when executed by the at least one computing processor, enable the computing system to:
obtain an interrupt notification, the interrupt notification indicating that an item is positioned within a detection region of an interrupt sensor;
obtain 2D data from a 2D camera, the 2D data including a representation of the item;
obtain 3D data from a depth detection camera, the 3D data including depth data associated with the item;
transform the 2D data and 3D data to a reference 3D coordinate space based on positions of the plurality of cameras and positions of the plurality of depth detection cameras to associate 3D coordinates with geometric features represented in the 2D data relative to the reference 3D coordinate space; and
determine item properties associated with the item based on the geometric features.
 
11. A computer-implemented method, comprising:
obtaining an interrupt notification, the interrupt notification indicating that an item is positioned within a detection region of an interrupt sensor;
obtaining two-dimensional (2D) data from a 2D camera of a plurality of cameras configured to capture 2D data in a package conveyance system, individual cameras being arranged at different positions with respect to markers within the package conveyance system, the 2D data including a representation of the item;
obtaining three-dimensional (3D) data from a depth detection camera of a plurality of depth detection cameras configured to capture depth data in the package conveyance system, individual depth detection cameras being arranged at different positions with respect to the markers, the 2D data and the depth data captured substantially simultaneously, the 3D data including depth data associated with the item;
transforming the 2D data and 3D data to a reference 3D coordinate space based on positions of the plurality of cameras and positions of the plurality of depth detection cameras to associate 3D coordinates with geometric features represented in the 2D data relative to the reference 3D coordinate space; and
determining item properties associated with the item based on the geometric features.
 
17. A non-transitory computer readable storage medium storing one or more sequences of instructions executable by one or more processors to perform a set of operations comprising:
obtaining an interrupt notification, the interrupt notification indicating that an item is positioned within a detection region of an interrupt sensor;
obtaining two-dimensional (2D) data from a 2D camera of a plurality of cameras configured to capture 2D data in a package conveyance system, individual cameras being arranged at different positions with respect to markers within the package conveyance system, the 2D data including a representation of the item;
obtaining three-dimensional (3D) data from a depth detection camera of a plurality of depth detection cameras configured to capture depth data in the package conveyance system, individual depth detection cameras being arranged at different positions with respect to the markers, the 2D data and the depth data captured substantially simultaneously, the 3D data including depth data associated with the item;
transforming the 2D data and 3D data to a reference 3D coordinate space based on positions of the plurality of cameras and positions of the plurality of depth detection cameras to associate 3D coordinates with geometric features represented in the 2D data relative to the reference 3D coordinate space; and
determining item properties associated with the item based on the geometric features.