US 12,008,790 B2
Encoding three-dimensional data for processing by capsule neural networks
Nitish Srivastava, San Francisco, CA (US); Ruslan Salakhutdinov, Pittsburgh, PA (US); and Hanlin Goh, Sunnyvale, CA (US)
Assigned to APPLE INC., Cupertino, CA (US)
Filed by Apple Inc., Cupertino, CA (US)
Filed on Mar. 31, 2020, as Appl. No. 16/836,028.
Claims priority of provisional application 62/904,890, filed on Sep. 24, 2019.
Prior Publication US 2021/0090302 A1, Mar. 25, 2021
Int. Cl. G06T 9/00 (2006.01); G06N 3/047 (2023.01); G06N 3/08 (2023.01); H04N 13/111 (2018.01); H04N 13/161 (2018.01)
CPC G06T 9/002 (2013.01) [G06N 3/047 (2023.01); G06N 3/08 (2013.01); H04N 13/111 (2018.05); H04N 13/161 (2018.05)] 30 Claims
OG exemplary drawing
 
1. A method, comprising:
defining a geometric capsule, wherein the geometric capsule is an encoded form of three-dimensional data that is interpretable by a capsule neural network, and the geometric capsule includes a three-dimensional feature representation and a pose;
determining multiple viewpoints relative to the geometric capsule;
determining a first appearance representation of the geometric capsule for each of the multiple viewpoints based on the three-dimensional feature representation for the geometric capsule;
determining a transform for each of the multiple viewpoints that moves each of the multiple viewpoints to a respective transformed viewpoint;
determining second appearance representations that each correspond to one of the transformed viewpoints based on the three-dimensional feature representation for the geometric capsule;
combining the second appearance representations to define an agreed appearance representation; and
updating the three-dimensional feature representation for the geometric capsule based on the agreed appearance representation.