US 11,776,420 B2
Augmented reality in wire harness installation
David Balfour, Greensboro, NC (US)
Assigned to B/E Aerospace, Inc., Winston-Salem, NC (US)
Filed by B/E Aerospace, Inc., Winston-Salem, NC (US)
Filed on Feb. 12, 2021, as Appl. No. 17/174,822.
Prior Publication US 2022/0262271 A1, Aug. 18, 2022
Int. Cl. G09B 19/00 (2006.01); G06V 20/20 (2022.01); G06T 19/00 (2011.01)
CPC G09B 19/003 (2013.01) [G06T 19/006 (2013.01); G06V 20/20 (2022.01)] 14 Claims
OG exemplary drawing
 
1. A computer apparatus comprising:
at least one processor in data communication with a memory storing processor executable code for configuring the at least one processor to:
receive one or more image streams;
identify one or more landmarks in each image stream, the one or more landmarks each corresponding to a fixed point on an aircraft seat frame;
orient and locate a 3D component model with respect to the landmarks;
retrieve a sequence of major installation steps and one or more sequences of interstitial installation steps, the major installation steps corresponding to wire harness placement points and each of the sequences of interstitial installation steps corresponding to individual fixation steps for one of the major installation steps;
render an overlay of the 3D component model to appear at a real-world location of a corresponding installation component during a major installation step in a sequence of major installation steps;
determine that the major installation step is complete via an object recognition algorithm operating on the one or more image streams to determine that the corresponding installation component occupies a space corresponding to the 3D component model;
render an overlay of a subsequent 3D component model to appear at a real-world location of a corresponding installation component during a subsequent major installation step in the sequence of major installation steps;
receive a request from an installer for a detailed installation step;
render a high-detail 3D component model; and
render the sequence of interstitial installation steps corresponding to the current major installation step.