US 11,775,835 B2
Room layout estimation methods and techniques
Chen-Yu Lee, Sunnyvale, CA (US); Vijay Badrinarayanan, Mountain View, CA (US); Tomasz Jan Malisiewicz, Mountain View, CA (US); and Andrew Rabinovich, San Francisco, CA (US)
Assigned to MAGIC LEAP, INC., Plantation, FL (US)
Filed by Magic Leap, Inc., Plantation, FL (US)
Filed on Apr. 9, 2020, as Appl. No. 16/844,812.
Application 16/844,812 is a continuation of application No. 15/923,511, filed on Mar. 16, 2018, granted, now 10,657,376.
Claims priority of provisional application 62/473,257, filed on Mar. 17, 2017.
Prior Publication US 2020/0234051 A1, Jul. 23, 2020
This patent is subject to a terminal disclaimer.
Int. Cl. G06N 3/084 (2023.01); G06N 3/04 (2023.01); G06V 20/00 (2022.01); G06V 10/44 (2022.01); G06V 10/94 (2022.01); G06V 20/20 (2022.01); G06F 18/214 (2023.01); G06F 18/2413 (2023.01); G06N 3/044 (2023.01); G06N 3/045 (2023.01); G06V 10/764 (2022.01); G06V 10/774 (2022.01); G06V 10/82 (2022.01); G06N 3/082 (2023.01)
CPC G06N 3/084 (2013.01) [G06F 18/214 (2023.01); G06F 18/2413 (2023.01); G06N 3/04 (2013.01); G06N 3/044 (2023.01); G06N 3/045 (2023.01); G06V 10/44 (2022.01); G06V 10/454 (2022.01); G06V 10/764 (2022.01); G06V 10/774 (2022.01); G06V 10/82 (2022.01); G06V 10/95 (2022.01); G06V 20/20 (2022.01); G06V 20/36 (2022.01); G06N 3/082 (2013.01)] 24 Claims
OG exemplary drawing
 
1. A system comprising:
non-transitory memory configured to store:
an image of a room; and
a neural network comprising:
an encoder-decoder sub-network comprising an encoder and a decoder; and
a classifier sub-network in communication with the encoder-decoder sub-network configured to classify a room type associated with the image;
a hardware processor in communication with the non-transitory memory, the hardware processor programmed to:
access the image;
predict, using the encoder, decoder, and the image, two-dimensional (2D) keypoint maps corresponding to room types;
determine, using the encoder, the classifier sub-network, and the image, a predicted room type from the room types; and
determine a predicted layout of the room in the image based at least in part on the 2D keypoint maps and the predicted room type.