US 12,380,652 B2
Floorplan generation based on room scanning
Feng Tang, Cupertino, CA (US); Afshin Dehghan, Sunnyvale, CA (US); Kai Kang, San Jose, CA (US); Yang Yang, Sunnyvale, CA (US); Yikang Liao, Sunnyvale, CA (US); and Guangyu Zhao, Bejing (CN)
Assigned to Apple Inc., Cupertino, CA (US)
Filed by Apple Inc., Cupertino, CA (US)
Filed on Nov. 15, 2022, as Appl. No. 17/987,256.
Application 17/987,256 is a continuation of application No. 17/146,582, filed on Jan. 12, 2021, granted, now 11,551,422.
Claims priority of provisional application 62/962,485, filed on Jan. 17, 2020.
Prior Publication US 2023/0075601 A1, Mar. 9, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G06T 19/00 (2011.01); G06F 18/21 (2023.01); G06F 18/214 (2023.01); G06F 18/24 (2023.01); G06N 3/045 (2023.01); G06T 7/50 (2017.01); G06T 7/73 (2017.01); G06T 11/00 (2006.01); G06T 15/20 (2011.01); G06T 19/20 (2011.01); G06V 10/22 (2022.01); G06V 20/00 (2022.01); G06V 20/10 (2022.01); G06V 20/64 (2022.01); G06V 30/262 (2022.01); H04N 7/18 (2006.01)
CPC G06T 19/00 (2013.01) [G06F 18/21 (2023.01); G06F 18/2148 (2023.01); G06F 18/217 (2023.01); G06F 18/24 (2023.01); G06N 3/045 (2023.01); G06T 7/50 (2017.01); G06T 7/73 (2017.01); G06T 11/00 (2013.01); G06T 15/205 (2013.01); G06T 19/003 (2013.01); G06T 19/20 (2013.01); G06V 10/22 (2022.01); G06V 20/10 (2022.01); G06V 20/36 (2022.01); G06V 20/64 (2022.01); G06V 30/274 (2022.01); H04N 7/183 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2210/04 (2013.01); G06T 2210/12 (2013.01); G06T 2210/56 (2013.01)] 17 Claims
OG exemplary drawing
 
1. A method comprising:
at an electronic device having a processor:
displaying a live camera feed comprising a sequence of images of a physical environment;
obtaining a three-dimensional (3D) representation of the physical environment that was generated based on depth data and light intensity image data of the physical environment obtained during the displaying of the live camera feed;
generating a live preview of a first floorplan of the physical environment based on the 3D representation of the physical environment utilizing a first process, wherein the first process for generating the first floorplan comprises generating an edge map by:
identifying walls in the physical environment based on the 3D representation;
identifying wall attributes in the physical environment based on the 3D representation;
identifying objects in the physical environment based on the 3D representation; and
generating the live preview of the first floorplan based on the edge map that includes the identified walls, the identified wall attributes, and the identified objects;
displaying the live preview of the first floorplan concurrently with the live camera feed; and
generating a second floorplan of the physical environment based on the 3D representation of the physical environment utilizing a second process that is different than the first process, wherein the second process comprises:
refining the identified walls and the identified wall attributes;
identifying the objects in the physical environment based on the 3D representation and generating 3D object representations corresponding to the identified objects; and
generating the second floorplan based on the refined identified walls, the refined identified wall attributes, and the 3D object representations associated with the identified objects.