| CPC G06F 30/13 (2020.01) [G06T 7/11 (2017.01); G06T 7/55 (2017.01); G06T 7/70 (2017.01); G06T 17/05 (2013.01); H04N 23/698 (2023.01); G06F 30/12 (2020.01); G06T 2200/08 (2013.01); G06T 2200/24 (2013.01); G06T 2207/20021 (2013.01); G06T 2207/20212 (2013.01); G06T 2207/30244 (2013.01)] | 26 Claims |
|
1. A computer-implemented method comprising:
recording, by a mobile computing device having one or more cameras and under control of a user, a plurality of first panorama images at a plurality of first acquisition locations in a plurality of first rooms of a building that are a subset of multiple rooms of the building, wherein each of the first panorama images is in a spherical format and includes 360 degrees of horizontal visual coverage around a vertical axis and has visual coverage of at least some walls of one of the first rooms that contains the respective first acquisition location for that first panorama image;
analyzing, by the mobile computing device, visual data of the first panorama images to generate room shapes of the first rooms, including:
generating, by the mobile computing device and for each of the first panorama images, visual data of that first panorama image to generate an initial room shape of the one room in which the respective first acquisition location for that first image is located and that includes representations of the at least some walls of that one room, including determining positions of at least one of a doorway or a non-doorway wall opening in the one room, and further including determining uncertainty information for positioning of at least one wall of the at least some walls of that one room;
combining, by the mobile computing device, and for each of at least one room of the first rooms that contains at least two acquisition locations of the first acquisition locations, the generated initial room shapes for the first panorama images acquired at the at least two acquisition locations to produce the room shape of that at least one room, including to reduce the determined uncertainty information for at least one wall of the produced room shape based on the combining; and
selecting, by the mobile computing device, and for each of at least one other room of the first rooms that contains only one acquisition location of the first acquisition locations, the generated initial room shape for the first panorama image acquired at that only one acquisition location to produce the room shape of that at least one other room;
generating, by the mobile computing device, a partial floor plan for the building that includes the room shapes of the first rooms positioned relative to each other, including:
determining, by the mobile computing device, relative positioning of the first rooms based at least in part on visual overlap of the first panorama images;
positioning, by the mobile computing device, and using the determined relative positioning and the generated room shapes for the first rooms, the generated room shapes of the first rooms relative to each other;
generating, by the mobile computing device, a binary segmentation mask of an area of the positioned generated room shapes of the first rooms, including determining separate binary segmentation masks for each of the first rooms, and combining the separate binary segmentation masks;
determining, by the mobile computing device, a polygon shape for the generated binary segmentation mask that represents the first rooms positioned relative to each other; and
adding, by the mobile computing device, information to the determined polygon shape to identify positions of walls and doorways and non-doorway wall openings of the first rooms, including adding information to the partial floor plan about uncertainty information about positioning of one or more walls;
determining, by the mobile computing device and based at least in part on the generated partial floor plan, guidance related to additional acquisition locations for the building at which to acquire additional images, including using the uncertainty information about the positioning of the one or more walls to identify at least one of the additional acquisition locations that will provide further visual data to reduce uncertainty about the positioning of the one or more walls;
presenting, by the mobile computing device and to the user in a graphical user interface (GUI) on the mobile computing device, a visual representation of the generated partial floor plan that includes visual indications of the uncertainty information about the positioning of the one or more walls, and visual indications overlaid on the generated partial floor plan related to the additional acquisition locations, and directions to the user related to the additional acquisition locations;
recording, by the mobile computing device and after the presenting, a plurality of additional second panorama images at a plurality of second acquisition locations in additional second rooms of the building, wherein the second acquisition locations include the additional acquisition locations;
analyzing, by the mobile computing device, additional visual data of the second panorama images to generate additional room shapes of the additional second rooms;
generating, by the mobile computing device, a completed floor plan for the building that includes the additional room shapes of the additional second rooms positioned relative to each other and to the room shapes in the generated partial floor plan of the first rooms, including generating and using segmentation masks for the additional room shapes; and
presenting, by the mobile computing device and to the user in the GUI, an updated visual representation of the generated completed floor plan, to enable further navigation of the building.
|