US 12,333,655 B2
Presenting building information using video and building models
Yuguang Li, Seattle, WA (US); Ivaylo Boyadzhiev, Seattle, WA (US); Romualdo Impas, Seattle, WA (US); Michael P. Caputo, Aurora, IL (US); Gregory J. Doerr, Palatine, IL (US); and Joshua Weisberg, Anacortes, WA (US)
Assigned to MFTB Holdco, Inc., Seattle, WA (US)
Filed by MFTB Holdco, Inc., Seattle, WA (US)
Filed on Feb. 21, 2023, as Appl. No. 18/112,352.
Application 18/112,352 is a continuation in part of application No. 17/589,842, filed on Jan. 31, 2022, granted, now 11,676,344.
Application 17/589,842 is a continuation in part of application No. 17/064,608, filed on Oct. 7, 2020, granted, now 11,238,652, issued on Feb. 1, 2022.
Application 17/064,608 is a continuation of application No. 16/681,787, filed on Nov. 12, 2019, granted, now 10,825,247, issued on Nov. 3, 2020.
Application 18/112,352 is a continuation in part of application No. 18/099,892, filed on Jan. 20, 2023.
Application 18/099,892 is a continuation of application No. 16/852,948, filed on Apr. 20, 2020, granted, now 11,574,473, issued on Feb. 7, 2023.
Prior Publication US 2023/0196684 A1, Jun. 22, 2023
Int. Cl. G06T 19/00 (2011.01); G06F 3/0482 (2013.01); G06T 15/50 (2011.01); G06T 17/05 (2011.01)
CPC G06T 19/003 (2013.01) [G06F 3/0482 (2013.01); G06T 15/506 (2013.01); G06T 17/05 (2013.01); G06T 19/006 (2013.01)] 32 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
selecting, by one or more computing systems, multiple types of information about an indicated building to simultaneously include in a displayed graphical user interface (GUI), the multiple types of information including a video having at least visual data from a succession of multiple locations at the indicated building, and further including one or more additional types of information that include a visual representation of a floor map of the indicated building; and
presenting, by the one or more computing systems and in the displayed GUI, the video and the one or more additional types of information, including presenting the video in a first pane of the GUI in a substantially real-time manner relative to capturing of the video by a presenter device at the indicated building that is moving through the indicated building and transmitting the video as it is captured, and including presenting the visual representation of the floor map in a second pane of the GUI simultaneously with the video and in a coordinated manner in which the visual representation of the floor map is repeatedly updated to reflect current visual data of the video including to visually indicate the succession of multiple locations, the presenting including:
(a) receiving, by the one or more computing systems, one or more current frames of the video from the presenter device for a current one of the multiple locations;
(b) analyzing, by the one or more computing systems, visual data of the one or more current frames to identify at least one of a room type of a room of the indicated building that is visible in the visual data of the one or more current frames, or a feature type for a structural element of the indicated building that is visible in the visual data of the one or more current frames;
(c) displaying, by the one or more computing systems, the visual data of the one or more current frames in the first pane, and a current version of the visual representation of the floor map in the second pane concurrently with the video, wherein the current version of the visual representation of the floor map includes at least a portion of the floor map of the indicated building showing a position of the current one location on the at least portion of the floor map and showing at least two-dimensional structural information for at least some of the indicated building;
(d) overlaying, by the one or more computing systems and on at least one of the first or second panes, a determined textual identifier for the identified at least one of the room type or the feature type; and
(e) repeating elements (a)-(d) until presentation of the video is ended.