US 12,008,665 B2
Methods and system for sensor-based layout generation
Arjun Mahanti, Brooklyn, NY (US); Matthew T. Kursmark, Brooklyn, NY (US); and Brian Stegall, Fox Point, WI (US)
Assigned to Block, Inc., Oakland, CA (US)
Filed by Block, Inc., San Francisco, CA (US)
Filed on Sep. 30, 2020, as Appl. No. 17/039,542.
Prior Publication US 2022/0101410 A1, Mar. 31, 2022
Int. Cl. G06Q 50/12 (2012.01); G06Q 10/02 (2012.01); G06Q 20/20 (2012.01); G06Q 30/0601 (2023.01); G06T 19/00 (2011.01)
CPC G06Q 50/12 (2013.01) [G06Q 10/02 (2013.01); G06Q 20/20 (2013.01); G06T 19/006 (2013.01); G06Q 30/0601 (2013.01)] 17 Claims
OG exemplary drawing
 
1. A method comprising:
generating data of a physical establishment of a merchant using one or more LiDAR sensors of a merchant device;
collecting, by the merchant device, an amount and a timing of a glare or reflection of at least one light emitted by the one or more LiDAR sensors;
analyzing the data to identify at least a first surface and a second surface as possible seating areas within the physical establishment, wherein analyzing the data comprises identifying at least the first surface and the second surface within the physical establishment based on a level of glare or reflection calculated from the amount and the timing of the glare or reflection of the at least one light;
determining, based at least in part on the amount and timing of the glare or reflection of the at least one light, one or more first coordinates of a first location of the first surface in the physical establishment and one or more second coordinates of a second location of the second surface in the physical establishment, wherein determining at least the first location and the second location comprises inputting the analyzed data to a first machine-trained model and receiving output of at least the first surface at the one or more first coordinates of the first location and the second surface at the one or more second coordinates of the second location, wherein the first machine-trained model is trained using historical data indicating at least one of size, flatness, or planar orientation of a plurality of items including seating areas;
presenting, on a display of the merchant device of the merchant, a graphical user interface comprising a representation of at least a portion of the physical establishment overlaid with: (i) a first indication that the first surface has been identified at the one or more first coordinates of the first location, and (ii) a second indication that the second surface has been identified at the one or more second coordinates of the second location;
receiving, via the graphical user interface, a first selection of the first surface, the first selection comprising a verification that the first surface represents a seating area;
receiving, via the graphical user interface, a second selection of the second surface, the second selection comprising an indication that the second surface does not represent a seating area;
generating, based at least in part on the first selection and the second selection, a representation of a current layout of the physical establishment including a representation of the seating area, wherein the representation of the current layout is generated based on the one or more first coordinates of the first location of the first surface and the one or more second coordinates of the second location of the second surface;
presenting the representation of the current layout via the graphical user interface;
based at least in part on inputting the representation of the current layout to a second machine-trained model, generating a representation of a suggested layout of seating areas in the physical establishment based on the one or more first coordinates of the first location of the first surface and the one or more second coordinates of the second location of the second surface, the representation of the suggested layout indicating that the first surface is a seating area at the first location, wherein the suggested layout represents at least a change in quantity or arrangement of seating areas; and
presenting the representation of the suggested layout on the display of the merchant device.