US 12,071,323 B2
Contactless elevator service for an elevator based on augmented datasets
Zafer Sahinoglu, Costa Mesa, CA (US); Kuan-Chuan Peng, Charlestown, MA (US); Alan Sullivan, Middleton, MA (US); and William Yerazunis, Acton, MA (US)
Assigned to Mitsubishi Electric Research Laboratories, Inc., Cambridge, MA (US)
Filed by Mitsubishi Electric Research Laboratories, Inc., Cambridge, MA (US)
Filed on Aug. 17, 2020, as Appl. No. 16/994,779.
Prior Publication US 2022/0048733 A1, Feb. 17, 2022
Int. Cl. B66B 1/46 (2006.01); B66B 1/28 (2006.01); B66B 1/34 (2006.01); B66B 3/00 (2006.01); G06F 3/14 (2006.01); G06F 18/214 (2023.01); G06N 3/04 (2023.01); G06N 3/08 (2023.01); G06V 10/44 (2022.01); G06V 20/62 (2022.01); H04N 23/54 (2023.01); G06V 30/10 (2022.01)
CPC B66B 1/468 (2013.01) [B66B 1/28 (2013.01); B66B 1/3461 (2013.01); B66B 3/002 (2013.01); G06F 3/14 (2013.01); G06F 18/214 (2023.01); G06N 3/04 (2013.01); G06N 3/08 (2013.01); G06V 10/44 (2022.01); G06V 20/62 (2022.01); H04N 23/54 (2023.01); B66B 2201/101 (2013.01); B66B 2201/103 (2013.01); B66B 2201/4615 (2013.01); B66B 2201/4623 (2013.01); B66B 2201/4638 (2013.01); B66B 2201/4653 (2013.01); G06V 30/10 (2022.01)] 30 Claims
OG exemplary drawing
 
1. A device for real-time contactless elevator service operation of an elevator, comprising:
a memory to store data, the data includes a trained neural network (TNN) model, wherein the TNN model is trained using augmented datasets as a synthetic training dataset via a training processor, to later perform elevator identifier recognition, such that the augmented data sets are generated from synthetic text images for each training elevator identifier of a set of training elevator identifiers for an elevator that corresponds to a training elevator template of a set of training elevator templates by an image transformation on the synthetic text images, wherein the synthetic text images are augmented with different geometrical parameters and visual parameters to a predetermined number of variations in appearance to the set of training elevator identifiers;
an image sensor to capture in real-time one or more images from a user;
a processor in real-time is to:
extract a text image portion from the one or more captured images using the TNN model, and detect at least an elevator identifier in the extracted text image portion using the extracted text image portion and the TNN model;
display the detected elevator identifier on a display for user confirmation or user cancellation, and upon user confirmation, generate a control command based the detected elevator identifier associated elevator service; and
output the control command to activate or not activate the elevator service based on the control command.