US 11,886,647 B2
In-air gesture control method based on visible light signals
Fan Wu, Shanghai (CN); Zimo Liao, Shanghai (CN); and Guihai Chen, Shanghai (CN)
Assigned to SHANGHAI JIAOTONG UNIVERSITY, Shanghai (CN)
Filed by SHANGHAI JIAOTONG UNIVERSITY, Shanghai (CN)
Filed on Sep. 21, 2022, as Appl. No. 17/949,951.
Prior Publication US 2023/0096340 A1, Mar. 30, 2023
Int. Cl. G06F 3/01 (2006.01); G06F 18/2413 (2023.01); G06F 3/03 (2006.01)
CPC G06F 3/017 (2013.01) [G06F 3/0325 (2013.01); G06F 18/24147 (2023.01)] 9 Claims
OG exemplary drawing
 
1. An in-air gesture control method based on visible light signals
comprising a light signal is emitted through a display device and its reflection signal with gestures is collected, and after analysis, gesture recognition is realized and used to generate corresponding control instructions, wherein: emitted light signals are in visible light bands;
a spatial resolution information in the display device to reduce user's visual effect;
wherein the analysis comprises the following steps:
i) performing short-time Fourier transform on reflected signals to convert data in time domain into frequency domain, amplitude sequence of f0=0 Hz, f1=

OG Complex Work Unit Math
and

OG Complex Work Unit Math
are effective features;
ii) segmenting the sequence of

OG Complex Work Unit Math
amplitudes in the frequency domain; when the amplitude of the frequency light signal is greater than the threshold, a gesture is detected, then the feature sequence needs to be processed and gesture recognition is performed; otherwise, the feature sequence is skipped and gesture recognition is not required; and
iii) after selecting the feature sequence, K-nearest neighbor classification method is used to classify and realize gesture recognition, specifically: choose the amplitude sequence E0(t) corresponding to f0=0 Hz, the amplitude sequence E1(t) corresponding to

OG Complex Work Unit Math
the product of the first derivative E0(t)′*E1(t)′, and the ratio of E1 to E2(t)(E2(t) is the amplitude sequence of

OG Complex Work Unit Math
as four key feature sequences, whereing each feature sequence is normalized using a zero-mean method, and dynamic time-warped is used to measure the distance.