US 12,433,533 B2
Method, computing apparatus, and computer program for analyzing sleeping state of user through sound information
Junki Hong, Daejeon (KR); Hong Hai Tran, Seoul (KR); Jinhwan Jung, Daejeon (KR); and Dongheon Lee, Seongnam-si (KR)
Assigned to ASLEEP CO., LTD, (KR)
Appl. No. 18/287,969
Filed by ASLEEP CO., LTD, Seoul (KR)
PCT Filed Dec. 30, 2022, PCT No. PCT/KR2022/021760
§ 371(c)(1), (2) Date Oct. 23, 2023,
PCT Pub. No. WO2023/128713, PCT Pub. Date Jul. 6, 2023.
Claims priority of application No. 10-2021-0194186 (KR), filed on Dec. 31, 2021.
Prior Publication US 2024/0081730 A1, Mar. 14, 2024
Int. Cl. A61B 5/00 (2006.01); A61B 7/04 (2006.01); G10L 25/18 (2013.01); G10L 25/66 (2013.01); H04R 19/04 (2006.01)
CPC A61B 5/4812 (2013.01) [A61B 5/7267 (2013.01); A61B 7/04 (2013.01); G10L 25/18 (2013.01); G10L 25/66 (2013.01); H04R 19/04 (2013.01); H04R 2201/003 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for analyzing a sleep state of a user based on sleep sound information, comprising the steps of:
acquiring the sleep sound information including a sound related to breathing and a body movement of the user through a microphone module provided in a user terminal;
pre-processing, via a processor, the sleep sound information;
converting, via the processor, the pre-processed sleep sound information to a plurality of spectrograms;
acquiring, via the processor, sleep state information by processing the plurality of spectrograms as an input of a sleep analysis model including a feature extraction model and a feature classification model, wherein the feature extraction model includes one or more neural networks including an encoder pre-trained through an autoencoder and the feature classification model includes one or more neural networks including a fully connected layer in at least one of the one or more neural networks, and wherein the sleep analysis model has been trained with a learning data set including a plurality of training spectrograms each of which is tagged with sleep stage information;
generating, via the processor, external environment adjustment information based on the sleep state information; and
transmitting the external environment adjustment information to one or more environment adjustment units, the one or more environment adjustment units is configured to adjust a sleep environment of the user by operating one or more environment adjustment modules based on the external environment adjustment information,
wherein the acquired sleep state information includes sleep stage information related to a sleep depth of the user,
wherein each of the plurality of spectrograms corresponds to a predetermined epoch,
wherein the one or more neural networks included in the feature extraction model are configured to extract a plurality of features each of which is based on each of the plurality of spectrograms, and
wherein the one or more neural networks included in the feature classification model are configured to estimate a plurality of sleep stages which are based on the plurality of the features as the sleep stage information.