US 12,451,226 B2
Inhaler training system and method
Sabine Häussermann, Munich (DE); Alexandros Sivris, Munich (DE); and Jonas Benedikt Natzer, Munich (DE)
Assigned to VisionHealth GmbH, Munich (DE)
Filed by VisionHealth GmbH, Munich (DE)
Filed on Apr. 18, 2023, as Appl. No. 18/302,015.
Application 18/302,015 is a continuation of application No. 16/955,802, granted, now 11,664,104, previously published as PCT/EP2018/086541, filed on Dec. 21, 2018.
Claims priority of application No. 17209861 (EP), filed on Dec. 21, 2017.
Prior Publication US 2023/0253086 A1, Aug. 10, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G06T 7/215 (2017.01); G06F 18/214 (2023.01); G06F 18/25 (2023.01); G06N 3/08 (2023.01); G06T 7/00 (2017.01); G06T 7/11 (2017.01); G06T 11/00 (2006.01); G06V 20/40 (2022.01); G06V 40/20 (2022.01); G16H 20/13 (2018.01)
CPC G16H 20/13 (2018.01) [G06F 18/214 (2023.01); G06F 18/251 (2023.01); G06N 3/08 (2013.01); G06T 7/0012 (2013.01); G06T 7/11 (2017.01); G06T 7/215 (2017.01); G06T 11/00 (2013.01); G06V 20/40 (2022.01); G06V 40/28 (2022.01); G06T 2207/10016 (2013.01); G06T 2207/30061 (2013.01)] 14 Claims
OG exemplary drawing
 
1. A computer-implemented method for providing feedback to a patient using an inhaler of a specific type for inhaling a therapeutic aerosol according to instructions for use provided to the patient for, or along with, the inhaler, the method comprising the steps of:
(a) obtaining a sequence of digital audio signals and a sequence of digital video signals emitted from the patient while using the inhaler;
(b) recognizing whether the patient has deviated from the instructions by processing the audio and video signals;
(c) if a deviation was recognized in step (b), recognizing a type of the deviation; and
(d) presenting to the patient one or more feedback messages comprised of:
(i) whether a deviation was recognized; and
(ii) if a deviation was recognized, identifying a type of deviation;
wherein steps (b) and (c) are performed by analyzing the audio signals and video signals using an artificial neural network trained to identify deviations, or by a method to detect a regular motion;
wherein steps (a) to (d) are performed by a portable feedback device for handheld use comprising:
a data processing system;
a data memory;
a camera;
a microphone;
a display; and
a speaker;
wherein the deviation to be recognized in step (b) comprises a deviation from a regular movement using a method to detect a regular motion comprising the steps:
(I) perform the following steps on a series of frames of said video signals of the patient using the inhaler;
(i) selecting a portion of a frame through which said regular motion passes through;
(ii) computing a mean brightness value within said portion of the frame;
(iii) storing said mean brightness value of said portion of said frame in a buffer;
(II) repeating steps (i)-(iii) for a sequence of frames comprising a plurality of frames, wherein said portion of a frame is identical in each frame creating a buffer of brightness values;
(III) calculating an average of a mean brightness of all values determined and stored in said buffer of brightness values;
(IV) subtracting said average of the mean brightness of all values from each value in said buffer to generate a zero centered buffer; and
(V) computing a zero crossing rate of said zero centered buffer.