US 12,080,261 B2
Computer vision and mapping for audio
Ilteris Canberk, Marina Del Rey, CA (US); Donald Giovannini, Venice, CA (US); and Sana Park, Los Angeles, CA (US)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on May 2, 2023, as Appl. No. 18/142,181.
Application 18/142,181 is a continuation of application No. 17/397,145, filed on Aug. 9, 2021, granted, now 11,670,267.
Application 17/397,145 is a continuation of application No. 16/724,209, filed on Dec. 21, 2019, granted, now 11,087,728, issued on Aug. 10, 2021.
Prior Publication US 2023/0267900 A1, Aug. 24, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G10H 1/00 (2006.01); G06T 7/90 (2017.01)
CPC G10H 1/0025 (2013.01) [G06T 7/90 (2017.01); G10H 1/0041 (2013.01); G06T 2207/10024 (2013.01); G06T 2207/30204 (2013.01); G10H 2220/101 (2013.01); G10H 2220/445 (2013.01); G10H 2220/455 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method of playing a musical composition, comprising:
initiating a computer vision and mapping application on a wearable device comprising a processor, a memory, a camera, a loudspeaker, and a display;
storing in the memory a plurality of preset token color values, each associated with a set of sound samples;
capturing with the camera a color image of a note matrix comprising one or more fields defined by rows and columns, wherein each row is associated with a musical note and each column is associated with a beat duration;
detecting with the application one or more tokens in the captured color image, wherein the one or more tokens comprises at least a first token characterized by a first token color value wherein the first token is detected at a first token position relative to the fields of the note matrix, and a subsequent token characterized by a subsequent token color value wherein the subsequent token is detected at a subsequent token position relative to the fields of the note matrix;
retrieving from the memory a first set of sound samples based on the first token color value and a subsequent set of sound samples based on the subsequent token color value;
selecting a first playable sound from the first set of sound samples based on the first token position and a subsequent playable sound from the subsequent set of sound samples based on the subsequent token position; and
playing through the loudspeaker a musical composition comprising the first playable sound and the subsequent playable sound in a sequence according to each column, in order of the note matrix.