US 12,293,471 B2
Cross reality system supporting multiple device types
Xuan Zhao, San Jose, CA (US); Ali Shahrokni, San Jose, CA (US); Daniel Olshansky, Mountain View, CA (US); Christian Ivan Robert Moore, Cupertino, CA (US); Rafael Domingos Torres, Boca Raton, FL (US); and Joel David Holder, Austin, TX (US)
Assigned to Magic Leap, Inc., Plantation, FL (US)
Filed by Magic Leap, Inc., Plantation, FL (US)
Filed on Jan. 7, 2022, as Appl. No. 17/571,172.
Application 17/571,172 is a continuation of application No. 17/071,907, filed on Oct. 15, 2020, granted, now 11,257,294.
Claims priority of provisional application 62/915,575, filed on Oct. 15, 2019.
Prior Publication US 2022/0130120 A1, Apr. 28, 2022
Int. Cl. G06T 19/00 (2011.01); G06F 3/01 (2006.01); G06T 5/50 (2006.01); G06T 5/70 (2024.01); G06T 7/73 (2017.01); H04L 67/131 (2022.01)
CPC G06T 19/006 (2013.01) [G06F 3/011 (2013.01); G06T 5/50 (2013.01); G06T 5/70 (2024.01); G06T 7/73 (2017.01); H04L 67/131 (2022.05); G06T 2200/04 (2013.01); G06T 2207/20201 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method of operating a portable electronic device of a first type in a cross reality system in which devices of multiple types interact with a remote localization service via at least one network, the method comprising:
capturing a plurality of images with one or more sensors on the portable electronic device of the first type, wherein the plurality of images comprise pixels depicting a three-dimensional (3D) environment of the portable electronic device of the first type at a time of capture, and the capturing is controlled by a native AR framework of the portable electronic device of the first type;
modifying, by an interface component of the portable electronic device of the first type, the plurality of images to normalize values of the pixels in the plurality of images captured with the one or more sensors on the portable electronic device of the first type with respect to values of pixels in images captured with electronic devices of a second type such that the values of the pixels in the plurality of images captured with the one or more sensors on the portable electronic device of the first type are more consistent with the values of pixels in images captured with electronic devices of the second type, wherein the native AR framework of the portable electronic device of the first type is not configured to operate with the remote localization service, and the electronic devices of the second type comprise native AR frameworks configured to operate with the remote localization service; and
causing information related to at least a subset of the plurality of modified images to be transmitted to the remote localization service, wherein the portable electronic device of the first type comprises one or more components configured to communicate with the remote localization service via the at least one network so as to transmit the at least a subset of the plurality of modified images to the remote localization service, and the interface component of the portable electronic device of the first type is configured to format a response from the remote localization service such that the formatted response can be used by the native AR framework of the portable electronic device of the first type to position virtual content on the device.