US 12,406,431 B2
Method and system for semantic-driven intelligent reconstruction of large-scene sparse light field
Lu Fang, Beijing (CN); Haiyang Ying, Beijing (CN); and Jinzhi Zhang, Beijing (CN)
Assigned to TSINGHUA UNIVERSITY, Beijing (CN)
Filed by TSINGHUA UNIVERSITY, Beijing (CN)
Filed on Aug. 23, 2023, as Appl. No. 18/454,513.
Claims priority of application No. 202211025358.0 (CN), filed on Aug. 25, 2022.
Prior Publication US 2024/0078744 A1, Mar. 7, 2024
Int. Cl. G06T 15/20 (2011.01); G06T 7/90 (2017.01)
CPC G06T 15/20 (2013.01) [G06T 7/90 (2017.01); G06T 2207/10024 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for semantic-driven intelligent reconstruction of a large-scene sparse light field, comprising:
acquiring a semantic primitive set of a multi-view image set, wherein the semantic primitive set comprises at least one semantic primitive, and the multi-view image set comprises at least one image;
acquiring a coordinate offset by inputting coordinate information and a feature vector corresponding to a first grid sampling point of the semantic primitive set into a first network model, and acquiring a second grid of the semantic primitive set based on the coordinate offset and geometric attribute information of the semantic primitive set;
acquiring first feature information of a second grid sampling point by inputting coordinate information and a feature vector corresponding to the second grid sampling point, and an observation angle value into a second network model, and acquiring second feature information of the semantic primitive set based on the first feature information, wherein the first feature information comprises a visibility value and a color value of the second grid sampling point; and
acquiring a light field reconstruction result of the multi-view image set based on an observation angle value of the semantic primitive set and third feature information extracted from the second feature information, wherein the third feature information comprises a spatial offset, a visibility value and a color value of the semantic primitive set.