US 12,293,534 B2
Imaging device, information processing device, imaging method, and information processing method
Shinichiro Gomi, Tokyo (JP); and Teppei Kurita, Tokyo (JP)
Assigned to SONY GROUP CORPORATION, Tokyo (JP)
Appl. No. 17/756,776
Filed by SONY GROUP CORPORATION, Tokyo (JP)
PCT Filed Dec. 4, 2020, PCT No. PCT/JP2020/045246
§ 371(c)(1), (2) Date Jun. 2, 2022,
PCT Pub. No. WO2021/117633, PCT Pub. Date Jun. 17, 2021.
Claims priority of application No. 2019-225887 (JP), filed on Dec. 13, 2019.
Prior Publication US 2023/0342963 A1, Oct. 26, 2023
Int. Cl. G06T 7/586 (2017.01); G01B 11/24 (2006.01); G02B 27/28 (2006.01); H04N 23/56 (2023.01); H04N 25/10 (2023.01)
CPC G06T 7/586 (2017.01) [G01B 11/24 (2013.01); H04N 23/56 (2023.01); H04N 25/10 (2023.01); G02B 27/288 (2013.01)] 8 Claims
OG exemplary drawing
 
1. An imaging device, comprising:
an imaging unit that includes
a plurality of lights, wherein
each of the plurality of lights is configured to emit light on a subject, and
a polarization direction of the light emitted by each of the plurality of lights is different, and
a polarization sensor configured to capture a first image of the subject that is simultaneously irradiated with the light from the plurality of lights; and
a central processing unit (CPU) configured to:
separate a pixel signal corresponding to each of the polarization directions from the captured first image;
generate a second image for each of the polarization directions;
calculate a normal line on a surface of the subject from the second image for each of the polarization directions by photometric stereo;
estimate a shape of the subject based on the normal line;
store correspondence information that associates each of the plurality of lights, the polarization directions, and directions of the plurality of lights with respect to the subject;
estimate a polarization model that indicates a correspondence relationship between the polarization directions and luminance of each pixel in the first image of the subject,
wherein the first image of the subject is irradiated with the light having the polarization direction based on luminance of each pixel in the second image for each of the polarization directions; and
calculate the luminance of each pixel in the second image for each of the plurality of lights based on the polarization model and the correspondence information.