US 12,112,525 B2
System and method for segment-aware semantic segmentation
Lior Shabtay, Ramat Gan (IL); and Alon Wander, Kiryat Gat (IL)
Assigned to GEOX GIS INNOVATIONS LTD., Petach Tikva (IL)
Filed by GEOX GIS INNOVATIONS LTD., Petach Tikva (IL)
Filed on Oct. 25, 2023, as Appl. No. 18/383,493.
Claims priority of application No. 297653 (IL), filed on Oct. 25, 2022.
Prior Publication US 2024/0135690 A1, Apr. 25, 2024
Prior Publication US 2024/0233345 A9, Jul. 11, 2024
Int. Cl. G06V 10/766 (2022.01); G06V 10/26 (2022.01); G06V 10/764 (2022.01); G06V 10/774 (2022.01); G06V 10/776 (2022.01); G06V 10/82 (2022.01); G06V 20/17 (2022.01); G06V 20/70 (2022.01)
CPC G06V 10/776 (2022.01) [G06V 10/26 (2022.01); G06V 10/764 (2022.01); G06V 10/774 (2022.01); G06V 10/82 (2022.01); G06V 20/17 (2022.01); G06V 20/70 (2022.01)] 15 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
receiving an image, a ground truth annotation mask and a segment ID mask, wherein the image comprises a plurality of pixels, the ground truth annotation mask comprises for each pixel a label of a correct class, and the segment ID mask comprises for each pixel a segment identifier identifying a segment to which the pixel belongs;
feeding the image to a machine learning model for semantic segmentation and computing an outcome;
computing a per-segment value for each one of the plurality of segments from the information in the outcome and in the ground truth annotation mask related to pixels associated with the one of the plurality of segments in the segment ID mask;
calculating a segment-aware value from a plurality of per-segment values;
using the segment-aware value for training the machine learning model for semantic segmentation; and
updating a plurality of parameters of the semantic segmentation machine learning model using the segment-aware value,
wherein the step of computing the per-segment value further comprises
computing, for each one of the plurality of pixels, a per-pixel loss value using the ground truth annotation mask and the outcome, wherein the outcome is a per class probability map containing for each pixel a probability of belonging to each class, and
computing a per-segment value for each one of the plurality of segments using a plurality of per-pixel loss values of pixels associated with the one of the plurality of segments, thereby creating a per-segment loss, and wherein the step of calculating a segment-aware value further comprises computing a segment aware-loss value as an aggregation of the per-segment loss values.