	US 12,238,312 B2
	Load balancing method for video decoding in a system providing hardware and software decoding resources
Kristine Strandby, Espergaerde (DK); Jeppe Jensen, Rodovre (DK); and Arthur Magaljan, Taastrup (DK)
Assigned to Canon Kabushiki Kaisha, Tokyo (JP)
Filed by CANON KABUSHIKI KAISHA, Tokyo (JP)
Filed on Jul. 19, 2023, as Appl. No. 18/355,005.
Application 18/355,005 is a continuation of application No. 18/069,874, filed on Dec. 21, 2022, granted, now 11,778,210.
Application 18/069,874 is a continuation of application No. 17/059,181, granted, now 11,563,961, issued on Jan. 24, 2023, previously published as PCT/EP2019/064140, filed on May 30, 2019.
Claims priority of application No. 18175601 (EP), filed on Jun. 1, 2018; and application No. 1815853 (GB), filed on Sep. 28, 2018.
Prior Publication US 2023/0370625 A1, Nov. 16, 2023
Int. Cl. H04N 19/42 (2014.01); H04N 7/18 (2006.01); H04N 19/44 (2014.01)

CPC H04N 19/42 (2014.11) [H04N 7/18 (2013.01); H04N 19/44 (2014.11)]

17 Claims

1. A load balancing method for video decoding in a video surveillance system which is configured to carry out a plurality of video decoding processes each corresponding to a stream of video data that originates from a different one of a plurality of video cameras, the system comprising resources for hardware and software decoding, wherein the resources for hardware decoding comprise a plurality of graphics processing units (GPUs) and the resources for software decoding comprise a video codec program module executable by at least one computer processing unit (CPU) core, the method comprising:

monitoring loads of the GPUs;

for a decoding process, carrying out load balancing comprising:

determining which GPUs are suitable GPUs for the decoding process;

determining a current load of each of the suitable GPUs;

determining the suitable GPU as a potential GPU, when the current load of the suitable GPU is less than a first threshold; and

determining the suitable GPU as an overloaded GPU, when the current load of the GPU is greater than or equal to the first threshold and less than a second threshold;

when there are no suitable GPUs, then carrying out the decoding process by software decoding using the video codec program module executed by the at least one CPU core;

when there is only one potential GPU, then carrying out the decoding process on the potential GPU;

when there are more than one potential GPU, then selecting one of the potential GPUs to execute the decoding process,

when there are no potential GPUs, but there is at least one overloaded GPU, the method comprises selecting the overloaded GPU or one of the overloaded GPUs to execute the decoding process, and

when there are no potential GPUs or overloaded GPUs, then carrying out the decoding process by software decoding using the video codec program module executed by the at least one CPU core.