US 12,013,929 B2
Stack pivot exploit detection and mitigation
Andrew Sandoval, San Antonio, TX (US)
Assigned to OPEN TEXT INC., Menlo Park, CA (US)
Filed by Open Text Inc., Menlo Park, CA (US)
Filed on Jan. 24, 2023, as Appl. No. 18/158,621.
Application 18/158,621 is a continuation of application No. 17/228,478, filed on Apr. 12, 2021, granted, now 11,593,473.
Application 17/228,478 is a continuation of application No. 17/088,285, filed on Nov. 3, 2020, granted, now 11,443,032, issued on Sep. 13, 2022.
Application 17/088,285 is a continuation of application No. 15/952,678, filed on Apr. 13, 2018, granted, now 10,853,480, issued on Dec. 1, 2020.
Prior Publication US 2023/0160860 A1, May 25, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 21/52 (2013.01); B01D 15/18 (2006.01); G01N 30/20 (2006.01); G01N 30/22 (2006.01); G01N 30/46 (2006.01); G01N 30/60 (2006.01); G06F 11/36 (2006.01)
CPC G06F 21/52 (2013.01) [B01D 15/1885 (2013.01); G01N 30/20 (2013.01); G01N 30/22 (2013.01); G01N 30/466 (2013.01); G01N 30/6043 (2013.01); G06F 11/3688 (2013.01); G01N 2030/202 (2013.01); G01N 2030/207 (2013.01); G06F 2221/033 (2013.01)] 14 Claims
OG exemplary drawing
 
1. A system for stack pivot exploit detection and mitigation, comprising:
a processor; and
memory storing instructions that, when executed by the processor, causes the system to perform a set of operations for stack pivot detection and mitigation, the set of operations comprising:
defining, for a process executing in an execution mode, thread information for the process, the thread information comprising a current memory pointer, a memory base, and a memory limit, the memory base and the memory limit defining a memory range for the process;
based on the current memory pointer for the process executing in the execution mode, generating a corresponding memory base pointer for the memory base, and a corresponding memory limit pointer for the memory limit;
comparing the current memory pointer for the thread information to the memory base pointer and the memory limit pointer to determine whether the current memory pointer is within the memory range;
when the current memory pointer is determined to be within the memory range, permitting the process to execute;
when the current memory pointer is determined to not be within the memory range, determining an occurrence of a stack pivot exploit and, in response, performing a remedial action; and
executing a series of checkpoints over time, each checkpoint determining the occurrence of the stack pivot, wherein the execution mode comprises a kernel execution mode, a 32-bit execution mode, and a 64-bit execution mode, each, the execution mode having a corresponding series of checkpoints for determining the occurrence of a stack pivot within each execution mode.