US 12,443,429 B2
Memory deduplication for encrypted virtual machines
Michael Tsirkin, Yokneam Illit (IL)
Assigned to Red Hat, Inc., Raleigh, NC (US)
Filed by Red Hat, Inc., Raleigh, NC (US)
Filed on Aug. 30, 2022, as Appl. No. 17/898,707.
Prior Publication US 2024/0069950 A1, Feb. 29, 2024
Int. Cl. G06F 9/455 (2018.01); G06F 12/14 (2006.01); G06F 21/64 (2013.01)
CPC G06F 9/45558 (2013.01) [G06F 12/1408 (2013.01); G06F 21/64 (2013.01); G06F 2009/45583 (2013.01)] 17 Claims
OG exemplary drawing
 
1. A method, comprising:
identifying a page in a private memory of an encrypted virtual machine to place into a public memory in a virtualization environment;
calculating a checksum for the page and storing the checksum in the private memory of the encrypted virtual machine;
subsequently to or contemporaneously with calculating and storing the checksum, passing the page to a hypervisor of the virtualization environment to place into the public memory;
subsequent to placement of the page into the public memory, calling the page via an application running in the encrypted virtual machine; and
in response to calling the page, performing, by the encrypted virtual machine, the steps of:
restoring the page into the private memory from the public memory;
calculating a verification checksum for the restored page;
comparing the calculated verification checksum against the calculated checksum stored in the private memory;
if, based on the comparing, the calculated verification checksum and the calculated checksum match, executing the application using the page; and
if, based on the comparing, the calculated verification checksum and the calculated checksum do not match, executing the application while preventing the usage of the restored page from the private memory.