acquiring training data including values of drilling parameters for a current stage of a drilling operation being performed within a subsurface formation;

training a reinforcement learning model to estimate values of the drilling parameters for a subsequent stage of the drilling operation to be performed, based on the acquired training data and a reward policy mapping inputs and outputs of the reinforcement learning model, wherein the reward policy maps the inputs and the outputs using an objective function that defines response values of one or more operating variables for the subsequent stage of the drilling operation based on current values of input parameters for the current stage of the drilling operation;

performing the subsequent stage of the drilling operation based on the values of the drilling parameters estimated using the trained reinforcement learning model;

calculating a difference between the estimated values and actual values of the drilling parameters, based on real-time data acquired as the subsequent stage of the drilling operation is performed within the subsurface formation and the reward policy associated with the reinforcement learning model;

retraining the reinforcement learning model using an on-policy reinforcement algorithm to refine the associated reward policy for at least one additional stage of the drilling operation to be performed, wherein the on-policy reinforcement algorithm refines the associated reward policy based on the actual values of the drilling parameters from performing the subsequent stage of the drilling operation based on the values of the drilling parameters estimated using the reward policy, and on the calculated difference; and

performing the at least one additional stage of the drilling operation using the retrained reinforcement learning model.