obtaining, by one or more processors of a calibration device, an error curve of an output from a device under test (DUT);

identifying, by the one or more processors, a target segment of the error curve;

determining, by the one or more processors, a compensation curve that corresponds to the error curve based on a compensation function that corresponds to the target segment of the error curve;

finding, by the one or more processors, a plurality of biquadratic coefficients that correspond to the compensation curve; and

adjusting, by the one or more processors, a plurality of DUT coefficients based on the plurality of biquadratic coefficients.