Weird jog causes rest of cut to shift by width of the jog

Are you cutting a slot just as narrow as the endmill?

Where possible avoid slotting and add geometry and cut as a pocket

and/or

and consider leaving a roughing clearance and taking a finishing pass.