Seems almost as though the cutter is removing more material than it should. Could be a symptom of runout. I would check for runout and see if it can be adjusted as shown by NYC CNC: https://www.nyccnc.com/measure-fix-tool-runout/
Also make sure you are using the precision collets that should have been in one of the setup boxes and not what came in the router box (if any was present).