So, I tested the X axis for backlash after tightening the belt even more, and here is a video. Jogged 100mm of movement each time.
No idea what’s going on. 
Seems to have lost -0.321 mm on the first pass and the passes after that are not consistent in returning to where it previously was, but at least no huge losses after the first pass?
Any ideas? Thank you 
Have you calibrated the steps for each motor? It appears you have it zeroed then when you return to the “zero” position, it is reading around -0.3mm. This could be due to the stepper motors not being calibrated and moving the distance specified in your command. Anything beyond that tenth of a mm measurement, I have a hard time trusting, unless your digital dial is accurate and precise to the hundredth, or thousandth of a mm.
Should belts be tightened from both ends to ensure equal tension in both belt segments underneath the bearing idlers, or just push the gantry to the end opposite of the tensioner and only tension one side?
If you slide the gantry all the way to one side and tighten the belt with the steppers powered down you will be fine and do not need to adjust both ends. Just make sure both Y axis belts have the same tension but be careful not to over tighten the belts as you can damage the stepper shafts with the larger belts. You will possibly snap or bend one.
There is enough slop in the Y carriages to cause a slight racking angle of the gantry if the steppers are powered off and not clamped at the rear of the machine before tightening the belts.
I have a standard SO3 that was not an issue for me when I adjusted the belts at the same time 1/4 turn at a time with the same tooth count through the clip. I don’t remember if we clamped the gantry in place though.
So, the new motor arrived today and carefully made sure all the pulleys were set correctly and loctited and tightened the belts, and things seem better, however circle shapes are still terrible. The way the X axis motor pulley comes from the factory doesn’t have the set screws clamping on the D shaft profile of the motor, so that was resolved by turning the pulley around.
Measuring square shapes seems accurate according to the drawing. Really lost on this but definitely needs solved asap. Circles are the last piece of the puzzle.
What happens if you create a close to circle polygon?
Could it be the software?
How many sides do you suggest? 8?
Depending on the size, I was thinking much higher. Approaching the tolerance for arcs.
You said the machine doesn’t use gcode? What kind of output do you get from CAM? Can you read it or is it proprietary?
Are the flats on the circles in line with the X and Y axes?
It’s proprietary; you can’t read it.
I already have the arc tolerance set for 5 µM in the design software.
The four flats are 90º rotated in the cardinal directions from each other:
What motor drivers are you using?
Try this.
Move Y in the positive direction say 30mm. Next take your indicator and put it against the X rail as close to the left Y rail as you can get it. Move Y in the negative direction in the smallest increment you can till your indicator registers movement. Note the difference in movement between the what the software told it to move and what the indicator showed. Also note if it is consistent from the first time you tell it to move to the second, third, forth, fifth, etc.
Next move Y again in the positive direction 30mm. Move your indicator so it is against the X rail as close to the right Y rail as you can get it. Move Y in the negative direction in the smallest increment you can till your indicator registers movement. Again note the difference between the software and what the indicator showed as well if the moves are consistent.
This should tell you which side is causing the problem and where to spend your time troubleshooting. My guess is that one side is not moving right away when you change directions while the other is lagging behind a bit.
What if the issue is on X instead of Y? Would I put the digital indicator in the same spot like I did in my previous video?
DM542T drivers at 1.69 amps. Motors are rated for 2 amps, so wasn’t sure if I should set them to the 2.03 amp setting or not. https://www.omc-stepperonline.com/digital-stepper-driver/digital-stepper-driver-10-42a-20-50vdc-for-nema-17-23-24-stepper-motor-dm542t.html
Which axis are the odd flat spots on your cut out parallel to?
I revised the photo here:
The top and bottom doesn’t even look too bad, but the left and right ones are definitely the ones that have been the real achilles heel.
And was the circle cut clockwise or counter(anti)-clockwise?
Clockwise
