The first thing I would check is that they’re not too loose. If you pull/push on the endmill, is there any play? Conveniently, @LiamN just wrote about this here.