Here is a post / video of me drilling 0.1875" deep holes in 6061 with an 0.0400" carbide drill bit without issue.
If you have not trammed your machine yet, you may want to. If not that, check runout?