I would give this a try, as these are my settings for a single flute (that’s what I use typically in aluminum) when boring holes adapted to what you want to try.
Climb cut
RPM = 10,000 (this will help allow for more chip evacuation)
Chipload = 0.001 (20 in/min)
Pitch = 0.5mm (0.02 in)
That should work. You could even share the file you’re working on to make sure nothing else is weird.