Photos of the damaged endmills?
Any galling on them?
My best successes w/ aluminum have been:
- using coated tools
- using single-flute tooling
- adjusting stepover so as to ensure a reasonable amount of material is being removed — I think this last is the problem in your case, and suspect @Ed.E has it aright
See: