At each operation you will want to leave enough uncut material that each remaining operation will be able to take a full chip at least, so for the 3 tools you mentioned, you would want to set up the 1/4" tool so that it both takes an ultimate pass of 0.01", and leaves an uncut thickness of material of 0.006" for the two succeeding finishing passes, then the 1/8" tool would cut a thickness of 0.004", leaving the final 0.002" to be cut by the single flute tool.
Paradoxically, larger tooling (which is more rigid) can result in better finishes, so it might be that you could try without the smaller tool.
All of this of course assumes that the belts are well-tensioned, pulley set screws securely in place, and the eccentric nuts properly adjust to hold the Delrin V wheels securely in place ensuring correct linear motion and that the carriage isn’t being shifted by cutting forces (I took some pretty heavy aluminum cuts w/ w/ decent finishes on my SO3 which has an HDZ, so this should be in your machine’s bailiwick).
I believe @wmoy has some tips on this in the videos at:
and @TDA of Precise Bits has shared some excellent information in his posts here as well.
And of course, pictures showing a freshly cut part and its workholding still in place on the machine and the matching file would be the biggest clues for us to examine.