Can I clean up a outside contour

Leave a roughing clearance and then take a full-depth finishing pass with a suitable tool — consider an HSS (high speed steel) tool, since steel is more easily sharpened than carbide).

Where possible avoid slotting and add geometry and cut as a pocket

and/or

and consider leaving a roughing clearance and taking a finishing pass.