Actually, that’s helpful enough! I’m fairly confident that’s your problem 
Hobby lobby stocks “K & S precision metals” brass sheet. Lots of reviews touting how soft it is would indicate it’s not a very machinable variety. Then checking K&S directly, they only ever reference 260 alloy sheets.
Alloy 260 is rough in my experience: