PCBs are often FR4 or Garolite. Those are friable (“easily crumbled”) materials (wood, Fiberglass, and carbon composite too). When using a CNC machine on them, they generate particles that are extremely dangerous to your lungs and health. Appropriate precautions should be taken; budgeting for an enclosure, dust collector and dust separator is critical.
In general, plastics and metals are fine - unless one mills them until they start smoking.