1610s, from quarry (n.2) + man (n.). Related: Quarrymen.
quarantine
quark
quarrel
quarrelsome
quarry
quarryman
quart
quarter
quarterback
quarterly
quartermaster