1590s, "sharp, poignant" (of pain or grief), from Latin pungentem (nominative pungens), present participle of pungere "to prick, pierce, sting," figuratively, "to vex, grieve, trouble, afflict" (from suffixed form of PIE root *peuk- "to prick").
Meaning "having powerful odor or taste" first recorded 1660s. Literal sense "sharp, pointed" (c. 1600) is very rare in English, mostly limited to botany. Middle English and early Modern English also had a now-obsolete verb punge "to prick, pierce; to smart, cause to sting," from Latin pungere. Related: Pungently.