buff (n.) Look up buff at Dictionary.com
1570s, buffe leather "leather made of buffalo hide," from Middle French buffle "buffalo" (15c., via Italian, from Latin bufalus; see buffalo (n.)).

The color term comes from the hue of buffalo hides (later ox hides). Association of "hide" and "skin" led c. 1600 to in the buff. Buff-colored uniforms of New York City volunteer firefighters since 1820s led to meaning "enthusiast" (1903).
The Buffs are men and boys whose love of fires, fire-fighting and firemen is a predominant characteristic. [N.Y. "Sun," Feb. 4, 1903]
buff (adj.) Look up buff at Dictionary.com
"well-built, hunky," 1980s, from buff (v.) "polish, make attractive."
buff (v.) Look up buff at Dictionary.com
"to polish, make attractive," 1885, in reference to the treatment of buff leather or else to the use of buff cloth in polishing metals, from buff (n.). Related: Buffed; buffing.