early 12c., "religious recluse, one who dwells apart in a solitary place for religious meditation," from Old French hermit, ermit "hermit, recluse," from Late Latin eremita, from Greek eremites, literally "person of the desert," from eremia "a solitude, an uninhabited region, a waste," from erēmos "uninhabited, empty, desolate, bereft," from PIE *erem- "to rest, be quiet" (source also of Sanskrit ramate "to rest;" Lithuanian rimti "to be quiet," Gothic rimis "rest," Old Irish fo-rimim "to set, lay"). The unetymological h- first appeared in Medieval Latin heremite.
Transferred sense of "person living in solitude" is from 1799. Related: Hermitic; hermitical. The hermit crab (1735) is said to be so called for its seeking out and dwelling in a solitary cell.