children's game, from plural of marble (n.); the game is recorded by that name by 1709 but is probably older (it was known in 13c. German as tribekugeln). It was originally played with small balls of polished marble or alabaster, later clay; the modern glass ones marbles the colored swirl date from the 1840s.
Meaning "mental faculties, common sense" (as in to lose or not have all one's marbles) is by 1927, American English slang, perhaps [OED] from earlier slang marbles "furniture, personal effects, 'the goods' " (1864, Hotten), a corrupt translation of French meubles (plural) "furniture" (see furniture).