back down (v.) Look up back down at Dictionary.com
in figurative sense of "withdraw a charge," 1859, American English, from notion of descending a ladder, etc.; from back (v.) + down (adv.).