early 15c., boh, "A combination of consonant and vowel especially fitted to produce a loud and startling sound" [OED, which compares Latin boare, Greek boaein "to cry aloud, roar, shout"]; as an expression of disapproval, 1884 (n.); hence, the verb meaning "shower (someone) with boos" (1885).
Booing was common late 19c. among London theater audiences and at British political events; in Italy, Parma opera-goers were notorious boo-birds. But the custom seems to have been little-known in America before c. 1910. To say boo "open one's mouth, speak," originally was to say boo to a goose.
To be able to say Bo! to a goose is to be not quite destitute of courage, to have an inkling of spirit, and was probably in the first instance used of children. A little boy who comes across some geese suddenly will find himself hissed at immediately, and a great demonstration of defiance made by them, but if he can pluck up heart to cry 'bo!' loudly and advance upon them, they will retire defeated. The word 'bo' is clearly selected for the sake of the explosiveness of its first letter and the openness and loudness of its vowel. [Walter W. Skeat, "Cry Bo to a Goose," in Notes and Queries, 4th series, vi, Sept. 10, 1870]