"spectator, one who stands near," 1610s, from by + agent noun from stand (v.). They have been innocent at least since 1829. Stander-by is from 1540s.
by-path
by-product
byre
by-road
Byronic
bystander
byte
byway
byword
Byzantine
Byzantium