fly-by-night (n.) Look up fly-by-night at Dictionary.com
1796, slang, said by Grose to be an old term of reproach to a woman signifying that she was a witch; used from 1823 in reference to anyone who departs hastily from a recent activity, especially while owing money. The different senses involve the two verbs fly.