1650s, from un- (1) "not" + grammatical. Related: Ungrammatically.
ungodly
ungovernable
ungoverned
ungraceful
ungracious
ungrammatical
ungrateful
ungual
unguarded
unguent
unguided