Quirky case is probably familiar to anyone with an interest in case systems. We could, however, considered similar lexically caused quirks for other things than case!
Consider some nouns that trigger exceptional voice marking on some, or even all vers? Maybe they cannot be direct objects, so whenever they are objects, they force some detransitivizing voice and some kind of oblique marking.
Some adjectives as complements, or maybe some nouns as verbs or objects, might cause unusual behaviours with regards to tense, aspect or mood. Maybe some noun as subject (or as object) always triggers some irrealis mood regardless of the reality of the VP and of the subject.
Some Semitic languages have some numbers mark opposite gender agreement. We could consider, though, verbs or adjectives that mismatch gender. Coming up with a historical reason for these might be interesting.
Quirky Adjective/Adverb Things
One could imagine certain nouns or verbs forcing comparatives or superlatives, or vice versa, some nouns or verbs blocking comparatives and superlatives (but making superlative or comparative meanings mandatory).
Some adjectives or verbs or adpositions could imaginably also require some noun in some relation to it to mark number exceptionally. Alternatively, some verb could break the number congruence if verbs have that in the language.
I guess by now the reader should be able to come up with some ideas for this as well, analogously to the ideas given above.