Different verbs could have different types of modal meaning in different moods - passive could be jussive for some verbs, permissive for some, a potential mood for some, and a precative imperative for some verbs.
Basically, I am thinking there could be a language where there is one primary modal distinction, and
- that this modal distinction is different for different verbs - but probably only three or four different classes of verbs or somesuch
- that this modal distinction primarily is expressed by what previously was a passive voice, and that this leads to the situation where the main modal distinction also leads to syntactical shenanigans.
Of course, other less prominent modes may be marked by more traditional ways that don't interfere with what historically has been voice marking. Possibly, voice marking retains its original meaning when other explicit voice marking is present. A new indicative passive may be forming in some manner.
One big question that should be obvious now, of course is: how do we get a verb for which the usual meaning of the passive is 'jussive' to mark potential or permissive or precative? Maybe the association with different moods is just a statistic thing - some verbs are likely to appear in contexts where precative is more likely, some are more likely to appear in collocations where jussive is more likely, and thus saying that 'go.pass' is permissive just says a thing about its most common use, and precative or potential or jussive use is actually inferred from the context, but we observe that it most often just happens to be permissive in the language and therefore pretend go.pass really is permissive. Alternatively, there may be subtle differences in use - have a statitical tendency to omit the agent with jussive and precative for verbs that don't default to jussive or precative meanings, have some weakening adverbs or such that make non-potentials potential, have some case marking that can go on a participant acquiring permission to turn non-permissive verbs into permissive ones.
Another idea that could fit with this is using negative passives as a kind of intensive negation.