I think part of it (besides everybody playing less) may be discussing the situation with the sound people. If they can turn the organist down, good. If not, other players may have to be turned up (not reccomended, as this may lead to a "volume war" among the musicians).
IMO, many keyboard players, including myself, are used to playing alone and 'covering all the bases' (playing melody, chords, bass lines, rhythms). It may take a while for them to get used to a guitar, bass, second keyboard, etc., hearing what they play, and ceding that sonic territory to them...