I'm with the depends on the situation camp.
A singer should always reserve the right to decline respectfully. That's why a psalmist should always be atuned to the spirit of God, and know when it is right for him or her to sing.
Certainly though, there are very respectful ways to decline a person of authority and one should think twice about it when addressing a pastor. But if you are really getting a bad vibe, then I say exercise the right you reserved.
Either opinion, yea or nay, is going to bring potential problems with it.