I've started playing the organ at the church I play at on Sunday. Just this past Sunday, the guest preacher said she was going to sing a praise song before her sermon and she needed me to follow her. The choir had just finished singing in F, I think, and then she gets up and sings in A. I can count the number of times I've played in A in the last 10 years with my fingers, but this time I was able to pick it up and play comfortably behind her after the first or second time around.
I like playing in flats, so Ab was quite familiar to me. Everything else was just relaxing and moving the chords I knew from Ab a half step up to A. It didn't take as long as I thought and now I can say I can play in A.
My point is that a lot of musicians are more comfortable playing in flats than sharps. So, if you know keys like Db, Eb, Ab and Bb, then you can easily learn D, E, A and B. And, if you know G, you can easily learn Gb. Take the time to program your fingers to a new key. It's worth it!