I think of the switch like a singer applying vibrato to his/her voice ... It would sound very annoying and it would be very difficult to sustain the note if he/she just sung straight (Imagine singing high notes without vibrato... it would sound like nails scratching a chalkboard)...
If you notice one thing about vibrato... It sounds good when it is applied gradually at the end of a note or phrase... you can't sing long passages or start notes in vibrato without sounding like a nervous sheep
The leslie switch is no different, that's why it seems like many organists are constantly switching on every other chord...
Besides that, it sounds very dramatic (especially with dominant and diminished chords, or during cadences) and it makes playing the organ look more complicated, thus making the organist look better!!!!