I use my pinky but it doesn't stay aligned with my other three fingers. Yet it makes it to the right fret with the right amount of force. My bass teacher tried in vain to get me to rein it in.
I wouldn't say that it's a handicap if you can play effectively, with the right speed, accuracy and force.