I also feel the same way. It's annoying to drop the controller to do that.