Uber drivers aren't invisible. It wouldn't take much effort to use the app to flag one down, note the license plate, and give them a nice little talking to. I doubt many part-timers would stick with it if they thought there was a chance for trouble.