When we use pour-on we just pour it on. I've never seen any instructions to rub it on with your hands. Liquid that gets stuck to your hands is liquid that isn't still on the animal's skin where it will have therapeutic effect.
But sure, it's completely safe for human beings to be exposed to ivermectin. WHO wouldn't recommend it otherwise.
The dose makes the poison; nothing is completely safe to be exposed to. The lethal dose of ivermectin is about 10 mg/kg orally, so it's about 250 times more poisonous than table salt, which from my point of view puts it in the "dangerous poisons" category. But that's about 30 times the standard therapeutic dose, so it's pretty unlikely to happen by accident.
However, that's the standard therapeutic dose for parasites. To work as an antiviral it needs to interfere with viral replication somehow, and since viruses replicate using normal somatic-cell metabolic processes, you'd probably have to take a high enough dose of ivermectin for it to interfere with normal somatic-cell metabolic processes. That, in turn, means you're going to see side effects you wouldn't see at the normal antiparasitic doses.
Nevertheless, there isn't in fact an epidemic of serious ivermectin poisoning, horse dewormers or no. Maybe one in ten million people in the US, less elsewhere.
But sure, it's completely safe for human beings to be exposed to ivermectin. WHO wouldn't recommend it otherwise.