To avoid the religion argument, let's switch it to a shirt with a big marijuana leaf on it.

In that case, yes. Unless the person is a voter in the act of voting.

I didn't think voters were allowed to have political clothing either? I've heard of some voters asked to turn masks and shirts inside out while they're at the polls.

The PDF linked from the post clarifies the rule for PA: "Wearing a t-shirt or button supporting a candidate, campaign, or political party (except voters in the act of voting)"

(Other jurisdictions may have other rules, of course)

If the colors/hat had become a signalling mechanism by the political movement and were being used to make a political statement, I would say wearing them would be in bad taste. Especially if a member of law enforcement or the military was doing it.

More directly, if the Rastafarian style hat was a prominent display in several recent political rallies for a single party, I think it should be absolutely called into equal question.

Rastafarianism is a religion. I assume religious attire is not restricted.

Is it religious attire if the person wearing it is not religious?

I think argument was that it was worn by an official.

If it was a police officer wearing that or a poll worker, that's not acceptable. If it was a voter in act of voting it is fine.