It is irrelevant who said what, only what was said. It is not possible to validly counter an argument by attacking the man, which is why ad hominem is a fallacy. The logical way to have approached this is to ignore the man and point out the argument from authority, also a fallacy.