It's substantially more noticeable in swing states because they're swing states and by definition divided roughly 50/50 on partisan lines (really usually 30/30/30[undecided/independent], but the latter varies from place to place). If a state is more heavily tilted one way than the other, the day-of vote is more likely to be just as representative of that tilt as the mail-in vote, especially if you stop paying attention when the state gets called (which was as soon as polls closed for many of the states whose tendencies were more obvious). To add, urban areas have been more heavily hit by covid (so are more likely to vote by mail) and even the in-person vote count in cities takes longer to count. Even in 2016, IIRC Clinton started to (ultimately unsuccessfully) close the gap in MI and WI later at night as the vote counts from Detroit and Milwaukee trickled in.
edit: to balance parens