I agree. But the thing is, some individual points change the course of sets, even entire matches.
Yes, of course they do. You're absolutely right. These pieces of luck are far more likely to get the opposition angry and flustered than just firing a good forehand past them. If they get mad, it leads to them playing worse. Soon enough, the match has turned around.
Things like bad calls, flukes and net chords are big mental blows to whoever comes off worse.
However it's important not to confuse this extremely interchangeable short-term luck with a player playing above his level, which is no co-incidence. That comes with the right mentality and belief in technique.