The three players who have beaten him since RG: Federer, Murray, Del Potro. On a lower level, huge servers can also be tough for him ie Isner, Raonic.
His matchup with Federer is very netural imo, and dependant on surface. Murray is the toughest matchup for him naturally imo, as he can nullify Nole's attacking weapons better than anyone, the matches inevitably turn into wars of attrition. Del Potro has only shown glimpses of this bad matchup potential, but I do believe that a fit/injury free Del Potro could be very tough for Novak for sure - he's already the only big hitter to beat Novak since January 2011 and did it twice. He can more than match Novak's baseline rhythm.
Big serves are a bad matchup because Nole likes to take big swings on return. Against the likes of Isner and Raonic, that's very hard to do. He's not wired to return defensively.
You seem to be a Del Po fan, like myself. Who or what style do you think is a bad match-up for Del Po?