your're right, but if it was me back then, i'd have roddick disqualified for that comment. that's not a pure insult, that's the absolute opposite of how a atp pro is supposed to behave. he'd have had to walk off the court if i'd been the umpire or linesman in the match, i'd have sent him packing.
He should have been given a violation code for that, and maybe even he was, during advertisment break or when the camera was elsewhere. He would have been disqualified if said "fuck you" three times - first time is a warning, second is a penalty point. It is relevant that the violation must be of the same kind, for example if he said "fuck you" twice and then got a warning for timewasting, he would be still on court.
As for James, he studies German linguistics - VERY unusual for a Brit