When he loses his movement. From what I saw in the Murray-Fed final, his movement was up with the best I've seen it.
Forget the serve, Federer not moving well loses all timing and precision. He starts spraying errors left, right and centre (and into the net
) continually. He can afford to have a low first serve percentage but move well. For example, Cinci SF and F he didn't serve as well as he can but moved beautifully and took the ball on so early and it made his serve look secondary. I personally don't think he's got many years left at the very top. Many people talk about Connors and Rosewall, about how they played into their late 30s and still had success. The game nowadays is much more physical though, and even if Fed's game is not as taxing on the body it'll eventually catch up with him. It's easy to overlook how many long and gruelling matches he's had to grind out because usually it looks quite effortless.
I predict majors will continue to be his most successful tournaments, but eventually biological decline will kick in. We thought it had already happened in 2008 when he looked to be a step slow but of late that seems to be more and more of a temporary blip in his form rather than something that continued.