Smoothness of the surface is a big factor. Smoother it is, less friction there is between ball and the court and less speed the ball will lose upon impact. One year, they painted Melbourne courts to make them smooth & fast, to help Aussies (Scud, mostly). Of course, that year Agassi won, beating Sampras on the way
Also, height of the bounce has impact on gameplay. It is not necessarily tied to court speed - clay courts have lower bounce than most hardcourts, yet are usually considered "slower".