How can a ripple wallet trust the server?

How can a wallet that uses ripple-lib interact with the ledger to be sure it is seeing the true consensus book? If an attacker has control of the wavy server, or the connection to it, what is preventing them from falsely verifying that funds have been received?

Is it just a matter of trusting the server operator + the certification authority + https? If you are currently connected to a legitimate server, are there any data that you can save that confirm that the same entity verified future transactions?