How to verify the signature of DHE_RSA server parameters (TLS 1.2)?
By reading the TLS 1.2 specs I have been able to manually decrypt/encrypt pretty much all the steps of a TLS connection (DHE_RSA_AES_CBC_128_SHA cipher suite).
One of things I am missing is how to verify the signature of the Diffie-Hellman Ephemeral parameters sent by the server. I was able to verify the Certificate chain, but not the initial DH parameters.
First of all, I compute signature^e mod n (e and n coming from the server Certificate). The result looks OK as its looks properly encoded (01FFFF...FF00...<hash>).
However, I am not sure what format to use to hash the DH parameters. I tried hashing the bytes as sent by the server and using DER encoding. I tried encoding just pubKey, and various mixes of p, g and pubKey. But I still cannot make the two hash numbers to match.
