Pitfalls of manual AES encryption for data transfer
Context
I've used OpenSSL to encrypt some socket communications.
I am however using some functionality from the windows API that prevents me from using OpenSSL's opaque builtin sockets, so I am buffering through their BIO_s_mem interfaces - that requires 4 extra copies for all data passing through (socket->read buffer->read BIO->OpenSSL state machine->write BIO->write buffer->socket, as opposed to socket->OpenSSL state machine->socket).
This is not efficient, especially seen as the goal is (among other things) large file transfers.
Idea
As a result I'm thinking I could establish an OpenSSL session, generate some symmetric keys, and use that secure session to share them. I would then open another channel (or just use the underlying socket which I have full control over) to transfer the bulk data using something like AES256-GCM - doing that 'manually' (using OpenSSL crypto libraries) would make it incredibly fast as it would no longer imply the inefficiencies of passing it through the OpenSSL state machine.
Problem, Goal & Question
I'm not a cryptographer. I don't even know what I don't know. All I know is if people can find side-channel attacks based on compression ratio of encrypted data, then any mistakes I make with this would probably be fatal.
I'm aware key generation has to be done very carefully - I've read through this, and I was thinking of using OpenSSL's facilities for the purpose. I'm also aware that AES is by itself secure, but a wrong implementation will easily throw that away.
This does however, on the surface, seem like a manageable challenge. Generate keys -> Run data through AES256-GCM. The upside is big, and this is a personal project that likely won't have the most proficient hackers (or any hackers) trying to mess with it. I just want it secure so that if people start to use it, they're not using something could be easily broken.
So, is there something obvious I'm missing? Something less than obvious?
Extra considerations, perhaps a correction on something wrong I might have said, some resource I should read before the attempt? All is appreciated, but if your contribution is "just don't do it", I ask you to please add some context as to what could go wrong so I get the picture.
