Understand HTTPS With a Python Flask Example(Self-signed Certificate)
Nowadays, server/client mode has become a very popular way in developing since it is a good and efficient way for product development and maintenance. Simple and fast definition, adaptive in communication type, no status/ connections, all these features make the HTTP become a common standard for server / client mode. But sometimes, this swift and nice communication might needs some protection to fulfill the requirements on security concerns. HTTPS was born for this and now is still improving. In this article, I’m going to introduce HTTPS in a not very professional way, but I believe that this could make you understand the HTTPS better and how do we apply these features in the codes.
If you have read any tutorials or essays about encrypted communication, you’ll find that “Alice” and “Bob” are usually involved in the story as main characters. Alice and Bob are generally good friends that often send letters to each other. In HTTP method, if Alice needs to write to Bob, she just need to write everything in the letter and use a carrier pigeon send to Bob. However the risk could be that during the flight, the letter could be hijacked by someone else, then the contents could be exposed to others. So if the contents are important, for example, Bob is proposing to Alice so he send out his debit card with password, it would be a disaster.
To make it secure, Alice and Bob might need to encrypt the contents of letters. If it is a basic way, they will try to encrypt the contents with a particular method then send the encrypted contents to each other. But the hackers now are usually very professional, they could easily find that predefined method and decrypted the contents. We call the method symmetric encryption, once you know the key, you can do the decryption and encryption in both side.
Bob and Alice then come out with a better method. Bob would ask Carrier Pigeon to go to Alice’s place with nothing. Then Alice would reply with a locker box, but leave the box open to Bob. Bob received the box, put the letter inside and then close the box then response to Alice. Alice keeps the key only on her side. So even if the Carrier Pigeon is hijacked, the hacker could not open the box unless destroy the box in physical way, but this would inform that the contents are exposed, and actually in cyber communication, the “physical” way is hard to apply since the box is a black box, usually the boundary between box and contents is not clear as the case, also the time cost would meet the expiration of the encrypted information. The only risk would be that the hacker could hijack the Carrier Pigeon at first and send his own box to Bob.
In this case, we need to introduce another important person — Ted, who is a famous, well known and trustworthy wise man, will never give out the signature information to others. Ted needs to give out his certificated signature to everyone, and when someone is sending out the box, he will help to sign on the box so that they would know that the box is exactly owned by the one they are communicating with. Finally even there are still many ways to improve, Alice and Bob find out a secured way to communicate. Technically, HTTPS method could be describe in the following picture, Bob is client, Alice is server and crt public is Ted.
Then let’s start with sample codes. To apply the HTTPS is simple in flask, except the flask environment run:
pip install pyOpenSSLThen in your entry portal, just apply :
app.run(ssl_context=’your_certificate_address’) If you are using browser as client, you will need to add the certificate in your browser environment settings also. If your client is also a coded part, you need to attach the certificate when send out the request also. In a python client, the code would be:
requests.request(“GET/POST/…”,”your_url",files=your_data, verify=your_certificate)But there is a very sad thing: Ted also has his own bills to pay and mouth to feed so he is not doing the certificate for free. Usually, to make the certificate secured enough, developers would need to pay for the protection. But if you are just play it around or in your case Internet is not allowed(public certificate requires internet to identify), I would recommend you the self-signed certificate. The aim is to create a free “Ted” and make this “Ted” at least worth trust for Alice and Bob. In development, openssl could help us to do this.
Firstly you need to install openssl and add it to you system environment path. You can choose to compile in its official website, but I would recommend you to download the compiled exe done by others, it saves you a lot of time. A worth mentioning thing is that you have tested other encryption demo before, your default system path of openssl.conf file might not point to the new installed one, please do it manually. When you get everything ready, we can apply those in your code now.
I’m a bit lazy so I’m not showing you all the steps in commandlines. Instead, I’ll give the codes of running them in python code. I think it can also help in some occasions when you want to apply some automation in generating an applying the certificates for you projects. We have 3 folders in the root directory : ca for “Ted”, server for server (Bob), client for client (Alice).
Firstly, let’s create out own “Ted”. When creating for Bob or Alice, we all need to refer to those files that can indicate the features of “Ted”.
subprocess.call([“openssl”,”genrsa”, “-out”, “./ca/ca-key.pem”, “1024”],shell=True)subprocess.run([“openssl”,”req”, “-new”, “-out”, “./certificate/ca/ca-req.csr”, “-key”, “./ca/ca-key.pem”],stdout=subprocess.PIPE,input=b’contry\nstate\nlocality_name\noranization_name\ncommon_name\nyour_ip\your_email\nPassword\nPassword\n’)subprocess.run([“openssl”, “x509”, “-req”, “-in”, “./ca/ca-req.csr”,”-out”,”./ca/ca-cert.pem”,”-signkey”,”./ca/ca-key.pem”,”-days”,”3650"])
Then we can create the digit signature for server (Bob), in the first subprocess call &run, the server is creating his own locker box and key, and in the second subprocess run, Bob is asking “Ted” to give a unique digit signature.
ip = your ip address:
subprocess.call([“openssl”,”genrsa”, “-out”, “./certificate/server/server-key.pem”, “1024”],shell=True)subprocess.run([“openssl”,”req”, “-new”, “-out”, “./server/server-req.csr”, “-key”, “./server-key.pem”],stdout=subprocess.PIPE,input=bytes((‘country\nstate\nlocality_name\noranization_name\ncommon_name\n{ip}\nyour_email_address\nPassword\nPassword\n’).format(ip=ip),encoding=”utf-8"))subprocess.run([“openssl”,”x509",”-req”,”-in”,”./server/server-req.csr”,”-out”,”./server-cert.pem”,”-signkey”,”./server/server-key.pem”,”-CA”,“./certificate/ca/ca-cert.pem”,”-CAkey”,”./ca/ca-key.pem”,”-CAcreateserial”,”-days”,”3650"])
The client side (Alice) steps are exactly the same. Please note that for “Ted” it is not import that who is Alice(client) and who is Bob(server). Since both of them could be the starter or receiver of the communication. If you want to apply the HTTPS in you browser side, you will need to transfer the certificate file to .p12 format, and add it in you browser environment settings. The code would be :
subprocess.run([“openssl”, “pkcs12”, “-export”, “-clcerts”, “-in”, “./(*)-cert.pem”, “-inkey”, “./(*)-key.pem”, “-out”, “./(*).p12”])
# (*) could be ca/server/client.When the files are ready, add the link to the flask running configuration:
app.run(debug=False, ssl_context=(“/server/server-cert.pem”,“/server/server-key.pem”), host=host,port=port)
Client side would be：
requests.request(“GET/POST/…”,”your_url",files=your_data, verify=“/client/client-cert.pem”,“/client/client-key.pem”))
Finally, the settings are done! Let’s do some testing. I’m using wireshark to “hijack” the communication. (Those painted part are information input when signed CA, blocked them for privacy) As you can see, the contents part are not able to re-construct to original one.
That all for my sharing on HTTPS. If you are careful enough, you might find out a very considerable issue: What if the hacker buy the Ted or fake the digit signature of Bob or Alice? Unfortunately these might happen. In fact, picking up a robust public certificate now is also a challenge for production, because to be a Ted, he or she need to be famous, well known and trustworthy, but for an organization which supply certificates except fames they need a very strong technical background. A more disappointing thing is that even for these qualified and famous Ted, there are still some huge power force Ted to spoil out. In the Xkeyscore project that unmasked by Edward Snowden, there are a lot of web communication encrypted by HTTPS were hijacked and decrypted by CIA, indicating that there are some “Ted”s controlled by the high layer entities. HTTPS has achieved a lot, but the war of information protection would never end.
