Skip to content

Commit b990af9

Browse files
guedoubzalkilani
guedou
authored and
bzalkilani
committed
Use a NSS Key Log file to decrypt a TLS session (secdev#3374)
* Use a NSS Key Log file to decrypt a TLS session * Decrypting TLS 1.2 using a known master secret * Test TLS 1.2 decryption using a NSS Key Log
1 parent 9af4225 commit b990af9

File tree

5 files changed

+240
-30
lines changed

5 files changed

+240
-30
lines changed

‎doc/notebooks/tls/notebook3_tls_compromised.ipynb

Lines changed: 151 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -4,15 +4,17 @@
44
"cell_type": "markdown",
55
"metadata": {},
66
"source": [
7-
"# The lack of PFS: a danger to privacy"
7+
"# The lack of PFS: a danger to privacy\n",
8+
"\n",
9+
"With TLS 1.2 and earlier, some cipher suites do not provide Perfect Forward Secrecy. Without this property, an attacker compromising the server private key can easily decrypt TLS traffic.\n",
10+
"\n",
11+
"In the following example, Scapy is used to decrypt a comunication made without PFS using the ciphersuite `TLS_RSA_WITH_AES_128_CBC_SHA`, giving the server private key stored in `raw_data/pki/srv_key.pem`."
812
]
913
},
1014
{
1115
"cell_type": "code",
1216
"execution_count": null,
13-
"metadata": {
14-
"collapsed": true
15-
},
17+
"metadata": {},
1618
"outputs": [],
1719
"source": [
1820
"from scapy.all import *\n",
@@ -22,9 +24,7 @@
2224
{
2325
"cell_type": "code",
2426
"execution_count": null,
25-
"metadata": {
26-
"collapsed": false
27-
},
27+
"metadata": {},
2828
"outputs": [],
2929
"source": [
3030
"record1_str = open('raw_data/tls_session_compromised/01_cli.raw', 'rb').read()\n",
@@ -36,7 +36,6 @@
3636
"cell_type": "code",
3737
"execution_count": null,
3838
"metadata": {
39-
"collapsed": false,
4039
"scrolled": true
4140
},
4241
"outputs": [],
@@ -49,23 +48,19 @@
4948
{
5049
"cell_type": "code",
5150
"execution_count": null,
52-
"metadata": {
53-
"collapsed": true
54-
},
51+
"metadata": {},
5552
"outputs": [],
5653
"source": [
57-
"# Suppose we possess the private key of the server\n",
58-
"# Try registering it to the session\n",
59-
"#key = PrivKey('raw_data/pki/srv_key.pem')\n",
60-
"#record2.tls_session.server_rsa_key = key"
54+
"# Supposing that the private key of the server was stolen,\n",
55+
"# the traffic can be decoded by registering it to the Scapy TLS session\n",
56+
"key = PrivKey('raw_data/pki/srv_key.pem')\n",
57+
"record2.tls_session.server_rsa_key = key"
6158
]
6259
},
6360
{
6461
"cell_type": "code",
6562
"execution_count": null,
66-
"metadata": {
67-
"collapsed": false
68-
},
63+
"metadata": {},
6964
"outputs": [],
7065
"source": [
7166
"record3_str = open('raw_data/tls_session_compromised/03_cli.raw', 'rb').read()\n",
@@ -76,9 +71,7 @@
7671
{
7772
"cell_type": "code",
7873
"execution_count": null,
79-
"metadata": {
80-
"collapsed": false
81-
},
74+
"metadata": {},
8275
"outputs": [],
8376
"source": [
8477
"record4_str = open('raw_data/tls_session_compromised/04_srv.raw', 'rb').read()\n",
@@ -89,34 +82,163 @@
8982
{
9083
"cell_type": "code",
9184
"execution_count": null,
92-
"metadata": {
93-
"collapsed": false
94-
},
85+
"metadata": {},
9586
"outputs": [],
9687
"source": [
88+
"# This is the first TLS Record containing user data. If decryption works,\n",
89+
"# you should see the string \"To boldly go where no man has gone before...\" in plaintext.\n",
9790
"record5_str = open('raw_data/tls_session_compromised/05_cli.raw', 'rb').read()\n",
9891
"record5 = TLS(record5_str, tls_session=record4.tls_session.mirror())\n",
9992
"record5.show()"
10093
]
94+
},
95+
{
96+
"cell_type": "markdown",
97+
"metadata": {},
98+
"source": [
99+
"# Decrypting TLS Traffic Protected with PFS\n",
100+
"\n",
101+
"When PFS is in action, the only way to break TLS 1.2 is to possess decryption keys. They can be retrieved by dumping the process memory, or making the TLS library to write then into a [NSS Key Log](https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/Key_Log_Format) (as allowed by OpenSSL, Chrome or Firefox).\n",
102+
"\n",
103+
"The data used in the following examples was retrieved the following commands:\n",
104+
"```\n",
105+
"cd doc/notebooks/tls/raw_data/\n",
106+
"\n",
107+
"# Start a TLS 1.12 Server using the s_server\n",
108+
"sudo openssl s_server -accept localhost:443 -cert pki/srv_cert.pem -key pki/srv_key.pem -WWW -tls1_2\n",
109+
"\n",
110+
"# Sniff the network and write packets to a file\n",
111+
"sudo tcpdump -i lo -w tls_nss_example.pcap port 443\n",
112+
"\n",
113+
"# Connect to the server using s_client and retrieve the secrets.txt file\n",
114+
"openssl s_client -connect localhost:443 -keylogfile tls_nss_example.keys.txt\n",
115+
"```\n",
116+
"\n",
117+
"## Decrypt a PCAP files\n",
118+
"\n",
119+
"Scapy can parse NSS Key logs, and use the cryptographic material to decrypt TLS traffic from a pcap file."
120+
]
121+
},
122+
{
123+
"cell_type": "code",
124+
"execution_count": null,
125+
"metadata": {},
126+
"outputs": [],
127+
"source": [
128+
"load_layer(\"tls\")\n",
129+
"\n",
130+
"conf.tls_session_enable = True\n",
131+
"conf.tls_nss_filename = \"raw_data/tls_nss_example.keys.txt\"\n",
132+
"\n",
133+
"packets = rdpcap(\"raw_data/tls_nss_example.pcap\")"
134+
]
135+
},
136+
{
137+
"cell_type": "code",
138+
"execution_count": null,
139+
"metadata": {},
140+
"outputs": [],
141+
"source": [
142+
"# Display the HTTP GET query\n",
143+
"packets[11][TLS].show()"
144+
]
145+
},
146+
{
147+
"cell_type": "code",
148+
"execution_count": null,
149+
"metadata": {},
150+
"outputs": [],
151+
"source": [
152+
"# Display the answer containing the secret\n",
153+
"packets[13][TLS].show()"
154+
]
155+
},
156+
{
157+
"cell_type": "markdown",
158+
"metadata": {},
159+
"source": [
160+
"## Decrypt Manually\n",
161+
"\n",
162+
"Internally, the `conf.tls_session_enable` parameter makes Scapy follows TCP records, such as Client Hello or Server Hello, and updates `tlsSession` objects.\n",
163+
"\n",
164+
"Scapy inner behavior is illustrated by the following example."
165+
]
166+
},
167+
{
168+
"cell_type": "code",
169+
"execution_count": null,
170+
"metadata": {},
171+
"outputs": [],
172+
"source": [
173+
"# Read packets from a pcap\n",
174+
"load_layer(\"tls\")\n",
175+
"\n",
176+
"packets = rdpcap(\"raw_data/tls_nss_example.pcap\")\n",
177+
"\n",
178+
"# Load the keys from a NSS Key Log\n",
179+
"nss_keys = load_nss_keys(\"raw_data/tls_nss_example.keys.txt\")"
180+
]
181+
},
182+
{
183+
"cell_type": "code",
184+
"execution_count": null,
185+
"metadata": {},
186+
"outputs": [],
187+
"source": [
188+
"# Parse the Client Hello message from its raw bytes. This configures a new tlsSession object\n",
189+
"client_hello = TLS(raw(packets[3][TLS]))\n",
190+
"\n",
191+
"# Parse the Server Hello message, using the mirrored client_hello tlsSession object\n",
192+
"server_hello = TLS(raw(packets[5][TLS]), tls_session=client_hello.tls_session.mirror())\n",
193+
"\n",
194+
"# Configure the TLS master secret retrieved from the NSS Key Log\n",
195+
"server_hello.tls_session.master_secret = nss_keys[\"CLIENT_RANDOM\"][\"Secret\"]\n",
196+
"\n",
197+
"# Parse remaining TLS messages\n",
198+
"client_finished = TLS(raw(packets[7][TLS]), tls_session=server_hello.tls_session.mirror())\n",
199+
"server_finished = TLS(raw(packets[9][TLS]), tls_session=client_finished.tls_session.mirror())"
200+
]
201+
},
202+
{
203+
"cell_type": "code",
204+
"execution_count": null,
205+
"metadata": {},
206+
"outputs": [],
207+
"source": [
208+
"# Display the HTTP GET query\n",
209+
"http_query = TLS(raw(packets[11][TLS]), tls_session=server_finished.tls_session.mirror())\n",
210+
"http_query.show()"
211+
]
212+
},
213+
{
214+
"cell_type": "code",
215+
"execution_count": null,
216+
"metadata": {},
217+
"outputs": [],
218+
"source": [
219+
"# Display the answer containing the secret\n",
220+
"http_response = TLS(raw(packets[13][TLS]), tls_session=http_query.tls_session.mirror())\n",
221+
"http_response.show()"
222+
]
101223
}
102224
],
103225
"metadata": {
104226
"kernelspec": {
105-
"display_name": "Python 2",
227+
"display_name": "Python 3 (ipykernel)",
106228
"language": "python",
107-
"name": "python2"
229+
"name": "python3"
108230
},
109231
"language_info": {
110232
"codemirror_mode": {
111233
"name": "ipython",
112-
"version": 2
234+
"version": 3
113235
},
114236
"file_extension": ".py",
115237
"mimetype": "text/x-python",
116238
"name": "python",
117239
"nbconvert_exporter": "python",
118-
"pygments_lexer": "ipython2",
119-
"version": "2.7.13"
240+
"pygments_lexer": "ipython3",
241+
"version": "3.9.1"
120242
}
121243
},
122244
"nbformat": 4,

‎doc/notebooks/tls/raw_data/tls_nss_example.keys.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# SSL/TLS secrets log file, generated by OpenSSL
2+
CLIENT_RANDOM c43c799f04ad31e397ee4fe14c8819a19bf5951bbc545cada407c6c7589e60ab b599798159244555ddd10d80b5552a37d327fd6e661f3520194c28ef6e8bb0af6e3fb4d4f9945a61e83a41f2345fa27a

‎doc/notebooks/tls/raw_data/tls_nss_example.pcap

4.11 KB
Binary file not shown.

‎scapy/layers/tls/session.py

Lines changed: 74 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
TLS session handler.
99
"""
1010

11+
import binascii
1112
import socket
1213
import struct
1314

@@ -24,6 +25,53 @@
2425
from scapy.layers.tls.crypto.hkdf import TLS13_HKDF
2526
from scapy.layers.tls.crypto.prf import PRF
2627

28+
# Typing imports
29+
from scapy.compat import Dict
30+
31+
32+
def load_nss_keys(filename):
33+
# type: (str) -> Dict[str, bytes]
34+
"""
35+
Parses a NSS Keys log and returns unpacked keys in a dictionary.
36+
"""
37+
keys = {}
38+
try:
39+
fd = open(filename)
40+
except FileNotFoundError:
41+
warning("Cannot open NSS Key Log: %s", filename)
42+
return {}
43+
else:
44+
with open(filename) as fd:
45+
for line in fd:
46+
if line.startswith("#"):
47+
continue
48+
data = line.strip().split(" ")
49+
if len(data) != 3 or data[0] != data[0].upper():
50+
warning("Invalid NSS Key Log Entry: %s", line.strip())
51+
return {}
52+
53+
try:
54+
client_random = binascii.unhexlify(data[1])
55+
except binascii.Error:
56+
warning("Invalid ClientRandom: %s", data[1])
57+
return {}
58+
59+
try:
60+
secret = binascii.unhexlify(data[2])
61+
except binascii.Error:
62+
warning("Invalid Secret: %s", data[2])
63+
return {}
64+
65+
# Warn that a duplicated entry was detected. The latest one
66+
# will be kept in the resulting dictionary.
67+
if data[0] in keys:
68+
warning("Duplicated entry for %s !", data[0])
69+
70+
keys[data[0]] = {"ClientRandom": client_random,
71+
"Secret": secret}
72+
return keys
73+
74+
2775
# Note the following import may happen inside connState.__init__()
2876
# in order to avoid to avoid cyclical dependencies.
2977
# from scapy.layers.tls.crypto.suites import TLS_NULL_WITH_NULL_NULL
@@ -366,6 +414,10 @@ def __init__(self,
366414
self.server_rsa_key = None
367415
# self.server_ecdsa_key = None
368416

417+
# A dictionary containing keys extracted from a NSS Keys Log using
418+
# the load_nss_keys() function.
419+
self.nss_keys = None
420+
369421
# Back in the dreadful EXPORT days, US servers were forbidden to use
370422
# RSA keys longer than 512 bits for RSAkx. When their usual RSA key
371423
# was longer than this, they had to create a new key and send it via
@@ -546,7 +598,14 @@ def compute_master_secret(self):
546598
log_runtime.debug("TLS: master secret: %s", repr_hex(ms))
547599

548600
def compute_ms_and_derive_keys(self):
549-
self.compute_master_secret()
601+
# Load the master secret from an NSS Key dictionary
602+
if self.nss_keys and self.nss_keys.get("CLIENT_RANDOM", False) and \
603+
self.nss_keys["CLIENT_RANDOM"].get("Secret", False):
604+
self.master_secret = self.nss_keys["CLIENT_RANDOM"]["Secret"]
605+
606+
if not self.master_secret:
607+
self.compute_master_secret()
608+
550609
self.prcs.derive_keys(client_random=self.client_random,
551610
server_random=self.server_random,
552611
master_secret=self.master_secret)
@@ -894,15 +953,25 @@ def __init__(self, _pkt="", post_transform=None, _internal=0,
894953
self.tls_session.ipdst = tcp.underlayer.dst
895954
except AttributeError:
896955
pass
956+
957+
# Load a NSS Key Log file
958+
if conf.tls_nss_filename is not None:
959+
if conf.tls_nss_keys is None:
960+
conf.tls_nss_keys = load_nss_keys(conf.tls_nss_filename)
961+
897962
if conf.tls_session_enable:
898963
if newses:
899964
s = conf.tls_sessions.find(self.tls_session)
900965
if s:
966+
if conf.tls_nss_keys is not None:
967+
s.nss_keys = conf.tls_nss_keys
901968
if s.dport == self.tls_session.dport:
902969
self.tls_session = s
903970
else:
904971
self.tls_session = s.mirror()
905972
else:
973+
if conf.tls_nss_keys is not None:
974+
self.tls_session.nss_keys = conf.tls_nss_keys
906975
conf.tls_sessions.add(self.tls_session)
907976
if self.tls_session.connection_end == "server":
908977
srk = conf.tls_sessions.server_rsa_key
@@ -1110,3 +1179,7 @@ def toPacketList(self):
11101179
conf.tls_sessions = _tls_sessions()
11111180
conf.tls_session_enable = False
11121181
conf.tls_verbose = False
1182+
# Filename containing NSS Keys Log
1183+
conf.tls_nss_filename = None
1184+
# Dictionary containing parsed NSS Keys
1185+
conf.tls_nss_keys = None

‎test/tls.uts

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1565,3 +1565,16 @@ assert [type(x) for x in pkt.msg] == [TLSServerHello, TLSCertificate, TLSCertifi
15651565
# see test/tls/tests_tls_netaccess.uts
15661566

15671567

1568+
###############################################################################
1569+
####################### Decrypt packets from a pcap ##########################
1570+
###############################################################################
1571+
1572+
bck_conf = conf
1573+
conf.tls_session_enable = True
1574+
conf.tls_nss_filename = scapy_path("doc/notebooks/tls/raw_data/tls_nss_example.keys.txt")
1575+
1576+
packets = rdpcap(scapy_path("doc/notebooks/tls/raw_data/tls_nss_example.pcap"))
1577+
assert b"GET /secret.txt HTTP/1.0\n" in packets[11].msg[0].data
1578+
assert b"z2|gxarIKOxt,G1d>.Q2MzGY[k@" in packets[13].msg[0].data
1579+
1580+
conf = bck_conf

0 commit comments

Comments
 (0)