Get Your Insecure PostgreSQL Passwords to SCRAM
Download as PPTX, PDF0 likes1,698 views
Jonathan S. Katz discusses password management in PostgreSQL, emphasizing the shift towards using SCRAM-SHA-256 for enhanced security over previous methods like plaintext and MD5. The presentation details the vulnerabilities associated with older password mechanisms and explains how SCRAM provides a more secure authentication process by allowing mutual verification without password exposure. Attendees will learn how to transition systems to implement SCRAM-SHA-256 effectively.
Get Your Insecure PostgreSQL Passwords to SCRAM
- 1. Get Your Insecure Passwords to SCRAM Jonathan S. Katz PGCon 2020 May 6, 2020
- 2. aka "A Tale of Two Hippos" 2
- 3. " …and an Elephant" 3
- 4. " …Guest Starring Blue Elephant" 4
- 5. • Vice President of Platform Engineering, Crunchy Data • Previously: Engineering leadership in startups • Longtime PostgreSQL community contributor • Advocacy & various committees for PGDG • @postgresql + .org content • Director, PgUS • Conference organization + speaking • @jkatz05 About Me 5
- 6. Market Leading Data Security Crunchy Data is the leader in PostgreSQL security. Common Criteria certification and essential security enhancements make Crunchy Certified PostgreSQL the trusted open source PostgreSQL distribution for the enterprise. Cloud Ready Data Management Whether deploying to public or private clouds, Crunchy Data provides market leading, open source, Kubernetes- based technology solutions, giving your team the choice and flexibility for how you deploy your data. Leader in Open Source Enterprise PostgreSQL Crunchy Data gives organizations the technology, support, and confidence to enjoy the power and efficiency of open source PostgreSQL.
- 7. • You’ll understand the evolution of how password management in PostgreSQL • If you use PostgreSQL’s password mechanisms in production, you should want to start using SCRAM-SHA-256 • You will know how to upgrade your systems to use SCRAM-SHA-256 By The End of This Talk 7
- 8. Let’s Log Into PostgreSQL!
- 9. 9 $ psql -h localhost app Password for user jkatz: psql: FATAL: password authentication failed for user "jkatz"
- 10. 10 $ psql -h localhost –U jkatz app Password for user jkatz: psql (12.2) Type "help" for help. app=#
- 11. It Just Works!
- 12. …but, what just works?
- 13. • It’s very easy to take password management for granted when it just works • Before version 10, offered two methods of storing passwords and validating passwords • plain • crypt • md5 How Do Passwords Work in PostgreSQL? 13
- 15. Meet grayhippo 15 This is "grayhippo" grayhippo loves PostgreSQL, and loves storing large amounts of data. grayhippo's password is "datalake"
- 16. Meet redhippo 16 This is "redhippo" redhippo also loves PostgreSQL. redhippo likes to play pretend, and in particular, likes to pretend to be grayhippo.
- 17. • Stored in plain text • Communicated in plain text • Method was kept available while drivers were updating to “md5” method. • Dropped in PostgreSQL 10* • “Plaintext” validation is still requested for some methods, such as LDAP PostgreSQL Plaintext Passwords 17 password
- 18. Plaintext Password Authentication Flow 18 Hi! I want to authenticate. I’m “grayhippo”
- 19. Plaintext Password Authentication Flow 19 Hi grayhippo! Please tell me your password
- 20. Plaintext Password Authentication Flow 20 My password is “datalake”
- 21. Plaintext Password Authentication Flow 21 Confirmed! You can access PostgreSQL as “grayhippo”
- 22. And all was well.
- 23. …except we know it’s not.
- 24. The Eavesdropper 24 My password is “datalake”
- 25. What Could Go Wrong? 25 Hi! I want to authenticate. I’m “grayhippo”
- 26. … 26 Hi grayhippo! Please tell me your password
- 29. …oh 29 LOL! I got your data
- 30. What About TLS?
- 31. • Supersedes Securely Socket Layers (SSL) • PostgreSQL refers to “SSL” everywhere in the configuration • Performs a “secure handshake” between two parties and encrypts all traffic between them • Use the “hostssl” authentication type to require TLS connections • Can be used as a method of authentication in PostgreSQL…but that’s a different talk • (PostgreSQL 12: clientert=verify-full – allows for two-factor authentication using certificates + another authentication method!) Transport Layer Security 31
- 32. The Eavesdropper…With TLS 32 My password is “datalake” T L S
- 33. 33 …
- 34. 34 💡
- 35. If At First You Don’t Succeed… 35 Hey, can you get me a dump of the entire database?
- 36. … 36 pg_dumpall
- 37. 37 -- -- Roles -- CREATE ROLE grayhippo;ALTER ROLE grayhippo WITH NOSUPERUSER INHERIT NOCREATEROLE NOCREATEDB LOGIN NOREPLICATION NOBYPASSRLS PASSWORD 'datalake'; …and if we inspect the file
- 38. Don’t Forget Manners! 38 Thanks! Oh your password is in this file, but don’t worry I won’t do anything bad.
- 39. Perhaps MD5 Can Do Better
- 40. • Derived by computing md5 hash of the combination of the plaintext password and the username • Stored in hexadecimal format with "md5" prepended to it • Communicated using the PostgreSQL "md5 protocol" • PostgreSQL server sends a 4 byte random salt • Connecting client computes md5 hash of password + username, then appends the 4 byte salt and computes another md5 hash PostgreSQL MD5 Password Authentication 40 "md5" + md5(password + username)
- 41. MD5 Authentication Flow 41 Hi! I want to authenticate. I’m “grayhippo”
- 42. MD5 Authentication Flow 42 Hi grayhippo! Let’s use the MD5 method. Here are some random bytes: “r4nd”
- 43. MD5 Authentication Flow 43 "md5" + md5("datalake" + "grayhippo") = md55133cd3d6af5b408fa0bd7fbbde1dedd
- 44. MD5 Authentication Flow 44 "md5" + md5("md55133cd3d6af5b408fa0bd7fbbde1dedd " + "r4nd") = md558dc9038c1b6284fabdb4eefa421a881
- 46. MD5 Authentication Flow 46 Confirmed! You can access PostgreSQL as “grayhippo”
- 47. Is redhippo Out Of Luck?
- 49. 49 Hi! I want to authenticate. I’m “grayhippo”
- 50. 50 Hi grayhippo! Let’s use the MD5 method. Here are some random bytes: "n0n0"
- 53. Nope! 53 Meh.
- 54. If At First You Don’t Succeed… 54 Hey, can you get me a dump of the entire database?
- 55. … 55 pg_dumpall
- 56. 56 -- -- Roles -- CREATE ROLE grayhippo;ALTER ROLE grayhippo WITH NOSUPERUSER INHERIT NOCREATEROLE NOCREATEDB LOGIN NOREPLICATION NOBYPASSRLS PASSWORD 'md55133cd3d6af5b408fa0bd7fbbde1dedd'; …and if we inspect the file
- 57. 57 Hi! I want to authenticate. I’m “grayhippo”
- 58. 58 Hi grayhippo! Let’s use the MD5 method. Here are some random bytes: "n0n0"
- 59. 59 "md5" + md5("md55133cd3d6af5b408fa0bd7fbbde1dedd " + "n0n0") = md5863c562e50cbee7c19662191fe737157
- 61. 61 Confirmed! You can access PostgreSQL as “grayhippo”
- 62. 62 Really? LOL!
- 63. • CREATE ROLE grayhippo PASSWORD 'datalake'; • If redhippo can read the server logs, redhippos now has the plain text password • Alternatives • Via psql: Use password • Via other: pre-hash the password with the formula: "md5" + md5(password + username) • …but both of these methods still log the md5 hashed password to the server logs • SELECT rolname, rolpassword FROM pg_authid; • Requires a privileged user, but if redhippo has said privileges… Other Ways redhippo Can Get the Password 63
- 64. These Password Methods Need to SCRAM!
- 65. • Defined in RFC 5802 • Part of the Simple Authentication and Security Layer (SASL) family (RFC 4422) • PostgreSQL uses RFC7767 which specifies the use of the SHA-256 hashing algorithm • Designed to be easy to implement and use all standard, accepted methods • SHA-256 (RFC 6234) • HMAC (RFC 2014) • SASLprep (RFC 4013), based on stringprep (RFC 3434) • …and we'll talk about how PostgreSQL does this • Allows for two parties to verify they both know a secret without exchanging the secret Salted Challenge Response Authentication Mechanism 65
- 66. Two Parties Can Verify They Both Know a Secret Without Exchanging the Secret
- 68. • In PostgreSQL need to build a "SCRAM verifier" • A client can build the SCRAM verifier and transmit it to the server. • Even in this form, an eavesdropper will be unable to access the secret Creating a Password For SCRAM 68 <DIGEST>$<ITERATIONS>:<SALT>$<STORED_KEY>:<SERVER_KE Y>
- 69. • PostgreSQL 10 introduced support for "SCRAM-SHA-256" • PostgreSQL 11 introduced support for "SCRAM-SHA-256-PLUS", which is used for channel binding (more on that later) Building a SCRAM Secret - DIGEST 69 "SCRAM-SHA-256"
- 70. • Iterations is the number of times the HMAC signature is computed with hashing function (SCRAM-SHA-256) • We'll start talking about the full computation of this in two slides • By default, PostgreSQL uses 4096 iterations. If you built your own secret, you can customize the number of iterations. Building a SCRAM Secret - ITERATIONS 70 "4096"
- 71. • A cryptographically secure randomly generated value • PostgreSQL defaults to 16 bytes • Ultimately stored in base64 representation Building a SCRAM Secret - SALT 71 "s+1VLTv5oCfNEymVKi01Fw=="
- 72. • Before creating the hashed password, it needs to be normalized using SASLPrep* • *SASLprep is for UTF-8 encoded strings, but PostgreSQL accepts multiple encodings…so we perform a modified SASLprep • If password is ASCII or is not UTF-8 encoded, this step is skipped • 4 Steps: • Step 1: Remove non-ASCII space characters. If password ends up being empty, then return original password • Step 2: Normalize the password using NFKC form. If empty, then return original password • Step 3: Check for prohibited characters. If any are detected, then return original password • Step 4: Bi-directional characters. If any prohibited Bi-directional characters detected, return original password. If there is a "RandALCat" character and either a) at least one "LCat" character OR b) does not end with a "RandALCat" character, return original password • If we get through that, return normalized password! Building a SCRAM Secret – SASLPrep the Password 72 It's still "datalake"
- 73. • The first iteration for generating the salted password uses the following formula: • HMAC using SHA-256 with "password" as the key signing a message with "salt" concatenated with a 32-bit value with final bit set to 1 • Store this value as an aggregator • For the remaining iterations: • Calculate the HMAC using SHA-256 with "password" as the key signing a message that is composed of HMAC calculated from the previous iteration • XOR this HMAC with the aggregator (aggregator = aggregator XOR HMAC) Building a SCRAM Secret – Generate the Salted Password 73 In base64 for this example, the value is: "oJZnL+tf7yE1QWp5fykec7xEgw1cwfyr5Jh3SudbWio="
- 74. • The SHA-256 hash of the "Client Key", stored in a base64 representation • …the "Client Key" is a HMAC using the salted password as the key and "Client Key" as the message Building a SCRAM Secret – STORED KEY 74 "z+tqsnBaCmgSJQBLf1cPOmq7n80PW3aRjzeuAGkkuq4="
- 75. Stored Key 75 Plaintext Password Salted Hashed Password Client Key Salt Hash Iterations HMAC HMAC "Client Key" SHA-256 Stored Key
- 76. • A HMAC using the salted password as the key and "Server Key" as the message Building a SCRAM Secret – SERVER KEY 76 "9ZWHsv+XZAmrpkiEnDPm05wg55jLRrITWspTqsz1zD4="
- 77. Server Key 77 Plaintext Password Salted Hashed Password Server Key Salt Hash Iterations HMAC HMAC "Server Key"
- 78. • You can just use "password" from psql • If your PostgreSQL driver supports it, its password creation facilities • It may delegate it to libpq, which has said facilities Building a SCRAM Secret – "Easy Button" 78 That sounds much simpler…
- 79. Send the SCRAM Secret to PostgreSQL 79 ALTER ROLE grayhippo PASSWORD 'SCRAM-SHA- 256$4096:s+1VLTv5oCfNEymVKi01Fw==$z+tqsnBaCmgSJQBLf1cPOmq7n 80PW3aRjzeuAGkkuq4=:9ZWHsv+XZAmrpkiEnDPm05wg55jLRrITWspTqsz 1zD4='; (We'll let redhippo listen)
- 80. • Identifying information sent: • Stored Key: A hash of a HMAC using the computed salted password and "Client Key" • Server Key: A HMAC using the computed salted password and "Server Key" • In other words, we did not send the original secret in any recognizable form • redhippo can still try to brute force the password or try other offline attacks, but with a high number of iterations (and good password selection), this is hard. We Don't Care That redhippo Listens? 80 Just wait until my quantum computer boots up…
- 82. • The goal of SCRAM authentication is for two parties to verify that each ones knows the shared secret • To prevent against replay attacks, one-time nonces are used for the session being authenticated SCRAM Authentication 82 Just to save space, pretend I'm listening on the next few slides
- 83. SCRAM Authentication Flow 83 Hi! I want to authenticate. I’m “grayhippo”
- 84. SCRAM Authentication Flow 84 Hi grayhippo! Let’s use the SASL method. Here are some available methods: "SCRAM-SHA-256"
- 85. SCRAM Authentication Flow 85 OK. I choose "SCRAM-SHA-256". Here is a nonce I generated: "FVazuC8Hjl46XPPCs2L9RFhq"
- 86. SCRAM Authentication Flow 86 Cool. Here is my nonce added to your nonce: "FVazuC8Hjl46XPPCs2L9RFhqMxMjEzh5txq6gZv6iuW813Aa" Here is the number of iterations to use to computed a salted password: "4096" Here is the salt for you to use: "s+1VLTv5oCfNEymVKi01Fw==" Send me proof that you know the password.
- 87. This is the magical step!
- 88. SCRAM Authentication Flow: Generating Proof 88 My password is "datalake" so from that I am going to computed the salted password using the "SHA-256" hashing method, the "s+1VLTv5oCfNEymVKi01Fw==" salt, and applying the method for "4096" iterations. Basically, like generating the salted hash password. Which was "oJZnL+tf7yE1QWp5fykec7xEgw1cwfyr5Jh3SudbWio=" in base64.
- 89. Recall: Client Key 89 Plaintext Password Salted Hashed Password Client Key Salt Hash Iterations HMAC HMAC "Client Key"
- 90. Recall: Stored Key 90 Plaintext Password Salted Hashed Password Client Key Salt Hash Iterations HMAC HMAC "Client Key" SHA-256 Stored Key
- 91. SCRAM Authentication Flow: Generating Proof 91 From that I can derive the client key (see password generation) which is "eq9WtJSH7PLTxsdv8SJQqJA4+6SU3f03nWE1TW4Ppqg=" And the stored key which is "z+tqsnBaCmgSJQBLf1cPOmq7n80PW3aRjzeuAGkkuq4=" I can create the client signature by taking a HMAC using the SHA- 256 hashing method with the stored key and the message containing information from the authorization sessions headers
- 92. Client Signature 92 Stored Key Client Signature $AUTHENTICATION_MESSAGE HMAC “Authentication Message” is composed of some of the one-time information shared between the server and client
- 93. SCRAM Authentication Flow: Generating Proof 93 And I can generate the client proof by calculating client key XOR client signature
- 94. SCRAM Authentication Flow: Sending Proof 94 I'm sending you the client proof that I built up. It's: "MhvH1xAaVVhUrp4d5TCyjIiMsmlNQQAFTgI57AksP7g=" Also, here is our combined nonce.
- 95. SCRAM Authentication Flow: Server Verification 95 Okay, I can compute the client signature because I know the stored key and I have the authentication session info, too. I can get what you think the client key by calculating client proof XOR client signature If the SHA-256 digest of your client key matches the stored key, then I know you know the password! Plaintext Password Hashed Password Client Key Salt Iterati ons "Client Key" Stored Key
- 96. SCRAM Authentication Flow: Server Verification 96 But…I still need to prove to you that I know the password. I'm going to send you my server signature which is an HMAC using the SHA-256 method using my stored server key and the authorization session info as the message.
- 97. Server Signature 97 Server Key Server Signature $AUTHENTICATION_MESSAGE HMAC “Authentication Message” is composed of some of the one-time information shared between the server and client
- 98. SCRAM Authentication Flow 98 Here's my server signature: "DOCb6oniIvWOCY0aDWKGtaUtUip3wgcQQA1PO5dp1dU="
- 99. Technically, grayhippo is Authenticated At This Point. But, can grayhippo trust the PostgreSQL server?
- 100. SCRAM Authentication Flow: Client Verification 100 First let me generate the server key. This is just like generating the SCRAM secret, and I have the salted password available. I can then generate an HMAC using the SHA-256 method and the server key and the authorization session info as the message. If this value matches your server signature, then I know you know my password!
- 101. We Never Exchanged the Secret!
- 102. • Identifying information sent: • Stored Key: A hash of the HMAC using the computed salted password and "Client Key" • Server Key: A HMAC using the computed salted password and "Server Key" • A combined client/server nonce that can only be used once for this session • Information about the session from the authorization headers Recall What We Transmitted 102 Grr…
- 104. Looks Like redhippo is Stuck!
- 105. But…What About a Rogue Server?
- 106. • Recall: As the final step of SCRAM, the server sends a server signature to the client. • The client is authenticated, but what if the server is not who you think it is? Case #1: Server "Claims" To Know Secret 106 Here's my server signature, you're authenticated now!
- 107. Case #1: Server "Claims" To Know Secret 107 Awesome, here is all of my secret information
- 108. • This is why the client must also verify the server, to ensure the server actually knows the client's password. Case #1: Server "Claims" To Know Secret 108 Thanks! I'm going to go sell this for a lot of money and buy up so many fruits and vegetables!
- 109. • Recall: We can use TLS to secure the connection endpoints between the client and server • But…how do we know that the server sending the signature is the same instance we originally connected to? Case #2: Elephant-in-the-Middle Attack 109 TLS
- 110. Case #2: Elephant-in-the-Middle Attack 110 TLS OK. I choose "SCRAM-SHA- 256". Here is a nonce I generated: Hmm…I want to get in on this…
- 111. Case #2: Elephant-in-the-Middle Attack 111 TLS OK, here's a bunch of stuff that looks normal because your connection is already of a secure socket so this must appear to be normal…nothing weird going on here…
- 113. • Introduced in PostgreSQL 11, channel binding ensures that the TLS handshake is still the same when the client and server are identifying each other • Specified for SCRAM in RFC 5802, TLS channel bindings in RFC 5929 • PostgreSQL uses "tls-server-end-point", which uses a hash of the server certificate to bind the channels • A server that supports channel binding advertises it as "SCRAM-SHA-256- PLUS" • A client then requests to use channel binding (and in PostgreSQL 13 client can require to use channel binding!) Channel Binding 113
- 114. Channel Binding 114 TLS OK. I choose "SCRAM-SHA-256-PLUS". Here is a channel binding header with your certificate. And the nonce Oh no…I won't be able to prove that I can validate that certificate…
- 115. Upgrading to SCRAM
- 116. • As of this presentation, all client drivers and libraries support SCRAM except for the Swift driver • https://wiki.postgresql.org/wiki/List_of_drivers#Drivers • In postgresql.conf set password_encryption to scram-sha-256 • Keep md5 as your authentication method in pg_hba.conf until all your users have re-hashed their passwords • ...have your users re-hash their passwords. Easiest way is password • Once all of your users have re-hashed their password, go back to pg_hba.conf and switch your authentication method to scram-sha-256 Upgrading to SCRAM 116
- 117. • As of this presentation, all client drivers and libraries support SCRAM except for the Swift driver https://wiki.postgresql.org/wiki/List_of_drivers#Drivers Driver Support for SCRAM 117
- 118. Conclusion
- 119. • It's good to question "it just works" as this can lead to better solutions • Beyond PostgreSQL, SCRAM is a general purpose solution. Consider implementing it in your other applications where you need to verify secrets • Example PostgreSQL Password Creator: https://gist.github.com/jkatz/e0a1f52f66fa03b732945f6eb94d9c21 • Example (poorly written) SCRAM implementation with PostgreSQL: https://gist.github.com/jkatz/7444eda78a6fff18ab5d74c024e3761d • …oh, upgrade your passwords to use SCRAM. Now. • Also look into using "clientcert=verify-full" if you're running PostgreSQL 12 – two factor authentication! Not All Password Methods Are the Same 119
- 120. Thank You! 120
- 121. Thank You Jonathan S. Katz jonathan.katz@crunchydata.com @jkatz05
Editor's Notes
- #74: PBKDF2 – Password-Based Key Derivation Function