Overcoming the Perils of Kafka Secret Sprawl (Tejal Adsul, Confluent) Kafka Summit 2020
0 likes870 views
The document discusses the perils of secret sprawl and clear text secrets in Apache Kafka configurations. It proposes using a key management system (KMS) to securely store secrets and introducing a configuration provider that allows replacing secrets in configuration files with references to the KMS. The Kafka Improvement Proposals (KIPs) 297 and 421 aim to address this by externalizing secrets to providers and automatically resolving secrets during configuration parsing. Key recommendations include selecting a KMS, moving secrets to it, adding the configuration provider, and replacing secrets with indirection tuples pointing to the KMS.
![KIP-297
Config Transformer
${ provider [:path] : key }](https://image.slidesharecdn.com/s035tejaladsulkafkasumittejaladsul2-200910050434/85/Overcoming-the-Perils-of-Kafka-Secret-Sprawl-Tejal-Adsul-Confluent-Kafka-Summit-2020-48-320.jpg)
![KIP-297
Config Transformer
${ provider [:path] : key }](https://image.slidesharecdn.com/s035tejaladsulkafkasumittejaladsul2-200910050434/85/Overcoming-the-Perils-of-Kafka-Secret-Sprawl-Tejal-Adsul-Confluent-Kafka-Summit-2020-49-320.jpg)
![KIP-297
Config Transformer
${ provider [:path] : key }
Config Provider
KMS](https://image.slidesharecdn.com/s035tejaladsulkafkasumittejaladsul2-200910050434/85/Overcoming-the-Perils-of-Kafka-Secret-Sprawl-Tejal-Adsul-Confluent-Kafka-Summit-2020-50-320.jpg)
![KIP-297
Config Transformer
${ provider [:path] : key }
Config Provider
KMS
secret
key](https://image.slidesharecdn.com/s035tejaladsulkafkasumittejaladsul2-200910050434/85/Overcoming-the-Perils-of-Kafka-Secret-Sprawl-Tejal-Adsul-Confluent-Kafka-Summit-2020-51-320.jpg)
Overcoming the Perils of Kafka Secret Sprawl (Tejal Adsul, Confluent) Kafka Summit 2020
- 1. Overcoming the Perils of Kafka Secret Sprawl Tejal Adsul | Security Engineer | @TejalAdsul
- 2. Agenda
- 3. Agenda • Security implications of clear text secrets and secret sprawl • Key Management Systems • Kafka Config Provider
- 4. Secrets
- 6. Secrets Digital Certificates API keys Username and Passwords
- 7. How do we use secrets in Apache Kafka
- 8. Apache Kafka Security 101 Plaintext Plaintext Producer ConsumerPlaintext B R O K E R B R O K E R
- 9. Apache Kafka Security 101 Producer ConsumerPlaintext B R O K E R B R O K E R Plaintext Plaintext
- 10. Apache Kafka Security 101 Producer ConsumerB R O K E R B R O K E R Network Attacks
- 11. Apache Kafka Security 101 Security Goal Reduce or Minimize Potential Vulnerable Points
- 12. ENCRYPTION Apache Kafka Security 101 Producer B R O K E R
- 13. Authentication Apache Kafka Security 101 Producer B R O K E R Client Authentication
- 14. Authentication Apache Kafka Security 101 Producer B R O K E R Server Authentication
- 15. # list of brokers used for bootstrapping knowledge about the rest of the cluster # format: host1:port1,host2:port2 ... bootstrap.servers=localhost:9092 # specify the compression codec for all data generated: none, gzip, snappy, lz4, zstd compression.type=none #SASL Configuration sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username=”kafka-producer" password=”producer-secret"; Kafka Producer Configuration File Apache Kafka Security 101 Producer
- 16. Kafka Broker Configuration File Apache Kafka Security 101 B R O K E R # The id of the broker. This must be set to a unique integer for each broker. broker.id=0 # The address the socket server listens on. It will get the value returned from listeners=SASL_SSL://:9092 # SASL Configuration sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username=“admin” password=“admin-secret” user_kafka-producer=”producer-secret";
- 17. TLS Configuration Keystore Truststore Keystore Truststore Apache Kafka Security 101 Producer B R O K E R
- 18. Kafka Broker Configuration File Apache Kafka Security 101 B R O K E R # The id of the broker. This must be set to a unique integer for each broker. broker.id=0 # The address the socket server listens on. It will get the value returned from listeners=SASL_SSL://:9092 # SASL Configuration sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username=“admin” password=“admin-secret” user_kafka-producer=”producer-secret"; ssl.keystore.location=/var/private/ssl/kafka.server.keystore.jks ssl.keystore.password=ZtFCq5EYdcV30GwL ssl.truststore.location=/var/private/ssl/kafka.server.truststore.jks ssl.truststore.password=y77VVeQYqC8e8ceB
- 19. Security Implications of Clear Text Secrets
- 25. Clear text secrets in Configuration File Syndrome
- 29. Secret Sprawl
- 39. Encryption Audit logs Rotations and Version Control Access Control
- 40. KMS
- 42. KIP-297 Externalizing Secrets for Connect Configuration
- 43. KIP-297
- 44. KIP-297
- 45. KIP-297
- 48. KIP-297 Config Transformer ${ provider [:path] : key }
- 49. KIP-297 Config Transformer ${ provider [:path] : key }
- 50. KIP-297 Config Transformer ${ provider [:path] : key } Config Provider KMS
- 51. KIP-297 Config Transformer ${ provider [:path] : key } Config Provider KMS secret key
- 52. Config Provider KIP-297 databaseSecret$123 Vault Config Provider Vault db_password /var/private/vault db_password = databaseSecret$123 db_password = ${ vault : /var/private/vault : db_password }
- 53. ${ vault : /var/private/vault : db_password } Config Provider KIP-297 databaseSecret$123databaseSecret$123 Config Transformer Vault Config Provider Vault {/var/private/vault:db_password} db_password
- 55. Configure a Configuration Provider config.providers = vault, file // Vault Config Provider config.providers.vault.class = org.vault.configs.GenericVaultConfigProvider config.providers.vault.param.token = /run/secrets/vault-token config.providers.vault.param.uri = 168.229.49.188 // File Config Provider config.providers.file.class = org.apache.configs.FileConfigProvider KIP-297 B R O K E R
- 56. KIP-421 Automatically resolve external configurations.
- 57. KIP-421 Configuration Config Def Abstract Config configuration parsed & validated configuration Name : SSL_KEYSTORE_PASSWORD Type: PASSWORD Importance: High Default_Value: null
- 58. KIP-421 broker.id = 0 security.inter.broker.protocol = SSL ssl.keystore.location = /var/private/ssl/kafka.server.keystore.jks ssl.keystore.password = YB3PgSf7M2t3mUo9 B R O K E R
- 59. KIP-421 broker.id = 0 security.inter.broker.protocol = SSL ssl.keystore.location = /var/private/ssl/kafka.server.keystore.jks ssl. keystore.password = ${vault:ssl.keystore.password} B R O K E R
- 60. KIP-421 B R O K E R broker.id = 0 security.inter.broker.protocol = SSL ssl.keystore.password = ${vault:ssl. keystore.password} config.providers = vault config.providers.vault.class = com.org.apache.configs.VaultConfigProvider config.providers.vault.param.token = /run/secrets/vault-token config.providers.vault.param.uri = 168.229.49.188
- 61. KIP-421 Abstract Config broker.id = 0 security.inter.broker.protocol = SSL ssl.key.password = ${vault:ssl.keystore.password} config.providers = vault config.providers.vault.class = com.org.apache.configs.VaultConfigProvider config.providers.vault.param.token = /run/secrets/vault-token config.providers.vault.param.uri = 168.229.49.188
- 62. KIP-421 Abstract Config Vault Config Provider vault.param.token = /run/secrets/vault-token vault.param.uri = 168.229.49.188
- 63. KIP-421 Abstract Config broker.id = 0 security.inter.broker.protocol = SSL ssl.keystore.password = ${vault:ssl.keystore.password} config.providers = vault config.providers.vault.class = com.org.apache.configs.VaultConfigProvider config.providers.vault.param.token = /run/secrets/vault-token config.providers.vault.param.uri = 168.229.49.188 ${}
- 64. KIP-421 Abstract Config ssl.keystore.password = ${vault:ssl.keystore.password} Config Transformer
- 65. KIP-421 Abstract Config ssl.keystore.password = YB3PgSf7M2t3mUo9 Config Transformer
- 66. KIP-421 Abstract Config ssl.keystore.password = YB3PgSf7M2t3mUo9 broker.id = 0 security.inter.broker.protocol = SSL Config Def
- 67. KIP-421 Abstract Config ssl.keystore.password = YB3PgSf7M2t3mUo9 broker.id = 0 security.inter.broker.protocol = SSL Config Def
- 68. Securing Passwords in Log File Config DefConfiguration TYPE
- 69. Recap 1. Select Key Management System 2. Move Secrets to KMS 3. Add the Configuration provider to Kafka 4. Replace the secrets in configuration file with indirection tuple
- 70. ssl.keystore.password = YB3PgSf7M2t3mUo9 ssl.keystore.password = YB3PgSf7M2t3mUo9 Config Transformer Vault Config Provider Vault ssl.keystore.password ssl.keystore.password YB3PgSf7M2t3mUo9YB3PgSf7M2t3mUo9 ssl.keystore.password=${vault:ssl.keystore.password} ssl.keystore.password=YB3PgSf7M2t3mUo9 1 2 3 45 6 7 8 Abstract Config Config Def
- 72. Demo
- 73. Tejal Adsul
- 74. Reference • Apache Kafka Security 101 • Secret Sprawl • KIP-297: Externalizing Secrets for Connect Configurations • KIP-421: Automatically resolve external configurations • Confluent Secret Protection