This guide provides a comprehensive approach to securing Apache Kafka on Debian systems, covering authentication, encryption, authorization, network isolation, and monitoring.
Before starting, ensure you have:
sudo apt install openjdk-11-jdk).kafka_2.13-3.5.2).Update your package list and install essential tools for certificate management and firewall configuration:
sudo apt update
sudo apt install -y openjdk-11-jdk keytool ufw
SASL (Simple Authentication and Security Layer) ensures only authorized clients/brokers can access Kafka. We’ll use SCRAM-SHA-256 (stronger than PLAIN) for authentication.
Create a JAAS file (/etc/kafka/kafka_server_jaas.conf) to define users and credentials. Replace admin and securepassword with strong values:
KafkaServer {
org.apache.kafka.common.security.scram.ScramLoginModule required
username="admin"
password="securepassword"
user_admin="securepassword";
};
Edit /opt/kafka/config/server.properties (adjust path if Kafka is installed elsewhere) to enable SASL:
# Enable SASL for inter-broker and client communication
listeners=SASL_SSL://0.0.0.0:9093
security.inter.broker.protocol=SASL_SSL
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-256
sasl.enabled.mechanisms=SCRAM-SHA-256
sasl.jaas.config=/etc/kafka/kafka_server_jaas.conf
# SSL settings (refer to Section 4 for SSL details)
ssl.keystore.location=/etc/kafka/ssl/kafka.keystore.jks
ssl.keystore.password=keystore_password
ssl.key.password=key_password
ssl.truststore.location=/etc/kafka/ssl/kafka.truststore.jks
ssl.truststore.password=truststore_password
Tell Kafka to use the JAAS file by adding this to /etc/kafka/kafka-env.sh:
export KAFKA_OPTS="-Djava.security.auth.login.config=/etc/kafka/kafka_server_jaas.conf"
SSL/TLS encrypts data in transit between clients and brokers, preventing eavesdropping.
Use keytool to create a keystore (for brokers) and truststore (for clients):
# Create keystore (replace "localhost" with your broker’s hostname in production)
keytool -genkey -alias kafka -keystore /etc/kafka/ssl/kafka.keystore.jks -keyalg RSA -validity 365 -storepass keystore_password -keypass key_password -dname "CN=localhost, OU=IT, O=YourCompany, L=City, ST=State, C=US"
# Export certificate from keystore
keytool -export -alias kafka -file /etc/kafka/ssl/kafka.crt -keystore /etc/kafka/ssl/kafka.keystore.jks -storepass keystore_password
# Create truststore and import the certificate
keytool -import -alias kafka -file /etc/kafka/ssl/kafka.crt -keystore /etc/kafka/ssl/kafka.truststore.jks -storepass truststore_password -noprompt
Add the following to server.properties (already included in Section 3.2):
# Enable SSL for all listeners
listeners=SASL_SSL://0.0.0.0:9093
security.inter.broker.protocol=SASL_SSL
# Keystore/truststore paths and passwords
ssl.keystore.location=/etc/kafka/ssl/kafka.keystore.jks
ssl.keystore.password=keystore_password
ssl.key.password=key_password
ssl.truststore.location=/etc/kafka/ssl/kafka.truststore.jks
ssl.truststore.password=truststore_password
# Restrict protocols and ciphers for stronger security
ssl.enabled.protocols=TLSv1.2,TLSv1.3
ssl.cipher.suites=TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384
ACLs (Access Control Lists) restrict user access to Kafka topics. Use the kafka-acls.sh tool to define permissions.
Add this to server.properties:
authorizer.class.name=kafka.security.authorizer.AclAuthorizer
allow.everyone.if.no.acl.found=false
super.users=User:admin # Grant admin full access
Grant admin read/write access to topic my_topic:
/opt/kafka/bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 \
--add --allow-principal User:admin --operation Read --operation Write --topic my_topic
Grant user1 read-only access to my_topic:
/opt/kafka/bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 \
--add --allow-principal User:user1 --operation Read --topic my_topic
Restrict access to Kafka ports using ufw (Uncomplicated Firewall) to prevent unauthorized connections.
Allow only necessary ports (e.g., 9093 for SASL_SSL, 2181 for ZooKeeper if used):
sudo ufw allow 9093/tcp # SASL_SSL port
sudo ufw allow 2181/tcp # ZooKeeper port (if applicable)
sudo ufw enable
Check active rules to ensure only allowed ports are open:
sudo ufw status verbose
Apply changes by restarting Kafka and testing authentication/encryption.
sudo systemctl restart kafka
sudo systemctl status kafka # Ensure service is running
Use Kafka’s console producer/consumer to verify secure communication:
# Producer (replace "admin" and "securepassword" with your credentials)
/opt/kafka/bin/kafka-console-producer.sh --broker-list localhost:9093 --topic my_topic \
--producer.config /opt/kafka/config/client.properties
# Consumer (same credentials as producer)
/opt/kafka/bin/kafka-console-consumer.sh --bootstrap-server localhost:9093 --topic my_topic --from-beginning \
--consumer.config /opt/kafka/config/client.properties
Note: The client.properties file should include:
security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-256
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
username="admin" \
password="securepassword";
ssl.truststore.location=/etc/kafka/ssl/kafka.truststore.jks
ssl.truststore.password=truststore_password
/var/log/kafka/server.log for suspicious activity.server.properties, JAAS files, and SSL certificates.0.0.0.0) in listeners to reduce exposure.By following these steps, you can significantly enhance the security of your Kafka deployment on Debian, protecting against unauthorized access, data breaches, and man-in-the-middle attacks.