Rasa Kafka 消息代理测试环境全指南从无认证到 SASL/TLS 的 Docker 一键部署【免费下载链接】rasa Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants项目地址: https://gitcode.com/GitHub_Trending/ra/rasa导读本篇文章围绕 Rasa 开源仓库中的 Kafka 测试环境目录系统讲解如何用 Docker Compose 在本地搭建五套覆盖不同安全等级的 Kafka 消息代理broker用于验证 Rasa 对话系统中的 Kafka 事件代理KafkaEventBroker集成。读完本文你将掌握每套配置的端口规划、ZooKeeper 协作机制、SASL_PLAIN / SASL_SCRAM 认证与 TLS 加密的组合方式、JAAS 凭证文件写法、TLS 证书生成流程以及如何在 Rasa 的endpoints.yml中按对应协议接入这些 broker快速搭建可复现的本地测试环境。测试环境概览一套目录五套配置在 Rasa 仓库中Kafka 相关测试环境统一存放在 test_environments/message_and_event_brokers/kafka 下。上级目录 test_environments/message_and_event_brokers/README.md 明确指出该目录用于设置并运行各种消息与事件代理而 Kafka 是其中唯一已提供完整配置的服务。每一套子配置都自带独立的README.md与docker-compose.yml其端口规划遵循一个清晰规律Kafka broker监听在909x端口x为 1~9取决于所启用的认证组合ZooKeeper监听在218x端口x同样随认证组合变化。具体端口映射如下表配置目录认证方式传输加密Kafka 端口ZooKeeper 端口no_authentication无无90922181sasl_plain/no_tlsSASL_PLAIN无90932181sasl_plain/with_tlsSASL_PLAINTLS9094 / 90952181sasl_scram/no_tlsSASL_SCRAM (SHA-256 / SHA-512)无9096 / 90972186 / 2187sasl_scram/with_tlsSASL_SCRAM (SHA-256 / SHA-512)TLS9098 / 90992188 / 2189从源码结构看这种每套认证组合一个独立子目录 独立端口 独立 README的组织方式是为了让开发者可以在同一台机器上同时启动多套 broker互不冲突便于针对不同安全场景分别做集成测试。理解 ZooKeeper 在 Kafka 集群中的角色在所有配置中Kafka 与 ZooKeeper 都以成对方式出现。二者的关系是ZooKeeper 提供分布式协调服务Kafka 负责实际的数据流与客户端连接。具体到 ZooKeeper 的职责见 kafka/README.mdLeader 选举为 Kafka broker 集群执行领导节点选举服务发现与集群拓扑管理让每个 broker 知道谁加入了集群、谁退出了集群、哪个 broker 宕机了以及某个 topic/partition 组合的首选 leader 是谁Topic 生命周期跟踪记录 topic 何时被创建或删除维护 topic 列表集群一致性视图总体上为 Kafka 集群提供一份同步一致的集群状态视图。在 no_authentication/docker-compose.yml 中可以看到两者的标准协作方式ZooKeeper 通过ZOOKEEPER_CLIENT_PORT: 2181暴露客户端端口Kafka broker 通过KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181指向它并以depends_on: zookeeper: condition: service_healthy确保 ZooKeeper 健康检查通过后才启动 broker两个容器都通过nc -z探活zookeeper: image: confluentinc/cp-zookeeper:7.3.2 container_name: zookeeper-no-authentication environment: ZOOKEEPER_CLIENT_PORT: 2181 ZOOKEEPER_TICK_TIME: 2000 healthcheck: test: nc -z localhost 2181 || exit 1 interval: 10s retries: 10 start_period: 15s timeout: 10s所有配置统一使用 Confluent 官方镜像confluentinc/cp-kafka:7.3.2与confluentinc/cp-zookeeper:7.3.2这意味着测试环境与真实生产环境的 Kafka 行为一致Rasa 客户端连接方式也与生产相同。配置一无认证 Kafka端口 9092最简单的一档配置位于 no_authentication面向broker 不要求客户端做任何形式认证的测试场景。Kafka 监听9092客户端使用localhost:9092连接。启动命令极简docker-compose up -d完整的 docker-compose.yml 内容如下version: 3.8 services: zookeeper: image: confluentinc/cp-zookeeper:7.3.2 container_name: zookeeper-no-authentication environment: ZOOKEEPER_CLIENT_PORT: 2181 ZOOKEEPER_TICK_TIME: 2000 healthcheck: test: nc -z localhost 2181 || exit 1 interval: 10s retries: 10 start_period: 15s timeout: 10s kafka-broker: image: confluentinc/cp-kafka:7.3.2 container_name: kafka-broker-no-authentication ports: - 9092:9092 depends_on: zookeeper: condition: service_healthy environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092 KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1 KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1 healthcheck: test: nc -z localhost 9092 || exit 1 interval: 30s retries: 10 start_period: 15s timeout: 10s几个关键环境变量的作用KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT声明监听器PLAINTEXT使用明文协议不做任何加密与认证KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092向客户端通告的连接地址这就是客户端能通过localhost:9092访问的原因KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1/KAFKA_TRANSACTION_STATE_LOG_*单 broker 环境下将内部 topic 的副本因子降为 1避免因副本不足导致启动失败。配置二SASL_PLAIN 认证 无 TLS端口 9093位于 sasl_plain/no_tls用于broker 要求客户端认证、但传输不加密的测试场景。Kafka 监听9093通信走不安全的明文连接。启动方式同样是一条命令docker-compose up -d该配置预置了两个测试用户定义在broker_jaas.conf中用户密码adminadmin-secretalicealice-secret客户端连接时需要使用上述任一用户名/密码。完整的 docker-compose.yml 如下version: 3.8 services: zookeeper: image: confluentinc/cp-zookeeper:7.3.2 container_name: zookeeper-sasl-plain-no-tls environment: ZOO_MY_ID: 1 ZOOKEEPER_CLIENT_PORT: 2181 ZOOKEEPER_TICK_TIME: 2000 ZOOKEEPER_SASL_ENABLED: false healthcheck: test: nc -z localhost 2181 || exit 1 interval: 10s retries: 10 start_period: 15s timeout: 10s kafka-broker: image: confluentinc/cp-kafka:7.3.2 container_name: kafka-broker-sasl-plain-no-tls ports: - 9093:9093 depends_on: zookeeper: condition: service_healthy environment: KAFKA_LISTENERS: SASL_PLAINTEXT://:9092 KAFKA_ADVERTISED_LISTENERS: SASL_PLAINTEXT://localhost:9093 KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 ZOOKEEPER_SASL_ENABLED: false KAFKA_INTER_BROKER_LISTENER_NAME: SASL_PLAINTEXT KAFKA_SASL_ENABLED_MECHANISMS: PLAIN KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: PLAIN KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 KAFKA_OPTS: -Djava.security.auth.login.config/etc/kafka/broker_jaas.conf volumes: - ./broker_jaas.conf:/etc/kafka/broker_jaas.conf healthcheck: test: nc -z localhost 9093 || exit 1 interval: 30s retries: 10 start_period: 15s timeout: 10sJAAS 凭证文件用户从哪来这里的用户并非来自数据库而是由 Java 的 JAASJava Authentication and Authorization Service登录模块定义。挂载的 broker_jaas.conf 内容如下KafkaServer { org.apache.kafka.common.security.plain.PlainLoginModule required usernameadmin passwordadmin-secret user_adminadmin-secret user_alicealice-secret; }; Client{};要点解读KafkaServer段通过PlainLoginModule声明了 broker 自身的认证身份username/password用于 broker 之间通信以及可认证的客户端用户列表user_admin、user_aliceClient{}空段表示 broker 作为 ZooKeeper 客户端时无需凭据该文件通过volumes挂载到容器内/etc/kafka/broker_jaas.conf再由KAFKA_OPTS中的-Djava.security.auth.login.config参数指定加载。注意KAFKA_LISTENERS: SASL_PLAINTEXT://:9092与KAFKA_ADVERTISED_LISTENERS: SASL_PLAINTEXT://localhost:9093的差异容器内部监听 9092而对外通告为宿主机的 9093这正是端口映射9093:9093生效后客户端访问localhost:9093的原理。配置三SASL_PLAIN 认证 TLS 加密端口 9094 / 9095位于 sasl_plain/with_tls同时启用认证与传输加密。它又细分为两个子场景ssl_all_conectionsbroker 证书 SAN 设为0.0.0.0客户端可接受来自任意 IP 的 broker 证书仅用于测试严禁用于生产ssl_localhostbroker 证书 SAN 设为localhost客户端仅接受运行在 localhost 上的 broker 证书。两个场景分别暴露端口 9094 与 9095客户端连接 URL 分别为localhost:9094与localhost:9095可用用户仍是admin/admin-secret与alice/alice-secret。以 ssl_localhost/docker-compose.yml 为例它展示了 Kafka 同时配置 SASL_SSL 与 PLAINTEXT 双监听器的典型写法version: 3.8 services: zookeeper: image: confluentinc/cp-zookeeper:7.3.2 container_name: zookeeper-sasl-plain-tls-localhost environment: ZOO_MY_ID: 1 ZOOKEEPER_CLIENT_PORT: 2181 ZOOKEEPER_TICK_TIME: 2000 ZOOKEEPER_SASL_ENABLED: false healthcheck: test: nc -z localhost 2181 || exit 1 interval: 10s retries: 10 start_period: 15s timeout: 10s kafka-broker: image: confluentinc/cp-kafka:7.3.2 container_name: broker-sasl-plain-tls-localhost ports: - 9095:9095 - 29095:29095 depends_on: zookeeper: condition: service_healthy environment: KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 ZOOKEEPER_SASL_ENABLED: false KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 KAFKA_LISTENERS: SASL_SSL://0.0.0.0:9095, PLAINTEXT://0.0.0.0:29095 KAFKA_ADVERTISED_LISTENERS: SASL_SSL://localhost:9095, PLAINTEXT://localhost:29095 KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT KAFKA_SASL_ENABLED_MECHANISMS: PLAIN KAFKA_OPTS: -Djava.security.auth.login.config/etc/kafka/broker_jaas.conf KAFKA_SSL_KEYSTORE_FILENAME: kafka.server.keystore.jks KAFKA_SSL_KEYSTORE_CREDENTIALS: ssl_keystore_credentials KAFKA_SSL_KEY_CREDENTIALS: ssl_key_credentials KAFKA_SSL_ENABLED_PROTOCOLS: TLSv1.2,TLSv1.1,TLSv1 KAFKA_LOG4J_ROOT_LOGLEVEL: DEBUG volumes: - ./broker_jaas.conf:/etc/kafka/broker_jaas.conf - ./server.keystore.jks:/etc/kafka/secrets/kafka.server.keystore.jks - ./ssl_key_credentials:/etc/kafka/secrets/ssl_key_credentials - ./ssl_keystore_credentials:/etc/kafka/secrets/ssl_keystore_credentials healthcheck: test: nc -z localhost 9095 || exit 1 interval: 30s retries: 10 start_period: 15s timeout: 10s该配置体现了三个值得注意的设计点双监听器SASL_SSL://0.0.0.0:9095供客户端做认证 TLS 连接PLAINTEXT://0.0.0.0:29095供 broker 之间内部通信KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT从而避免内部流量也承担认证开销证书与密码分离keystoreserver.keystore.jks通过KAFKA_SSL_KEYSTORE_FILENAME指定keystore 与私钥密码分别存放在ssl_keystore_credentials、ssl_key_credentials两个挂载文件中由容器内机制读取协议兼容KAFKA_SSL_ENABLED_PROTOCOLS: TLSv1.2,TLSv1.1,TLSv1同时启用三个 TLS 版本便于在不同客户端环境下测试。理解 SANSubject Alternative Name该配置的父级 README 专门讲解了 SAN 证书的作用SAN 证书允许一张证书同时保护多个主机名或 IP。例如证书 SAN 为194.3.5.1时仅当连接的 Kafka broker IP 是194.3.5.1时客户端才接受证书证书 SAN 为localhost时仅当连接的 broker 主机名是localhost时客户端才接受证书证书 SAN 为rasa.com时仅当连接的 broker 主机名是rasa.com时才有效证书 SAN 为0.0.0.0时客户端接受任意主机名或 IP 的证书——方便测试但文档明确警告DO NOT USE THIS IN PRODUCTION。客户端连接 TLS broker 的两条路径要让客户端成功连接启用 TLS 的 broker必须把 CA 证书加入客户端的信任池以验证 broker 身份加入操作系统证书池将ca-cert安装到运行客户端的那台机器的系统证书池加入客户端内存证书池仅在客户端运行期间生效具体做法需查阅所用 Kafka 客户端库的证书管理文档。如果你希望跳过 broker 身份验证也可以指示客户端跳过证书校验。此时 TLS 加密通信仍然生效只是 broker 的身份不再被验证适用于本地开发场景。配置四SASL_SCRAM 认证 无 TLS端口 9096 / 9097位于 sasl_scram/no_tls按哈希算法再拆分为两档scram_sha_256使用 SCRAM-SHA-256Kafka 端口 9096ZooKeeper 端口 2186scram_sha_512使用 SCRAM-SHA-512Kafka 端口 9097ZooKeeper 端口 2187。SCRAMSalted Challenge Response Authentication Mechanism的显著特点是服务端不存储明文密码而是存储加盐的挑战-响应凭证安全性高于 PLAIN 机制。也因此SCRAM 用户无法像 PLAIN 那样写在 JAAS 文件里而是需要在 ZooKeeper 上通过kafka-configs命令动态创建。以 scram_sha_512/README.md 为例启动流程分为三步# 1. 先启动 ZooKeeper docker-compose up -d zookeeper # 2. 在 ZooKeeper 上创建 kafkabroker 与 kafkaclient 两个用户 docker exec -it zookeeper-sasl-scram-sha-512 bash cd /etc/kafka/client KAFKA_OPTS-Djava.security.auth.login.configzookeeper_client_jaas.conf kafka-configs --zookeeper localhost:2187 --alter --add-config SCRAM-SHA-512[iterations4096,passwordpassword] --entity-type users --entity-name kafkabroker KAFKA_OPTS-Djava.security.auth.login.configzookeeper_client_jaas.conf kafka-configs --zookeeper localhost:2187 --alter --add-config SCRAM-SHA-512[iterations4096,passwordpassword] --entity-type users --entity-name client # 3. 退出容器后启动 broker exit docker-compose up -d kafka-broker其中iterations4096指定 SCRAM 迭代次数passwordpassword是两用户共用的测试密码。SCRAM 的 SHA-256 版本操作完全一致只需把端口与算法名替换为 2186/SCRAM-SHA-256。由于 SCRAM 场景下 ZooKeeper 自身也启用了 SASLscram_sha_512/docker-compose.yml 中 ZooKeeper 容器通过一组KAFKA_OPTS开启 quorum 认证并挂载三份 JAAS 配置zookeeper: image: confluentinc/cp-zookeeper:7.3.2 container_name: zookeeper-sasl-scram-sha-512-no-tls ports: - 2187:2187 environment: ZOOKEEPER_SERVER_ID: 1 ZOOKEEPER_CLIENT_PORT: 2187 ZOOKEEPER_TICK_TIME: 2000 ZOOKEEPER_LOG4J_ROOT_LOGLEVEL: DEBUG KAFKA_OPTS: -Djava.security.auth.login.config/etc/kafka/secrets/zookeeper_server_jaas.conf -Dquorum.auth.enableSasltrue -Dquorum.auth.learnerRequireSasltrue -Dquorum.auth.serverRequireSasltrue -Dquorum.cnxn.threads.size20 -Dzookeeper.authProvider.1org.apache.zookeeper.server.auth.SASLAuthenticationProvider -Dzookeeper.authProvider.2org.apache.zookeeper.server.auth.DigestAuthenticationProvider -DjaasLoginRenew3600000 -DrequireClientAuthSchemesasl -Dquorum.auth.learner.loginContextQuorumLearner -Dquorum.auth.server.loginContextQuorumServer volumes: - ./zookeeper_server_jaas.conf:/etc/kafka/secrets/zookeeper_server_jaas.conf - ./zookeeper_client_jaas.conf:/etc/kafka/client/zookeeper_client_jaas.conf kafka-broker: image: confluentinc/cp-kafka:7.3.2 container_name: kafka-broker-sasl-scram-sha-512-no-tls ports: - 9097:9097 depends_on: - zookeeper environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:2187 KAFKA_ADVERTISED_LISTENERS: SASL_PLAINTEXT://localhost:9097 KAFKA_MIN_INSYNC_REPLICAS: 1 KAFKA_SASL_ENABLED_MECHANISMS: SCRAM-SHA-512 KAFKA_SECURITY_INTER_BROKER_PROTOCOL: SASL_PLAINTEXT KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: SCRAM-SHA-512 KAFKA_AUTO_CREATE_TOPICS_ENABLE: true KAFKA_OFFSETS_RETENTION_MINUTES: 172800 KAFKA_LOG4J_LOGGERS: kafka.authorizer.loggerDEBUG,kafka.controllerDEBUG KAFKA_LOG4J_ROOT_LOGLEVEL: DEBUG KAFKA_SUPER_USERS: User:kafkabroker;User:kafkaclient KAFKA_ZOOKEEPER_SASL_ENABLED: true KAFKA_ALLOW_EVERYONE_IF_NO_ACL_FOUND: false KAFKA_OPTS: -Dzookeeper.sasl.clienttrue -Dzookeeper.sasl.clientconfigClient -Djava.security.auth.login.config/etc/kafka/secrets/conf/kafka_server_jaas.conf volumes: - ./broker_jaas.conf:/etc/kafka/secrets/conf/kafka_server_jaas.conf对应的 zookeeper_server_jaas.conf 同时定义了 ZooKeeper 服务端、quorum 服务端与 quorum learner 三套身份Server { org.apache.zookeeper.server.auth.DigestLoginModule required user_adminpassword; }; QuorumServer { org.apache.zookeeper.server.auth.DigestLoginModule required user_zookeeperpassword; }; QuorumLearner { org.apache.zookeeper.server.auth.DigestLoginModule required usernamezookeeper passwordpassword; };broker 侧的关键参数包括KAFKA_SASL_ENABLED_MECHANISMS: SCRAM-SHA-512启用机制、KAFKA_SUPER_USERS: User:kafkabroker;User:kafkaclient声明超级用户、KAFKA_ALLOW_EVERYONE_IF_NO_ACL_FOUND: false表示未配置 ACL 时不允许任意用户访问。配置五SASL_SCRAM 认证 TLS 加密端口 9098 / 9099位于 sasl_scram/with_tls是安全等级最高的一档scram_sha_256Kafka 端口 9098ZooKeeper 端口 2188TLS 证书位于ssl/子目录scram_sha_512Kafka 端口 9099ZooKeeper 端口 2189TLS 证书位于ssl/子目录。启动流程与配置四相同只是容器名、端口与算法名不同。以 scram_sha_512/README.md 为例# 启动 ZooKeeper docker-compose up -d zookeeper # 创建用户 docker exec -it zookeeper-scram-sha-512-tls bash cd /etc/kafka/client KAFKA_OPTS-Djava.security.auth.login.configzookeeper_client_jaas.conf kafka-configs --zookeeper localhost:2189 --alter --add-config SCRAM-SHA-512[iterations4096,passwordpassword] --entity-type users --entity-name kafkabroker KAFKA_OPTS-Djava.security.auth.login.configzookeeper_client_jaas.conf kafka-configs --zookeeper localhost:2189 --alter --add-config SCRAM-SHA-512[iterations4096,passwordpassword] --entity-type users --entity-name client # 退出并启动 broker exit docker-compose up -d kafka-broker客户端连接时需配置用户名/密码为kafkaclient/passwordSASL 机制为SCRAM-SHA-512可选启用 TLS 加密或跳过证书校验并把ssl/目录下的ca-cert导入客户端证书池。scram_sha_512/docker-compose.yml 中 broker 的关键片段kafka-broker: image: confluentinc/cp-kafka:7.3.2 container_name: kafka-broker-scram-sha-512-tls ports: - 9099:9099 - 29099:29099 depends_on: - zookeeper environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:2189 KAFKA_LISTENERS: SASL_SSL://0.0.0.0:9099, PLAINTEXT://0.0.0.0:290929 KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT KAFKA_ADVERTISED_LISTENERS: SASL_SSL://localhost:9099, PLAINTEXT://localhost:29099 KAFKA_SSL_ENABLED_PROTOCOLS: TLSv1.2,TLSv1.1,TLSv1 KAFKA_SSL_KEYSTORE_FILENAME: server.keystore.jks KAFKA_SSL_KEYSTORE_CREDENTIALS: ssl_keystore_credentials KAFKA_SSL_KEY_CREDENTIALS: ssl_key_credentials KAFKA_SASL_ENABLED_MECHANISMS: SCRAM-SHA-256 KAFKA_AUTO_CREATE_TOPICS_ENABLE: true KAFKA_OFFSETS_RETENTION_MINUTES: 172800 KAFKA_LOG4J_LOGGERS: kafka.authorizer.loggerDEBUG,kafka.controllerDEBUG KAFKA_LOG4J_ROOT_LOGLEVEL: DEBUG KAFKA_SUPER_USERS: User:kafkabroker;User:kafkaclient KAFKA_ZOOKEEPER_SASL_ENABLED: true KAFKA_ALLOW_EVERYONE_IF_NO_ACL_FOUND: false KAFKA_OPTS: -Dzookeeper.sasl.clienttrue -Dzookeeper.sasl.clientconfigClient -Djava.security.auth.login.config/etc/kafka/secrets/conf/kafka_server_jaas.conf volumes: - ./ssl:/etc/kafka/secrets - ./broker_jaas.conf:/etc/kafka/secrets/conf/kafka_server_jaas.conf与 PLAINTLS 配置类似这里同样采用 SASL_SSL 对外、PLAINTEXT 对内的双监听器结构并把ssl/目录含 keystore 与密码文件整体挂载到容器。需要留意的是仓库该文件中KAFKA_LISTENERS的 PLAINTEXT 端口写作290929而KAFKA_ADVERTISED_LISTENERS为29099两者不一致属于仓库自带的笔误若实际运行该配置请以29099为准SHA-256 对应目录 scram_sha_256/docker-compose.yml 中该值是正确的29098。深入 TLS 证书体系CA、keystore 与 SANSASL_SSL / SSL 场景的父级 README 详细解释了这套测试环境背后的证书体系使用RSA 算法生成用于签名/验证/加密/解密数据的公私钥证书颁发机构CA是签发签名其他实体证书的可信实体。CA 的私钥用于签署其他实体的证书必须妥善保护、不得共享CA 的公钥即CA certificate下发给客户端客户端用它验证收到的证书确实由该 CA 签名而非恶意实体伪造TLS 通信需要三样东西CA 私钥、CA 公钥ca-cert、由 CA 私钥签名的 broker 证书。握手过程为客户端连接 broker 时broker 发送自己的证书客户端用 CA 证书验证该证书校验通过后客户端方可连接。每个 TLS 子目录都包含一组预生成证书与密钥文件ssl_localhost/README.md 给出了完整清单docker-compose.ymlKafka 与 ZooKeeper 容器编排文件server.keystore.jks存放服务器证书与 CA 证书的 keystoreca-certCA 证书用于签署服务器证书必须作为受信证书导入 keystore客户端也应导入它以验证 broker 身份ca-keyCA 私钥用于生成ca-certcert-requestbroker 的证书签名请求必须先经 CA 签名才能使用signed-server-certbroker 的已签名证书必须导入 keystoressl_keystore_credentials/ssl_key_credentialskeystore 密码与 CA 私钥密码文件broker_jaas.confbroker 的 JAAS 配置包含客户端可用的用户名密码。这些预生成证书仅用于测试不得用于生产。文档同时注明证书有效期至 2024 年 3 月 30 日若证书过期可按下文流程重新生成。重新生成 TLS 证书的完整命令链如果预生成证书已过期或你需要为其他 SAN 生成新证书ssl_localhost/README.md 与 ssl_all_conections/README.md 提供了可直接复制的完整命令链。以 localhost 版SANlocalhost为例# 1. 创建 CA 公私钥公钥即 CA 证书 openssl req -x509 -newkey rsa:4096 -keyout ca-key -out ca-cert -days 365 -nodes -subj /CNlocalhost/OUAtom/ORasa/LBerlin/STGermany/CGE -passin pass:123456 -passout pass:123456 # 2. 创建受 storepass 与 keypass 保护的服务器 keystoreSAN 设为 localhost keytool -dname CNlocalhost,OUAtom,ORasa,LBerlin,SGermany,CGE -keystore server.keystore.jks -alias localhost -validity 365 -genkey -keyalg RSA -storetype pkcs12 -ext SANIP:localhost -storepass 123456 -keypass 123456 # 3. 生成证书签名请求 keytool -keystore server.keystore.jks -alias localhost -certreq -file cert-request -storepass 123456 -keypass 123456 -ext SANIP:localhost # 4. 用 CA 私钥签署请求 openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-request -out signed-server-cert -days 365 -CAcreateserial -passin pass:123456 # 5. 将 CA 证书导入 keystore用于解封 broker 发送的已签名证书 keytool -keystore server.keystore.jks -alias CARoot -import -file ca-cert -storepass 123456 -keypass 123456 # 6. 将已签名证书导入 keystore keytool -noprompt -keystore server.keystore.jks -alias localhost -import -file signed-server-cert -storepass 123456 -keypass 123456 -ext SANDNS:localhost若需生成接受任意 IP的版本只需把第 2、3、6 步中的SANIP:localhost/SANDNS:localhost全部替换为SANIP:0.0.0.0即 ssl_all_conections 的做法。命令中的密码统一为123456与ssl_keystore_credentials、ssl_key_credentials文件内容对应-days 365表示证书一年有效可按需调整。故障排查工具箱父级 README 提供了一组针对 TLS 场景的排查命令查看 keystore 内容确认证书与别名是否正确导入keytool -list -v -keystore server.keystore.jks -storepass 123456 -keypass 123456检查私钥是否受密码保护openssl rsa -check -in ca-key -passin pass:123456验证 CA 证书能否解开已签名证书openssl verify -CAfile ca-cert signed-server-cert验证 TLS 连接是否工作按需选择 TLS 版本# TLS 1.0 openssl s_client -debug -connect localhost:29092 -tls1 # TLS 1.1 openssl s_client -debug -connect localhost:29092 -tls1_1 # TLS 1.2 openssl s_client -debug -connect localhost:29092 -tls1_2与 Rasa Kafka 事件代理打通endpoints.yml 配置搭建好上述任一 broker 之后关键在于让 Rasa 作为客户端接入。Rasa 的 Kafka 事件代理实现在 rasa/core/brokers/kafka.py 的KafkaEventBroker类中它基于confluent-kafka客户端库其构造函数支持与上文五套配置一一对应的参数urlbootstrap 服务器地址如localhost:9092security_protocolPLAINTEXT、SSL、SASL_PLAINTEXT、SASL_SSL四选一默认SASL_PLAINTEXTsasl_mechanismPLAIN、GSSAPI、OAUTHBEARER、SCRAM-SHA-256、SCRAM-SHA-512默认PLAINsasl_username/sasl_passwordSASL 认证凭据ssl_cafile/ssl_certfile/ssl_keyfileCA 证书、客户端证书、客户端私钥路径ssl_check_hostname是否校验证书与 broker 主机名匹配用于防范中间人攻击partition_by_sender是否按 sender_id 分区默认False随机分区。Rasa 官方文档 docs/docs/event-brokers.mdx 的 Kafka 章节对此有完整说明仓库中的测试端点文件提供了四套可直接对照的示例。对应上文五种 broker 配置endpoints.yml写法如下① 无认证 brokerPLAINTEXT对应端口 9092参见 kafka_plaintext_endpoint.ymlevent_broker: type: kafka security_protocol: PLAINTEXT topic: topic url: localhost client_id: kafka-python-rasa② SASL_PLAIN 无 TLSSASL_PLAINTEXT对应端口 9093参见 kafka_sasl_plaintext_endpoint.ymlevent_broker: type: kafka security_protocol: SASL_PLAINTEXT topic: topic url: localhost partition_by_sender: True sasl_username: username sasl_password: password sasl_mechanism: PLAIN把sasl_mechanism换成SCRAM-SHA-256或SCRAM-SHA-512、并指向 9096/9097 端口即可对接 SCRAM 无 TLS 场景。③ 仅 TLSSSL参见 kafka_ssl_endpoint.ymlevent_broker: type: kafka security_protocol: SSL topic: topic url: localhost ssl_cafile: CARoot.pem ssl_certfile: certificate.pem ssl_keyfile: key.pem ssl_check_hostname: True④ SASL TLSSASL_SSL对应端口 9094/9095、9098/9099参见 kafka_sasl_ssl_endpoint.ymlevent_broker: type: kafka security_protocol: SASL_SSL topic: topic url: localhost sasl_username: username sasl_password: password sasl_mechanism: PLAIN ssl_cafile: CARoot.pem ssl_certfile: certificate.pem ssl_keyfile: key.pem ssl_check_hostname: True这里ssl_cafile应指向测试环境目录中的ca-cert即 CA 公钥客户端用它验证 broker 证书ssl_check_hostname: True时客户端会额外校验 broker 主机名与证书 SAN 是否一致——这与上文证书 SAN 决定客户端接受哪个主机名的机制直接呼应。仓库还通过 kafka_invalid_sasl_mechanism.yml 与 kafka_invalid_security_protocol.yml 两个测试文件验证了非法sasl_mechanism与非法security_protocol会被正确拒绝说明这些参数在 KafkaEventBroker 中是受约束的合法值。实践建议与注意事项综合五套配置与 Rasa 接入方式给出以下实操建议按测试目标选档仅验证事件流转选无认证9092验证账号密码认证选 SASL_PLAIN 无 TLS9093模拟生产安全要求选 SASL_SCRAM TLS9098/9099。SASL_SCRAM 因不落明文密码比 PLAIN 更适合接近生产的测试同一台机器可并行启动多套五套配置端口互不重叠9092~9099 / 2181、2186~2189可同时运行对比测试SCRAM 场景务必先建用户再启 brokerkafka-configs创建用户依赖 ZooKeeper 先行就绪顺序颠倒会导致 broker 因找不到用户凭据而启动失败TLS 场景先装 CA 证书客户端必须信任ca-cert才能完成握手若只想快速验证功能可让客户端跳过证书校验但需明白此时身份未被验证预生成证书有有效期仓库证书标注有效期至 2024 年 3 月过期后按上文的opensslkeytool命令链重新生成即可生产环境禁止使用 0.0.0.0 SAN 证书也禁止使用测试 JAAS 中公开的默认用户名密码。总结Rasa 仓库提供的 Kafka 测试环境目录 是一套精心组织的、覆盖从零安全到SASL_SCRAM TLS 全量安全的 Docker 化 Kafka 测试矩阵。每套配置都配有可独立运行的docker-compose.yml、JAAS 凭证文件、预生成 TLS 证书与逐步操作文档端口规划清晰、互不冲突配合 rasa/core/brokers/kafka.py 的KafkaEventBroker与 endpoints.yml 示例 即可在本地完整复现 Rasa 与 Kafka 在各种认证组合下的事件代理集成测试。理解其中的监听器规划、JAAS 机制、SCRAM 用户引导流程与 TLS 证书链原理也能直接迁移到真实生产环境的 Kafka 安全配置中。【免费下载链接】rasa Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants项目地址: https://gitcode.com/GitHub_Trending/ra/rasa创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考