Skip to content

Add mutual SSL support for pipe sinks#18080

Open
Caideyipi wants to merge 3 commits into
apache:masterfrom
Caideyipi:codex/pipe-mutual-ssl
Open

Add mutual SSL support for pipe sinks#18080
Caideyipi wants to merge 3 commits into
apache:masterfrom
Caideyipi:codex/pipe-mutual-ssl

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Summary

This PR extends the pipe thrift SSL sink path to support mutual SSL authentication, following the general thrift client mTLS support added in #18026.

Changes

  • Add pipe sink/connector SSL key-store parameters:
    • sink.ssl.key-store-path
    • sink.ssl.key-store-pwd
    • connector.ssl.key-store-path
    • connector.ssl.key-store-pwd
  • Let iotdb-thrift-ssl-sink / iotdb-thrift-ssl-connector read both sink and connector SSL aliases for:
    • SSL enable flag
    • trust store path/password
    • key store path/password
  • Pass the optional key store path/password through:
    • IoTDBSslSyncSink
    • IoTDBSyncClientManager
    • IoTDBSyncClient
    • DataNode sync sink/client manager
    • ConfigNode sync sink/client manager
  • Preserve existing one-way SSL behavior: key-store parameters are optional, but if either key-store path or password is specified, both must be specified.
  • Keep async thrift sink behavior explicit by rejecting all SSL attributes there, including the new key-store attributes.
  • Extend legacy pipe sink to accept connector/sink SSL aliases and to pass key-store settings into both its thrift client and SessionPool.
  • Hide ssl.key-store-pwd in pipe parameter value masking.

Tests

  • Added UT coverage for:
    • ssl.key-store-pwd value hiding.
    • DataRegion sync SSL sink accepting mutual SSL parameters.
    • DataRegion sync SSL connector accepting connector-prefixed SSL aliases.
    • DataRegion sync SSL sink rejecting incomplete key-store parameters.
    • Async thrift sink rejecting SSL key-store parameters.
    • Legacy pipe sink accepting connector-prefixed mutual SSL aliases.
    • ConfigRegion sink accepting mutual SSL parameters.
    • ConfigRegion sink rejecting incomplete key-store parameters.
  • Added IT coverage:
    • IoTDBPipeMutualSSLIT starts a receiver with thrift SSL client auth enabled, creates an iotdb-thrift-ssl-sink pipe with trust/key stores, and verifies pipe data transfer through a mutual-SSL receiver.

Local Verification

  • mvn -Ddevelocity.off=true spotless:apply -pl iotdb-api/pipe-api,iotdb-core/datanode,iotdb-core/confignode,integration-test -P with-integration-tests
  • git diff --check

I also attempted targeted UT execution with mvn -Ddevelocity.off=true test -pl iotdb-api/pipe-api -Dtest=PipeParametersTest ..., but the local Windows environment failed to start/continue the JVM due to native memory/pagefile exhaustion (There is insufficient memory for the Java Runtime Environment to continue). The same environment issue also affected broader compile/test attempts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant