[fix](be) Use trusted JDBC catalog metadata for connection tests#62464
[fix](be) Use trusted JDBC catalog metadata for connection tests#62464CalvinKirs wants to merge 1 commit intoapache:masterfrom
Conversation
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: Prevent BE test_jdbc_connection from trusting caller-supplied JDBC driver metadata and validation SQL. BE now fetches trusted JDBC catalog metadata from FE by catalog id, and the JDBC tester verifies the driver checksum before loading the driver jar.
### Release note
Harden JDBC connection testing so the backend only loads trusted driver metadata from the catalog.
### Check List (For Author)
- Test: FE unit test / Manual test
- FE unit test: ./run-fe-ut.sh --run org.apache.doris.datasource.jdbc.JdbcExternalCatalogTest
- Manual test: build-support/check-format.sh
- Manual test: cd fe && mvn -pl fe-core,be-java-extensions/jdbc-scanner -am validate -DskipTests -Dmaven.build.cache.enabled=false -rf :fe-core
- No BE unit test: ./run-be-ut.sh --run --filter=JdbcUtilsTest.* -j 4 failed in CMake dependency detection (missing OpenMP_C)
- Behavior changed: Yes (JDBC connection test now ignores caller-supplied driver metadata and SQL, and uses trusted catalog metadata instead)
- Does this need documentation: No
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
|
/review |
There was a problem hiding this comment.
Found 1 blocking issue.
- Blocking:
CREATE JDBC CATALOGnow fails duringcheckWhenCreating()whentest_connection=trueand a BE is alive.CatalogFactory.createCatalog()callscatalog.checkWhenCreating()beforeCatalogMgr.createCatalogImpl()registers the catalog, but the new BE test path sends onlycatalog_idandFrontendServiceImpl.getJdbcTestConnectionInfo()resolves it fromEnv.getCurrentEnv().getCatalogMgr(). For a brand-new catalog that lookup returns null, so the BE-side connectivity test rejects the request even though the FE already has the trusted in-memory properties. The new unit tests only cover request/table builders and do not exercise this creation-time path.
Critical checkpoint conclusions:
- Goal of the task: Partially achieved. The PR removes caller-controlled JDBC metadata from the BE test path, but it breaks JDBC catalog creation/testing in the default
test_connection=trueflow. - Minimality/focus: Mostly focused, but the new trusted-metadata lookup adds a lifecycle dependency on catalog registration that the previous code path did not have.
- Concurrency: No new lock-order or shared-state concurrency issue was identified in the reviewed paths.
- Lifecycle management: Blocking issue present. The new FE lookup assumes the catalog already exists in
CatalogMgr, which is false whilecheckWhenCreating()is still running. - Configuration changes: No new config items.
- Compatibility / parallel paths: No blocking protocol-compatibility issue found in the optional new RPC/proto fields, and existing JDBC scan paths still use FE-provided checksum metadata.
- Special conditional checks: No additional blocker found.
- Test coverage: Insufficient for this change. Added FE unit tests do not cover the creation-time BE connection test path that regressed.
- Observability: Existing returned status/error messages are adequate for this failure path.
- Transaction / persistence / data writes: Not applicable for this review.
- FE-BE variable passing: The new
catalog_idpropagation exists, but it does not work for the creation-time path because the catalog is not yet registered. - Performance: The extra FE RPC is acceptable for a non-hot connection-test path.
- Other issues: None beyond the blocker above were confirmed.
Overall opinion: request changes until the creation-time lifecycle regression is fixed.
| } | ||
|
|
||
| private TTableDescriptor buildTestConnectionThrift() throws DdlException { | ||
| public PJdbcTestConnectionRequest buildTestConnectionRequest(TOdbcTableType tableType) { |
There was a problem hiding this comment.
Blocking: this switches the BE connectivity test to resolve the catalog back out of CatalogMgr, but checkWhenCreating() runs before the new catalog is registered there. The create flow is CatalogFactory.createCatalog(...)->catalog.checkWhenCreating() first, and only later CatalogMgr.createCatalogImpl(...)->createCatalogInternal(...) adds the catalog. In that window FrontendServiceImpl.getJdbcTestConnectionInfo() will see getCatalog(request.getCatalogId()) == null, so CREATE CATALOG ... PROPERTIES('type'='jdbc', ...) now fails whenever test_connection=true and a BE is available. Please keep using the already-validated in-memory catalog properties during creation, or add a separate trusted path that does not depend on the catalog already existing in CatalogMgr.
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: Prevent BE test_jdbc_connection from trusting caller-supplied JDBC driver metadata and validation SQL. BE now fetches trusted JDBC catalog metadata from FE by catalog id, and the JDBC tester verifies the driver checksum before loading the driver jar.
Release note
Harden JDBC connection testing so the backend only loads trusted driver metadata from the catalog.
Check List (For Author)
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)