Commit d872075
authored
[PECOBLR-2461] Add comprehensive MST transaction E2E tests (#775)
* Add comprehensive MST transaction E2E tests
Replaces the prior speculative test skeleton with 42 tests across 5
categories:
- TestMstCorrectness (18): commit/rollback/isolation/multi-table
atomicity/repeatable reads/write conflict/parameterized DML/etc.
- TestMstApi (6): DB-API-specific — autocommit, isolation level,
error handling.
- TestMstMetadata (6): cursor.columns/tables/schemas/catalogs inside
a transaction, plus two freshness tests asserting Thrift metadata
RPCs are non-transactional (they see concurrent DDL that the txn
should not see).
- TestMstBlockedSql (9): MSTCheckRule enforcement. Some SHOW/DESCRIBE
commands throw + abort txn, others succeed silently on Python/Thrift
(diverges from JDBC). Both behaviors are explicitly tested so
regressions in either direction are caught.
- TestMstExecuteVariants (2): executemany commit/rollback.
Parallelisation:
- Each test uses a unique Delta table derived from its test name so
pytest-xdist workers don't collide on shared state.
- Tests that spawn concurrent connections to the same table
(repeatable reads, write conflict, freshness) use xdist_group so
the concurrent connections within a single test don't conflict with
other tests on different workers.
Runtime: ~2 minutes on 4 workers (pytest -n 4 --dist=loadgroup),
well within the existing e2e budget.
Co-authored-by: Isaac
Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
* Fix TestMstBlockedSql: SHOW COLUMNS and DESCRIBE QUERY are blocked
CI caught that the initial "not blocked" assertions were wrong — the
server returns TRANSACTION_NOT_SUPPORTED.COMMAND for SHOW COLUMNS
(ShowDeltaTableColumnsCommand) and DESCRIBE QUERY (DescribeQueryCommand)
inside an active transaction.
The server's error message explicitly lists the allowed commands:
"Only SELECT / INSERT / MERGE / UPDATE / DELETE / DESCRIBE TABLE are
supported." DESCRIBE TABLE (basic) remains the only DESCRIBE variant
that is allowed.
Earlier dogfood runs showed SHOW COLUMNS / DESCRIBE QUERY succeeding —
likely because the dogfood warehouse DBR is older than CI. Aligning
tests with the current/CI server behavior.
Co-authored-by: Isaac
Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
* Address PR review comments
- test_auto_start_after_commit: assert the rolled-back id=2 is NOT
present (use _get_ids set equality instead of just row count).
- test_auto_start_after_rollback: same pattern — assert the
rolled-back id=1 is NOT present.
- test_commit_without_active_txn_throws: match specific
NO_ACTIVE_TRANSACTION server error code to ensure we're catching
the right exception, not an unrelated one.
Add _get_ids() helper for checking the exact set of persisted ids.
Verified 42/42 pass against pecotesting in ~1:36 (4 workers).
Co-authored-by: Isaac
Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
---------
Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>1 parent b088a35 commit d872075
1 file changed
Lines changed: 692 additions & 522 deletions
0 commit comments