Skip to content

Fix ObjectDisposedException race in PublishTestResultsCommand#5516

Open
avishmsft wants to merge 1 commit intomasterfrom
users/avishjain/fix-publish-test-results-race
Open

Fix ObjectDisposedException race in PublishTestResultsCommand#5516
avishmsft wants to merge 1 commit intomasterfrom
users/avishjain/fix-publish-test-results-race

Conversation

@avishmsft
Copy link
Copy Markdown
Contributor

@avishmsft avishmsft commented Mar 30, 2026

Context

ObjectDisposedException when multiple ##vso[results.publish] commands are emitted in the same step. The root cause is a concurrency bug:
PublishTestResultsCommand is a singleton whose Execute() method stores inputs in instance fields and then starts an async task. The command serialization lock (_commandSerializeLock) releases when
Execute() returns, but the async task continues in the background. A subsequent Execute() call overwrites the instance fields and reinitializes shared singleton services (via GetService()), causing the
first async task to read corrupted state — disposed VssConnection objects, wrong test runner, wrong file list, etc.


Description

Provide a concise summary of the changes introduced in this PR.
1. GetService → CreateService for per-invocation service isolation:
Changed GetService() to CreateService() for ITestDataPublisher, ILegacyTestRunDataPublisher, ITestResultsServer, IFeatureFlagService, and ITestRunPublisher. GetService returns a cached singleton from
HostContext._serviceInstances — a second Execute() call would reinitialize the same instance (overwriting _connection, _featureFlagService, etc.) while the first async task is still using it.

  1. Instance field capture as locals before async boundary:
    Capture all: mutable instance state as local variables in Execute() before the async continuation, and pass them as parameters to PublishTestRunDataAsync() and helper methods.

Risk Assessment (Low / Medium / High)

Low

Unit Tests Added or Updated (Yes / No)

No — Existing ResultsCommandTests (3 tests) continue to pass


Additional Testing Performed

List manual or automated tests performed beyond unit tests (e.g., integration, scenario, regression).


Change Behind Feature Flag (Yes / No)

 No — This is a concurrency correctness fix


Tech Design / Approach

Per-invocation service instances via CreateService + local variable capture.


Documentation Changes Required (Yes/No)

No


Logging Added/Updated (Yes/No)

No — Existing logging is sufficient. No new code paths were added; only the concurrency model was fixed.

Telemetry Added/Updated (Yes/No)

No — Existing telemetry via PublishEventsAsync is preserved


Rollback Scenario and Process (Yes/No)

Yes — Rollback is a single commit revert


Dependency Impact Assessed and Regression Tested (Yes/No)

Yes — No new dependencies introduced

@avishmsft avishmsft force-pushed the users/avishjain/fix-publish-test-results-race branch 2 times, most recently from ffe2d2a to 016204d Compare March 31, 2026 15:48
@avishmsft avishmsft marked this pull request as ready for review March 31, 2026 15:50
@avishmsft avishmsft requested review from a team as code owners March 31, 2026 15:50
@avishmsft
Copy link
Copy Markdown
Contributor Author

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@avishmsft avishmsft force-pushed the users/avishjain/fix-publish-test-results-race branch 3 times, most recently from a7ed563 to 59dcc7a Compare March 31, 2026 18:30
@avishmsft avishmsft added the bug label Apr 2, 2026
Rebased on latest master (includes _isDetectTestRunRetry feature).

Changes from original:
- ResultsCommandExtension: GetService->CreateService for publishers,
  capture all instance fields as locals before async boundary
  (including new _isDetectTestRunRetry), use locals throughout async methods
- TestDataPublisher: CreateService for ITestResultsServer and IFeatureFlagService
- Parser: new overload accepting IFeatureFlagService from caller,
  fallback to GetService when not provided
- TestDataPublisher passes its FeatureFlagService to parser

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@avishmsft avishmsft force-pushed the users/avishjain/fix-publish-test-results-race branch from c289d4e to 31021cd Compare April 2, 2026 09:25
@avishmsft
Copy link
Copy Markdown
Contributor Author

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

private IExecutionContext _executionContext;

//publish test results inputs
private List<string> _testResultFiles;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove these as class variables and jsut make them local variable in execute method so that multiple execution do not conflict...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants