Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repositories using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Contributing to HOBL Developer Scenarios

This guide covers conventions, patterns, and requirements for modifying or creating developer scenarios in the HOBL codebase. For general HOBL usage (lab setup, DUT configuration, running tests), see README.md.

Scenario Structure
Platform Parity
PowerShell Coding Standards (Windows)
Shell Script Standards (macOS)
Timing and Metrics
Prep Version Bumping
Error Handling
Path Handling
Architecture Awareness
ETW Phase Markers (Windows)
Python Path Resolution
Visual Studio Integration
Checklist Before Submitting a PR

Scenario Structure

Each developer scenario has both a Windows and macOS variant. The folder layout follows this pattern:

scenarios/
  windows/<scenario>/
    <scenario>.py                    # Python class (orchestrator)
    __init__.py
    <scenario>_resources/
      <scenario>_prep.ps1            # One-time setup (install deps, clone repos)
      <scenario>_run.ps1             # Timed workload execution
      <scenario>_teardown.ps1        # Cleanup (optional)
  MacOS/mac_<scenario>/
    mac_<scenario>.py
    __init__.py
    mac_<scenario>_resources/
      mac_<scenario>_prep.sh
      mac_<scenario>_run.sh

Python Orchestrator (`<scenario>.py`)

The Python file defines a class that inherits from scenarios.app_scenario.Scenario and controls the scenario lifecycle:

Method	Purpose
`setUp()`	Checks `prep_version`, uploads resources to DUT, runs prep script if needed
`runTest()`	Executes the run script in a loop (default iteration count set via `Params.setDefault`)
`tearDown()`	Calls base tearDown, then runs teardown script if one exists
`kill()`	Emergency cleanup of processes

Key class attributes:

module = __module__.split('.')[-1]     # Auto-derived scenario name
prep_version = "2"                      # Controls prep re-execution (see below)
resources = module + "_resources"       # Resource folder name

Script Roles

Prep script — Installs dependencies (SDKs, tools, repos). Runs once per prep_version. Should be idempotent where possible.
Run script — The timed workload. Called multiple times (loop iterations). Must produce metrics CSV.
Teardown script — Optional cleanup after all iterations complete.

Platform Parity

Windows and macOS scripts for the same scenario must be functionally equivalent:

Time the same phases (e.g., if Windows times build, macOS must also time build)
Use the same CSV key names (e.g., both use build_time, not build_real on one and build_time on the other)
Compute scenario_runtime the same way — always wall-clock time
macOS scripts may capture extra detail (e.g., user, sys, cputime via /usr/bin/time -p), but scenario_runtime must match semantically

Before modifying one platform, check the other platform's script to keep them aligned.

PowerShell Coding Standards (Windows)

Script Template

Every Windows prep/run script should follow this structure:

param(
    [string]$logFile = ""
)

$scriptDrive = Split-Path -Qualifier $PSScriptRoot
if (-not $logFile) { $logFile = "$scriptDrive\hobl_data\<scenario>_run.log" }

[Console]::OutputEncoding = [System.Text.Encoding]::UTF8

function log {
    [CmdletBinding()] Param([Parameter(ValueFromPipeline)] $msg)
    process {
        if ($msg -Match " ERROR - ") {
            Write-Host $msg -ForegroundColor Red
        } else {
            Write-Host $msg
        }
        Add-Content -Path $logFile -encoding utf8 "$msg"
    }
}

function check {
    param($code)
    if ($code -ne 0) {
        " ERROR - Last command failed." | log
        Exit $code
    }
}

function checkWinget {
    param($code)
    if ($code -eq 0) {
        "Winget command succeeded" | log
    } elseif ($code -eq -1978335189) {
        "Package already installed (this is OK)" | log
    } elseif ($code -eq -1978335215) {
        "No applicable upgrade found (this is OK)" | log
    } else {
        " ERROR - Winget command failed with exit code: $code" | log
        Exit $code
    }
}

Execution Policy

Scripts that use pyenv install (which calls Expand-Archive) must set execution policy early:

$executionPolicy = Get-ExecutionPolicy -Scope Process
if ($executionPolicy -eq "Restricted" -or $executionPolicy -eq "Undefined") {
    Set-ExecutionPolicy -ExecutionPolicy Unrestricted -Scope Process -Force -ErrorAction Stop
}

Shell Script Standards (macOS)

macOS scripts use /bin/sh and follow this pattern:

#!/bin/sh

LOG_DIR="/Users/Shared/hobl_data"
METRICS_FILE="$LOG_DIR/mac_<scenario>_results.csv"

check_status() {
    if [ $? -ne 0 ]; then
        echo " ERROR - $1 failed"
        exit 1
    fi
    echo "✓ $1 successful"
}

check_command() {
    if command -v "$1" >/dev/null 2>&1; then
        echo "✓ $1 is available"
        return 0
    else
        echo " ERROR - $1 is not available"
        return 1
    fi
}

Timing and Metrics

Run scripts must produce a CSV

Every run script writes a <scenario>_results.csv to the hobl_data directory in key,value format:

scenario_runtime,45.23
build_time,45.23

scenario_runtime is required — it represents total wall-clock time of the timed workload
Additional phase timings are encouraged (e.g., build_time, test_time)

Windows timing pattern

$buildDuration = Measure-Command {
    dotnet build Aspire.slnx --no-restore > "$logDir\build.log" 2>&1
}
check($lastexitcode)
$buildTime = [math]::Round($buildDuration.TotalSeconds, 2)

macOS timing pattern

build_start=$(date +%s)
# ... build commands ...
build_end=$(date +%s)
build_time=$((build_end - build_start))

Metrics summary banner

Print a human-readable summary at the end of each run:

========================================
.NET Aspire Metrics Summary
========================================
Build Time:  45.23s
scenario_runtime (total): 45.23s
========================================

Prep Version Bumping

Each scenario's Python file contains a prep_version string:

prep_version = "2"

This controls whether the prep script re-runs on the DUT. The HOBL framework checks this value and only re-runs prep when the version changes.

Rules

Whenever prep or run scripts are modified in a PR, increment prep_version by 1 in the corresponding scenario's Python file
Increment for both platforms if both were changed, or just the affected one
This only needs to happen once per PR, not per commit
Forgetting to bump means DUTs won't pick up your script changes until someone manually clears the prep status

Error Handling

Error message format

All error messages must use this exact format:

 ERROR - <description>

Note the spacing: leading space, ERROR, space, dash, space. This is required because the log function pattern-matches on " ERROR - " to colorize output.

✅ Correct:

" ERROR - Last command failed." | log
" ERROR - dotnet not found on PATH" | log
Write-Host " ERROR - Unsupported architecture" -ForegroundColor Red

❌ Wrong:

"ERROR: something failed" | log
"Error - bad path" | log

Helper functions

Use the standard helpers consistently:

Function	Use when
`check($lastexitcode)`	After any command where `$LASTEXITCODE` indicates success/failure
`checkWinget($lastexitcode)`	After `winget install` (handles "already installed" codes)
`checkCmd($result)`	After commands returning "True"/"False" strings
`checkGitClone $code $path`	After `git clone` (handles "already exists" case)
`checkSetLocation $path`	Before `Set-Location` to verify path exists
`check_status "description"`	macOS equivalent of `check`

Path Handling

Windows: Drive-relative paths

Never hardcode C:\ or any drive letter. Scripts may run from any drive.

# Derive drive from script location
$scriptDrive = Split-Path -Qualifier $PSScriptRoot

# Use for all HOBL paths
$logFile = "$scriptDrive\hobl_data\scenario.log"
$repoDir = "$scriptDrive\aspire"

System paths are the exception — $env:ProgramFiles, $env:USERPROFILE, $env:TEMP, etc., resolve correctly on their own.

macOS: Fixed paths

macOS scripts use /Users/Shared/hobl_data and /Users/Shared/hobl_bin as standard locations.

Architecture Awareness

Windows scenarios must detect and handle both x64 and ARM64:

$osInfo = Get-CimInstance Win32_OperatingSystem
$arch = $osInfo.OSArchitecture
$processorArch = $env:PROCESSOR_ARCHITECTURE

if ($arch -eq "64-bit" -and $processorArch -eq "AMD64") {
    $isARM64 = $false
    $logSuffix = "x64"
} elseif ($arch -match "ARM" -or $processorArch -match "ARM") {
    $isARM64 = $true
    $logSuffix = "ARM64"
} else {
    " ERROR - Unsupported architecture: $arch" | log
    Exit 1
}

Use $isARM64 to conditional-branch where binaries, workarounds, or configurations differ between architectures.

ETW Phase Markers (Windows)

HOBL includes a manifest for the scenario phase marker provider at:

scenarios/windows/_dev_tools/HOBL-Scenario-Phases.man

This manifest maps provider GUID 9f0f6e2e-8d06-4d2f-b8f5-6f1f2d5a1c01 event ID 1 to two Unicode payload fields:

Marker
Scenario

Registering the manifest for WPA decoding

Run these commands in an elevated PowerShell terminal:

$manifestPath = "<repo_root>\scenarios\windows\_dev_tools\HOBL-Scenario-Phases.man"
wevtutil um $manifestPath
wevtutil im $manifestPath
wevtutil gp "HOBL-Scenario-Phases" /ge:true /gm:true

Then reopen WPA and reload the ETL file. Generic Events should decode payload text for this provider when metadata matches.

Marker naming convention

Use this standard marker format:

phase.run_prep.start / phase.run_prep.end
phase.run_build.start / phase.run_build.end
phase.run_test.start / phase.run_test.end

This keeps phase attribution consistent across all developer scenarios and avoids naming collisions with prep/teardown script concepts.

Emitting markers in run scripts

For Windows scenario run scripts, define an ETW EventSource with:

Name: HOBL-Scenario-Phases
GUID: 9f0f6e2e-8d06-4d2f-b8f5-6f1f2d5a1c01
Event ID: 1
Payload: marker string and scenario name

Emit marker pairs immediately around each timed run phase. Example:

Write-RunPhaseMarker "phase.run_build.start"
# build work
Write-RunPhaseMarker "phase.run_build.end"

Python Path Resolution

HOBL uses pyenv-win for Python version management. Do not replace it with winget install of Python.

# WRONG — returns the pyenv shim, not the real .exe
$python = Get-Command python

# CORRECT — returns the actual python.exe path
$pythonExeRaw = pyenv which python 2>$null
$pythonExe = $pythonExeRaw.Trim()

This matters when passing paths to tools like CMake (-DPython3_EXECUTABLE) which require the real executable.

Visual Studio Integration

Use vswhere.exe to discover Visual Studio — never hardcode installation paths.

$vsInfo = getVSVersion -product $vsProduct
$actualVSPath = $vsInfo.Path
$vsDevCmd = Join-Path $actualVSPath "Common7\Tools\VsDevCmd.bat"

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Contributing

Contributing to HOBL Developer Scenarios

Table of Contents

Scenario Structure

Python Orchestrator (`<scenario>.py`)

Script Roles

Platform Parity

PowerShell Coding Standards (Windows)

Script Template

Execution Policy

Shell Script Standards (macOS)

Timing and Metrics

Run scripts must produce a CSV

Windows timing pattern

macOS timing pattern

Metrics summary banner

Prep Version Bumping

Rules

Error Handling

Error message format

Helper functions

Path Handling

Windows: Drive-relative paths

macOS: Fixed paths

Architecture Awareness

ETW Phase Markers (Windows)

Registering the manifest for WPA decoding

Marker naming convention

Emitting markers in run scripts

Python Path Resolution

Visual Studio Integration

Checklist Before Submitting a PR

FilesExpand file tree

CONTRIBUTING.md

Latest commit

History

CONTRIBUTING.md

File metadata and controls

Contributing

Contributing to HOBL Developer Scenarios

Table of Contents

Scenario Structure

Python Orchestrator (<scenario>.py)

Script Roles

Platform Parity

PowerShell Coding Standards (Windows)

Script Template

Execution Policy

Shell Script Standards (macOS)

Timing and Metrics

Run scripts must produce a CSV

Windows timing pattern

macOS timing pattern

Metrics summary banner

Prep Version Bumping

Rules

Error Handling

Error message format

Helper functions

Path Handling

Windows: Drive-relative paths

macOS: Fixed paths

Architecture Awareness

ETW Phase Markers (Windows)

Registering the manifest for WPA decoding

Marker naming convention

Emitting markers in run scripts

Python Path Resolution

Visual Studio Integration

Checklist Before Submitting a PR

Python Orchestrator (`<scenario>.py`)