feat(e2e-testing): auth-server client

[y3rsh]

Overview

This work adds and refines e2e-testing support for the robot auth-server HTTP API: a typed AuthClient, Pydantic response models under e2e-testing/automation/clients/auth_models/, is_alive() (probes GET /auth/settings with full envelope validation), and two walkthrough scripts:

• e2e-testing/scripts/check_auth.py: Rich panels, pretty-printed models, and assertions on the OAuth error body for a deliberate bad password (HTTP 400, error: invalid_grant, non-empty error_description), aligned with auth-server/tests/integration/test_oauth_flows.tavern.yaml.
• e2e-testing/scripts/check_auth_quiet.py: Same call sequence with no stdout - much easier to read the code but does the exact same thing as the other script.

Purpose

The purpose of this script and client is to show how we can start from the human-readable Redoc documentation for the auth server, understand the available API surface, and build a reusable client for it. In this case, we used an LLM to help generate a client that knows how to talk to the auth server’s endpoints and perform meaningful business actions against them. The included scripts are just lightweight convenience checks that exercise the client against the dev server, but that is not the primary long-term use case. The real goal is to have a reusable client that we can use for setup and end-to-end workflows on actual robots, including scripting common tasks like provisioning administrative users, regular users, and users with different scopes once builds are deployed to robots with the auth server. These clients are intended primarily for real robot setup and real robot end-to-end testing, not for expanding CI coverage against dev servers, since we already have Tavern tests covering that space and any additional CI-focused API testing should likely be added there.

There is some implementation complexity here, especially around Pydantic-based model validation, which can feel a little magical at first glance, but that is not the main thing to focus on in this PR. The important part is the overall structure of the client and the pattern it gives us for building reusable API clients that support scripts, setup flows, and testing against real hardware.

Manual: dev auth-server locally

1. Install dependencies once (your usual monorepo robot Python setup, e.g. make setup-py, plus cd e2e-testing && make setup).

2. Start the auth-server (default dev port 33950, matches the scripts):

   cd auth-server
make dev

   Leave this running. (auth-server/Makefile sets dev_port to 33950 for make dev.)

3. Loud script (Rich output), second terminal:

   cd e2e-testing
uv run python scripts/check_auth.py localhost

[SyntaxColoring]

Cool!

It sounds like this is just a quick demo, but here's a warning about using Pydantic for this, if/when we take it further.

[SyntaxColoring]

Because of Pydantic footguns regarding nullable vs. omittable vs. defaultable fields, you might want to avoid Pydantic BaseModels for this kind of work.

The semantics that we're trying to implement for these PATCH JSON APIs are:

• Omitting a field from the PATCH request will leave its value unchanged.
• Setting a field to null in the PATCH request will set its value to null. Semantically, this generally means something like "disabled." For example, setting maxNumberOfLoginAttempts: null means to disable the login attempt limit.
• Setting a field to a regular value in the PATCH request, like 123, will of course just set it to that value.
• GET will always return the actual current values for everything, populating them with defaults if they've never been set.

But if you use this model to generate your PATCH requests, you won't be able to leave a value unset. e.g. if you omit max_number_of_login_attempts, your request will transmit an explicit 5. If you set it to None, your request will transmit an explicit null.

It's tempting to try to fix that by fiddling with the default values, and with options like exclude_unset=True. In my experience, those fixes don't work for every kind of field (e.g. try a field that's either an int or is omitted), and even if they do, they're subtle and easily broken.

You might want to try modeling the requests and responses as TypedDicts with NotRequired fields, and use Pydantic to parse them. (I have not tried this but it seems like it should work.) We can't do that in our server code because we need real Pydantic models in order to generate the OpenAPI spec, but that's not a concern for you on the consuming side.

I would also recommend using separate types for requests and responses, and not encoding server-side defaults like max_number_of_login_attempts=5.

[y3rsh]

Thank you for the detailed call out! Got things adjusted for this and documented the pattern. I think I like how it looks now a lot better.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 56.94%. Comparing base (d0a8de8) to head (1e9bfc6).
⚠️ Report is 54 commits behind head on edge.

Additional details and impacted files

@@            Coverage Diff             @@
##             edge   #21169      +/-   ##
==========================================
- Coverage   57.19%   56.94%   -0.25%     
==========================================
  Files        3977     3977              
  Lines      326193   326229      +36     
  Branches    46378    46381       +3     
==========================================
- Hits       186568   185784     -784     
- Misses     139406   140226     +820     
  Partials      219      219              

Flag	Coverage Δ
app	45.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 129 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[SyntaxColoring]

Thanks!