test(aggregation): Improve interactive plotter#641
test(aggregation): Improve interactive plotter#641ValerianRey merged 25 commits intoSimplexLab:mainfrom
Conversation
Implement Gradient Vaccine (ICLR 2021) as a stateful Jacobian aggregator. Support group_type 0 (whole model), 1 (all_layer via encoder), and 2 (all_matrix via shared_params), with DEFAULT_GRADVAC_EPS and configurable eps. Add Sphinx page and unit tests. Autogram is not supported; use torch.manual_seed for reproducible task shuffle order.
- Add GOVERNANCE.md documenting technical governance structure - Add CODEOWNERS file defining project maintainers - Add CODE_OF_CONDUCT.md referencing Linux Foundation CoC These files are required for PyTorch Ecosystem membership. --------- Co-authored-by: Valérian Rey <31951177+ValerianRey@users.noreply.github.com>
- Use group_type "whole_model" | "all_layer" | "all_matrix" instead of 0/1/2 - Remove DEFAULT_GRADVAC_EPS from the public API; keep default 1e-8; allow eps=0 - Validate beta via setter; tighten GradVac repr/str expectations - Fix all_layer leaf sizing via children() and parameters() instead of private fields - Trim redundant GradVac.rst prose; align docs with the new API - Tests: GradVac cases, value regression with torch.manual_seed for GradVac - Plotter: factory dict + fresh aggregator instances per update; legend from selected keys; MathJax labels and live angle/length readouts in the sidebar This commit includes GradVac implementation with Aggregator class.
…hting GradVac only needs gradient norms and dot products, which are fully determined by the Gramian. This makes GradVac compatible with the autogram path. - Remove grouping parameters (group_type, encoder, shared_params) from GradVac - Export GradVacWeighting publicly
Seed is already set to 0 because of the autoused fix_randomness fixture declared in conftest.py
Add a Grouping example page covering all four strategies from the GradVac paper (whole_model, enc_dec, all_layer, all_matrix), with a runnable code block for each. Update the GradVac docstring note to link to the new page instead of the previous placeholder text. Fix trailing whitespace in CHANGELOG.md.
|
Ideally, this should be split in 3 PRs, in order:
|
- The plan is to add it back in another PR
I merged main into this and removed grouping example (59e7942), so now this only does the improvement of the plotter. I'll also update the PR title and message accordingly. |
|
/opencode:Plan please review this. |
|
Now I have both versions. Let me analyze the changes: Code Review: Interactive Plotter ImprovementsSummary of Changes
|
The interactive plotter is only for torchjd devs (not people who pip install the library), so it's fine if it doesn't work with python 3.10. So IMO this is not an issue. |
ValerianRey
left a comment
There was a problem hiding this comment.
Really nice. Few nitpicks and we can merge.
| "AlignedMTL-min": lambda: AlignedMTL(scale_mode="min"), | ||
| "AlignedMTL-median": lambda: AlignedMTL(scale_mode="median"), | ||
| "AlignedMTL-RMSE": lambda: AlignedMTL(scale_mode="rmse"), |
There was a problem hiding this comment.
It seems like it would be nice to change the __str__ method of AlignedMTL to also include the scale_mode. For example, str(AlignedMTL(scale_mode="min")) would become AlignedMTL-min.
But this is quite independent from this PR, and should come in a different PR if you care about that! @rkhosrowshahi
|
We could make another PR that adds the possibility to change the number of gradients. A few things to keep in mind:
So it's not trivial, but if you're interested in doing that, we could land the PR quite easily @rkhosrowshahi |

Summary
Interactive plotter
tests/plots/interactive_plotter.pyandtests/plots/_utils.py: factory-based aggregators, checklist selection, improved gradient controls