Skip to content

Add common infrastructure for Rule-Based Testing#4977

Merged
thomas-bc merged 16 commits intonasa:develfrom
thomas-bc:rule-based-test-docs
Apr 15, 2026
Merged

Add common infrastructure for Rule-Based Testing#4977
thomas-bc merged 16 commits intonasa:develfrom
thomas-bc:rule-based-test-docs

Conversation

@thomas-bc
Copy link
Copy Markdown
Collaborator

@thomas-bc thomas-bc commented Apr 8, 2026

Related Issue(s)
Has Unit Tests (y/n)
Documentation Included (y/n)
Generative AI was used in this contribution (y/n)

Change Description

Preface to #3134

The goal here is to provide a “standard" pattern for doing Rule-Based Testing. This does not mean restricting to only that pattern, but I will next document how to work with this specific pattern, and provide tooling to work with it.

  • adds a TestUtils MACRO for defining Rule-Based Test rules. This enables the framework's ASSERT_EVENTS_* etc. macros to be usable within the rule body by... simply making the body of the rule delegate to a member function of the tester. this is all declared / handled by the macro, and users need simply to implement each body.
  • refactors Svc.Ccsds.ApidManager to use said common infrastructure

Future Work

  • Write How-To Guide
  • Identify potential tooling support. My plan for now:
    • fprime-util new --rule-based-testing --> bootstraps the RBT structure (TestState folder and class definition, Rules/*.cpp file)
    • I don't see much value in having a fprime-util new --rule for each rule, as we need to insert the MACROs inside the ComponentTester.hpp which would be very brittle and not much added value. Instruct users to copy-paste is fine imo.
    • Potentially fprime-util new --rule-group that creates the RuleGroup.cpp file and a TODO rule ?

AI Usage (see policy)

Help think through the architecture and refactor the ApidManager test code to the new architecture

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds shared infrastructure to simplify Rule-Based Testing (RBT) rules in F Prime unit tests, then refactors the Svc::Ccsds::ApidManager unit tests to use the new approach (shadow state + grouped rules implemented as tester methods).

Changes:

  • Introduces FW_RBT_DEFINE_RULE in TestUtils to generate rule structs that delegate to tester member functions (enabling existing ASSERT_* macros inside rule bodies).
  • Refactors ApidManager UTs into a TestState shadow-model and per-group rule implementation .cpp files.
  • Updates the ApidManager UT CMake registration to compile the new rule/state sources.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
TestUtils/RuleBasedTesting.hpp Adds a macro-based pattern for defining STest::Rule types that delegate to tester methods.
Svc/Ccsds/ApidManager/test/ut/TestState/TestState.hpp Introduces an ApidManager shadow-state model API for rule preconditions/actions.
Svc/Ccsds/ApidManager/test/ut/TestState/TestState.cpp Implements shadow-state helpers (sequence count tracking + random APID selection).
Svc/Ccsds/ApidManager/test/ut/Rules/GetSeqCount.cpp Adds rule implementations for exercising getApidSeqCountIn.
Svc/Ccsds/ApidManager/test/ut/Rules/ValidateSeqCount.cpp Adds rule implementations for exercising validateApidSeqCountIn.
Svc/Ccsds/ApidManager/test/ut/ApidManagerTestMain.cpp Updates tests to instantiate/apply the new nested rule types and run randomized scenarios.
Svc/Ccsds/ApidManager/test/ut/ApidManagerTester.hpp Refactors the tester to use FW_RBT_DEFINE_RULE and the new ApidManagerTestState.
Svc/Ccsds/ApidManager/test/ut/ApidManagerTester.cpp Removes legacy embedded rule/helper implementations from the tester .cpp.
Svc/Ccsds/ApidManager/CMakeLists.txt Adds new UT source files (shadow state + rule groups) to the unit test target.

Comment thread Svc/Ccsds/ApidManager/test/ut/Rules/GetSeqCount.cpp Outdated
Comment thread Svc/Ccsds/ApidManager/test/ut/TestState/TestState.cpp
Comment thread Svc/Ccsds/ApidManager/test/ut/TestState/TestState.hpp Outdated
LeStarch
LeStarch previously approved these changes Apr 9, 2026
Copy link
Copy Markdown
Collaborator

@bocchino bocchino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! I like the simplification into one macro. I just had one suggested revision to the naming in the macro.

Comment thread TestUtils/RuleBasedTesting.hpp Outdated
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
LeStarch and others added 2 commits April 14, 2026 15:03
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Comment thread TestUtils/RuleBasedTesting.hpp Outdated
Comment thread TestUtils/RuleBasedTesting.hpp Outdated
@thomas-bc thomas-bc requested a review from bocchino April 14, 2026 23:35
Copy link
Copy Markdown
Collaborator

@bocchino bocchino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@thomas-bc thomas-bc merged commit b40cdc0 into nasa:devel Apr 15, 2026
53 checks passed
@thomas-bc thomas-bc deleted the rule-based-test-docs branch April 15, 2026 17:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants