tests/run-tests.py: Run tests-with-regex-output as normal tests. #17768
+5
−6
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dpgeorge
force-pushed
the
tests-run-tests-regex-output-not-special
branch
from
0e6611f to
bd3b157
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #17768 +/- ##
=======================================
Coverage 98.38% 98.38%
=======================================
Files 171 171
Lines 22283 22283
=======================================
Hits 21924 21924
Misses 359 359 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Some tests (currently given by the `special_tests` list) have output which must be mached via a regex, because it can change from run to run (eg the address of an object is printed). These tests are currently classified as `is_special` in the test runner, which means they get special treatment. In particular they don't set the emitter as specified by `args.emit`. That means these tests do not run via .mpy or using the native emitter, even if those options are given on the command line. This commit fixes that by considering `is_special` as different to `tests_with_regex_output`. The former is used for things like target feature detection (which are not really tests) and when extra command line options need to be passed to the unix micropython executable. The latter (now called `tests_with_regex_output`) are specifically for tests that have output to be matched via regex. The `thread_exc2.py` test now needs to be excluded when running using the native emitter, because the native emitter doesn't print traceback info. And the `sys_settrace_cov.py` test needs to be excluded because set-trace output is different with the native emitter. Signed-off-by: Damien George <[email protected]>
dpgeorge
force-pushed
the
tests-run-tests-regex-output-not-special
branch
from
bd3b157 to
1401fdb
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Some tests (currently given by the
special_testslist) have output which must be mached via a regex, because it can change from run to run (eg the address of an object is printed). These tests are currently classified asis_specialin the test runner, which means they get special treatment. In particular they don't set the emitter as specified byargs.emit. That means these tests do not run via .mpy or using the native emitter, even if those options are given on the command line.This commit fixes that by considering
is_specialas different totests_with_regex_output. The former is used for things like target feature detection (which are not really tests) and when extra command line options need to be passed to the unix micropython executable. The latter (now calledtests_with_regex_output) are specifically for tests that have output to be matched via regex.The
thread_exc2.pytest now needs to be excluded when running using the native emitter, because the native emitter doesn't print traceback info.Testing
Tested locally using the unix port. Will be more thoroughly tested by CI.