Skip to content

Combine all tests into a single suite to increase parallelism. NFC#26365

Open
sbc100 wants to merge 1 commit intoemscripten-core:mainfrom
sbc100:test_runner
Open

Combine all tests into a single suite to increase parallelism. NFC#26365
sbc100 wants to merge 1 commit intoemscripten-core:mainfrom
sbc100:test_runner

Conversation

@sbc100
Copy link
Collaborator

@sbc100 sbc100 commented Feb 27, 2026

My motivation for this change is to allow for increased parallelism that is not limited to how
many tests are in a give module. I'm hoping to do more splitting up of test_other.py, for
example, and I don't want to loose parallism.

After this change all tests are combined into a single parallel test suite. For example:

$ ./test/runner jslib other.test_a* core0.test_a* --skip-slow
Test suites: ['test_core', 'test_jslib', 'test_other']
Running test_core: (88 tests)
Using 88 parallel test processes
[88/88] test_asan_vector (test_core.core0.test_asan_vector) ... ok

----------------------------------------------------------------------
Ran 88 tests in 3.296s

OK (skipped=5)
Running test_jslib: (54 tests)
Using 54 parallel test processes
[54/54] test_jslib_aliases_closure_wasm64 (test_jslib.jslib.test_jslib_aliases_closure_wasm64) ... ok

----------------------------------------------------------------------
Ran 54 tests in 5.287s

OK
Running test_other: (41 tests)
Using 41 parallel test processes
[41/41] test_abspaths (test_other.other.test_abspaths) ... ok

----------------------------------------------------------------------
Ran 41 tests in 5.210s

OK
Total core time: 228.881s. Wallclock time: 13.795s. Parallelization: 16.59x.
====================

TEST SUMMARY
    test_core: 88 run, 0 errors, 0 failures, 5 skipped
    test_jslib: 54 run, 0 errors, 0 failures, 0 skipped
    test_other: 41 run, 0 errors, 0 failures, 0 skipped

After:

$ ./test/runner jslib other.test_a* core0.test_a* --skip-slow
Running 183 tests
Using 128 parallel test processes
[183/183] test_abspaths (test_other.other.test_abspaths) ... ok

----------------------------------------------------------------------
Ran 183 tests in 7.459s

OK (skipped=5)
Total core time: 301.490s. Wallclock time: 7.459s. Parallelization: 40.42x.

Note the wall clock time is less since we ran all the tests in parallel.

If you try to mix parallel and non-parallel test module you now get an error. e.g.:

$ ./test/runner core0 benchmark
runner: error: attempt to mix parallel and non-parallel test modules

@sbc100 sbc100 changed the title Combine all tests into a single test suite. NFC Combine all tests into a single suite to increase parallelism. NFC Feb 28, 2026
This allows all tests to be run in parallel.  For example:

```
Test suites: ['test_core', 'test_jslib', 'test_other']
Running test_core: (88 tests)
Using 88 parallel test processes
[88/88] test_asan_vector (test_core.core0.test_asan_vector) ... ok

----------------------------------------------------------------------
Ran 88 tests in 3.296s

OK (skipped=5)
Running test_jslib: (54 tests)
Using 54 parallel test processes
[54/54] test_jslib_aliases_closure_wasm64 (test_jslib.jslib.test_jslib_aliases_closure_wasm64) ... ok

----------------------------------------------------------------------
Ran 54 tests in 5.287s

OK
Running test_other: (41 tests)
Using 41 parallel test processes
[41/41] test_abspaths (test_other.other.test_abspaths) ... ok

----------------------------------------------------------------------
Ran 41 tests in 5.210s

OK
Total core time: 228.881s. Wallclock time: 13.795s. Parallelization: 16.59x.
====================

TEST SUMMARY
    test_core: 88 run, 0 errors, 0 failures, 5 skipped
    test_jslib: 54 run, 0 errors, 0 failures, 0 skipped
    test_other: 41 run, 0 errors, 0 failures, 0 skipped
```

After:

```
$ ./test/runner jslib other.test_a* core0.test_a* --skip-slow
Running 183 tests
Using 128 parallel test processes
[183/183] test_abspaths (test_other.other.test_abspaths) ... ok

----------------------------------------------------------------------
Ran 183 tests in 7.459s

OK (skipped=5)
Total core time: 301.490s. Wallclock time: 7.459s. Parallelization: 40.42x.
```

Note the wall clock time is less since we ran all the tests in parallel.

If you try to mix parallel and non-parallel test module you now get an
error.  e.g.:

```
$ ./test/runner core0 benchmark
runner: error: attempt to mix parallel and non-parallel test modules
```
@sbc100 sbc100 requested a review from kripken March 3, 2026 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants