Skip to content

test: reduce impact of flaky HTTP server tests#42926

Merged
aduh95 merged 1 commit intonodejs:masterfrom
tniessen:flaky-test-http-server-x-timeout-keepalive
May 2, 2022
Merged

test: reduce impact of flaky HTTP server tests#42926
aduh95 merged 1 commit intonodejs:masterfrom
tniessen:flaky-test-http-server-x-timeout-keepalive

Conversation

@tniessen
Copy link
Member

@tniessen tniessen commented Apr 30, 2022

Mark the tests as flaky on macOS and increase the timeout for other platforms (namely, FreeBSD).

Context: #42741 (comment)

Median success rate (refer to patch and separate tables above):

osx11 freebsd12-x64
master (2000ms) 91.0 % 83.8 %
patched (5000ms) 71.3 % 100 %

Refs: #42741

@tniessen tniessen added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 30, 2022
@tniessen tniessen requested a review from richardlau April 30, 2022 13:06
@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. test Issues and PRs related to the tests. labels Apr 30, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 30, 2022
@nodejs-github-bot

This comment was marked as outdated.

Sign in to view
@tniessen
Copy link
Member Author

tniessen commented Apr 30, 2022 •
edited
Loading

Is PASS,FLAKY not the right configuration? node-test-commit-osx still fails with this. Should it be SKIP?

Removed the file extensions, maybe that works.

@tniessen tniessen force-pushed the flaky-test-http-server-x-timeout-keepalive branch from 35014d7 to 06c1abe Compare April 30, 2022 14:06
@targos
Copy link
Member

targos commented Apr 30, 2022

Removed the file extensions, maybe that works.

Yep:

node/test/parallel/parallel.status

Lines 3 to 5 in 02e0c17

# To mark a test as flaky, list the test name in the appropriate section
# below, without ".js", followed by ": PASS,FLAKY". Example:
# sample-test : PASS,FLAKY

@tniessen tniessen added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 30, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 30, 2022
@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Apr 30, 2022

CI: https://ci.nodejs.org/job/node-test-pull-request/43796/

aduh95
aduh95 approved these changes Apr 30, 2022
View reviewed changes
Copy link
Contributor

@aduh95 aduh95 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The flakiness is also annoying on the GHA runner, and unfortunately this PR won't help with that I'm afraid. Oh well, at least it's improving things for the Jenkins CI, so LGTM.

@github-actions github-actions bot mentioned this pull request May 1, 2022
Open
29 tasks
@tniessen
Copy link
Member Author

tniessen commented May 1, 2022

The flakiness is also annoying on the GHA runner, and unfortunately this PR won't help with that I'm afraid.

@aduh95 I am not sure about that. #42741 (comment) suggested that GitHub Actions might respect the list of flaky tests.

@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented May 1, 2022

CI: https://ci.nodejs.org/job/node-test-pull-request/43804/

@tniessen
Copy link
Member Author

tniessen commented May 1, 2022

CI is failing because of nodejs/build#2917.

@tniessen tniessen added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label May 1, 2022
@github-actions github-actions bot mentioned this pull request May 2, 2022
Open
30 tasks
@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented May 2, 2022

CI: https://ci.nodejs.org/job/node-test-pull-request/43810/

@tniessen tniessen added the commit-queue Add this label to land a pull request using GitHub Actions. label May 2, 2022
@tniessen
Copy link
Member Author

tniessen commented May 2, 2022

CI is green but GitHub seems to be thinking that Jenkins is still running, and the commit queue refuses to run...

@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented May 2, 2022

CI: https://ci.nodejs.org/job/node-test-pull-request/43826/

@aduh95 aduh95 removed the commit-queue Add this label to land a pull request using GitHub Actions. label May 2, 2022
@aduh95 aduh95 merged commit 0fc455a into nodejs:master May 2, 2022
@aduh95
Copy link
Contributor

aduh95 commented May 2, 2022

Landed in 0fc455a

@tniessen
Copy link
Member Author

tniessen commented May 2, 2022

Thank you @aduh95 :)

This was referenced May 3, 2022
Open
Open
Open
RafaelGSS pushed a commit that referenced this pull request May 10, 2022
@tniessen @RafaelGSS
test: reduce impact of flaky HTTP server tests
72b90fd 
Refs: #42741

PR-URL: #42926
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
@RafaelGSS RafaelGSS mentioned this pull request May 10, 2022
Merged
@daeyeon daeyeon mentioned this pull request May 23, 2022
Merged
@danielleadams
Copy link
Contributor

danielleadams commented Jun 26, 2022

This PR is dependent on #42846, so marking as blocked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lpinca lpinca lpinca approved these changes

@targos targos targos approved these changes

@aduh95 aduh95 aduh95 approved these changes

@richardlau richardlau Awaiting requested review from richardlau

Assignees

No one assigned

Labels

author ready PRs that have at least one approval, no pending requests for changes, and a CI started. needs-ci PRs that need a full CI run. test Issues and PRs related to the tests.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@tniessen @nodejs-github-bot @targos @aduh95 @danielleadams @lpinca