Fix: Instrumented tests are executed in Drone CI again#16612
Open
Fix: Instrumented tests are executed in Drone CI again#16612
Conversation
PhilLab
changed the title
PhilLab
commented
Feb 28, 2026
|
|
||
| debug { | ||
| enableUnitTestCoverage = project.hasProperty("coverage") | ||
| enableAndroidTestCoverage = project.hasProperty("coverage") |
Contributor
Author
There was a problem hiding this comment.
@tobiasKaminsky with this, Drone now actually runs the instrumented tests again. See https://drone.nextcloud.com/nextcloud/android/28585/1/3 as a proof of failure, i.e. broken tests on master which were never detected.
I'll go ahead addressing the test failures. But there is several other things I noticed:
- We should have Drone only run the instrumentation tests. Currently, drone CI runs everything for which coverage is enabled. Why this is chosen as filter, I don't know. But what this means is that Drone wastes time on the unit tests which we already run elsewhere.
- Unit tests were the only tests Drone ran since Chore: Improve Gradle Configuration #15859. Another weird side-effect of that PR.
- Easiest for saving drone time would be to disable the coverage for unit tests in the line above, but I guess this is not desired?
- Actually, what about the coverage reporting? It doesn't appear anymore. It looks broken, in several ways: - I am not seeing the coverage report on any PR I have recently checked. Don't know when this stopped. Many of the pulls on Codecov just show errors: https://app.codecov.io/github/nextcloud/android/pulls
- The unit test job seems to still upload to Codecov, but prints out wrong links in the log, e.g.https://app.codecov.io/github/***/android/commit/2c1438fba2fda500d3ec2e330d2aa62284ef8e2c (log) .
- You need to replace the
***withnextcloud, then it works. Is this redaction of secrets at play - if so, I surely hope, no secret token has simply "nextcloud" as value?
- Coverage reporting should also be extended by the instrumentation tests. I don't see them uploading anything
PhilLab
commented
Feb 28, 2026
|
|
||
| if [ $TYPE = "IT" ]; then | ||
| FOLDER=app/build/reports/androidTests/connected/flavors/gplay | ||
| FOLDER=app/build/reports/androidTests/connected/debug/flavors/gplay |
Contributor
Author
There was a problem hiding this comment.
@tobiasKaminsky these adjustments are needed so the report files are actually found. But reporting still is broken, which you can see e.g. in the log of https://drone.nextcloud.com/nextcloud/android/28585/1/3:
It does upload the report to https://www.kaminsky.me/nc-dev/android-integrationTests/28585-IT-stable-10-44 but it fails to create the expected Github comment notifying the developer of this.
The error is:
{
"message": "Must have admin rights to Repository.",
"documentation_url": "https://docs.github.com/rest/issues/comments#create-an-issue-comment",
"status": "403"
}
2c1438f to
d904e65
Compare
The Drone CI job is running ./gradlew createGplayDebugCoverageReport This only executes tests with coverage. In 5fd2e29, the line testCoverageEnabled = project.hasProperty("coverage") was replaced with enableUnitTestCoverage = project.hasProperty("coverage") which only left the unit tests with coverage Enabling the coverage for androidTest (the instrumented tests) makes Drone run them again. Closes #16350 Signed-off-by: Philipp Hasper <vcs@hasper.info>
The drone tests simply ended with the log line: stable-IT test failed, but no output was generated. Maybe a preliminary stage failed. The reason was that the script didn't use the correct folder Signed-off-by: Philipp Hasper <vcs@hasper.info>
1. SetOnlineStatusBottomSheetIT: Checked for the wrong view, at the wrong point in time 2. SetStatusMessageBottomSheetIT: Fragment transactions are scheduled asynchronously, so you couldn't set the status inside the same scenario where show() is shown. Now, the setPredefinedStatus() method instead pre-stores the status and only updates the adapter if available Signed-off-by: Philipp Hasper <vcs@hasper.info>
PhilLab
force-pushed
the
ph/fix_instrumented_testing
branch
from
d904e65 to
ffaabe4
Compare
|
APK file: https://www.kaminsky.me/nc-dev/android-artifacts/16612.apk |
|
blue-Light-Screenshot test failed, but no output was generated. Maybe a preliminary stage failed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Drone CI job is running ./gradlew createGplayDebugCoverageReport This only executes tests with coverage.
In 5fd2e29 (PR #15859), the line
was replaced with
which only left the unit tests with coverage
Enabling the coverage for androidTest (the instrumented tests) makes Drone run them again.
Closes #16350
🏁 Checklist