Conversation
Scan the built Kafka Connect distribution zip for known vulnerabilities using Trivy. This runs alongside the existing tests on PRs and pushes to main/version branches, and also on release candidate tags, giving visibility into CVEs before a release vote starts. - Table output on all runs (visible in CI logs) - SARIF upload to GitHub Security tab on push events Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Match the convention used in spark-ci.yml for third-party actions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
i think its better to do this in the docker image publishing step, similar to what Superset is doing |
|
@kevinjqliu I'm not sure I follow. How's this relate to publishing docker images? This is to identify CVEs in the Kafka Connect connector itself. thanks. |
|
Looking at how Trivy is used in Superset, its scanning the docker image only. In this PR, we're unpacking the jars to scan. I think it makes more sense to build a kafka connect image and then use Trivy to scan the image. Just my preference |
Not really clear why we would add the infra to build a docker image just to use trivy. Seems like a direct scan is more parsimonious |
rymurr
left a comment
rymurr
left a comment
There was a problem hiding this comment.
LGTM, I just want to see if we can avoid 2x build w/o over complicating the CI job
| ~/.gradle/wrapper | ||
| key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }} | ||
| restore-keys: ${{ runner.os }}-gradle- | ||
| - name: Build Kafka Connect distribution |
There was a problem hiding this comment.
do we really have to build this twice? Does it make sense to do the scan in the same job as the tests? Not a strong preference, just seems unnecessary
There was a problem hiding this comment.
good point. changed this behaviour: d52b1b9
| scan-ref: '/tmp/kafka-connect-scan' | ||
| scanners: 'vuln' | ||
| severity: 'CRITICAL,HIGH' | ||
| ignore-unfixed: true |
There was a problem hiding this comment.
just to make sure I understand: the scan build will break when there is a hihg severity bug? A minor nit would be more comments in the file.
There was a problem hiding this comment.
At the moment the scan is report-only and won't fail the build.
I've also added inline comments explaining the behaviour.
Thinking about it some more, what'd be the ideal behaviour be? If a PR introduces a CVE, ought the build fail? Or maybe log a PR comment?
Address review feedback: move the vulnerability scan steps into the existing kafka-connect-tests job (gated on JVM 21) instead of running a separate job that duplicates checkout, setup, and compilation. Also adds inline comments explaining the scan behaviour and explicit exit-code: '0' to ensure the scan is report-only (the default would fail the build on findings).
Just looking at general patterns from the apache repos. https://grep.app/search?f.repo.pattern=apache%2F&q=uses%3A+aquasecurity%2Ftrivy-action The only instance of From the docs, it seems like Filesystem scan and Container image scan are similar in that they both scan for Vulnerabilities, Misconfigurations, Secrets, and Licenses. I think it would be helpful here if you can run the change on your fork repo and see if the fs trivy scan catches the currently open CVE for kafka connect (#15440) |
3 participants
Summary
checkcompletes, it buildsdistZip, unpacks it, and scans the bundled jars for CRITICAL/HIGH CVEsContext
Discussion on dev@ mailing list: https://lists.apache.org/thread/kbf98950pzstzgon92st7mh9vrbv5yhb
Confluent Marketplace requires a Trivy scan before listing connectors. This has previously caught CVEs that needed patching (e.g. #14985). Running the scan in CI catches vulnerabilities during development and — critically — on RC tags before the release vote starts, when fixes can still be applied.
This is independent of #15212 (adding the KC artifact to the release process) and can land in either order.