Create a template workflow to execute `poetry update`

[ArBridgeman]

Summary

Dependabots has been opening many security issues for our transitive/phantom dependencies. These are not of interest to our customers, & they can update their own poetry.lock in many cases.

Additionally, it may be that the prod/dev/main dependencies have a wide enough range to already allow the non-vulnerable version.

To Dos

Part 1: Workflow Template Creation

Let's create a workflow template in exasol/toolbox/templates/github/workflows/ that:

• Runs on a weekly schedule (ideally not same day as the project-keeper or use a different Slack channel)
• Executes poetry update
• If there are changes to the poetry.lock file:
  • It commits the updated poetry.lock file to a branch
  • Opens a PR to merge that branch to main
  • Communicates the PR to our team channel

Notes:

• It does more than we need, but the analogous project-keeper workflow might be helpful to reference. It looks like they might have stayed with or switched back to daily runs. We can check with @antonireus. For now, it's ok if we deviate as we're building up this tool.
• Each project uses the noxconfig.py::PROJECT_CONFIG to define specific values, while it's not been fully thought through for this task, you might need values for that. In that case, our templates use Jinja2 to render variables in them. The default values are defined in github_template_dict.

Part 2: Using the Workflow Template in the PTB

• Set this up for use in the PTB as an active workflow by using
  poetry run -- tbx workflow install <your-workflow's-short-name>
• For the Slack credential setup in GitHub, you'll likely need to ask @ckunki (or @ArBridgeman )

Note:

• <your-workflow's-short-name> is defined here in the code and visible with poetry run -- tbx workflow list.

Part 3: Testing

• (optional) The PTB workflow would be temporarily modified so it uses a workflow_call to verify that it works via the usual CI pipeline (before we merge it)
• Might adapt test/integration/tools/workflow_test.py (improvements always welcome 😄)
• Might adapt test/unit/tools/test_template.py

Part 4: Write Up What Another PTB-based Project Would Need to Do

• Add this to existing documentation: https://exasol.github.io/python-toolbox/main/user_guide/features/github_workflows/configuration.html
• Add an entry to the changelog (per the PTB PR guidelines). It'd be ideal to link to the configuration information you added too.

Notes:

• rst pages can link to rst & md pages by location references or tags.
• But md pages usually link to URLs of formatted documentation. Thus, if a new rst page is added, we may end up with a chicken-and-egg issue, as the URL doesn't exist yet. In such case, we adapt the changelog later, like when the release is made.

[ArBridgeman]

Known limitations;

• Does not include any automated changelog updates.
• Does not modify the pyproject.toml - there is a common (but not universal) practice in Python to change the existing pyproject.toml range to exclude a vulnerable one. This is technically not required as the latest one should always be taken on a poetry update (pip) & users would have their own auditing tools to alert them to re-lock.
• Does not separate concerns for developers - if multiple, even non-vulnerable packages are updated, then the commit includes all of those when the CI is run. This would mean, if the tests fails, that a developer needs to go back to identify what went wrong and may need to downpin.
• Does not address vulnerabilities that require pyproject.toml modifications -> developers would need to manually do that. But it would implicitly alert that this needs to be done.
• Does not create or close Security Issues for vulnerabilities.
• Does not manage if more than 1 PR has been opened, e.g. developer did not merge one(s) from the previous week(s). Does not limit open PRs.

[ArBridgeman]

Other comments:

• In PTB version 5.1.0, we uncoupled the jobs in the checks.yml, so all the checks should run. It may be (though it seems slightly flakey), that the Changelog Update Check step should often fail until a developer updates the PR with a changed Changelog.
• We currently require slow-checks.yml to be manually approved in most repos. A developer should trigger this, as it's required for the PR to merge anyways.
• We should write up how these PRs could/should be resolved in the documentation. Information may include also which developers (the maintainers) handle it.