Skip to content

Comments

Recreate firewall on unhealthy condition#63

Open
Honigeintopf wants to merge 35 commits intomainfrom
firewall-health-check
Open

Recreate firewall on unhealthy condition#63
Honigeintopf wants to merge 35 commits intomainfrom
firewall-health-check

Conversation

@Honigeintopf
Copy link
Collaborator

@Honigeintopf Honigeintopf commented Nov 4, 2024

Description

Closes #62.

This pr introduces the functionality for deleting firewalls if they exceed the firewallHealthTimeout which for now is set to 20 minutes.
Integration tests where added to make sure everything works as intended.

CA were updated, otherwise it is not possible to deploy to mini-lab.

@Honigeintopf Honigeintopf requested a review from a team as a code owner November 4, 2024 14:31
@Honigeintopf Honigeintopf linked an issue Nov 4, 2024 that may be closed by this pull request
@Honigeintopf Honigeintopf changed the title Firewall health check Firewall delete on unhealthy condition Nov 4, 2024
@Honigeintopf Honigeintopf requested a review from Gerrit91 November 4, 2024 14:33
@Gerrit91 Gerrit91 changed the title Firewall delete on unhealthy condition Recreate firewall on unhealthy condition Nov 4, 2024
Copy link
Contributor

@Gerrit91 Gerrit91 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for coming up with a PR for this.

@github-project-automation github-project-automation bot moved this to Review in Development Jun 5, 2025
@Gerrit91 Gerrit91 removed the status in Development Jun 13, 2025
@Gerrit91 Gerrit91 moved this to Upcoming in Development Oct 20, 2025
Comment on lines 38 to 43
if fw.Status.Phase == v2.FirewallPhaseCreating && timeSinceReconcile > allocationTimeout {
c.log.Info("create timeout reached")
return firewallConditionStatus{CreateTimeout: true}
}

if seedConnected && unhealthyTimeout != 0 && created && timeSinceReconcile > unhealthyTimeout {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check if allocationTimeout is set to be able to disable this check

Honigeintopf and others added 3 commits January 23, 2026 12:37
Co-authored-by: Gerrit <Gerrit91@users.noreply.github.com>
Co-authored-by: Gerrit <Gerrit91@users.noreply.github.com>
Co-authored-by: Gerrit <Gerrit91@users.noreply.github.com>
@Gerrit91 Gerrit91 moved this from Upcoming to In Progress in Development Jan 26, 2026
Co-authored-by: Gerrit <Gerrit91@users.noreply.github.com>
@Gerrit91
Copy link
Contributor

Test needs adaption (fake one of the unhealthy conditions).

@Honigeintopf
Copy link
Collaborator Author

Honigeintopf commented Feb 4, 2026

I changed a line in the code to only apply health timeout once we have a non-zero seed reconcile timestamp and made possible to specify 0s as timeout which translates to disabling the deletion.

@Gerrit91 Gerrit91 mentioned this pull request Feb 9, 2026
@Honigeintopf
Copy link
Collaborator Author

Okay the issue with using FirewallPhaseRunning:

  1. Phase = Running (machine phoned home)
  2. But Connected, SeedConnected, DistanceConfigured haven't been set to True yet (monitor not updated)
  3. !allConditionsMet is true even though conditions never degraded - they were never fully met in the first place

So either we go ahead and fix when a fw is running( I wouldn't do that) or we say hey there is a new fw condition when the fw was ready once i.e. it finished progressing

@Gerrit91
Copy link
Contributor

Is it an issue if the firewall is phoned home and entered the running phase and the firewall is unhealthy until the firewall controller connects? It should not take longer than a minute anyway?

@Honigeintopf
Copy link
Collaborator Author

No, it's not an issue. During the window between phoned-home and firewall-controller-connecting, the FirewallHealthy condition(It's a new one) hasn't been set yet (it's only set once ALL conditions are met for the first time)

@Gerrit91
Copy link
Contributor

Okay, I see now where you want to go, I will comment in the code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

Firewall health check

3 participants