Core: Handle bulk deletion exceptions in CatalogUtil#15464
Open
sumtiogo wants to merge 1 commit intoapache:mainfrom
Open
Core: Handle bulk deletion exceptions in CatalogUtil#15464sumtiogo wants to merge 1 commit intoapache:mainfrom
sumtiogo wants to merge 1 commit intoapache:mainfrom
Conversation
7fc52f2 to
62b5ff3
Compare
62b5ff3 to
8ec13df
Compare
nastra
reviewed
Feb 27, 2026
|
|
||
| import static org.assertj.core.api.Assertions.assertThat; | ||
| import static org.assertj.core.api.Assertions.assertThatThrownBy; | ||
| import static org.assertj.core.api.AssertionsForClassTypes.assertThatCode; |
Contributor
There was a problem hiding this comment.
Suggested change
| import static org.assertj.core.api.AssertionsForClassTypes.assertThatCode; | |
| import static org.assertj.core.api.Assertions.assertThatCode; |
nastra
reviewed
Feb 27, 2026
| .deleteFiles( | ||
| Iterables.transform( | ||
| removedPreviousMetadataFiles, TableMetadata.MetadataLogEntry::file)); | ||
| try { |
Contributor
There was a problem hiding this comment.
rather than adding this try-catch block, we should probably just call deleteFiles() here directly, which internally already catches exceptions.
deleteFiles(
io,
removedPreviousMetadataFiles.stream()
.map(TableMetadata.MetadataLogEntry::file)
.collect(Collectors.toSet()),
"metadata",
true)
with that you can also remove the parallel delete in the else block
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I recently opened Issue #15456 regarding this missing exception handling. Since the fix is relatively straightforward and I have already verified it with a unit test locally, I went ahead and created this PR to facilitate the discussion and review process.
Fixes #15456
Why are the changes needed?
In
CatalogUtil.deleteRemovedMetadataFiles, there is an asymmetry in how exceptions are handled between bulk operations and single-file deletions.When the
FileIOsupports bulk operations,deleteFilesis called directly without failure suppression. However, implementations ofSupportsBulkOperations.deleteFilescan and do throw exceptions (e.g.,BulkDeletionFailureExceptioninS3FileIOwhen S3DeleteObjectsfails). Because this cleanup operation is intended to be best-effort, a system-level or network exception here should be caught and logged rather than bubbling up and potentially crashing the main calling process. The single-file deletion (elsebranch) already handles this correctly usingTasks.suppressFailureWhenFinished().What changes were proposed in this pull request?
SupportsBulkOperations.deleteFilescall in atry-catchblock withinCatalogUtil.deleteRemovedMetadataFiles.BaseTransaction).How was this patch tested?
testDeleteRemovedMetadataFilesBulkDeletionFailureinTestCatalogUtilusing Mockito to simulate aRuntimeExceptionduring bulk deletion and verified that it is safely caught without throwing../gradlew spotlessApplyand./gradlew :iceberg-core:testlocally.