Add Reader max window size and Writer window size#150
Open
cce wants to merge 4 commits intoDataDog:1.xfrom
Open
Add Reader max window size and Writer window size#150cce wants to merge 4 commits intoDataDog:1.xfrom
cce wants to merge 4 commits intoDataDog:1.xfrom
Conversation
…mit memory usage during compression and decompression. More details available in RFC 9659. - NewWriterLevelDictWindowSize added to set specific window size (in bytes) - NewReaderDictMaxWindowSize added to set a maximum window size (in bytes) that this decompressor will allow.
…4.8 due to facebook/zstd#2442, fixed in v1.4.9 by facebook/zstd#2451
Viq111
reviewed
Mar 28, 2025
Collaborator
Viq111
left a comment
There was a problem hiding this comment.
Thanks for your contribution and the thorough test cases !
Unfortunately as this is using experimental (per zstd's documentation) features and disallow dynamic linking, we wouldn't be able to merge to the 1.x. I'll leave the PR open and if the upstream zstd repositority makes this parameter non-experimental, we can revisit and merge it
| package zstd | ||
|
|
||
| /* | ||
| #define ZSTD_STATIC_LINKING_ONLY 1 |
Collaborator
There was a problem hiding this comment.
🫤 Unfortunately this means only static linking is allowed which is a deal breaker for many users of this library that are using dynamic linking. As this would break compatibility, we can't merge it to 1.x.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
NewWriterLevelDictWindowSizeandNewReaderDictMaxWindowSizeto limit memory usage during compression and decompression.More documentation on window sizes are in RFC 9659 and in zstd.h:
Resolves #123.