Add sort and exclude options for folder auto-discovery in toc.yml

[barkbay]

This is my very first code change in C# and in this project, so bear with me! Code was generated with the help of an AI agent but I tried to review and understand it with my 🧠 .

This PR adds two options to folder entries in toc.yml for auto-discovered files:

sort — control file ordering

Please refer to this comment for the original need.

toc:
  - folder: api-versions
    sort: desc

Valid values: asc, ascending, desc, descending. Defaults to ascending.

• When no sort is specified, files are sorted alphabetically (A-Z), preserving existing behavior.
• When sort is set, natural sort order is used so version numbers sort correctly (3_2_0 comes before 3_10_0).
• index.md is always placed first regardless of sort order.
• When explicit children are defined, sort has no effect — the user-defined order is respected.
• Unrecognized sort values emit an error diagnostic and default to ascending.

exclude — filter out specific files

toc:
  - folder: docs
    exclude:
      - draft.md
      - internal-notes.md

• Excluded file names are matched case-insensitively.
• Only applies to auto-discovery (no effect when children are explicitly defined).

Test plan

• Default sort order is ascending when sort is omitted
• sort: desc / sort: descending orders files Z-A
• sort: asc / sort: ascending orders files A-Z
• index.md always placed first even with descending sort
• folder + file combo preserves sort value
• Explicit children ignore sort order
• Unrecognized sort value emits an error diagnostic
• Natural sort handles version numbers correctly
• Exclude filters out specified files
• Exclude supports multiple files
• Exclude is case-insensitive
• Exclude can remove index.md when explicitly listed
• No exclude includes all files (baseline)

[github-actions]

🔍 Preview links for changed docs

• docs/configure/content-set/navigation.md

[reakaleek]

This generally LGTM.

Thank you for the contribution!

Is this enough for sorting versions? I expect unintended behaviour when we go past 10.x versions. This might actually be already true for minor versions beyond x.10.0.

e.g.

3.0.0
3.1.0
3.2.0
3.10.0

but if we sort this ascending or descending. It will become

ascending:

3.0.0
3.1.0
3.10.0
3.2.0

or

descending:

3.2.0
3.10.0
3.1.0
3.0.0

[shainaraskas]

@barkbay if you intend to use this for ECK, it should be paired with some sort of exclusion logic so we can hide current

[barkbay]

Is this enough for sorting versions? I expect unintended behaviour when we go past 10.x versions. This might actually be already true for minor versions beyond x.10.0.

Good catch 🤦 I updated the PR with 10f4ab3 to use natural sort order (comparing numeric segments as integers) when sort is explicitly set. This ensures version-numbered files sort correctly, for example 3_2_0 comes before 3_10_0, not after.

To be conservative, natural sort is only used when sort is explicitly specified in YAML. Folders without sort still use the original alphabetical sorting, so existing behavior is preserved and nothing breaks.

I also added NaturalStringComparer with its own set of unit tests covering version numbers, mixed prefixes, edge cases...

[barkbay]

@barkbay if you intend to use this for ECK, it should be paired with some sort of exclusion logic so we can hide current

@shainaraskas good point! Should I create a new PR? I'm asking as it feels like a separate concern from sort order. Happy to open a follow-up PR.

[barkbay]

@barkbay if you intend to use this for ECK, it should be paired with some sort of exclusion logic so we can hide current

@shainaraskas good point! Should I create a new PR? I'm asking as it feels like a separate concern from sort order. Happy to open a follow-up PR.

Maybe not a big change, so it I decided to implement in a20edc2 Happy to also create a dedicated PR.

[shainaraskas]

I'll let dev opine on whether exclude would need to go in alone but this makes sense to me

[Copilot]

Pull request overview

This PR extends toc.yml folder auto-discovery to support configurable file ordering (sort) and file filtering (exclude) when children are not explicitly defined, including natural sorting for version-like filenames.

Changes:

• Parse and carry sort / exclude from toc.yml into FolderRef, validate sort, and apply sorting + filtering during folder auto-discovery.
• Introduce SortOrder, parsing helpers, and a NaturalStringComparer to support natural ordering when sort is explicitly set.
• Add/expand tests and update navigation documentation to describe the new options.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
tests/Navigation.Tests/Isolation/NaturalStringComparerTests.cs	Adds tests for natural string comparison behavior (currently contains compile errors).
tests/Navigation.Tests/Isolation/FolderSortOrderTests.cs	Adds tests for sort behavior in folder auto-discovery.
tests/Navigation.Tests/Isolation/FolderExcludeTests.cs	Adds tests for exclude behavior in folder auto-discovery.
src/Elastic.Documentation.Configuration/Toc/TableOfContentsYamlConverters.cs	Extends YAML converter to deserialize sort and exclude for folder items.
src/Elastic.Documentation.Configuration/Toc/TableOfContentsItems.cs	Adds SortOrder, parsing extensions, NaturalStringComparer, and extends FolderRef.
src/Elastic.Documentation.Configuration/Toc/DocumentationSetFile.cs	Applies sort/exclude during folder auto-discovery and emits diagnostics for invalid sort.
docs/configure/content-set/navigation.md	Documents the new sort and exclude options for folder auto-discovery.

[Copilot]

new string(same) does not compile because string has no constructor that takes a string. If you need a different instance with the same content (to exercise the non-ReferenceEquals path), create it via same.ToCharArray() (or similar) instead.

[barkbay]

I'm not sure I understand this. According to Claude string has an implicit conversion to ReadOnlySpan<char>, and there is a string(ReadOnlySpan<char>) constructor.

When compiling new string(same) the compiler sees:

1. No string(string) constructor — doesn't match
2. same implicitly converts to ReadOnlySpan — matches string(ReadOnlySpan) — compiles

This code works perfectly fine with .NET 10

[barkbay]

@copilot new string(string) compiles fine on .NET Core 2.1+ because string implicitly converts to ReadOnlySpan, which matches the string(ReadOnlySpan) constructor. The tests pass. Can you confirm?

[Copilot]

new string(a) does not compile because there is no string constructor that accepts a string. If the goal is to compare equal values with different references, construct from a char[] or another explicit copy mechanism.

[barkbay]

Same as #2870 (comment)

[Copilot]

ParseNumber uses long.Parse(...) which can throw OverflowException if a filename contains a numeric segment larger than Int64.MaxValue (and would fail the docs build when sort is set). Consider using long.TryParse with a deterministic fallback comparison (e.g., compare digit-run lengths, then ordinal compare) to keep sorting total and non-throwing.

Suggested change

	return long.Parse(s[start..index], CultureInfo.InvariantCulture);
	var segment = s.Substring(start, index - start);
	if (long.TryParse(segment, NumberStyles.None, CultureInfo.InvariantCulture, out var value))
	return value;
	// Fallback for values that do not fit in Int64 (e.g., very long digit runs).
	// Returning a fixed sentinel keeps comparisons deterministic and non-throwing.
	return long.MaxValue;

[barkbay]

Valid point. A filename with a huge numeric segment would throw. However, this is extremely unlikely for documentation file names. If we want to be defensive, the long.TryParse fallback is reasonable. But the suggested fallback of returning long.MaxValue would make all oversized numbers compare equal, which isn't great 🤷 A better fallback might be to compare by digit-run length first (longer = larger), then lexicographically.

[barkbay]

Digging into this, the overflow would only matter for filenames with numeric segments larger than 9,223,372,036,854,775,807 (19+ digits). Do we think this is realistic for documentation file names?

[Copilot]

When resolving a FolderRef that has explicit children, the returned FolderRef drops the Exclude value (new FolderRef(..., folderRef.Sort)), so the resolved model no longer reflects the original YAML configuration. Even if exclude is ignored for explicit children, consider passing through folderRef.Exclude for consistency/debuggability.

Suggested change

	return new FolderRef(fullPath, pathRelativeToContainer, resolvedChildren, context, folderRef.Sort);
	return new FolderRef(fullPath, pathRelativeToContainer, resolvedChildren, context, folderRef.Sort, folderRef.Exclude);

[barkbay]

This is intentional, exclude only applies to auto-discovery. Carrying it through when children are explicit would suggest it has an effect when it doesn't. Tried to clarify this in 80fae41

[Copilot]

new string(same) does not compile (no string ctor overload taking string). Use a safe way to create a distinct string instance (e.g., from same.ToCharArray()) or remove the redundant allocation if reference identity isn't important here.

[barkbay]

Same as #2870 (comment)

[Copilot]

@barkbay I've opened a new pull request, #2880, to work on those changes. Once the pull request is ready, I'll request review from you.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

---

You can also share your feedback on Copilot code review. Take the survey.