[Repo Assist] Add llms.txt/llms-full.txt generation

[github-actions]

🤖 This is an automated PR from Repo Assist, an AI assistant for this repository.

Closes #951

Summary

Adds a --generatellmstxt flag to fsdocs build and fsdocs watch. When enabled, two files are written to the output root:

• llms.txt — a Markdown index with links to all documentation pages and API reference entries, following the [llmstxt.org]((llmstxt.org/redacted) convention
• llms-full.txt — same as above but with full page content included after each entry

This makes it easy to add documentation context for F# projects to LLMs and AI coding assistants.

Usage

dotnet fsdocs build --generatellmstxt
```

## Implementation

The generated files reuse the search index data (`latestApiDocSearchIndexEntries` and `latestDocContentSearchIndexEntries`) that is already in memory, so there's no additional parsing overhead. The files are regenerated whenever the search index is regenerated (on initial build and on file changes in watch mode).

The format is:

```
# {collection-name}

## Docs

- [Page title]((example.com/redacted)
...

## API Reference

- [Namespace.Type]((example.com/redacted)
...

Trade-offs

• Uses existing in-memory data; no new dependencies.
• llms-full.txt can be large for projects with extensive API docs. Users can choose which file to expose.
• Watch mode regenerates both files on every content or project change, which is appropriate for iterative editing.

Test Status

• ✅ dotnet build src/fsdocs-tool/fsdocs-tool.fsproj -c Release — succeeded, 0 errors
• ✅ dotnet test tests/FSharp.ApiDocs.Tests/FSharp.ApiDocs.Tests.fsproj -c Release — 68 passed, 0 failed

Generated by Repo Assist

To install this workflow, run gh aw add githubnext/agentics/workflows/repo-assist.md@ee50a3b7d1d3eb4a8c409ac9409fd61c9a66b0f5. View source at https://github.com/githubnext/agentics/tree/ee50a3b7d1d3eb4a8c409ac9409fd61c9a66b0f5/workflows/repo-assist.md.

[github-actions]

✅ Pull request created: #980

[dsyme]

/repo-assist change the flag in this PR to --llms

[github-actions]

✅ Commit pushed: 468ac35

[github-actions]

🤖 This is an automated response from Repo Assist.

Done! Renamed --generatellmstxt to --llms in both the code (BuildCommand.fs) and RELEASE_NOTES.md.

Generated by Repo Assist for issue #980

To install this workflow, run gh aw add githubnext/agentics/workflows/repo-assist.md@828ac109efb43990f59475cbfce90ede5546586c. View source at https://github.com/githubnext/agentics/tree/828ac109efb43990f59475cbfce90ede5546586c/workflows/repo-assist.md.

[dsyme]

/repo-assist We need some kind of testing added for this

[github-actions]

✅ Commit pushed: a8c94e0

[github-actions]

🤖 This is an automated response from Repo Assist.

Done! Added testing for the --llms flag. Here's what was changed:

src/fsdocs-tool/BuildCommand.fs
Extracted the core text-generation logic from the private generateLlmsTxt local function into a new internal module LlmsTxt with two public functions:

• LlmsTxt.buildSection — builds one markdown section (Docs or API Reference)
• LlmsTxt.buildContent — takes collectionName + all search index entries, returns (llms.txt content, llms-full.txt content)

The generateLlmsTxt call site is simplified to just call LlmsTxt.buildContent and write the results.

tests/FSharp.Literate.Tests/DocContentTests.fs
Added 8 unit tests for LlmsTxt.buildContent:

1. Correct header is generated
2. Empty entries → header only
3. Docs and API Reference sections are separated correctly
4. llms.txt (index mode) does not include content body
5. llms-full.txt includes content body
6. Blank content is skipped in full mode
7. ## Docs section is omitted when no content entries exist
8. ## API Reference section is omitted when no apiDocs entries exist

All 8 tests pass (dotnet test --filter "FullyQualifiedName~LlmsTxt" → 8/8 passed).

Generated by Repo Assist for issue #980

To install this workflow, run gh aw add githubnext/agentics/workflows/repo-assist.md@828ac109efb43990f59475cbfce90ede5546586c. View source at https://github.com/githubnext/agentics/tree/828ac109efb43990f59475cbfce90ede5546586c/workflows/repo-assist.md.

[nojaf]

I think it makes more sense to have this as fsproj/msbuild configuration rather than a new feature flag.

[nojaf]

Having to pass --llm on everything invocation doesn't seem the way to go for me.
I'd rather enable this once and then never forget about it.

[dsyme]

Having to pass --llm on everything invocation doesn't seem the way to go for me. I'd rather enable this once and then never forget about it.

Yup agreed. It's possible it should be on by default too? Gven use of LLMs is now normal.

[nojaf]

It's possible it should be on by default too?

Seems like a reasonable default yes.

[dsyme]

/repo-assist update this PR taking this into account:

I think it makes more sense to have this as fsproj/msbuild configuration rather than a new feature flag.

Please make sure all docs are updated.

[github-actions]

✅ Commit pushed: 761d83b

[github-actions]

🤖 This is an automated response from Repo Assist.

Done! Replaced the --llms CLI flag with a (FsDocsGenerateLlmsTxt) MSBuild property.

What changed:

• src/fsdocs-tool/ProjectCracker.fs — Added FsDocsGenerateLlmsTxt to CrackedProjectInfo. Reads from the project file (default: true) and is propagated through crackProjects.
• src/fsdocs-tool/BuildCommand.fs — Removed the --llms CLI option. The generateLlmsTxt function now checks the MSBuild property value instead.
• docs/styling.md — New "LLM-Friendly Output" section documenting the property and how to opt out.
• RELEASE_NOTES.md — Updated entry to reflect the new configuration approach.

Behaviour:

Scenario	Result
Default (no property set)	llms.txt and llms-full.txt are generated ✅
(FsDocsGenerateLlmsTxt)true(/FsDocsGenerateLlmsTxt)	Generated ✅
(FsDocsGenerateLlmsTxt)false(/FsDocsGenerateLlmsTxt)	Not generated ❌

The property can be set in the project file (.fsproj) or in Directory.Build.props for repo-wide control. All 8 existing LlmsTxt unit tests still pass.

Generated by Repo Assist for issue #980

To install this workflow, run gh aw add githubnext/agentics/workflows/repo-assist.md@828ac109efb43990f59475cbfce90ede5546586c. View source at https://github.com/githubnext/agentics/tree/828ac109efb43990f59475cbfce90ede5546586c/workflows/repo-assist.md.

[dsyme]

/repo-assist Make the default for the MSBuild property for llms text to be on by default please and adjust test and documentation for this. Update release notes

[github-actions]

Commit pushed: 882ef16

Generated by Repo Assist

[nojaf]

/repo-assist

I'm still a little torn about the _template.md, llms.txt will have most benefit if the links lead to markdown files. But most users might not have _template.md file so when upgrading our new feature will be sub optimal.

Can you add some sort of warning message that if no _template.md file was found.
And this is recommended.

Also fix the build!

[dsyme]

/repo-assist See comment from @nojaf above.

Also about this:

Preferring _template.md for API docs when FsDocsGenerateLlmsTxt = true means that for users who have both _template.html and _template.md, API docs will now be generated as markdown (reference/type.md) rather than HTML (reference/type.html). The HTML nav links in _template.html already reference the API pages using clean URLs (InUrl = ""), so they should still work — but this is worth verifying. If the behaviour change is undesirable, the ordering can be conditioned on the absence of _template.html rather than the presence of _template.md.

This is indeed a problem - turning on FsDocsGenerateLlmsTxt should also generate markdown - it shouldn't switch to markdown.

I didn't actually see in the docs anything about how to turn on markdown documentation generation and whether that can be done in addition to HTML or other output, so we probably have documentation gaps too.

Do what you can but we might have to rethink this if it gets too complex

[nojaf]

@dsyme I believe it happens now if the user has a _template.md file.
This file needs to exists, but can be empty.
Maybe we should always generated markdown when FsDocsGenerateLlmsTxt is enabled?
And assume empty template if no _template.md is present?

[github-actions]

Commit pushed: c8d602b

Generated by Repo Assist

[github-actions]

🤖 This is an automated response from Repo Assist.

I've pushed fixes addressing @dsyme's and @nojaf's feedback:

Changes made

1. Fix: FsDocsGenerateLlmsTxt no longer switches API docs from HTML to markdown

The template ordering for API docs has been restored to HTML-first, regardless of FsDocsGenerateLlmsTxt. Previously, enabling this property would prefer _template.md over _template.html for API reference generation — causing users with both templates to get .md files instead of .html files. Now it correctly keeps HTML as primary and only uses markdown if no HTML template is found.

The doc content layer already generates .md alongside .html when FsDocsGenerateLlmsTxt is enabled (via the defaultMdTemplate passed to docContent.Convert), so that path remains correct.

2. Add: Warning when no _template.md found

When FsDocsGenerateLlmsTxt is enabled but no _template.md exists in the docs folder, a note is printed:

note: FsDocsGenerateLlmsTxt is enabled but no '_template.md' was found in 'docs'.
      Consider adding a '_template.md' to generate Markdown pages alongside HTML.
      Markdown output is more suitable for LLM consumption (see (fsprojects.github.io/redacted)

3. Fix: Build errors in tests

The LlmsTxt.buildContent calls in DocContentTests.fs were missing the two new boolean arguments (docContentUsesMarkdown and apiDocUsesMarkdown) — all 16 calls updated to pass false false.

4. Fix: Code scanning alert

Added StringComparison.Ordinal to the StartsWith call flagged at line 1317.

---

Note on "also generate markdown for API docs": Currently API docs can only be generated in one format per run (HTML or Markdown). Generating both in the same run would require a second pass through ApiDocs.GenerateMarkdownPhased, which is a larger architectural change. For now, apiDocUsesMarkdown in llms.txt will only be true when the user has a markdown template but no HTML template. This is worth a follow-up issue if generating both is desired.

Test status: Build succeeds, all 116 literate tests pass.

Generated by Repo Assist for issue #980

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/repo-assist.md@afb00b92a9514fee9a14c583f059a03d05738f70

[dsyme]

@nojaf Not sure this feature is worth it if it only kicks in when generating markdown docs (and not HTML).

wdyt should we ask it to do the work to allow two doc type generations? I guess that's the solution?

[nojaf]

@dsyme I would do the following:

FsDocsGenerateLlmsTxt is on by default, and because it is on we generate markdown as well regardless of the presence of _template.md.

If you turn of FsDocsGenerateLlmsTxt and have no _template.md it does not generate markdown.

[nojaf]

/repo-assist right now the PR has a conflict in the RELEASE_NOTES.md file.
You need to fix that, but also realize that a single PR should typically lead to a single entry. Right now you are incrementally adding entries there but that does not make much sense for a feature that we never shipped yet.

[github-actions]

Commit pushed: 2cd7946

Generated by Repo Assist

[nojaf]

@dsyme this works for me.
If it works for you we can merge this.

[dsyme]

@nojaf so good, thank you!