Skip to content

OpenVINO EP registration, LMEvaluator handles model with single logits#2347

Merged
jambayk merged 4 commits intomainfrom
jambayk/llmeval-ov
Mar 5, 2026
Merged

OpenVINO EP registration, LMEvaluator handles model with single logits#2347
jambayk merged 4 commits intomainfrom
jambayk/llmeval-ov

Conversation

@jambayk
Copy link
Contributor

@jambayk jambayk commented Mar 5, 2026

Describe your changes

  • OpenVINO EP requires openvino.dll to be on PATH. importing openvino package helps do that if it's not already on PATH since python doesn't seem to discover it automatically from the site-packages directory.
  • LMEvaluator with ortgenai backend now handles models that only return the logits for the final token. Tasks such as hellaswag require logits for more than one last token.

Checklist before requesting a review

  • Add unit tests for this change.
  • Make sure all tests can pass.
  • Update documents if necessary.
  • Lint and apply fixes to your code by running lintrunner -a
  • Is this a user-facing change? If yes, give a description of this change to be included in the release notes.

(Optional) Issue link

@jambayk jambayk enabled auto-merge (squash) March 5, 2026 08:13
@jambayk jambayk merged commit 79220f6 into main Mar 5, 2026
11 checks passed
@jambayk jambayk deleted the jambayk/llmeval-ov branch March 5, 2026 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants