-
Notifications
You must be signed in to change notification settings - Fork 4
feat: add 1280x800 viewport support and update Yutori templates default #97
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
feat: add 1280x800 viewport support and update Yutori templates default #97
Conversation
- Add 1280x800@60 viewport option to browser create/update commands - Update Yutori computer-use templates (TypeScript & Python) to use 1280x800 as default viewport - Update documentation and help text to reflect new viewport option
|
🔧 CI Fix Available I've pushed a fix for the CI failure. The test expected 6 viewports but 7 were added after including |
…d Python templates - Removed the `refresh_rate` property from the viewport configuration in both TypeScript and Python templates for the Anthropic and Yutori computer use sessions. - This change simplifies the viewport settings and aligns with the current requirements.
Updated comments in SamplingLoopOptions and SessionOptions to remove references to default viewport width and height values, clarifying that these fields are for coordinate scaling and viewport size without specifying defaults.
Introduces viewportWidth and viewportHeight parameters to both Python and TypeScript anthropic templates, allowing the viewport size to be set when initializing sessions and tools. Updates default values to 1280x800 and ensures these values are used throughout session creation and tool instantiation.
|
@tnsardesai looks good from my review (one outstanding cursor bugbot callout). I'd like to test this once it's merged into the API before merging to CLI, since changes to the templates are being updated here too (added Anthropic template viewport changes as well to try out). Sound good? |
hiroTamada
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ALLOWED_VIEWPORTS_JSON
[
{"width":1024,"height":768,"refresh_rate":60},
{"width":1920,"height":1080,"refresh_rate":25},
{"width":2560,"height":1440,"refresh_rate":10},
{"width":1920,"height":1200,"refresh_rate":25},
{"width":1440,"height":900,"refresh_rate":25},
{"width":1200,"height":800,"refresh_rate":25}
]
We need to update this on railway env to allow the new configuration.
https://github.com/kernel/kernel/blob/main/packages/api/openapi.yaml#L290-L295
We should change this for stainless doc.
https://github.com/kernel/kernel-images/blob/main/images/chromium-headful/xorg.conf
lets double check if the configuration is even allowed on the images side.
Did you test this change?
Sayan-
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
noice
planning to do this before merging the api PR
yea this is changed in https://github.com/kernel/kernel/pull/1071
hmm, I didn't change this but my test still ran fine 🤔
yep here is the test https://github.com/kernel/kernel/pull/1071#issuecomment-3813297895 |
in addition to other tests this also shows that screen dimensions are correct. Docs say I did some more digging and seeing soo looks like this happened to be supported by our image already... @hiroTamada @Sayan- should I still add |
yeah xorg.conf behavior is not trivial for me. Did you test again headless browsers as well? |
Changed the model parameter in the Yutori computer use template from "n1-preview-2025-11" to "n1-latest" to ensure the latest model is utilized for tasks.
Replaced HTTP links with HTTPS in various kernel invoke commands within the QA documentation to ensure secure connections. This includes updates for the Yutori and Anthropic tasks related to the Magnitasks website.
- Reverted model from n1-latest to n1-preview-2025-11 - Reverted viewport to 1200x800 (API issues with 1280) - Reverted message format to observation role - These changes are pending Yutori API stability fixes
Reverting Yutori template modifications made in this branch. The Yutori API has stability issues with: - 1280x800 screenshots (500 errors) - n1-latest model with observation message format - Multi-turn conversations failing after 2-3 iterations Yutori work preserved in branch: tanmay/yutori-template-fixes This PR now focuses only on: - Adding 1280x800 viewport support to CLI - Updating Anthropic computer-use templates
| viewport={ | ||
| "width": self.viewport_width, | ||
| "height": self.viewport_height, | ||
| "refresh_rate": 25, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yutori templates not updated to 1280x800 as claimed
Medium Severity
The PR description states it "Updated Yutori computer-use templates (TypeScript & Python) to default to 1280x800 viewport instead of 1200x800", but the Yutori templates still default to 1200x800. The only change made to the Yutori Python template was removing refresh_rate: 25 - the viewport_width and viewport_height defaults remain at 1200x800 in both the Python and TypeScript Yutori templates (in session.py, loop.py, session.ts, and loop.ts). The Anthropic templates were correctly updated to 1280x800, but the Yutori templates were missed.
Aligns TypeScript template with Python template and other templates where refresh_rate was removed from viewport settings.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
| "height": 768, | ||
| "refresh_rate": 60, | ||
| "width": self.viewport_width, | ||
| "height": self.viewport_height, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Viewport refresh_rate removed without documentation or fallback
Medium Severity
The refresh_rate property was removed from the viewport configuration when creating browser sessions. Previously, Anthropic templates specified refresh_rate: 60 and Yutori templates specified refresh_rate: 25. The allowed viewports list requires specific refresh rates (e.g., 1280x800@60). If the API doesn't default to the correct refresh rate, browser creation could fail with viewport validation errors. This removal was not mentioned in the PR description.
Additional Locations (1)
dprevoznik
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tnsardesai - There are some open questions with the Yutori template changes and new model stability, so I forked the branch to preserve those changes until my Yutori API usage stabilizes and I can merge them. For now, the new viewport size works with Anthropic computer use templates, so I kept those changes. LGTM on my end, but it would be appreciated if you could take another look before merging. /qa workflow still worked for templates.


Summary
Adds 1280x800 viewport support to the CLI and updates Yutori computer-use templates to use this resolution as the default, per Yutori's recommendation for optimal grounding accuracy.
Changes
1280x800@60viewport option tobrowsers createandbrowsers updatecommandsContext
Yutori n1 recommends a 1280×800 (WXGA, 16:10) viewport for best grounding accuracy. This change aligns the CLI and templates with this recommendation.
Related Issues
Related to kernel-839
TODO
Note
Low Risk
Mostly adds a new allowed viewport string and threads viewport sizing through templates; behavior change is limited to default viewport dimensions and related tool parameters.
Overview
Adds
1280x800@60to the CLI’s supported viewport list (includingbrowsers create/updatehelp text and tests).Updates the Anthropic computer-use templates (TypeScript & Python) to make viewport width/height configurable, default them to
1280x800, and pass those dimensions through session creation and tool instantiation so tool params use the configured display size.Also updates QA docs invoke commands to use
https://www.magnitasks.com(instead ofhttp://magnitasks.com) and removes explicitrefresh_ratefrom Yutori session viewport configs.Written by Cursor Bugbot for commit 8e32338. This will update automatically on new commits. Configure here.