Skip to content

Testing: use synctest for timing-dependent tests#756

Open
La002 wants to merge 8 commits intomodelcontextprotocol:mainfrom
La002:synctest
Open

Testing: use synctest for timing-dependent tests#756
La002 wants to merge 8 commits intomodelcontextprotocol:mainfrom
La002:synctest

Conversation

@La002
Copy link
Contributor

@La002 La002 commented Jan 17, 2026

Testing: use synctest for timing-dependent tests

Converted timing-based tests to use Go 1.25's synctest for instant, deterministic execution with simulated time.

Moved converted tests to *_go125_test.go files. Tests using real I/O (exec.Command, httptest) remain timing-based as they require actual system operations.

Fixes #680

Converted tests with time.After/time.Sleep to use Go 1.25's synctest
package in new *_go125_test.go files for instant, deterministic execution.

Tests using real I/O remain timing-based.

Fixes modelcontextprotocol#680
@La002 La002 changed the title Moved tests using time to use synctest in new files Testing: use synctest for timing-dependent tests Jan 17, 2026
@La002
Copy link
Contributor Author

La002 commented Jan 25, 2026

@findleyr @jba following up, is this still relevant ? Are any changes required?

@La002
Copy link
Contributor Author

La002 commented Jan 28, 2026

@maciej-kisiel Pushed with the changes discussed here

Copy link
Contributor

@maciej-kisiel maciej-kisiel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR and and patience with the review. I added some comments.

defer cs.Close()

// Let the connection establish properly first
time.Sleep(30 * time.Millisecond)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be replaced with synctest.Wait()?

Copy link
Contributor Author

@La002 La002 Feb 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added 246ae8b

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the test still pass if you remove the Sleep? If so, I would propose to remove it altogether (the comment may be left to explain the synctest.Wait call).

// Wait for keepalive to detect the failure and close the client
// Check periodically with simulated time advancement
for i := 0; i < 40; i++ { // 40 iterations * 25ms = 1 second total
time.Sleep(25 * time.Millisecond)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have the background why this loop was introduced, but wouldn't a single sleep longer than the keepalive time be sufficient?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it might have been useful for earlier test, to exit early and not have to wait the whole duration of whatever longer sleep time would have been specified. But now with synctest, time is not an issue, so a sufficient sleep time of 1s should be good enough

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me try to be more precise: could this loop be replaced with:

time.Sleep(51 * time.Millisecond) // One millisecond more than the keepalive duration.
if _, err := cs.CallTool(...); !errors.IsError(err, ErrConnectionClosed) {
  t.Errorf(...)
}

Copy link
Contributor Author

@La002 La002 Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should not be ideally. When the connection is closed it should need some buffer time to detect the failure. (2-3x atleast should have been given)
But I am unable to test that here because of this potential issue, when we do CallTool() - its also capable of detecting error , whether KeepAlive mechanism detected it or not.
I tried this below test for example (without synctest), and this still passes.

func TestKeepAliveFailure(t *testing.T) {
	// same csetup.....
	defer cs.Close()

	// Let the connection establish properly first
	//time.Sleep(30 * time.Millisecond)

	// simulate ping failure
	ss.Close()
	
	time.Sleep(10 * time.Nanosecond) // net wait from session closure is 10 ns
	_, err = cs.CallTool(ctx, &CallToolParams{
		Name:      "greet",
		Arguments: map[string]any{"Name": "user"},
	})
	if errors.Is(err, ErrConnectionClosed) {
		return // Test passed
	}
	t.Errorf("expected connection to be closed by keepalive, but it wasn't. Last error: %v", err)
}

So a bigger wait of 51ms is also passing.
Is this expected?

@La002
Copy link
Contributor Author

La002 commented Feb 5, 2026

replied to comments @maciej-kisiel

Copy link
Contributor

@maciej-kisiel maciej-kisiel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the changes. There are still 3 unresolved comment threads, please take a look.

defer cs.Close()

// Let the connection establish properly first
time.Sleep(30 * time.Millisecond)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the test still pass if you remove the Sleep? If so, I would propose to remove it altogether (the comment may be left to explain the synctest.Wait call).

// Wait for keepalive to detect the failure and close the client
// Check periodically with simulated time advancement
for i := 0; i < 40; i++ { // 40 iterations * 25ms = 1 second total
time.Sleep(25 * time.Millisecond)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me try to be more precise: could this loop be replaced with:

time.Sleep(51 * time.Millisecond) // One millisecond more than the keepalive duration.
if _, err := cs.CallTool(...); !errors.IsError(err, ErrConnectionClosed) {
  t.Errorf(...)
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Testing: use synctest throughout

3 participants