Linux Dedicated Server reaches 100% CPU due to busy polling in Transport (poll/select loop in Unity Update) #4087
Description
Environment:
OS: Linux (x86_64)
Tested on: Ubuntu 20.04 / 22.04 (kernel 5.x / 6.x)
Unity: 2022.3.34f1c1 (IL2CPP build)
Mirror: 96.8.5
Run 30 Dedicated Servers
Transport:
KcpTransport
ThreadedKcpTransport (reduced but still observable)
Build type: Dedicated Server (headless, -batchmode -nographics)
Load:
Even with 0 or very few clients
CPU already near 100%
Description
On Linux Dedicated Server, Mirror causes near 100% CPU usage even when there are no connected clients or very low traffic.
Profiling shows that CPU time is spent almost entirely in kernel polling syscalls, indicating a busy-poll loop caused by calling non-blocking socket Poll() / select() from Unity’s per-frame Update() loop.
This behavior does not reproduce on Windows, or is far less severe.
Observed Behavior
Server process consumes ~100% of one core
CPU usage remains high even when:
No players connected
No network traffic
CPU usage drops only when:
Artificial delays are introduced
Network polling frequency is manually throttled
A dedicated blocking I/O thread is implemented
Profiling Evidence (perf)
Using perf top / perf record on Linux:
Overhead Shared Object Symbol
14.80% [kernel] [k] do_poll.constprop.0
9.03% [kernel] [k] _raw_spin_lock_irqsave
8.58% [kernel] [k] x64_sys_call
7.20% [kernel] [k] copy_user_enhanced_fast_string
6.72% [kernel] [k] sock_poll
5.29% [kernel] [k] tcp_poll
4.37% [kernel] [k] __fdget
3.18% [kernel] [k] __fget_files
3.09% [kernel] [k] do_sys_poll
3.02% [kernel] [k] srso_alias_safe_ret
2.97% [kernel] [k] __lock_text_start
2.62% [kernel] [k] __fget_light
2.57% [kernel] [k] fput
2.46% [kernel] [k] fput_many
2.13% [kernel] [k] do_syscall_64
1.57% [kernel] [k] poll_freewait
1.36% [kernel] [k] __pollwait
1.25% [kernel] [k] remove_wait_queue
1.18% [kernel] [k] entry_SYSCALL_64_after_hwframe
1.16% libc.so.6 [.] __libc_calloc
1.14% [kernel] [k] __raw_callee_save___pv_queued_spin_unlock
0.99% [kernel] [k] pipe_poll
I tried to modify the script, but after running for a while, the issue of CPU usage at 100% still persists
ThreadedKcpTransport.cs:
protected override int GetWaitTimeoutMs()
{
if (ServerActive() )
{
return server.connections.Count == 0 ? 3000 : 1;
}
else
{
return 1;
}
}ThreadedTransport.cs:
protected virtual int GetWaitTimeoutMs()
{
return 1;
}
bool ThreadTick()
{
// was the device put to sleep?
if (sleepTimer != null &&
sleepTimer.Elapsed.TotalSeconds >= sleepTimeoutInSeconds)
{
Debug.Log("ThreadedTransport: entering sleep mode and stopping/disconnecting.");
ThreadedServerStop();
ThreadedClientDisconnect();
sleepTimer = null;
// if the device was put to sleep, end the thread gracefully.
// all threads must end, otherwise putting down the device would
// slowly drain the battery after a day or more.
return false;
}
// early update the implementation first
ThreadedClientEarlyUpdate();
ThreadedServerEarlyUpdate();
// process queued user requests
ProcessThreadQueue();
// late update the implementation at the end
ThreadedClientLateUpdate();
ThreadedServerLateUpdate();
// save some cpu power.
int waitMs = GetWaitTimeoutMs();
// Debug.LogError("sleep :" + waitMs);
Thread.Sleep(waitMs);
return true;
}Extensions.cs
public static bool ReceiveFromNonBlocking(this Socket socket, int ConnectedCount, byte[] recvBuffer, out ArraySegment<byte> data, ref EndPoint remoteEP)
{
data = default;
int pollTimeout = ConnectedCount == 0 ? 100000 : 0; // 100ms: 0;
try
{
int size = socket.ReceiveFrom(recvBuffer, 0, recvBuffer.Length, SocketFlags.None, ref remoteEP);
data = new ArraySegment<byte>(recvBuffer, 0, size);
return true;
}
catch (SocketException e)
{
if (e.SocketErrorCode == SocketError.WouldBlock) return false;
throw;
}
}using UnityEngine;
using System.Diagnostics;
using Mirror;
using System.Threading;
public class ServerIdleFrameLimiter : Singleton<ServerIdleFrameLimiter>
{
public override void Init()
{
base.Init();
Application.targetFrameRate = -1;
QualitySettings.vSyncCount = 0;
idleFrameTicks = Stopwatch.Frequency / idleFps;
lastFrameTicks = Stopwatch.GetTimestamp();
}
public int idleFps = 5;
long lastFrameTicks;
long idleFrameTicks;
public void OnUpdate()
{
// Headless Server
if (!Application.isBatchMode)
return;
if (!NetworkServer.active)
return;
if (NetworkServer.connections.Count > 0)
return;
long now = Stopwatch.GetTimestamp();
long elapsed = now - lastFrameTicks;
if (elapsed < idleFrameTicks)
{
int sleepMs = (int)((idleFrameTicks - elapsed) * 1000 / Stopwatch.Frequency);
if (sleepMs > 0)
{
//Log.Info($"Sleep ： {sleepMs}");
Thread.Sleep(sleepMs);
}
}
lastFrameTicks = Stopwatch.GetTimestamp();
}
}