[Bug]: Using NVPAIR on local machine remotely accessed - causes some issues

[Kaotic3]

PAIR version or commit

0.1.1

Affected component

Desktop application

Environment

OS: Windows 10
Architecture: x64
GPU: 4090 RTX
Model: Qwen 3.6 27B
Cluster Size: 4 nodes

Steps to reproduce

Have a desktop running windows 10, then use Sunshine and Vibeshine to dual monitor that remotely using moonlight into a Beelink device - running Windows 11, then connect that via a TESmart KVM to a dual monitor, 3 device, 1 keyboard and mouse setup.

Interesting fact - you would think that complexity would have an impact anyway - it doesn't. Without NVPAIR this system works flawlessly, no bitrate issues, no frames dropped - it is actually quite impressive when you get it running.

Expected behavior

I understood when looking at hte documentation that NVPAIR would see that your GPU was being used and bypass it - so I was expecting to be able to game without any issues.

Actual behavior

When I start the game I get warnings about lowering my bitrate and everything gets a latency in the realms of about 1-2 secs, and I don't mean "in-game" lag, I mean mouse movement lag across my desktop. It was actually tricky to shut down the game so that I could turn off NVPAIR. This is definitely an NVPAIR issue, when I turned it off (by right clicking in the taskbar) the game worked flawlessly again.

I have no idea why it would do this, or why it would impact on the bitrate - I suspect that it is Sunshine / Vibeshine / Moonlight that are providing those warnings - but I do not understand why NVPAIR would be impacting it. Hopefully you guys know.

Sanitized logs or screenshots

Confirmations

• I searched existing issues for duplicates.
• This is not a security vulnerability.
• I agree to follow the Code of Conduct.

[somtri]

@Kaotic3, PAIR on versus off is a useful result. We still need to separate the desktop process from a managed engine or active workload.

Please test the same game scene three times. Keep Sunshine, Vibeshine, Moonlight, display layout, and bitrate settings unchanged.

1. Quit PAIR and confirm its processes have exited.
2. Start PAIR with all managed engines stopped.
3. Start PAIR with the engine and model state that normally causes the lag.

For each run, include the start and end time, whether lag occurred, and a Task Manager capture. Show CPU, GPU, GPU engine, disk, and network use for PAIR, its managed engine, Sunshine, and the game.

Please also attach the sanitized PAIR logs from the same time range. State which engine and model were loaded, and whether an inference request was active. Remove usernames, hostnames, IP addresses, personal paths, personal data, private prompts, credentials, tokens, private keys, and real cluster identities.

This should show which of the three states introduces the latency.

[Kaotic3]

I did try this earlier this evening.

With no PAIR - no problems, no lag.
With PAIR, but no engines loaded (by which I took it to mean, no Ollama, no LMStudio loaded), no lag.

Here is an interesting thing, I had PAIR running, but had turned off LMStudio, no lag, but then I turned it on - to see whether it would cause the lag. It did not.

I then tested it, with LMStudio running, but freshly loaded - so it had never been used as an AI node - no lag, I added LMStudio to all of my other nodes - no lag.

I then ran an AI job, utilising multiple agents across 3 active nodes. I did not start the game during this process, I let it finish. Once it finished, I loaded the game - lag.

The logs do not show anything different while this process happens - so I don't see any value in posting them - there are no warnings, there are no errors, there are no entries that are different to the standard entries that appeared when it booted up. I am running Qwen3.6 27b on all active nodes, on a Desktop 4090, a Desktop 3090, and a Laptop 5090. Each with 24GB of VRAM.

The AI work is fine, and my gaming on the 4090 is fine.

The issue appears to be that once the node has been loaded for AI usage, at least via LMStudio - that despite the "idea" that the GPU would be released for gaming - it is not being released, and it is causing lag as perhaps the two are competing for limited resources.

This may be an issue with LMStudio, but I raise it that NVPAIR is about the idea of "when you want to use your GPU, then you can, and it will simply bypass the node" - which doesn't quite work, if LMStudio is not unloading from the GPU....it would really need NVPAIR to unload the LMStudio loaded model for you to use the GPU wouldn't it - and if it is meant to be doing that - then it doesn't appear to be doing that?

[somtri]

@Kaotic3, that narrows it down. The lag starts only after an inference job has loaded a model, and it stays after the job ends.

That matches how PAIR works today. PAIR sends new requests away from a GPU that is busy, but it does not unload a model on its own once LM Studio has it in memory, and its busy signal is GPU utilization, not VRAM. LM Studio keeps the model loaded on its own. A model loaded on demand stays for 60 minutes by default, and one loaded with lms load stays until you unload it (https://lmstudio.ai/docs/developer/core/ttl-and-auto-evict). So after the job, the game and Sunshine's encoder are probably competing with the resident Qwen model for the 4090's 24 GB.

Could you check that with one more run on the 4090?

1. Run the AI job and let it finish.
2. Run lms ps and note which models are loaded. Also note Dedicated GPU memory in Task Manager > Performance > GPU.
3. Run lms unload --all, then note Dedicated GPU memory again.
4. Start the game in the same scene as before.

If the lag is gone after step 3, the resident model is the cause. For now, you can unload before gaming or set a shorter idle TTL in LM Studio. The trade-off is that the next AI request on that node has to load the model again.

PAIR has no feature that frees a node's GPU when you start using it locally. Whether it should is a question for the maintainers.