Unlock Nvidia GPU to transcode more than 2 concurrent streams
If you are running Plex, you’ve noticed that transcoding takes up alot of CPU power. Specially if you’re transcoding 4K videos, which you shouldnt do anyway. Plex removes tone mapping when transcoding 4K HDR, so your movies or TV shows will look bland.
Now you want to offset some of that transcoding to your Nvidia GPU. You want to use Plex’s hardware transcoding. Transcoding audio will still use the CPU and not the GPU.
Nvidia uses NVENC to transcode, but its capped to two concurrent transcodes. This limitation is only applied to consumer GPU’es. You have to patch the GPU to remove the restricitons.
Keylase have made an amazing patch and released it on his github.
Table Of Contents
My Plex server
CPU: I7-3770K
GPU: GTX 1050TI 4GB
RAM: 16GB DDR3
The GPU can transcode up to 14 1080p/10Mbit to 720p/4Mbit. Using only my CPU it will only trancode 1, maybe 2 streams. So you see the huge difference. GTX 1050TI aslo support h265 transcodig.
The patch
- Navigate to keylase’s github
- Download appropriate patch for your GPU and do a clean install.
- Save the x64 library patch and x86 library patch by right clicking and saving.
- Download the Win_1337_Apply_Patch tool.
- Apply x64 library patch to corresponding file in
%WINDIR%\system32\with the Win_1337_Apply_Patch tool. - Apply x86 library patch to corresponding file in
%WINDIR%\SysWOW64\in the same way as previous one.
Session Limit bump
From jantenhove’s github: When using the patch from https://github.com/keylase/nvidia-patch/ on Windows the NvEnc session limit is raised for CUDA encoding sessions, but not for other surface types. There is however a work-around. When you first ‘enable’ the sessions using CUDA surface based encoders, they will remain enabled for other types of surfaces until a reboot.
Open up CMD and navigate to the download dir, and execute the .exe. You will get this output.
When you reboot your server, you will have to run the session bump again. Otherwise the patch wont work. Create a scheduled task, which runs the .exe on boot.
Thats all you have to do to unlock your Nvidia GPU. Now your GPU will transcode past the limitation.
