Tips for new h265 Settings

[Godzilla62]

Someone has also reported blockyness issue NVENC (H265) with Nvidia, Nvidia has generated bug report,but it has been 6 months

https://devtalk.nvidia.com/default/topic/1033976/video-codec-sdk/nvenc-quality-blocky-amp-jumpy-blacks/

I've played around with CQ mode high quality with CQ settings of i19 p23 b25 while encoding Thor with only a couple of areas of blotchyness Encoding Frame rate was above 150, I don't know of a way to set individual frame CQ parameters in DVDFab. I still had aq mode set, but not sure if it has any affect while using CQ mode.

Funny thing I encoded The Last Jedi and it looked horrible, in desperation I set the aq-strength to 2 and it came out good, Go figure.

I also have MV precision set to qpel

[Godzilla62]

I've been reading and learning, ordered a Nvidia GTX 1060 3Gb Model. Pascal vs Maxwell architecture of the 960. See if any difference with DVDFab. Hopefully better compression and background (less blocking).
I've also stumbled across another post where the person was using the edge level detection, this has yielded good results on most blu-rays. Here are the parameters that are sent to the nvenc

NVEncC64.exe --vbrhq 7000 --codec h265 --preset quality --cu-max 32 --bframes 0 --ref 4 --gop-len 90 --lookahead 15 --strict-gop --aq --cuda-schedule auto --colormatrix bt709 --colorprim bt709 --transfer bt709 --vpp-edgelevel strength=10,threshold=14,black=5 --vpp-deband --mv-precision q-pel --cabac

I have to give credit to user Comparison on the Videohelp.com forum, although I had to change some parameters, but not the edge detection settings. Terminology varies so greatly, strange that edge detection yielded better backgrounds and less blockyness.

I still have to do around 7000bps to get close to lossless. but 8k-10kps with h264 for same results. 5500k yeilded poor backgrounds @ 1080p , but older movies converted to blu-ray can and should be shrunk down 1600x???? resolution @ 5500bps, this removes some of the fine grain I've found. (Unless it is a quality conversion)
Frame conversion rates with the GTX 960 is 100-133fps, Hope to double that with GTX 1060.

H265 CPU conversion too slow on my machine at 5 to 10fps and it is not a slow machine passmark 12,000 so GPU hardware conversion needed, disk space not a issue.

I hope the DVDFab developers and other h265 advocates find this information helpful

[Godzilla62]

GTX 1060 Here, Been using it last couple of days, 80% faster than GTX 960, Better encoding also
Unfortunately DVDfab still having blocking issues with same bitrate encodes. You don't need anything special with Nvidia Pascal architecture. Still no b-frame support, but you can set reference frames to 8 960 was limited to 4
aq Mode on
aq-strength 9 (not even sure if it's using this) - Nvenc seems to have adjusted aq mode, works better with pascal.
Allmost lossless encodes at 8kbps streams
Staxrip Log

NVEncC (x64) 4.29 (r977) by rigaya, Dec 17 2018 12:04:34 (VC 1900/Win/avx2)
OS Version     Windows 10 x64 (17134)
CPU            Intel Xeon(R) X5670  @ 2.93GHz (12C/12T)
GPU            #0: GeForce GTX 1060 3GB (1152 cores, 1809 MHz)[PCIe2x16][418.81]
NVENC / CUDA   NVENC API 8.1, CUDA 10.1, schedule mode: auto
Input Buffers  CUDA, 36 frames
Input Info     avcuvid: VC-1, 1920x1080, 24000/1001 fps
Vpp Filters    copyDtoD
Output Info    H.265/HEVC main @ Level auto
1920x1080p 1:1 23.976fps (24000/1001fps)
Encoder Preset quality
Rate Control   VBRHQ
Bitrate        8000 kbps (Max: 22000 kbps)
Target Quality auto
Initial QP     I:20  P:23  B:25
VBV buf size   auto
Lookahead      on, 15 frames, Adaptive I, B Insert
GOP length     150 frames
B frames       0 frames
Ref frames     8 frames, LTR: off
AQ             on
CU max / min   32 / auto
Others         mv:Q-pel

NVEncC64.exe --avhw cuda --vbrhq 8000 --codec h265 --preset quality --cu-max 32 --bframes 0 --ref 8 --gop-len 150 --lookahead 15 --strict-gop --max-bitrate 22000 --aq --cuda-schedule auto --colormatrix bt709 --colorprim bt709 --transfer bt709 --mv-precision q-pel --cabac --aq-strength 9