This is what happens to high values (sun/sky) in HDRI when Gamma is changed from 1. They get much much more intense than shadows.
https://www.desmos.com/calculator/prabidnxeo
Also, since everything works fine with area lights, as you say, that means there's nothing wrong with materials.
I don't see anything that much wrong with materials in V4.

Cheers
Milan