I've experienced this before. Here are a few tips:
First, these scans aren't meant to be stretched 50 cm in height. You will inevitably get stepping. I'm guessing you were doing that in your example to really highlight the stepping. Displacement (especially when coming from a bitmap) is pixel based. It will read over each pixel and push the geometry up or down. So the bigger you stretch your texture or the higher you increase the displacement height, you will start seeing the individual pixels separating from their neighbors (stepping). In addition to that, you have the sample size set to 8k when the texture is 4k. The shader is looking at the same pixel twice in essentially the same spot, resulting in even more stepping.
8192x8192
4096x4096
A few ways to fix this:
1. These types of displacement maps are not meant to be focused on. Don't expect to be able to focus on a single leaf on the ground and have it be photo real.
2. Stay within a reasonable height. A few sites even provide the accurate height to use.
3. Use the same sample size as your image size.
4. If it's feeling muddy still, try tiling the texture.
5. That bright sunlight doesn't really help. These things look pretty good when they can cast shadows on themselves. Full illumination highlights their flaws.
6. If for some reason you're getting weird ripping or you still see some stepping, try setting the filter type to Gaussian.
Filter Type None
Filter Type Gaussian
With all that in mind, you should be able to get this:
Hope this helps.
