I believe there is a problem with the model and the displacement.
I have to filter out what exactly causes this. I hope it is not a matter of too much calculation.
In any case I applied displacement to a different surface in my model with smooth and it worked.
Only problem it looks strange with some shadows. I tried it again with a couple of walls and it worked just fine.