Hi refresh rate is critical to "presence", to trick our brains to believing the environment is real so we experience life like emotions (fear, joy etc). As soon as this becomes "less-presence" then we feel uncomfortable which leads to nausea, dizziness etc. Some people can get their VR legs with extended use but this is not all users.
The Hz refresh rate is also just one part of gaining presence in VR, resolution, motion-to-photon, latency etc all contribute to eye fatigue (which leads to nausea) if they are below a threshold. That minimum threshold is where we are today, we want to keep improving on it to get closer to natural presence as much as possible. That is years off though.