Now here's something interesting I haven't tried before.
If I face West (towards my PC monitor, etc), I experience some gyro drift so that I have to recenter about once every two minutes, depending what I'm looking at.
However, today I tried facing North and a little further away from my PC and I experienced hardly any gyro drift, even after at least 15 minutes.
So, it is possible that the drift does indeed have something to do with magnetic field disruption to the Pimax Magnetometer.
However, I can't think of why this would manifest itself as continuous drift rather than just an initial offset?
So it seem that a positive step to reducing drift would be a user calibration routine.
It would have to be user based because everyone's environment would be different.
We would need calibration for Magnetometer, Gyroscope and Accelerometer. Similar to how it's done for the PSMove controller in the PSMoveService calibration features.
Food for thought I think.