I didn't find it drifting if I stayed on a single patch, but changing patch re-triggers the sequencer regardless of whether it's on the beat or not. Makes it almost impossible to use the inbuilt sequencer in any live context.
I tend to sequence everything from an external source anyway (MPC) but I can see how it would be a problem for others.