So I talked to support and they helped me figure out what was going on.
There are two ways of modulating the filter cutoff with keyboard tracking.
1. There is a hardwired connection, which you can access from the filter menu that pops up when you turn the cutoff knob. The tab is called "Filter Misc." If you set KEY > LPF FREQ there to 127 it scales more or less perfectly regardless of the filter, etc.
2. You can use the mod matrix to create your own connection. Like the hardwired connection it still sends CV to the analog components, but the voltages are determined digitally, and therefore slightly different from the pure analog path. If you set NOTE NUM there to modulate CUTOFF at a rate of 127 it scales wonkily, and differently depending on the filter you choose.
Even though we have two ways of doing what appears to be the same thing here and I expected them to yield the same result, it's not the same modulation. For example, classic full keyboard tracking is centered at middle C, i.e. a middle C yields the same filter cutoff regardless of whether KEY > LPF FREQ is on or not. The mod matrix modulation version of that is centered at 0v, which means it's centered around C0, not middle C. Where it gets confusing is that the max voltage that the mod matrix sends isn't exactly the same as the max voltage between the hardwired KEY > LPF FREQ connection, so it doesn't quite turn out how you'd expect by maxing out the mod matrix rate regardless of the key center. But since there is a way of getting the result I was looking for, I have no problem with this. Although it is confusing and I'm sure I won't be the only one to try the mod matrix version and think the filters don't track correctly, Sequential has no intention of changing this behavior at this point in time. I guess the mod matrix is really meant more for gettin' crazy