I agree that you do not need polyAT for every sound, especially because P5/10 can only route it per voice to cut-off frequency. Plus, polyphonic expression if it is meant to be controlled (vs randomly applied) reduces the number of meaningful fingers per hand to 1-2.
What you could do is get a more compact controller, such as the Linnstrument 128, and use that one to control any notes you would like to have polyAT, while using your regular keys for anything else. Because you’d apply it to specific notes, the isomorphic non-piano layout will not be a problem. Despite its small physical range for (poly)aftertouch, the control thereof is pretty good imo.
In case you used it to control a keyless sound source, add in a MIDI merger to merge the MIDI input coming in from both controllers. Linnstrument is compatible with a wide range of MIDI sound sources and MIDI modes (except 2.0). With MPE you will have polyphonic sliding of up to 96 semitones as an additional expressive option - as well as Yabs and Yrel. For the P5/10 you would only have monophonic pitch sliding limited to the maximum pitch bend range of the Prophet, which can still be nice to have.
LS is not cheap but I personally find it entirely worth it, it’s a very robust design:
https://www.rogerlinndesign.com/support/support-linnstrument