Is there a reason you need P4 specifically? You should still be able to get it by typing it into the version box (rather than just using the drop down). I suspect the recommendation is to use P5 if P4 was taken off the drop down.
If you just type in "9.7P4" you don't get the option to download it? I do but it could be because I am an employee. You may need to connect with the account SE to pull that version down or perhaps open a non technical case to get access to the code along with perhaps why it was pulled.
It is "working as designed". If you are looking for a specific version, you just type it in. The support site redesign tries to avoid long dropdown lists - it limits the list to recommended releases.
It isn't clear that the dropdowns will be incomplete. I was confused by that as well, so some verbiage similar to "the dropdown below may not have your specific version. If you need a specific version, type it in below."