Create a DLL that is injected into that process. Determine which API the application is using to determine the memory size, hook that API for this application and fake the return value to that which makes it work correctly. This is essentially what any type of application compatibility setting would have done if none are avaialble.
8bc7c0ec02c0e404c0cc0680f7018827ebee