I tried your test case and ran into the same problem. Seems that when the values are compared, the trailing spaces are removed (or added until the length is the same). Actually the same effect can happen in SQL Server when varchar data type is used so this may be by design. Quickly cannot think anything else than replacing trailing spaces with a special character that can be later removed (like ascii 255).
The need to optimize rises from a bad design. My articles[^]