.NET Core is alarmingly finicky in how it's built as it defaults to a larger cross-platform "I don't need the .NET Framework" build instead of simply a native one as .NET Framework did. Try and run
dotnet publish -r win-x64 --self-contained false -o bin/
Then check your bin folder. Should be far smaller with far less files with the limitation that it only runs on Windows. For a small Linux-only build, you can use
dotnet publish -r linux-x64 --self-contained false -o bin/
Whilst this is smaller, you will now need the .NET 5 Framework (Not to be mistaken with the .NET Framework Framework) installed as you did with regular .NET Framework (One of the reasons your one is so much larger is that it bundles the entirety of the Framework in it as well as it assumes it's not natively installed)
-= Reelix =-