Once game pass starts being more expensive than buying the games I want, I'll just go back to doing that
You may not have that option. The business model here is to burn cash, get consumers used to gamepass, then get games onto gamepass exclusively (likely in exchange for higher payouts from the service). Once we are at that point, which may be years away, prices will rise and there won't be another avenue to play most games.
This is the model right now for shows, and some movies, they are produced for streaming services and are only available on those services.
Most games already don't get physical releases. All that needs to happen to eliminate choice is that gamepass makes publishers a better offer than Steam - then there isn't a digital release either.