Population growth compounds, though. Once you’ve colonized a million worlds, the next million would come in a fraction of the time of the first million, and the next 2 million in less time than that, then 4 million in the same time as before, etc. Like grains of rice on a chessboard. Totally feasible to fill a galaxy if FTL travel is achieved.
More likely - FTL is impossible so each species is stuck in their own solar system.