Because human space flight looks powerful but is actually super weak. The chemical reactions we use are dangerously powerful of course but compared to the physics of gravity, required reaction mass etc, they're barely enough to get us off the planet. Those Saturn moon rockets were the most crazy impressive things we'd ever made but NASA couldn't get us to and from the moon until they did things like remove the seats from the lunar module. That's how close it was.

And the amount of money being spent similarly is a lot yet is also barely enough. The US briefly hit 4% of GDP on the moon program, a massive effort for something other than a world war. It only happened because the space program was essentially a military operation that was an extension of WW2. Space mostly still is today.

In short, we can just barely do space stuff. Every gram and every dollar is tight.

You are not mistaken in the idea. It's doable and preferable for biosecurity etc.