The biggest issue I have with this is that it tacitly assumes that we have the ability to choose what we believe. Barring that there are some additional issues.

I don't think P1 or P3 have been sufficiently demonstrated. For one thing I think we're double counting utility because implicitly a belief that accepts death as a terminus to life is seen as having infinite negative /marginal/ utility relative to a belief that doesn't see death as a terminus to life. Therefore we can't say that a belief which doesn't see death as a terminus to life as having infinite /positive/ marginal utility without switching frames to view the opposite belief as our default/0 point.

It's important to understand it in marginal terms because this shows how in absolute terms we actually still have cumulative positive utility even if believing that death is the ultimate endpoint somehow precludes the possibility of infinitely more positive utility in the future.

More importantly though, it's not clear how a belief that death is the terminus to life or a belief in naturalism results in infinite negative marginal utility relative to the contrary beliefs. It seems like you might be conflating someone's belief about how much utility is possible with the actual utility attained. If I believe that death is the ultimate endpoint then that may lead me to conclude that any possible utility I can accumulate stops at the point of my death (not obvious that this follows though) and therefore that I may only attain finite cumulative utility. If death ends up not being the endpoint then I may end up attaining infinite positive utility over an infinite timespan. Why would my belief that entails I can only achieve finite utility prevent me from attaining infinite utility unless we're saying that believing is a prerequisite for eternal life in which case this is sounding more and more like Pascal's wager.

But further still, why would my belief about death or naturalism even cause me to conclude that my ability to accumulate personal utility ends after death? Are you saying that the examples you give at the end such as trying to leave the world a better place for your children are invalid for Atheist Naturalists? A belief in the possibility of eternal life doesn't seem to be required for someone to have or even for them to justify desires they may have about the future state of the world after their death. I would point out though that if we're appealing to the values that people demonstrate having in every day life outside of any philosophy or theology that people seem to care much more about the immediate future after their death then about the long term future and their concern seems to dwindle exponentially the further out into the future you go. Therefore it would be reasonable to conclude that although most people implicitly have values that extend after their death that nevertheless they also seem to implicitly believe that their possible utility is bounded.