Depends. Do you want him to stay on as a songwriter/lyricist? Because you could begin with that as a compliment. Then lead into saying you all would like to pursue the possibility that lead vocals might sound better with a different singer (like maybe a female lead? I say this because a lot of guys have never even considered whether or not a female lead would sound better & can't really argue against it without hearing it).

If you'd rather drop the guy, even as a songwriter, well, then just be up front about it. If the person can actually sing (as in, hits the right notes consistently) but their tone color just doesn't fit, well, there's no amount of arguing that can make that work.