Hemoglobin carries oxygen to every cell in your body. Low hemoglobin means less oxygen is getting to your cells. Different cells can go do long without the optimal level of oxygen they need, and so, those parts of your body get damaged. The amount of damage done to different parts of the body determines the effect on your overall health. Since certain parts of your body have really important functions and a high demand of oxygen (your brain, your heart) this could have especially bad long term effects.