Generally polar and non-polar are rather subjective terms, as like most things this is on a spectrum and thus depends on where the line (in electronegativity) is drawn.

The dipole of a molecule just gives the average distribution of electrons across a molecule. It does not give you information about the individual bonds.

That being said for simple (diatomic) molecules the rule does apply (polar bonds = polar molecule and vice versa).

Ozone is bent, hence it has a dipole. Just like e.g. water, which also has a dipole. Carbon dioxide on the other hand does not have a permanent dipole, but can have an induced dipole (through radiation absorption or collision with other molecules, where it also gets bent)