A couple options off the top of my head

• Add orientation prediction to the bounding box
• Add keypoints for the 4 actual corners as a prediction
• Postprocess boxes with classical techniques, looking for the outermost corners that fit certain properties
• Do everything classically, and deal with the difficulties you have mentioned in your comment.

The first two require annotations of attributes for each box, and will be predicted directly by the model. Though note that you don't have to do this for every label, you can just not train parts of the model when certain attributes are unlabeled.

Both will require some care in modeling, e.g. orientation can have a loss condition at 360 degrees that you'll want to handle, and regressing keypoints can be done well and not well, reference how corners are modeled. And then of course you'll need to postprocess the model's outputs to align/visualize on an image.