It sounds like the tape may be too weak. I don't know if you have access to 3M Command strips. There are several kinds but I am thinking of the ones used to hang pictures. Thelr large picture hanging strips hold up to 16 pounds and will come off the wall and your panels without any damage or residue.

You could also potentially use like an industrial strength Velcro.