There is an easier way.

I took a 1/8 inch thick by 1 inch wide piece of aluminum and made bends so that the piece would bolt to two bolts already on the door and goes around the door handle. It takes a bit of work to get the piece bent exactly but once it's bent correctly and the door panel is put back on the door, the door handle piece should slide in so that it is touching the inside part of the aluminum.

My door closes like a normal car now without the door panel moving at all.