I tried something interesting after grinding down my pop riveter failed to give the clearance, I went to Lowes and got a piece of hardware, its hex and used to connect two threaded rod together. Anyway, I then lopped one edge off of it at an angle . Then put this over the pop rivet with the angle at the base of the pop rivet, which holds the pop rivet firm in the hold, but brings the stem out at a slight angle, and used my pop riveter there, Worked like a charm, and pulled the stems off cleanly.
*EDIT after the fact*
I just realized we may not be talking about the same places, i used my track attaching the leading edge wing assembly to the main spar on an RV-8, another very tight pop rivet location.