There are literally thousands of airplanes flying that have punched holes in them.
This may be true, but there is one
big difference in the context of this question regarding RV construction.
Those thousands of airplanes are for the most part not flush riveted and they were not dimple countersunk with dimple dies that have a pilot larger in diam. than the punched hole.
The dimple die sets we typically use in RV construction have pilots sized for a close fit in a #40 or #30 hole.
Prepunched parts from Vans have holes that are punched undersize from #30 or 40 (they are actually sized as small as posible and still be able to get a cleco in them). The pilot on the dimple die sets we use is bigger than the hole dimension and will not fit in the hole without being forced in. This is stretching the hole diameter before the dimple is even beginning to form.
It is well known in aircraft construction that the stretching that takes placed when dimple countersinking raises the risk for cracks at the holes. Particularly in large dimples such as for the #8 flush screws used on RV's. Some builders have had this problem on the larger dimples if they have dimpled without proper deburring, etc.
If you choose to dimple countersink without final drilling where specified, you are raising the risk level of radial cracks from dimples.
It is true their are probably quite a few RV's now flying that were built this way, and the builders will proudly tell you that it has worked out fine. As Dan already mentioned, the real test may be to ask them again 10 - 15 years from now.
If you are a builder that is not interested in doing some long term fatigue testing and then accepting the result (whatever that will be), I recommend you spend the extra few hours and final drill prepunch parts when recommended.