How difficult it is also comes down to how "stock" the RV is. If you've got non-standard stuff, it'll make inspectors nervous. There are Canadian requirements that don't exist in the US, too. Gascolator, for example. But most US aircraft have them, and even if you had to install one it wouldn't be *that* hard. Another gotcha is castellated nuts vs. nylocks, some builders don't seem to know when you use one or the other, and you can get set back if you have to change a bunch of them.
I think many people who have had trouble have contributed to it themselves at least in part... They expect that because the plane flew fine for 100+ hours in the US that they should be able to just bring it in and start flying, and they get mad, annoyed, etc. with anyone who tells them otherwise. The attitude given is directly linked to how poor their service is from each agency they need to deal with.
If you go into the process with your eyes open, knowing what the regs are and what's required, allow for each of those things to happen in turn without trying to rush things or get mad at someone who appears to be dragging their feet, you'll have an airplane here and flying in no time.
For my import, I flew the aircraft back registered under its Canadian markings. The import inspection found two things to fix: (1) there was no up elevator stop, and (2) the inspector found a crack in one of my rudder pedal weldments. I had both fixed in a couple of weeks. I took a few more weeks to swap out the older radio for a new iCom, and then a re-inspection gave me my paperwork. It was two months total from first landing in Canada to first takeoff post-import.