I started flying with my kids when my daughter was 2 (oldest) and my son was a few months. I held off on my daughter precisely because I only had a tandem airplane up until then and was waiting for something different. In this case it was a C170. The wait was worth it.
Based on my kids (which are not your kids), I would not fly with kids under 4 in a tandem. They were always yanking the headset off or needing some other kind of attention. In fact they ideal is a 4 seater, because then someone can sit with them and you can concentrate on flying.
If/when you do fly with young kids, a few things to keep in mind:
1. They are 3. They can't remember what they did yesterday. They aren't going to remember this forever. Keep in mind who the flight is for (hint: it's not them.)
2. Most will react they same way they react in a car. After a few minutes the novelty wears off, they get bored, hungry, fall asleep, etc....
3. A good headset is must. Most "kids" headsets are just adult headsets with a smaller mic and/or headband. Most aren't that great. I spent a good part of my childhood riding around with my dad in loud airplanes with crappy or no headset. I wouldn't trade a second of it because it was time with my Dad, but at 43 I have hearing problems. If they kids can't keep the headset on, we don't fly.
4. Know where that "pilot isolate" switch is on the intercom. In a pinch you can also pull their mic jack. This is essential when they start singing Frozen songs at the top of their lungs while you are trying to hear ATC. (And they will!)
You said were having trouble getting the car seat in. Would you jury-rig a car seat or use some cushions to make a homebrew booster in your minivan for the ride to the airport? So why would you do this in your airplane?
I'm not saying don't fly with your kids. Etched in my brain is my then 3 year old daughter repeating my radio calls. "Six tree cha-wie, conto!" (Six three Charlie, landing Oconto) I'm saying understand why you are doing it, and take the time to do it right. It's not a race.
DEM
Ps. If it is race, you already lost. My parents had the world's only 5 place Aeronca Champ. Dad flew, mom in the backseat, sister 1 on her lap, sister 2 in the baggage sling, and pregnant with me.