It can be done safely, given the right training and equipment. As evidence, I offer the story of Ed Long, generally accepted as the highest time pilot in the history of the world (about 65,000 hours).
Ed flew year round, mostly in a Super Cub, on a statewide power line patrol.
The controllers at Dannelly (MGM) gave him a special VFR when ducks were thumbing a ride. Admittedly he knew every post, pole, tower, pasture, and farm strip along his routes, as well as all the real airports, but he rarely ventured above 200 feet in the Cub, and considered 1000 and 3 to be a fine day.
One engine failure, no crashes. Gave me my first tailwheel dual, in a Great Lakes. In these parts, we all miss him.