Why were kids spending the night at the rectory? A little common sense is in order.
As far as scout campouts...maybe go on some of them until you are comfortable.
Nowadays, the Church and the Scouts have great training programs and rules which prevent individual adults from being with individual kids, and they enforce the heck out of those rules. If a local troop doesn't enforce those, the parent who received the training has to notice. If they don't notice, then kids are at risk. It takes multiple adults not caring enough for a kid to be hurt.
This message has been edited 3 time(s).