I have been regularly swimming for years. Usually 4 times a week. Each time 1 mile in freestyle stroke and then about 5 -10 minutes in backstroke.
To me the downside of swimming is the water in the ear. No convenient way to get water out of ears sooner right after swimming. I don't like wearing earbuds which can trap sweat in ears. How did you get water out of ears?