I remained in denial about Brady's status, until he won his 7th title in Tampa. My reasoning was similar to yours: He beat teams/players I liked and always seemed to get the most absurdly fortuitous bounces of the ball. And I obviously favored Montana due to his N.D. connection and the teams I loved in the 80's.
It pained me to accept that he was the G.O.A.T.