It was only natural for a lot of people who weren't "in the know" about the situation, or who had blind loyalty to Wililngham, to listen to the hype that the mainstream media had been spewing out there. The media can be quite powerful when it comes to influencing people.
I remember the sheer number of people who were screaming "Notre Dame is racist because they fired their Black head coach!" during those days, and it took several more years until they finally realized how much egg was on their faces when Lionel completely bombed at Washington.
God only knows, he almost destroyed our program after two years of the absolute worst recruiting in ND history. The class of 2004 was bad enough as it is (only 6 players had any meaningful playing time), and the class of 2005 would have been even worse if Weis hadn't snagged about 5 more serviceable players (including Paul Duncan and Mike Turkovich, who basically doubled the number of offensive linemen that Willingham had brought in for the 04 and 05 classes).
2007 was a rotten year, but anyone at the helm would have had similar results, since there was literally no developed junior and senior leadership, much less actual numbers...