He wanted nothing to do with Willingham. He first tried to hire John Gruden, which was nixed by Malloy. He then hired George O'Leary. After that fell through he had to be goaded to hire TW by others due to the desperation of the job search.
I think his defense of him in 2004 was more personal than professional, as Willingham was actually fired by the board of trustees, and not the school administration. He had to know how this looked for him in his position as well. Thankfully, he stepped down anyway several years later.
I am not sure he ever liked it at ND, however. Too much politics, and he never seemed fully comfortable with the size and scope of the football program.