While I personally think they should be in, don't lose to two unranked teams in the middle of the year, and don't miss your weak conference championship.
Of course, this also sews dissension in the ACC ranks, and creates animosity toward ND, which isn't lost.