He is not a great coach. His teams are generally soft and they don't play good defense. I used to get frustrated when an ND player would fail to stop the ball or give a poor effort on defense, yet wouldn't get pulled. Good coaches do that. It simply isn't and won't be a priority with Brey. He's focused on offense and some of his worst defensive players have been some of his best offensive players, like Luke Harangody or post-injury Chris Thomas.
Defense and physicality aren't dependent upon getting great athletes. John MacLeod had far superior defensive teams than most of Brey's teams. To put it in perspective, 25 years ago, MacLeod had a walk-on from across the street at St. Joe's HS, Pete Miller, who often started and often matched up with the best opposing Big East players, like Ray Allen and Kerry Kittles. Walk-on Pete Miller, with all those physical limitations, was a far superior defender than anyone on the current squad. That's not a function of player talent or athleticism, it's about coaching and accountability on both ends of the floor.