Regardless of vaccination status, you can spread Covid. Being vaccinated or not only helps to keep them from being more sick should they get infected. If they are vaccinated or not will not help to keep those children from getting Covid.
You let the kid stay and you limit visitors for everyone, if you want to require a negative test to visit I am good with that too. But kicking them out over this is not appropriate.