If kids' Xbox One and you couple's devices are paired in same wifi network, Yes, they play games

while you couple can listen to Spotify music. But I think your question is using Spotify as a standalone service on your device on different networks and you use your Xbox credential to sign in. This way I never tried. I think we both can try it and let's know the result to each other. :).