I think you are correct. I also get your point. But, the 5th is pretty clear that you have a right not to incriminate yourself, not that you have a right never to speak. Maybe a 1st Amendment argument can be made: The right to speak also includes the right not to speak. SCOTUS has used logic like that in the past.