Opposition to immigration is driven by dislike for the kind of people who are immigrating.
Spanish-speakers, brown people, residents of shitholes, etc. The real threat they pose is to the sense of American identity that so many people need to maintain. We will not be a bilingual nation! We need a wall or we will be overrun by billions of foreigners!
It’s really not that complicated.