Cheaper and faster processes in construction (just like in mfg) will likely lead to productivity increases and more of a reallocation of resources. It'll take less people to build a building, but more buildings will be erected. This is a process that has been going on since the cotton gin or before. The word "robot" is a trigger word, but it's just another machine, at least at this juncture.
What really sent jobs overseas and such was what I'd describe as neo-slavery. The wages and conditions in China and elsewhere were outright appalling but getting better. Imagine being the guy hand feeding a press 12 hrs a day, if you fuck up and lose a hand, you lose your job and your family starves. You have no prospects of retirement as the air quality will kill you off long before then. Expendable people.