The point to which you and Pinker and all secular humanists will always be oblivious is that the non-material factors are even more important than the material ones you or he may cite. A society in which suicide rates are spiking isn't "progressing" or "improving," regardless if lifespans are increasing overall. A society in which an epidemic of loneliness is now a major healthcare concern isn't "improving," regardless if we've eradicated polio or scarlet fever.
And I'd be remiss if I didn't point an underlying premise to all this that deserves questioning. That premise is that the reason we have fewer wars or more rights for women or whatever you wish to cite in that realm is due to moral improvement, which drove these changes. Kind of like how implicit bias folks get it backwards when they argue that we must change how people think and then they will act differently, when, in fact, it's other way around. Did segregation laws fall because there was a broad consensus that blacks were equal to whites? No, of course not. Laws changed and over time that shifted the way most people thought about race. Did an abomination like WWI a century ago happen because people were less morally advanced? No, of course not. It's much more complicated than that.