I find the ascent much harder. Like being on a stair climber all day with someone's knee on your chest.
Most people will tell you the descent is harder especially if you're doing something serious like Everest. The climb down is more dangerous because you've spent most of your mental and physical energy reaching the summit and you have little left in the tank for the descent. So you're prone to making bad mental decisions (not clipping into fixed ropes properly) or physically just not able to descend because your legs feel like wet spaghetti noodles.
I've never done anything that taxed me that much that the descent was a problem. Having said that the worst fall I've ever had was on a descent from the Lady MacDonald ridge walk (picture below). I didn't fall here, but just over the other side out of the picture. Made a stupid mistake and put my foot in the wrong spot, went head over hills down the side. Fortunately, I landed about 10 feet down and hit backpack first. The pack was full of 2 litre water bottles so that broke most of the fall and left me with a few scrapes and bruises. I was more embarrassed than hurt.