actually looking back, each hard had an extremely hard part for certain characters. Usopp's greatest durability check was against Mr.4 and Ms. Merry Christmas ( strange cause she's a desert mole), Luffy's was in EL ( only his willpower got him off the ground after beating Lucci) and THriller Bark (Nightmare Luffy which took tremendous willpower, going gear 3 AFTER NL, AND then using gear 2 +gear 3).
I think Skypiea was one of the least because:
A. Everyone was laid out. thats not making them try hard. if Mihawk came and beat the shit out of Koby, Koby didn't exactly try hard did he, especially since he was stomped.
B. Luffy took ENel down EASILY. About 4 attacks did him in ( gomu gomu no tako hanabi, gomu gomu no bazooka, gomu gomu no rifle, and gomu gomu no ougen rifle).
Being taken down easily isn't pushing you to your limits, so know. if its which villain was strongest or most destructive towards the mugiwara, Enel would be number 1 since he EFFORTLESSLY owned them.
IMO EL was the overall most draining and pushed them to the limits. Chopper had to go ALL THE WAY and use third rumble ball to go monster form. Nami almost died, and had to use her strategies to their max to beat Kalifa. Luffy couldn't even move after beating Lucci, and only with the combined motivation of his crew and willpower could he move. Zoro almost died if not for Asura and his demon ki. Usopp.... had to try to sneak past Jyabura, and had his heart stop

.