Gus Poyet admitted an early red card was pivotal to his side’s 3-0 defeat at Selhurst Park.
The Albion boss admitted he had “no problems” with referee Mick Russell’s decision to send off Lewis Dunk.
But he felt his side might have got something from the game had they taken a good chance straight after half-time.
Craig Mackail-Smith hit the post when clean through before Glenn Murray, who had already headed home from a corner, and Owen Garvan converted second-half spot kicks.
Albion used Andrea Orlandi and Liam Bridcutt as emergency left-backs with Inigo Calderon, who started in place of the injured Wayne Bridge and Marcos Painter, switched from the left to a central berth when Dunk departed.
Adam El-Abd missed the game and Orlandi suffered a broken rib in the first half.
Poyet, whose side kicked off the derby unbeaten in seven games, said: “You come into the game on a good run, plenty of confidence, the team organised and after five minutes everything goes away for many reasons, one because of the sending off.
“We were struggling with defenders in the squad.
“The one thing you don’t want to lose is one of your defenders so that’s it, that’s the story of the game.”
He added: “In a game where you play 80-plus minutes with ten men, they scored from a corner and two penalties.
“In terms of the way we defended, the way we tried to come back into the game in the second half, there were key moments.
“The sending-off was the biggest, the goal just before half-time.
“And I still think the best chance of the game is Mackail-Smith’s.
“If we had scored as soon as we started the second half it’s a different story.”
Poyet felt Murray might have been offside when he went through to win the first penalty.