Paulo Fonseca's side, now level with Inter on eight points in seventh place, secured a crucial victory thanks to Gabbia's late winner. The goal came after Federico Dimarco had equalized Christian Pulisic's impressive 10th-minute strike just before the half-hour mark.
"They played with a lot of courage today and I think we deserved to win, I can't remember another team that have created so many problems for Inter like we did today," said Fonseca to DAZN.
Milan's win snapped Inter's run of consecutive derby victories at six and stopped their rivals from taking top spot in Serie A on goal difference ahead of surprise early league leaders Torino.
They will have renewed belief after a difficult start to the campaign, with the pressure relieved on new coach Fonseca whose team were worthy winners.
"They were the better team tonight, we didn't play as a team and that's not something you can usually say about us," said Inzaghi.
Milan end derby hoodoo
Advertisement
Milan were booed off the field after being thumped by Liverpool on Tuesday but fans were fully back on their side after a strong display as the nominal away side at a pulsating San Siro.
They deserved to win after creating the best chances to win the match in the second half and were nearly denied the three points by a fine performance from Inter goalkeeper Yann Sommer.
The Swiss pulled off a sensational save to stop Rafael Leao heading Milan ahead seconds after the restart before diving low to keep out Reijnders 20 minutes later.
Leao again cursed his luck in the 75th minute when at the end of a rapid counter-attack Sommer kept out his close range finish, and seconds later Tammy Abraham flashed a great chance just wide.
Advertisement
But Gabbia sparked a wild party among the Milan fans who could finally celebrate ending their derby hoodoo in the most dramatic fashion.
"It was pretty difficult when things weren't going well, but this win will really help push us forward," said Milan youth product Gabbia to DAZN.
"There are always tough moments in life, all we can do is not have regrets at the final whistle and that's what happened today."
Roma protests
Goals from Artem Dovbyk, Paulo Dybala and Tommaso Baldanzi gave Roma their first win of the season against Udinese who had been surprise leaders before this weekend but drop down to third, a point behind Torino.
But the match was a sideshow due to the vociferous protests at De Rossi's dismissal on Wednesday which will continue unless new coach Ivan Juric can ensure continue with the sort of display which moved Roma up to 10th on six points.
"The boys were very sad about Daniele being fired, they were very honest about it and I appreciated that," said Juric after the match.
Supporters hurled abuse at the team from the stands while the entire Curva Sud section of the Olimpico, where the hardcore supporters stand, was empty for the first half an hour of the match.
By the time those fans started pouring into the stadium Roma were ahead thanks to Dovbyk's goal, but that didn't stop loud chants of De Rossi's name ringing around the stadium.
The anger at De Rossi being fired has been such that CEO Lina Souloukou, who resigned on Saturday morning, had been placed under police protection after online threats made against her and family.
Listen to the latest songs, only on JioSaavn.com
Earlier, Albert Gudmundsson netted a brace of penalties on his Fiorentina debut to fire his new team to their first win of the campaign, 2-1 against Lazio.
(This story has not been edited by staff and is auto-generated from a syndicated feed.)