The 30-year-old defender was introduced for Lewis Dunk in the 20th minute after he suffered an injury.
Balogun was introduced with the game still level but the Seagulls went on to cause a huge upset by defeating the Premier League big boys 3-2.
Coach Chris Hughton and his team bounced back from an opening day defeat to Watford to hold on for famous win.
Glenn Murray, Duffy, and Pascal Gross were the scorers for Brighton while Romelu Lukaku and Paul Pogba both foud the net for Jose Mourinho's side.
Balogun was not afforded any minutes in Brighton's opening day loss to Watford but put in a disciplined display for the Seagulls as they claimed their first points of the 2018/19 Premier League season.
The ex-Mainz defender came up against Lukaku several in aerial battles as Brighton tried to hold on to a win.
Balogun was signed as a reinforcement from Germany before the start of the 2018 FIFA World Cupin Russia.
Balogun will hope to continue getting minuted on the pitch when Brighton travel to face Liverpool in their next Premier League fixture scheduled for Saturday, August 25.