Rashford, who scored the winning spot-kick when United knocked PSG out in 2019, drilled home a low shot three minutes from the end which won a thrilling Group H contest at a near-empty Parc des Princes.
United had taken the lead in the 23rd minute when Bruno Fernandes slotted home from the spot at the second attempt, but Anthony Martial inexplicably headed Neymar's corner into his own net 10 minutes after the break.
The rest of the match was an end-to-end affair with both sides opting to go for the win rather than hold on for the draw, and Rashford took the points after twice being denied by Keylor Navas earlier in the match.
"Of course it's a nice feeling. More importantly we win the game and we're in a strong position now in the group," Rashford told BT Sport.
"It's the first game, the concentration and intensity we showed today we have to take forward in the next games in the Premier League."
The win sets United up nicely for next week's home clash with RB Leipzig, who are top of the group after beating Istanbul Basaksehir 2-0 in Germany.
Ronald Koeman got his first Champions League campaign as Barcelona coach off to a winning start with a confident win over Ferencvaros.
The Hungarians, the first team from their country to make the group stage in 11 years, started brightly and even had the ball in the net via Tokmac Nguen only for the powerful 12th-minute strike to be ruled out for offside.
Messi won and converted the penalty to give the Catalans a 27th-minute lead and the match was as good as done when Ansu Fati clipped home Frenkie de Jong's sumptuous pass just before the break.
Philippe Coutinho lashed home the third before Ihor Kharatin pulled one back from the penalty spot after Gerard Pique was sent off for pulling back Nguen as he charged towards goal.
Pique's red card means he will miss next week's game away at Juventus, a greater test of Barca's capabilities.
Pedri and Ousmane Dembele finished the scoring for the 10-man hosts as they took control of Group G, finishing the opening round top but level on points with Juve.
Immobile downs Dortmund
Andrea Pirlo's side looked comfortable in their 2-0 win at Dynamo Kiev in what was the Juve coach's first ever Champions League match in the dugout, with Alvaro Morata scoring twice to take the points for the Italians.
It was a good night for Italy as Ciro Immobile got revenge on his old club Borussia Dortmund by scoring in Lazio's 3-1 win over the Bundesliga club in Rome, their first Champions League match in 13 years.
Immobile managed just three goals in 24 Bundesliga games for Dortmund in 2014/15, but last season's European Golden Shoe winner outshone Dortmund's star forward Erling Braut Haaland by netting the opening goal for the hosts after just six minutes.
Lazio went two ahead midway through the first half thanks to Marwin Hitz deflected Luis Alberto's whipped corner into his own goal.
Haaland pulled one back with a thumping finish but Immobile laid on an easy finish for Akpa Akpro to hand Lazio a big win.
Simone Inzaghi's are level on three points with Club Brugge in Group F following their dramatic 2-1 win at Zenit Saint Petersburg earned thanks to Charles De Ketelaere 93rd minute winner.
Chelsea are still waiting for a £220 million ($284 million) spending spree to pay off they began their campaign with an uninspiring 0-0 Group E draw with Sevilla at Stamford Bridge.
The Blues have now won just two of their opening six games of the season and the only consolation of a cagey encounter for Lampard was a rare clean sheet.
All four sides in the group are on a single point after Rennes drew 1-1 with Krasnodar in their first ever Champions League match in front of 5,000 passionate fans in Brittany.
Sehrou Guirassy gave those supporters hopes of a dream start when he opened the scoring from the penalty spot 10 minutes after the break.
However Cristian Ramirez levelled almost instantly with a powerful drive to secure a good draw for the Russian side.