Bodo/Glimt, playing their 11th European game of the season, stunned the Italian giants who struggled in temperatures of two degrees on an artificial pitch just inside the Arctic Circle.
Erik Botheim scored a hat-trick for the Norwegians with captain Patrick Berg, Ola Solbakken and Amahl Pellegrino piling on the misery for Mourinho.
Bodo/Glimt won the Norwegian title for the first time in their 104-year history last season.
Their fans have become great curiosities in the sport for their habit of waving giant, yellow toothbrushes in the stands.
Carlos Perez scored a consolation goal for a much-changed Roma side who had won their first two games in Group C, scoring eight goals and conceding just one.
However, Mourinho would have had one eye on this weekend's Serie A clash with Napoli who top the table in Italy with eight wins in eight games.
"If I had played the first team, someone might get injured, then we'd concede four or five goals against Napoli on Sunday and that decision would've been considered wrong too," said Mourinho.
"The team that Bodo put out today had more quality than our second string."
Roma skipper Pellegrini told Sky Sport Italia there were lessons to be learned.
"I can only apologise and get back on track stronger than before. We said what we needed to in the locker room and it remains there," he said.
"Sometimes you need a slap to learn and we got a slap in the face today, one we fully deserved. We've got to make sure that we learn from this embarrassment."
Mourinho's former employers Tottenham also lost, going down 1-0 at Vitesse Arnhem in Group G.
Tottenham manager Nuno Espirito Santo changed his starting line-up completely with the likes of Harry Kane, Son Heung-min, Cristiano Romero and Hugo Lloris not travelling to the Netherlands.
The second-string side were eventually undone by Maximilian Wittek's 78th-minute goal.
"It was a tough match against a good team," said Nuno whose team are third in the group at the halfway stage, three points behind leaders Rennes.
"The Europa Conference League is like this, there are always tough moments."