The 55-year-old Klinsmann dropped a bombshell on Tuesday with a Facebook announcement he was quitting after just 76 days, despite agreeing last November to coach Hertha until the end of the season.
"It's going to be difficult now" for Klinsmann to coach again in the Bundesliga, Matthaeus, 58, wrote of his fellow 1990 World Cup winner in daily newspaper Bild in mid-week.
"Clubs who are looking for a coach will think hard.
"If you quit after 10 weeks, after this big investment, after this great euphoria that you yourself created, then the name 'Klinsmann' has probably dropped a peg or two at other clubs."
Klinsmann's sudden exit came in the wake of last Saturday's woeful 3-1 defeat at home to Mainz, which left Hertha six points from the relegation places in the German top-flight.
"Jurgen Klinsmann's tasteless departure from Hertha Berlin will mean that he will almost certainly never again be able to show his face as a coach in Germany," Bild insisted in Friday's editorial.
"Can you put the fate of a club in Klinsmann's hands? Hertha answered the question with 'no'."
In a video post on Tuesday, Klinsmann, who coached hosts Germany to third at the 2006 World Cup and took the USA to the last 16 of the Brazil 2014 finals, admitted his exit was "questionable".
He said he should have "taken more time, spoken more with" Hertha's senior figures, hinting at a power struggle and said he wanted to return to his previous role on the club's supervisory board.
'Loss of credibility'
However, in a press conference on Thursday, Hertha's muti-millionaire investor Lars Windhorst swiftly blocked the idea.
"The manner of his departure is unacceptable," fumed the entrepeneur Windhorst, who holds a 49.9 percent stake in Hertha.
"We cannot continue a purposeful cooperation with him in this way."
Windhorst also slammed the former forward's behaviour as "something you can do in your youth, but not as an adult", prompting magazine Kicker to comment "Windhorst and Klinsmann: like a father with a pubescent son".
Klinsmann's assistant Alexander Nouri is now in charge as caretaker coach, starting with Saturday's league match at bottom side Paderborn.
The swift departure of Klinsmann also followed massive investment in the January transfer window.
A reported figure of 77 million euros ($83 million) was spent on fresh talent, including AC Milan striker Krzysztof Piatek and Brazil forward Matheus Cunha from league rivals RB Leipzig.
Although Windhorst left open the possibility of working with Klinsmann in the future, he added "by leaving, he has lost a lot of credibility here at Hertha".
Both of Klinsmann's two coaching jobs in the Bundesliga have been brief as he was sacked by Bayern Munich in 2009 after just 10 months in charge.
An issue for Klinsmann at Hertha was he wanted to be an English-style manager, responsible for coaching the team and also for player transfers.
In Germany, the buying and selling of players is normally handled by a sports director or a team manager, not the coach.
However, Klinsmann wanted autonomy - which put him in dispute with Hertha's general manager Michael Preetz.
Refering to Liverpool's German manager Jurgen Klopp, who is on the verge of bringing the Premier League title to Merseyside for the first time in 30 years, Bild had a direct message for Klinsmann.
"With regard to the coach/manager position, one has to say: You are not like Klopp, you are only Klinsmann."