City's quest for a third straight league title is realistically over after they fell 16 points behind Liverpool, who also have a game in hand on Guardiola's men.
However, the bookmakers still believe City are the favourites to retain the League Cup and FA Cup and face Real Madrid in a blockbuster Champions League last 16 tie.
"To finish second, there always has to be something inside of us all to move forward and improve," said Guardiola.
"You cannot play good games in the knockout games if you are not running well in the Premier League.
"It's not one day, you are bad and then good. You have to maintain the level as high as possible."
City's quest for cup glory will be aided by the return of Aymeric Laporte after five months out with a serious knee injury.
The Frenchman has been badly missed at the heart of the City defence and he could make his return on Tuesday at Sheffield United.
"Every day Aymeric is getting better," added Guardiola. "I have an idea for him but I have to think today and tonight and then we’ll see."
In contrast to City's disappointing league campaign, Sheffield United have massively overachieved on their return to the top flight.
The Blades are seventh in the table and in the running to finish in the top four.
Guardiola is a big fan of United manager Chris Wilder with the use of overlapping centre-halves from his three-man defence catching the eye.
"I'm incredibly impressed," said Guardiola. "If you can improve watching other games and other teams and other managers, then this is the one.
"I have never seen some of the movements that they are doing. There's a reason why they are in the position they are.
"I admire how they play and how they are doing, with a high percentage of the same players from the Championship, League One and now in the Premier League. I like it."