The protest follows several demonstrations last month in the West African state over the outcome of recent parliamentary elections, which the president won, as well as over coronavirus restrictions.
On Friday, tens of thousands of people amassed in a central square in Bamako, AFP journalists said, to the blare of vuvezela horns.
The protest organisers, an opposition alliance called Rally of the Patriotic Forces of Mali (RFP), put the number of protesters at one million people.
A police official told AFP some 20,000 people attended the demonstration.
Some protesters waved banners bearing slogans such as "IBK, get out," using the acronym by which the president is popularly known.
Others marched with banners demanding the liberation of Soumaila Cisse, a former prime minister and opposition leader who was kidnapped in volatile central Mali on March 25 while campaigning for the parliamentary election.
Mali's government has struggled to quell jihadist violence that first broke out in north in 2012, and which has killed thousands of soldiers and civilians to date.
The conflict has since spread to the centre of the country, as well as neighbouring Burkina Faso and Niger.
"We are here to demand the departure of President IBK," said Issa Kaou N'Djim, an RFP official.
Addressing the large crowd in Bamako, he called on people to stay in the streets until Keita agrees to step down.
The president, who enjoys strong international support, has not yet responded to the protests.
- Opposition alliance
The RFP alliance that organised the rally is a newly formed grouping of opposition parties and civil-society groups.
Among its leaders are Cheick Oumar Sissoko, a former culture minister, and Mahmoud Dicko, a prominent imam and politician.
A religious hardliner, Dicko was considered an ally of President Keita before he entered politics several months ago.
Dressed in white and wearing a turban, the imam told protesters on Friday that the president's government was guilty of "mismanagement, embezzlement, lies".
"If he does not listen to us, he will see worse than today," Dicko said.
The RFP on Friday also urged the "implementation of all necessary actions" to release kidnapped opposition leader Somaila Cisse.
According to security officials, Cisse is suspected to have been captured by jihadists.
Keita -- who was first elected in 2013 and then won a second five-year term in 2018 -- has been under pressure to find solutions to Mali's cycle of violence.
The president admitted to contacts with jihadists in February, after having long resisted the option which many view as way necessary in order to stop the bloodshed.
Mali also held a parliamentary election in March, which had long been delayed, mainly over security concerns.
Holding the election was seen as a step towards solving the country's crisis.
There are hopes, for example, that new MPs will endorse changes to the constitution that will promote decentralisation -- another policy which experts which argue could reduce conflict.
The parliamentary poll provoked several protests last month, however, after the results for several seats were disputed.