The no-confidence motion poses the biggest challenge yet to Ennahda, which first took power following the Arab Spring uprising, but was forced to step down in 2013 after a series of protests.
Pressure on the government has mounted in recent weeks after opponents called for the resignation of Prime Minister Elyes Fakhfakh over an alleged conflict of interest.
On Sunday, Ennahda, the main party in the ruling coalition, said it supported calls to change the government.
Ennahda said that it wants "to hold talks to form a strong new government because this government lost credibility after suspicion of conflict of interest involving the prime minister," Imed Khmiri a senior official in Ennahda told Reuters.
Fakhfakh has rejected accusations of corruption after an independent member of parliament published documents last month indicating that companies the prime minister owns shares in had won deals worth 44 million dinars ($15 million) from the state.
Mohammed Ammar from Attayar party, said that a group of parliamentary blocs agreed to start the process of withdrawing confidence from the speaker, citing numerous violations, poor management and unilateral decisions that serve partisan interests.
These blocs include the Tahya Tounes, Attayar, Chaab and Reform parties, which are in the coalition with Ennahda.
The Free Constitutional party led by Abir Moussi, a supporter of former President Zine El Abidine Ben Ali overthrown by the 2011 revolution, has campaigned to oust Ghannouchi for weeks.
Moussi's party accused Ghannouchi of serving the Muslim Brotherhood’s agenda and foreign allies, including Turkey and Qatar.
Ghannouchi has rejected these accusations, arguing that Tunisians want a government focused on economic and social policy, not political in-fighting.
Procedures for withdrawing confidence require the signature of 73 members of the house for it to be put to a public vote. To succeed, 109 lawmakers will need to vote for no-confidence.
The five parties have about 90 members.