Suhail Shaheen, a spokesman for Taliban's political office in Qatar, said "the intentional delays in the release of our prisoners violates the peace agreement, therefore we call back our technical team back from Kabul."
Early on Tuesday the militant group broke off talks with the Afghan government on a prisoner exchange by calling it a series of "fruitless meetings" that were seen as crucial steps in peace talks being brokered by the United States after it agreed on a troop withdrawal pact with the militants.
Their decision represented the latest setback for the U.S.-led effort to end America's longest war and decades of strife in Afghanistan that also has been hampered by violence and a leadership feud between the country's top politicians.
The February pact between the United States and the Taliban, under which U.S.-led international forces will withdraw in phases in exchange for Taliban security guarantees, is the best chance yet of ending the 18-year U.S. military involvement in Afghanistan.
But peace hinges on talks between the U.S.-backed Afghan government and the militants. A prisoner exchange is meant to build confidence on both sides for those talks.
A spokesman for the government said it would continue working on the prisoner release plan.
"We ask the Taliban to not sabotage the process by making excuses now," said Javid Faisal, a spokesman for the National Security Council in Kabul.
The Taliban suspension of the talks could lead to an escalation of violence, which in turn could threaten the plan to withdraw U.S. troops, a major objective of U.S. President Donald Trump.
A three-member Taliban team had arrived in Kabul last month to begin the prisoner exchange process. It was not clear whether the delegation members came from Qatar or traveled to Kabul from provinces in Afghanistan.
Last week, Afghan officials said they would release 100 Taliban prisoners who were sick or over the age of 50.
In exchange, the Taliban were expected to free 20 members of the Afghan security forces. Ultimately both sides would aim to release the 6,000 prisoners they are holding.
But the Taliban, according to the Afghan officials wanted 15 commanders accused of carrying out brutal attacks to be released this week, a demand rejected by the Afghan government.
Despite the setback over the prisoner releases, U.S. Secretary of State Mike Pompeo on Tuesday said progress had been made since he visited Kabul on March 23 to press Afghan President Ashraf Ghani and former chief executive Abdullah Abdullah to end a feud over the results of a disputed September election.
"We've made some progress, but we see them posturing in the media, we see statements that come out," he told a State Department news conference.
While the feud has persisted, Pompeo's visit and his announcement of a $1 billion cut in U.S. aid to Afghanistan appeared to have an impact, with Ghani on March 26 announcing a delegation for peace talks with the Taliban that won Abdullah's endorsement.
Pompeo reiterated a call for those negotiations to start."I'm confident in the days ahead we'll have things that look like steps backward, but I'm also hopeful that all the parties are sincere and wanting what's good for the Afghan people," he said.