But a minister in the Yemeni government backed by Saudi Arabia dismissed the Iran-aligned Houthis' pullout as a "show" meant to "misinform the international community."
The move, yet to be verified by the United Nations, is the first major step in implementing the pact reached last year by the government and the Houthis for a truce and troop withdrawal in Hodeidah, a lifeline for millions of Yemenis.
U.N. teams were overseeing the Houthi redeployment in Saleef, used for grain, as other teams headed to the second port of Ras Isa, used for oil, to start implementing the withdrawal from there, according to the witness.
A dozen trucks carrying Houthi fighters, armed with rocket-propelled grenade launchers and machine guns, departed from Saleef. Two ships were docked at the port and operations were running normally, said the witness who was at the facility.
"The coast guards have taken over in Saleef," he said. "They and U.N. officials have started checking equipment at the port."
The Houthis said their "unilateral step shows our commitment to implement the Hodeidah agreement and to achieving peace," after four years of fighting in the Arab nation. The group called on the U.N. to press the Saudi-led coalition to take "similar steps."
However, Yemeni Information Minister Muammar al-Iryani told Reuters the pullout was "a flagrant show."
"It's an attempt to misinform the international community ahead of a meeting of the U.N. Security Council" on Yemen, he said. "A group of (Houthi) militiamen left and they were replaced by others wearing coast guard police uniforms."
The U.N. Redeployment Coordination Committee (RCC) has said the Houthis would make an "initial unilateral redeployment" from the ports between May 11 and May 14.
It said the redeployment would enable the United Nations to take a leading role in supporting the local Red Sea Ports Authority in managing the ports and enhance U.N. checks on cargoes. It would also allow humanitarian corridors to be reopened.
There has been no comment so far from the Saudi-led Sunni Muslim military coalition that has massed forces outside Houthi-held Hodeidah, which handles the bulk of Yemen's imports and aid supplies.
Western allies, which supply arms and intelligence to the coalition, have pushed for an end to the war.
The British ambassador to Yemen reacted sharply to the Yemeni government's skepticism about Houthi withdrawal. "The Yemeni cynics who criticize everything the other side does even if it is positive and who say the UN are naive seem to be saying the only solution is perpetual war in Yemen," Michael Aron said in a Twitter post. He said a U.N. presence in the ports would prevent arms smuggling.
Hodeidah became the focus of the war last year when the coalition twice tried to seize its port to cut off the main supply line of the Houthis, whom they accuse of smuggling Iranian weapons, including missiles that have targeted Saudi cities. The group and Tehran deny the accusations.
U.N. ASSESSMENT NEXT WEEK
It was not clear if U.N. special envoy Martin Griffiths had secured agreement between the two sides over the main sticking point regarding which local authorities would control the ports and city under U.N. supervision after both sides withdraw.
The coalition had disputed an earlier unilateral withdrawal by the Houthis from Hodeidah port in December, saying they had handed it over to coast guard members loyal to the group.
A U.N. source told Reuters on Saturday that the RCC would announce its assessment of the Houthi redeployment next week.
Under the first phase, the Houthis would pull back five km (three miles) from the ports over the next four days. Coalition forces, currently massed four km from Hodeidah port on the edges of the city, would retreat one km from "Kilo 8" and Saleh districts.
In the second phase, both sides would pull troops 18 km outside the city and heavy weapons 30 km away.
The United Nations secured the Hodeidah deal at peace talks in Sweden, the first in two years, to avert a full-scale assault on the port that risked triggering mass famine.
The pact is also a trust-building step to pave the way for wider political negotiations to end the conflict, widely seen in the region as a proxy between Saudi Arabia and Iran.
The alliance led by Saudi Arabia intervened in Yemen in 2015 after the Houthis ousted the internationally recognized government of Abd-Rabbu Mansour Hadi.
The ceasefire in Hodeidah has largely held but the fighting escalated elsewhere in Yemen.
On Saturday, the Houthis' Masirah TV channel said "enemy air bombing" killed six children and a woman, and injured 11 children and five women, in the southern province of al-Dhale.
A spokesman for the Saudi-led coalition didn't immediately respond to a Reuters request for comment.