The residents told Reuters the children - two brothers and their relative - were in the area when the missile struck the school in Kadha district in the west of Taiz governorate.
Two military sources said 15 soldiers were killed. They said the school had previously been used by Houthi forces and was taken over after pro-government fighters seized Kadha last week.
A Houthi official did not immediately respond to a request for comment.
Fighting has recently escalated in the disputed Taiz in southwestern Yemen between fighters loyal to the internationally recognised government and the Houthi movement which ousted it from power in the capital Sanaa in 2014.
The clashes in Taiz and Marib in the north come as the United States and the United Nations intensify efforts for a ceasefire between a Saudi-led coalition and the Houthis to revive U.N.-sponsored peace talks stalled since late 2018.
The coalition intervened in Yemen in March 2015 against the Houthis but the war, which has killed tens of thousands of people, has been in a military stalemate for years. Taiz has remained effectively under siege.
The U.S. envoy to Yemen said on Friday that a "sound plan" for a nationwide truce had been before Houthi leadership for "a number of days" but that it appeared the group was prioritising its military offensive in Marib, the government's last stronghold.
Military sources told Reuters fighting in Marib had eased on Sunday.
The Houthis say they are fighting a corrupt system and foreign aggression. The Yemeni government and the Saudi-led coalition say the Houthis are backed by Riyadh's rival Iran.