The total death toll from the fire, whose cause remains unclear, could be higher, the IOM regional director for the Middle East and North Africa, Carmela Godeau, said in a Twitter post.
It was not immediately clear how many people were being held at the facility.
IOM was providing emergency healthcare for the injured, more than 90 of whom were in serious condition, and distributing food to those affected, Godeau added.
The organization estimates that thousands of migrants became stranded in Yemen after movement restrictions were imposed due to the coronavirus pandemic across the Gulf region and Horn of Africa.
Alongside refugees, about 100,000 migrants arrive in Yemen each year by sea from the Horn of Africa hoping to trek north into wealthy Saudi Arabia and beyond.
The six-year war in Yemen between a Saudi-led military coalition and the Iran-aligned Houthi movement has left Yemen a land of displacement, with 80% of the population reliant on aid.