"The three survivors told us they were 120 when they left Garabulli, in Libya, on Thursday night. After 10 to 11 hours at sea ... (the boat) started sinking and people started drowning," IOM spokesman Flavio Di Giacomo said on Saturday.
He said the people came mainly from west Africa, adding: "Ten women including a pregnant girl were aboard and two children, one of whom was only two months old."
An Italian military plane on sea patrol on Friday had first sighted the dinghy sinking in rough waters and had thrown two safety rafts into the water before leaving due to a lack of fuel, Rear Admiral Fabio Agostini told TV channel RaiNews24.
A helicopter dispatched from a naval ship had then rescued the three people, who were suffering from severe hypothermia and were taken to hospital on the island of Lampedusa.
"During this operation at least three bodies were seen in the water who appeared to be dead," Agostini said.
The Italian navy said it had alerted Libyan authorities who coordinated rescue operations, ordering a merchant ship to go to the site of the sinking, though rescue efforts had ceased after the search for the dinghy had proved fruitless.
According to the IOM, 2,297 migrants died or went missing in the Mediterranean last year, out of a total of 116,959 people who reached Europe by sea.
Arrivals in the first 16 days of 2019 totaled 4,449, almost all by sea, compared with 2,964 in the same period of 2018.
"As long as European ports will remain open ... sea-traffickers will continue to do business and kill people," Italian Interior Minister Matteo Salvini said in a Facebook post late on Friday.
Since Italy's populist government came to power in June, Salvini, leader of the anti-migrant League, has closed Italian ports to humanitarian vessels.
Italian news agency AGI reported on Saturday 13 migrants had arrived on Saturday in Lampedusa following 68 arrivals the previous day, the year's first two groups of arrivals.