"It was rush hour, and lots of civilians were inside the restaurant," he said.
Hawa Hassan, a nurse from Baidoa's regional hospital told Anadolu Agency that they had received three seriously injured civilians.
Hassan Hussein Mohammed, Southwest state minister for security, also confirmed reports of the attack.
No group has claimed responsibility for the attack yet. However, many attacks in the past have been claimed by Somalia-based insurgent group al-Shabaab.
Baidoa, the fourth largest city in Somalia, is located 246 kilometers (152 miles) southwest of capital Mogadishu.
More about: #Somalia