They say the 27-year-old Bangladeshi immigrant carried out the bombing in support of the Islamic State group.
He wounded himself and three others in Monday morning's attack.
Mr Ullah is accused of blowing up a crude device strapped to his body in an underpass at Manhattan's Port Authority Bus Terminal during the rush hour.
The New York Police Department (NYPD) tweeted that he was facing state charges including criminal possession of a weapon, supporting an act of terrorism and making a "terroristic threat".
The federal charges, announced later on Tuesday, include providing material support to a foreign terrorist organisation, using a weapon of mass destruction and bombing a public place.
More about: #New-York
