He is the first politician to quit following recently revealed claims of serious sexual abuse in Parliament.
The BBC understands fresh claims about his behaviour were raised on Wednesday, but Downing Street refused to comment.
Political editor Laura Kuenssberg said that sources close to him do not believe he is "some kind of predator", but that he had not felt that he could guarantee that he would be able to account for every encounter in his long ministerial career.
Theresa May said she appreciated the "serious manner" in which Sir Michael had considered his Cabinet role.
She also praised the "particular example you wish to set servicemen and women and others".
'Below high standards'
In his resignation letter, Sir Michael said: "A number of allegations have surfaced about MPs in recent days, including some about my previous conduct.
"Many of these have been false but I accept that in the past I have fallen below the high standards that we require of the Armed Forces that I have the honour to represent."
Sir Michael told the BBC it "was right" for him to resign and said: "The culture has changed over the years, what might have been acceptable 15, 10 years ago is clearly not acceptable now.
"Parliament now has to look at itself and the prime minister has made very clear that conduct needs to be improved and we need to protect the staff of Westminster against any particular allegations of harassment."
When asked if he thought he should apologise, Mr Fallon said: "I think we've all got to look back now at the past, there are always things you regret, you would have done differently."
He added that it had been a "privilege" to have been defence secretary over the past three and a half years.
Sir Michael Fallon's resignation letter
In response Mrs May accepted his resignation and paid tribute to "a long and impressive ministerial career - serving in four Departments of State under four prime ministers".
More about: #MichaelFallon
