Mr Tembenu also ordered the release of two men charged with having sex "against the order of nature". One human rights activist describes the move as a step in the right direction.
However, many Malawians are not happy with the decision and religious leaders have asked the government not to relent to pressure from Western donors by allowing same-sex relationships.
Since 2012, the US government has put more than $350m (£230m) into supporting gay rights groups in sub-Saharan Africa, according to the New York Times.
The arrest of the two men earlier this month was condemned by the United States and leading international human rights organisations.
Malawi is one of the poorest countries of the world and relies heavily on international aid.
Mr Tembenu said the country was a signatory to international laws which allowed homosexuality, the local Nyasa Times newspaper reports.
But he wants the international community to appreciate the "sensitiveness of the matter". As in several African countries, homosexuality is widely seen as taboo in Malawi. Both the Catholic Church and Muslim leaders have in the past described it as un-Godly.
The country needs to reconsider its position to align itself with international laws, human rights activist Billy Mayaya told the BBC.
"It is on the right track to abolish sodomy laws but the public needs to be sensitised to understand the secular nature of our country," he said.
"Many people here thought Malawi is a Christian country which is not true, we are a secular state," Mr Mayaya added.
The government announced a similar suspension on anti-homosexuality laws in 2012 following the arrest of two people who were charged with public indecency.
More about:
