The resolution, which was backed strongly by Israel, needed a two-thirds majority to pass on Thursday following an earlier vote in the assembly.
The proposal failed to cross the threshold, with 87 nations voting in favour and 57 voting against. Thirty-three countries abstained.
The earlier vote to require a two-thirds majority, which followed a procedural move requested by Kuwait, was much closer: 75-72, with 26 abstentions.
Hamas thanks UN member states
The resolution was one of US ambassador to the UN envoy Nikki Haley's final acts in the international body before she leaves her post at the end of the year.
The outgoing envoy, a staunch defender of Israel, had written to member states on Monday to urge them to vote for the US-drafted text, warning them: "The United States takes the outcome of this vote very seriously."
"Before the General Assembly can credibly advocate compromise and reconciliation between the Palestinians and Israel, it must on record, unambiguously and unconditionally, condemn Hamas terrorism," Haley told the body before the vote.
Al Jazeera's James Bays reporting from the United Nations said that it was a tense meeting as the General Assembly first had to decide how the resolution would be voted on.
"It had a vote on whether it will be a simple majority or a two-thirds majority. That result was very narrow but it went in favour of two-thirds, putting a much higher bar for Ambassador Haley's resolution.
"When it was finally voted on, she got a majority but she didn't get the two-thirds - a blow to the US ambassador," Bays said.
In an official statement, Hamas thanked UN member states "that stood by our people's resistance and the justice of their cause" and attacked Haley who it said "is known for her extremism and her positions that support the Zionist terrorism in Palestine".
Hamas spokesman Sami Abu Zahri described the vote as a "slap" to President Donald Trump's administration which has taken a firm pro-Israeli stance in addressing the Middle East peace process.
"The failure of the American venture at the United Nations represents a slap to the US administration and confirmation of the legitimacy of the resistance," Zahri wrote on Twitter.
Mahmoud Abbas, president of the Palestinian Authority in the occupied West Bank, also welcomed the resolution's defeat saying: "The Palestinian presidency will not allow for the condemnation of the national Palestinian struggle."
Israel's UN Ambassador Danny Danon said the countries that rejected the draft resolution should be ashamed. Israeli Prime Minister Benjamin Netanyahu commended those that voted in favour.
Ali Abunimah, co-founder of Electronic Intifada, an independent online news publication, said the failure of the proposal was significant.
"This resolution was really just an attempt to weaponise the UN against the Palestinian people, against their legitimate rights," he told Al Jazeera.
"The resolution itself was just transparently Israeli talking points - it didn't mention the military occupation, the siege of Gaza, Israel's daily attacks against Palestinians in Gaza and the West Bank. I think the world saw through it and they rightly rejected it."
The Gaza Strip, home to two million Palestinians, has been under a crippling Israeli blockade for more than a decade.
In 2006, Hamas beat Fatah in parliamentary elections in the Gaza Strip and, a year later, fighting between the rival factions broke out.
When Hamas eventually took control, Israel responded by enforcing a land, sea and air blockade on Gaza and banning its residents from working in Israel.
Egypt followed suit, effectively sealing the Strip - often described as the world's largest prison - from the outside world.
Gaza's continued isolation has devastated its economy, impoverished its population and left 60 percent without jobs, adequate electricity and health services.
More about: Hamas