More than 56% of voters rejected the idea amid a high turnout.
The Calgary vote leaves the Swedish capital Stockholm and an Italian bid involving Milan and Cortina D'Ampezzo as the only candidates left.
Interest in hosting the summer and winter Olympics has fallen as the cost of staging the events has increased.
The IOC will name the 2026 host city next June.
Which other cities have pulled out?
The Swiss city of Sion, Sapporo in Japan and Graz in Austria all withdrew from bidding for the 2026 Games earlier this year. Erzurum in Turkey has been eliminated from the bidding process by the International Olympic Committee (IOC).
Four European cities - Oslo in Norway, Lviv in Ukraine, Krakow in Poland and Stockholm in Sweden - also withdrew bids to host the 2022 Winter Olympics. Public opinion in Norway was reportedly divided after the cost of the 2014 Games in Russia's Sochi was put at $51bn (£31bn).
That left only Almaty in Kazakhstan and the eventual winner, the Chinese capital Beijing.
Several cities have also pulled out of the running to host the 2024 summer Games - in 2015 residents in Germany's Hamburg voted against hosting them and the US Olympic Committee ended Boston's bid after fierce local opposition.
In 2016, Italy withdrew its bid for Rome to host the 2024 summer Games after Rome's anti-establishment mayor said it would be "irresponsible" and the city council voted to oppose the candidacy.
And in 2017 Budapest in Hungary withdrew after 260,000 people signed a petition saying the money would be better spent elsewhere.
Rome had previously applied to host the 2020 Olympics but pulled out of that race because then prime minister Mario Monti feared Italy could not afford it.
What has the reaction in Calgary been?
The vote in Calgary was non-binding, but the city council is expected to respect the result when it meets on Monday.
Calgary councillor Sean Chu said the result showed people in Calgary had "had enough of the establishment telling us what to do".
"They tell you to spend millions, billions, it's good for you," he said.
Calgary has hosted the Winter Olympics before, in 1988.
The Calgary 2026 Bid Corporation said the bid had been a chance to "put us back on the map".
"It all made sense, and it still makes sense," it said.
The Canadian Olympic Committee also said it was disappointed with the result but would respect it.
Holding the Games "within our nation's culture of peace and inclusion, would have offered countless benefits to all", it said.
More about: Calgary