The protests were in response to the death of 20 police cadets in a car bomb attack in the capital, Bogotá.
President Iván Duque and his predecessor, Juan Manuel Santos, were among those in marching in Bogotá.
Mr Duque blamed the attack on the National Liberation Army (ELN), and has ruled out any restart of peace negotiations with the left-wing rebels.
The ELN has not said it was behind last Thursday's attack at the General Santander School, but authorities have named the car driver, who was also killed, as ELN explosives expert, Jose Aldemar Rojas.
Mr Duque told the rally in Bogotá: "Together, the government and society will work for Colombia and we will never give in when facing terrorism.
"Colombia is strong and will never be scared, it will never be intimidated by criminals."
During the event protesters, many wearing white clothes and waving white flags, chanted "no to violence" and "no more attacks on the police or the army", while handing flowers to security personnel.
In Medellín, images of those killed in the attacks were projected on to large screens in the city's Plaza de las Luces.
The ELN is Colombia's last major guerrilla group and began peace talks with the country's government in 2017.
Mr Duque suspended talks after taking office in August last year, demanding the ELN free all hostages and stop its attacks.
About 10 ELN commanders are believed to be in Cuba for a restart of peace talks.
Following the bombing - the deadliest attack in Colombia for 15 years - Mr Duque asked Cuba to extradite them.
Cuba has refused, citing security guarantees for the rebels that were agreed as part of the negotiations.
The latest attack has raised fears of renewed violence across the country.
Car bombs were frequent during the decades of civil war with the Revolutionary Armed Forces of Colombia. It signed a peace agreement in 2016.
More about: Colombia