All four had backed Senator Leila de Lima, the chief critic of Duterte's anti-narcotics crackdown, who since Friday has been remanded in police custody on drug-related charges.
Political observers said the sudden realignment indicated moves by Duterte to strengthen his control over the powerful 24-seat Senate. Dozens of lawmakers in the lower house have switched loyalties to his party since he took office last year.
The four senators who lost their posts had attended a rally on Saturday to mark 31 years since the "People Power" uprising that removed dictator Ferdinand Marcos. The demonstration turned into a march to denounce the war on drugs.
More than 7,700 people have died since Duterte unleashed his bloody campaign eight months ago, a third in police raids and sting operations. The authorities deny activists' claims that many of the other deaths were extrajudicial killings.
"If this is the price to pay for showing up at the streets ... for talking about democracy, if this is the price to pay, I'd gladly pay," said Paolo Benigno Aquino, who was voted out as head of the Senate education committee.
De Lima, a former justice minister who has called Duterte a "sociopathic serial killer", is accused of taking bribes from convicts who ran narcotics businesses in jails.
She dismisses the accusation as part of a vendetta by the president.
With De Lima detained, the Senate has a minority bloc of five members, which will now be led by Antonio Trillanes, another critic of the president.
He alleges Duterte has tens of millions of dollars of hidden wealth and operated a death squad when he was a city mayor. Duterte denies all of the allegations.
More about: