Investigators say Mr Pezão took bribes when he was deputy governor of Rio from 2007 to 2014.
He has denied taking any illicit funds.
Mr Pezão is the latest politician to be detained as part Operation Car Wash, a corruption investigation which has resulted in the arrest of a long list of high-ranking politicians and businessmen.
The allegations relate to the time when Mr Pezão served as deputy for the then-governor of Rio, Sergio Cabral. Cabral himself is currently in prison after he was found guilty of money laundering and corruption last year.
Prosecutors say Mr Pezão is suspected of taking 25m-reais-worth ($6.5m; £5m) in kickbacks between 2007 and 2015.
Brazilian TV station Gobo News showed live footage of federal police officers arriving at the governor's residence, Laranjeiras Palace.
Mr Pezão, from the Brazilian Democratic Movement, has just over a month left in office before being replaced by Wilson Witzel, who was elected last month.
Mr Witzel, a former federal judge, said that he was confident justice would take its course and that the transition of power between Mr Pezão and him would not be affected.
One of Mr Witzel's campaign promises was to create an anti-corruption hotline in Rio where citizens can air their suspicions.
Rio state has been at the centre of corruption allegations with 10 state lawmakers from a variety of parties arrested three weeks ago and charged with graft.
Operation Car Wash began in March 2014 as an investigation into allegations that executives at the state oil company Petrobras took kickbacks from construction firms in return for awarding them contracts at inflated prices.
It has since expanded into a massive probe which has resulted in top politicians and some of Brazil's richest businessmen being jailed, including former President Luiz Inácio Lula da Silva.
The inquiry has been led by Judge Sergio Moro, who has been picked as Brazil's next justice minister by far-right President-elect Jair Bolsonaro.
Mr Bolsonaro. who campaigned on a promise to crack down on corruption, won last month's election by a substantial margin. He will take office on 1 January.