Another 1,700 of the inmates are not US citizens and will possibly face deportation.
The changes come as part of an effort to reverse long jail terms for non-violent offenders, who make up about half of the federal prison population.
Many of these prisoners were released from federal custody on Friday. The rest will be released by Tuesday.
They are the first wave of what will be thousands set free because of new rules from the US Sentencing Commission.
On average, their jail terms are being reduced by about two years.
It is not yet clear exactly what will happen to the non-US citizen prisoners. A BBC inquiry to the Department of Justice was not answered.
This release is the biggest of its kind in US history, and up to 46,000 of the country`s 100,000 drug offenders could qualify.
President Barack Obama`s administration has been working to tackle what he sees as a lack of fairness in the system.
The planned releases continue a drive that started in 2013 when US Attorney General Eric Holder dropped mandatory minimum sentences for non-violent drug offenders.
Many leading Democrats and Republicans now believe that mandatory prison sentences that led to mass incarceration in the US have not been effective.
The US has almost a quarter of the world`s prison population - one in every 99 Americans is behind bars.
More about:
