On board is a team of South Korean experts investigating ways to help North Korea modernise its rail network.
The two Koreas have made significant progress in improving their relationship over the past year.
The hope is that it will eventually become easier to travel and trade across the border.
When the leaders of North and South Korea had their historic meeting in April, North Korea's Kim Jong-un asked for help with updating his country's railways, which he said were in an "embarrassing" state.
Some of the infrastructure dates back to the early 20th Century, so if it is ever to link up with the South it needs a full overhaul.
The engineers boarded the train in Dorasan, just north of Seoul, on Friday morning for the short journey to the Demilitarised Zone which has divided the Korean peninsula since the Korean War in the 1950s.
A banner displayed across the train as it headed over the heavily armed border described it as an Iron Horse, running towards an era of peace and prosperity.
At Panmun Station across the border, a North Korean engine took over to take them further north.
The 28 experts will live on the train for the next 18 days while inspecting 1,200km (745 miles) of track and railway infrastructure.
The BBC's Laura Bicker in Seoul says the engineers can only look at the North's network but they can't touch it just yet.
They also had to get special one-off clearance from the UN to get around sanctions, which prohibit taking fuel and certain equipment into North Korea.
Cho Myoung-gyon, who heads the Unification Ministry which oversees relations with North Korea, said the project was "intended to overcome division and open a new future of the Korean Peninsula".
"Through the one connected railway, the South and the North will prosper together and the ground for peace on the Korean Peninsula will be consolidated. The trains running on the track will also carry peace and prosperity with them to North East Asia and the world," South Korea's Yonhap news agency quoted him as saying.
But while inter-Korean relation-building has continued apace over the past year, talks with the US have stalled.
Pyeongyang and Washington have repeatedly accused each other of failing to honour the vague agreements about denuclearisation they made at their landmark summit in Singapore in June.
South Korea's President Moon Jae-in is determined that this railway project will go ahead, says our correspondent.
The US has given its support to the rail survey, but Mr Moon will have to have a number of difficult conversations if he's to persuade President Donald Trump that easing sanctions at this stage would be worthwhile so the project can go ahead, our correspondent says.