The visit is intended to cement the recent reconciliation between the long-time enemies.
A peace deal ending the 1998-2000 border conflict had never been fully implemented and there has been tension between the neighbours ever since.
The countries have also agreed to re-establish trade and diplomatic ties.
Their border has been closed since the war ended, cutting off members of the same families.
Why is this such a big deal?
The two countries have been in a state of "no war, no peace" since 2000, when a peace deal ended a war in which tens of thousands of people were killed.
A border commission set up under the peace agreement ruled that the town of Badme, the flashpoint for the conflict, was part of Eritrea but Ethiopia refused to accept this and so normal relations were never resumed.
The rivalry affected the whole region with each country normally taking opposite positions whatever the question. They also took rival sides in Somalia's long conflict - Eritrea was accused of backing Islamist groups, while Ethiopia, a US ally, supported the internationally recognised government.
Until now, Eritrea has always said that war could resume at any time and so had national conscription, which could last indefinitely. This is one of the main reasons why so many Eritreans try to leave the country and seek asylum in Europe.
President Afwerki's visit is a remarkable turnaround for the 72-year-old independence leader who has been isolated diplomatically and seen as secretive and paranoid.
Why is this happening now?
Since Mr Abiy took over as prime minister of Ethiopia three months ago, he has introduced a raft of changes.
As well as his overtures to Eritrea, Mr Abiy has lifted a state of emergency, freed political prisoners and announced economic reforms.
He inherited a country that has seen some of the fastest economic growth in the world in recent years, but has also been riven by years of protests by people who feel marginalised.
Image copyrightMELA GEBRE MEDHINImage captionJubilant Eritreans took to the streets of the capital, Asmara, last week as Ethiopia's prime minister visited
The previous government was accused by many of human rights violations - including torture and extrajudicial killing of political dissidents.
Mr Abiy is the country's first Oromo leader - the ethnic group at the centre of nearly three years of anti-government protests, which have left hundreds of people dead.
More about: Eritrea