The ministers said in a joint statement that they had "agreed to embark on consultations at the technical and political levels" and that such talks would be in "a spirit of brotherly relations and mutual understanding." Sudan also will participate in these talks.
Mohamed Amr, Egypt's foreign minister, arrived in Ethiopia on Sunday for talks with Tedros Adhanom, his Ethiopian counterpart, amid a sharp exchange of words between their governments over the dam.
The two ministers met four times on Monday in what they described as "intensive" and "spirited" discussions.
Amr told reporters in Cairo upon arrival from Ethiopia that Egypt will invite the water ministers of Sudan and Ethiopia to meet "as soon as possible."
He said his Ethiopia visit helped "remove what clouded relations between the two countries" even as Egypt's goal is to protect the water resources.
Ethiopia's growing economy frequently suffers from power cuts and needs more electrical capacity, but Nile-dependent Egypt fears the project will diminish its share of Nile River waters.
Ethiopia last month started to divert Nile waters to make way for its massive $4.2 billion hydro-electric dam dubbed the Grand Ethiopian Renaissance Dam, sparking concerns in Egypt.
In a televised meeting on June 3, Egyptian politicians suggested attacks against Ethiopia to sabotage the dam. A week later Egyptian President Mohammed Morsi warned that "all options are open" to challenge Ethiopia's dam project.
Ethiopia's parliament on Thursday unanimously ratified a new accord that replaces colonial-era deals that awarded Egypt veto powers over Nile projects.
Ethiopia's leaders say work on the dam will not stop even as consultations proceed. They say the findings of an experts' panel, which includes four international experts, show the dam will not significantly affect water flow to both Egypt and Sudan.
Associated Press writer Maggie Michael in Cairo contributed to this report.