Quevyn Winters had 19 points and a team-high nine rebounds while P.J. Torres, Derrick Colter and Jerry Jones each tallied 12 points.
Duquesne (1-2) led by as many as 10, 46-36, in the second half but James Madison cut the deficit to 81-79 with 1:22 remaining behind an Andre Nation 3-pointer.
Devon Moore nailed two free throws with 39 seconds left to tie the game and James Madison (1-1) missed two layups and a tip in over the final 12 seconds to send the game into overtime.
Colter hit a layup to give Duquesne its first overtime lead, 88-86, and Jones was fouled after recording a steal on James Madison's next possession.
Jones sank the ensuing free throws to seal the win.