The search team spotted the dog, named Kentucky, late Saturday when its eyes reflected off their headlamps, police Lt. John Matarese told The Associated Press.
Nearby, the team found the body of 51-year-old Bjoern Waalberg, who had apparently died hours earlier.
There was no indication of foul play, police said.
Waalberg, from the borough of Kinnelon, and his dog set out for a hike mid-afternoon Saturday at the 576-acre Apshawa Preserve in northern New Jersey. The man's wife alerted authorities just after 7 p.m. when he didn't return by nightfall as expected.
His body was found just before 10 p.m., his dog close by.
After police found the man's car in the parking lot, authorities said rescue teams were sent out to search the heavily wooded, hilly preserve, calling out the names of Waalberg and his dog.
West Milford Search and Rescue director Dina D'Argenio told The Record newspaper that Waalberg's wife also helped aid the search through the tracking of calls he made on his cellphone.
Kinnelon Councilman James Freda told the newspaper that Waalberg, a married father of three, was an avid outdoorsman.