Muriwai Beach near Auckland was closed after the fatal attack, one of only about a dozen in New Zealand in the past 180 years.
Pio Mose, who was fishing at the beach, told The New Zealand Herald he saw the swimmer struggle against the "huge" shark. He told the man to swim to the rocks, but it was too late.
"All of a sudden there was blood everywhere," Mose said. "... I was shaking, scared, panicked."
Police Inspector Shawn Rutene said in a statement that the swimmer, who was in his 40s, was about 200 meters (650 feet) offshore when the shark attacked. He said police went out in inflatable surf-lifesaving boats and shot at the shark, which they estimate was 12 to 14 feet long.
"It rolled over and disappeared," Rutene said, without saying whether police are certain that they killed the creature.
Police recovered the body of the swimmer, whose name was not immediately released because his relatives had yet to be notified.
About 200 people had been enjoying the beach during the Southern Hemisphere summer at the time of the attack. Police said Muriwai and other beaches nearby have been closed until further notice.
Police did not say what species of shark was involved in the attack.
Attacks are rare. Duffy estimated that only 12 to 14 people have been killed by sharks in New Zealand since record-keeping began in the 1830s.
"There are much lower levels of shark attacks here than in Australia," he said. "It's possibly a function of how many people are in the water" in New Zealand's cooler climate.
He said that during the Southern Hemisphere summer, sharks often come in closer to shore to feed and to give birth, although that doesn't necessarily equate to a greater risk of attack.
"Ninety-nine percent of the time they ignore people," he said. "Sometimes, people get bitten."
Around the world, sharks attacked humans 80 times last year, and seven people were killed, according to the University of Florida's International Shark Attack File. The death toll was lower than it was in 2011 but higher than the average of 4.4 from 2001 to 2010.