Zardari's visit, the first by a Pakistani head of state in seven years, is the most visible sign that the two countries have put behind them the enmity that followed the 2008 terror attacks in the Indian city of Mumbai.
Zardari will meet with Prime Minister Manmohan Singh before heading to the Ajmer Sharif shrine of a famous Sufi Muslim saint in India's western state of Rajasthan.
The two leaders will hold talks in private, with no aides present. Indian officials said all outstanding issues between the two sides would be discussed.
Singh is widely expected to raise India's concerns about security and insist that Pakistan show its seriousness about reining in terrorist groups that were behind the Mumbai attacks. India blamed the Lashkar-e-Taiba militant group and demanded that Islamabad crack down on them.
Earlier this week, United States slapped a $10 million bounty on Hafiz Mohammad Saeed, the group's founder, who operates openly in Pakistan.
Zardari will be accompanied by around 25 members of his family, including his son, Bilawal Bhutto Zardari, who has been anointed as his political heir.
India and Pakistan have fought three wars, two of them over control of the disputed region of Kashmir, since independence from Britain in 1947.
Every day, thousands of believers visit the shrine of Sufi saint Moinudin Chishti in Ajmer, 350 kilometers (220 miles) southwest of Delhi.