Thielemans made the announcement Wednesday when he cancelled a concert in Antwerp for a second time in three months for health reasons.
His manager, Veerle Van de Poel, said Thielemans "considers his energy too limited for a full concert. He doesn't want to disappoint his fans, hence his decision to stop playing."
The haunting sounds produced by his harmonica can be heard on movie classics like Hollywood's "Midnight Cowboy" and France's "Jean de Florette."
His most famous hit was the upbeat "Bluesette," which he composed in 1962, but kept adapting over the years.
Prime Minister Elio Di Rupo thanked Thielemans for "many magical moments."