The music of Yemenite Jews was particularly influential in the development of Israeli music because it was seen by early new immigrants as a link to their biblical roots. The music of the ancient Hebrews, wrote the musicologist A.Z. Idelsohn, "is preserved in memory and practice in various Jewish centers. There was a Yemenite Jewish community in Palestine before 1900, and the European pioneers who came in the 1920s were enamored of the Yemenite style. In the 1930s and 1940s, Yemenite singer Bracha Zefira researched and recorded many Yemenite songs, and also sang original compositions in the Yemenite style. An example is the song "Shtu HaAdarim" (Drink, the Flock), with words by Alexander Penn and music by Nahum Nardi.

Aharon Amram became the first to record Yemenite music using instruments from outside its tradition. Among the instruments he accompanied his traditional Yemenite singing with were guitar, violin, qanoun, trumpet, trombone and percussion instruments. Yemenite music reached a world audience in the 1980s as a result of the efforts of Israeli singer Ofra Haza, whose album Yemenite Songs became an international hit with world music fans. Ofra Haza grew up in a traditional Yemenite family who lived in Tel Aviv's  Hatikva neighborhood. She became famous for singing pop music, but later in her career became something of a cultural ambassador for her community, both in Israel and internationally. Several of her most famous tracks, such as "Im Nin'alu", were reworkings of traditional Yemenite songs, many composed by Rabbi Shalom Shabazi, a medieval poet and mystic whose spiritual and artistic achievements are universally revered in the Yemenite community. Shabazi's poetry dealt with both religious and secular themes, giving Yemenite music a wider lyrical range than many other forms of traditional Jewish music, which tend to be liturgical in nature.

​