Outrage has grown after the prize was handed this month to rap duo Kollegah and Farid Bang, who in their song "0815" say their bodies are "more defined than an Auschwitz prisoner".
Barenboim, 75, said in a statement that the rappers' lyrics are "clearly anti-Semitic, misogynist, homophobic and contemptuous of human dignity".
They constituted an "abuse" of free expression which society must "never tolerate", wrote the Jewish general music director of the Berlin State Opera and the Staatskapelle Berlin.
Barenboim said he, the Staatskapelle Berlin and the West-Eastern Divan Orchestra had jointly decided to return the award because "decency and humanity" must outweigh "commercial interests".
The two rappers this month won the Echo Music Award's Hip-Hop/Urban prize after selling more than 200,000 copies of their album "Young, Brutal and Handsome 3".
The award ceremony was held as Israel marked Yom HaShoah, its Holocaust Remembrance Day, dedicated to the six million Jews killed by the Nazis during World War II.
Protests were led by Campino of German rock band Die Toten Hosen, who spoke out during the ceremony, and numerous other artists and industry figures have since returned their Echo awards.
Both rappers have said they reject all anti-Semitism, and Kollegah has offered free "lifetime" tickets to the duo's Jewish fans.
Even before the rappers won the Echo prize, which is based on sales figures, the International Auschwitz Committee said their presence at the awards ceremony was a "slap in the face for Holocaust survivors".
Germany has been shocked by several recent anti-Semitic incidents, including an assault by a Syrian refugee on two men wearing Jewish kippa skullcaps in Berlin.
German Chancellor Angela Merkel denounced the emergence of "another form of anti-Semitism", aside from that by far-right groups, from refugees of Arab origin in Germany, in an interview with Israeli television broadcast on Sunday.
Merkel reaffirmed that the security of Jews and of the state of Israel was a central concern for Germany because of its "eternal responsibility" for the Holocaust.