Speaking to a meeting of young members of her Christian Democratic Union party, Merkel said the idea of people from different cultural backgrounds living happily "side by side" did not work.
She said the onus was on immigrants to do more to integrate into German society.
"This [multicultural] approach has failed, utterly failed," Merkel told the meeting in Potsdam, west of Berlin, yesterday.
Her remarks will stir a debate about immigration in a country which is home to around 4 million Muslims.
Last week, Horst Seehofer, the premier of Bavaria and a member of the Christian Social Union – part of Merkel's ruling coalition – called for a halt to Turkish and Arabic immigration.