Outgoing German chancellor Angela Merkel made an implicit call on politicians to overcome their differences on Sunday, as talks between parties to choose her successor got under way following last week's close election.

The centre-left Social Democrats (SPD) and their candidate Olaf Scholz narrowly won last Sunday's vote on 25.7 percent. Merkel's conservative CDU-CSU alliance plunged to an all-time low of 24.1 percent, as she prepares to leave the stage after 16 years in power.

While the result leaves the SPD in pole position to form a government, conservative leader Armin Laschet has also vowed to begin coalition talks in a last-ditch effort to keep the ailing CDU-CSU in power.

Speaking in front of party leaders at celebrations in Halle to mark German reunification in 1990, Merkel said the country once again had the opportunity to shape its next chapter.

"We can argue over exactly how in the future, but we know that the answer is in our hands, that we have to listen and speak with each other, that we have differences, but above all things in common," Merkel said, in a clear reference to negotiations at hand.

The composition of the next German government essentially hinges on which of the two main parties can persuade the Greens and the liberal FDP to sign up for a partnership.

With the SPD and the conservatives scrambling for the affections of the Greens and the FDP, the smaller parties find themselves in the position of potential kingmakers.

The two parties, not natural bedfellows, have aready held two sounding-out sessions, their leaders posting a much-shared selfie on social media in a show of unity after a meeting last week.

On Sunday, they held their first sessions with the major parties, striking a positive tone at the start of what promises to be a months-long process.

The SPD first held "very constructive" talks with the FDP, the Social Democrats' general secretary Lars Klingbeil said in a statement after the parties met.

His FDP counterpart Volker Wissing described the need for a reforming government to be formed quickly, while adding that the parties' "positions differ on important points".

By contrast, after speaking to the CDU and the CSU later on Sunday, Wissing said there were "few obstacles" to an agreement with the conservative bloc on policy.

The Greens' co-leader Robert Habeck, meanwhile, hinted at a preference for a coalition with the SPD over the conservatives, praising the election winners' "readiness to start over again" with a new dynamic.

Having secured a narrow win in the election last week, the SPD are in the ascendancy, their candidate for chancellor Scholz insisting in an interview with Der Spiegel Magazine on Friday that the result of the election was "clear".

"The CDU and CSU have suffered a historic defeat and have been voted out," he said.

But the conservatives are not giving up. CSU general secretary Markus Blume said Sunday that early talks gave the conservatives the "desire for more".

The CDU and CSU will each speak to the Greens on Tuesday.

In what was billed as perhaps her last major speech as chancellor, Merkel on Sunday appealed to her successors to defend democracy amid the scramble to form a government.

"We sometimes take our democratic accomplishments too lightly," Germany's long-standing leader said in her speech.

Referring to a neo-Nazi attack on a synagogue in the city where she was speaking two years previously, she called on the public to "reject radicalisation".

Diversity and difference were not threats to society, Merkel added, as Germany had shown in the years since the fall of the Berlin Wall.

The veteran politician, who lived in the communist east before reunification, was visibly moved as she described her own struggles with prejudice and called for more respect for the personal histories of east Germans.