Japan's Prime Minister Fumio Kishida reshuffled his cabinet on Wednesday, removing some ministers with links to the Unification Church in a bid to stem plunging support amid growing public outrage over the ruling party's ties to the controversial group.

Kishida, in office since last October, announced his new government team in a shake-up that came earlier than analysts had expected.

While key personnel like foreign minister Yoshimasa Hayashi and finance chief Shunichi Suzuki held on to their posts, some high-profile ministers were removed, including Nobuo Kishi, the younger brother of slain former premier Shinzo Abe, replaced as defence minister by Yasukazu Hamada.

In the month since Abe was gunned down, a spotlight has been turned on the Liberal Democratic Party's (LDP) long-standing ties to the Unification Church, with polls showing plunging approval ratings for Kishida with respondents citing a need to know just how close those ties might be.

Abe's suspected killer has said his mother was a Unification Church member bankrupted by donating to it, and blamed Abe for promoting it.

In the latest survey, his support had fallen to 46% from 59% just three weeks ago, public broadcaster NHK said on Monday, the lowest rating for Kishida since he became prime minister.

"He's basically doing damage control," said political commentator Atsuo Ito. "What people are really watching is the Unification Church."

The religious group itself is set to hold a rare news conference with foreign media late on Wednesday.

In other changes, Koichi Hagiuda, the trade minister, became head of the LDP's policy research council, a heavyweight job in the party. That appointment is seen as an attempt to appease members of the Abe faction, the party's biggest, though Hagiuda has publicly acknowledged attending an event held by a Unification Church-related group.