U.S. stocks closed modestly lower Wednesday after a deep morning plunge.

The dip ended a two-day rally to start the fourth quarter. The three major indexes have posted losses in all three of the previous quarters.

The Dow Jones Industrial Average fell 41.52 points, or 0.14%, to 30,274.80. The S&P 500 dropped 7.28 points, or 0.19%, to 3,783, and the Nasdaq Composite was down 27.77 points, or 0.25%, to 11,148.64.

Stocks rallied on Monday and Tuesday with the S&P 500 posting its biggest two-day gain since 2020. Stocks dropped sharply at the start of Wednesday trading after yields started rising on the benchmark 10-year Treasury bond, which climbed 0.14% to 3.76%.

At one point, the Dow was down more than 300 points Wednesday morning. But it rallied, along with the broader markets.

Some of the stocks that declined included Google's parent company Alphabet (GOOG), which closed at $102.22, down $0.19, or 0.19%. Shares of Twitter's (TWTR) fell $0.68, or 1.31%, to close at $51.32.

"It's a moment of pause for the market to reflect on how durable the rally the past two days actually could turn out to be," Yung-Yu Ma, chief investment strategist for BMO Wealth Management, told CNBC. "The market's making the assessment that it's really going to take a lot for the Fed to make a dovish pivot."