Stephen Colbert Planned To Leave 'The Colbert Report' Before He Was Offered 'The Late Show'

During a guest appearance in the season finale of “Comedians in Cars Getting Coffee,” hosted by Jerry Seinfeld, Stephen Colbert revealed that he was planning on leaving “The Colbert Report” even before he was picked to be David Letterman’s successor.

“I was ready to stop. I was going to stop whether or not [I got the gig],” Colbert told Seinfeld, who described Colbert as one of the greatest performers he’d ever seen on TV as they walked through sleepy Montclair, New Jersey. “The Letterman thing fell into my lap.”

In the early years of “The Colbert Report,” which premiered in 2005, Colbert used to be one of the first people and wouldn’t leave until around 11:00 p.m., he revealed to Seinfeld. “For the first couple of years, it was like I lived there,” he said.

As the two enjoyed a meal at the Bluestone Coffee Co., they discussed the idea of Seinfeld appearing on “The Late Show with Stephen Colbert.” Seinfeld assured Colbert that his appearance on “Comedians Getting Coffee” is more relaxing than if Seinfeld visited "The Late Show."

“To do a guest appearance on your show, it’s no f------g fun,” Seinfeld admitted. “ When I come on your show, I’m going to do a stand-up set. You know how long it’s going to take me to put that together? So you cannot compare the pain in the a-- of doing your show to the joy and pleasure of doing mine.”

On the drive back to Colbert’s home in a blue 1964 Morgan +4, Colbert revealed that he and Jimmy Fallon, host of NBC’s “The Tonight Show,” got drinks soon after it was announced that he was taking over for Letterman.

“I think nothing would be more boring than a late night war,” the Second City alum said. “We [he and Jimmy Fallon] laughed. We told stories. It was lovely.”

Now in its sixth season on Crackle, "Comedians in Cars Getting Coffee" features Seinfeld conducting very casual interviews with fellow comedians as they drive around in vintage cars. Seinfeld has interviewed everyone from Tina Fey to Aziz Ansari to Sarah Jessica Parker.

Here's the entire episode:

"The Late Show with Stephen Colbert" premieres on September 8, 2015.