The rumor that President Donald Trump‘s chief economic advisor, Gary Cohn, would leave the administration, widely circulated before being denied by the White House, is not the only reason why stocks dropped as much as they did Thursday, Wall Street veteran Art Cashin told CNBC.

A cloud over the Trump administration’s image and ability to enact its agenda drove the Dow Jones industrial average to close down 1.2 percent, or 274.14 points to 21,750.73 on Thursday. The S&P 500 index declined 1.5 percent to 2.430.01 and the Nasdaq composite dropped 1.9 percent to 6,221.91.

“No disrespect to Mr. Cohn, but what’s really affecting the market is not the fear that only he would leave,” Cashin, UBS director of floor operations at the New York Stock Exchange, said Thursday on “Closing Bell.”

An unverified Twitter account, which has since been debunked by reporters and the administration itself, claimed on Thursday that Cohn was set to resign, Reuters reported. The White House released a statement later assuring that Cohn “intends to remain in his position.”

Cashin said Cohn’s departure could catalyze a “mass exodus” that would hurt many investors’ confidence in the Trump administration.

“The one thing that’s been said, even by people who don’t favor the president, is that, ‘Well, at least he’s surrounded himself with some very capable people: The generals that he has, Wilbur Ross, Mnuchin,'” Cashin said. “People began to fear, as they did this morning, that if Cohn would leave, then others might follow, and he would no longer be surrounded by competent people, and that took a hit in the market.”

Tony Scherrer, director of research at Smead Capital Management, agreed that the nature of the Trump administration is the main driver behind the stock sell-off.

“Our CIO likes to call this ‘the world’s greatest reality TV show,’ in terms of what we’re seeing in Washington D.C.,” Scherrer said on “Power Lunch.”

“And when you have that, you have episodes that are sometimes very fun and encouraging in terms of what it means to business,” he said, “and you have other times that are kind of disconcerting, and that’s what we’ve got going on today.”

The president’s chief strategist, Steve Bannon, appeared to criticize Cohn and others in a revealing interview with The American Prospect published Wednesday, which further fueled the rumors of Cohn’s exit.