WASHINGTON—President Biden’s age and leadership abilities are under fresh scrutiny after a special counsel report on his handling of classified materials said he presented himself as an “elderly man with a poor memory,” and after he repeatedly confused the names of past and present European leaders at campaign events this week.

As he seeks a second term, the 81-year-old president has been dogged by questions about his age, with polling showing that a majority of Americans have concerns about his advanced years. His aides have sought to show him as vigorous and up to the job, but the report and his verbal flubs this week suggest his age remains a liability for his campaign.

An investigation released Thursday by special counsel Robert Hur, a former U.S. attorney from Maryland during the Trump administration, said that no criminal charges were warranted against Biden but noted he was careless in holding on to classified documents from his time as vice president. The report portrayed the president as having “diminished faculties in advancing age,” and displaying a “faulty memory” in interviews with investigators.

In combative remarks at the White House Thursday night, Biden pushed back against the report’s characterization of his recollections.

“My memory is fine. Take a look at what I’ve done since I’ve become president, ” Biden said. He appeared incensed when he noted the special counsel had asked about the death of his son, Beau Biden , saying, “it wasn’t any of their damn business.”

But his appearance included an additional flub when discussing the potential for a hostage deal in Gaza, in which he referred to Egyptian President Abdel Fattah Al Sisi as “the president of Mexico.”

Democrats accused Hur of a politically motivated hit-job, drawing parallels to former FBI Director James Comey ’s decision to recommend that Hillary Clinton not be prosecuted in 2016 over her use of a private email server while serving as secretary of state—but still accuse her of sloppily handling classified information. Jim Messina, a former adviser to President Obama, wrote on X that Hur “didn’t have a case against Biden, but he knew exactly how his swipes could hurt Biden politically.”

Republicans pounced on the report. Republican National Committee Chairwoman Ronna McDaniel said in a statement that “putting a ‘sympathetic,’ ‘elderly’ man who can’t remember major events from his own life back in the Oval Office would undoubtedly make America less safe.”

Paul Begala, a Democratic strategist and former adviser to President Bill Clinton , said that with the report, Biden is “being cleared legally and he’s being kneecapped politically.” He said Biden’s age is “his biggest liability. Every politician has competing master narratives—the positive and the negative.”

“Anything that feeds the negative master narrative is especially damaging,” he said.

The report also came the same week Biden confused current and past European leaders at campaign events. Speaking in Las Vegas on Sunday, he described a gathering with fellow world leaders after he took office in 2021 and referred to “Mitterrand from France,” instead of French President Emmanuel Macron . François Mitterrand was France’s leader from 1981 to 1995 and died in 1996.

During a New York fundraiser Wednesday, Biden spoke again about that meeting and described a moment when “ Helmut Kohl of Germany looked at me.” Kohl was chancellor of Germany from 1982 to 1998 and died in 2017.

White House press secretary Karine Jean-Pierre said Thursday that other officials have mixed up names from time to time, listing Republican officials she said had recently made flubs. “It happens to all of us and it is common,” she said. Campaign officials argued that the GOP attacks are nothing new.

The Biden campaign has recently attempted to turn the age issue around on likely Republican opponent Donald Trump , 77, highlighting an episode in which the former president referred to GOP rival Nikki Haley instead of former House Speaker Nancy Pelosi as he spoke about the Jan. 6, 2021, attack on the U.S. Capitol. Democrats also have pointed to Trump repeatedly mixing up Biden and Obama.

But voters remain more concerned about Biden’s age, according to surveys. A Wisconsin poll released this week by Marquette Law School showed that 61% of voters in the battleground state thought Biden was too old to be president, while just 29% thought Trump was too old.

“The most damaging thing that can happen to a politician is to have an existing negative suspicion confirmed. This is why this is challenging,” said James Carville, a veteran Democratic strategist and former adviser to President Clinton. “When people see this, they will say, ‘Look, Martha, I told you so.’”

Jean-Pierre defended Biden on Thursday, declaring that “his leadership, his experience, speaks for itself.” She said Biden will soon have his annual physical—the previous one was in February of last year—and that the results will be disclosed.

She also this week confirmed that Biden had declined an interview that would air during the Super Bowl on Sunday, passing up on a sizable televised audience for the second year in a row. Jean-Pierre said Biden “will find many other ways to communicate with Americans.” Biden has given fewer interviews and held fewer news conferences than his recent predecessors, according to Martha Joynt Kumar of the White House Transition Project.

In his report, Hur wrote that he didn’t think prosecutors could pursue a criminal case against Biden over the classified material, in part because there were some innocent explanations for Biden keeping the material that jurors might find convincing. “Mr. Biden would likely present himself to a jury, as he did during our interview of him, as a sympathetic, well-meaning, elderly man with a poor memory,” the report said.

Responding to a question Thursday night about that quote, Biden said, “I’m well-meaning, and I’m an elderly man, and I know what the hell I’m doing. I’ve been president and I put this country back on its feet.”

The report said Biden did “not remember when he was vice president, forgetting on the first day of the interview when his term ended (‘if it was 2013—when did I stop being Vice President?’) and forgetting on the second day of the interview when his term began (‘in 2009, am I still Vice President?’).”

“He did not remember, even within several years, when his son Beau died,” the report stated, adding that the president’s “memory appeared hazy when describing the Afghanistan debate that was once so important to him.”

White House special counsel Richard Sauber disagreed with “a number of inaccurate and inappropriate comments” in the report and wrote in a separate letter to Hur on Feb. 5 that the report’s treatment of Biden’s memory wasn’t “accurate or appropriate” and used “highly prejudicial language to describe a commonplace occurrence among witnesses: a lack of recall or years-old events.”

Sauber asked that Hur’s team revisit and revise the report’s descriptions of the president’s memory. He also noted that the report didn’t mention that the interview began the day after the Oct. 7 attack on Israel and said Biden was conducting calls with world leaders and his national security team in the lead-up.

Write to Ken Thomas at ken.thomas@wsj.com and Catherine Lucey at catherine.lucey@wsj.com