Commerce Secretary Howard Lutnick said he visited Jeffrey Epstein’s private island in 2012 with his wife and children, years after Lutnick said he had cut off ties with the convicted sex offender.

“My wife was with me, as were my four children and nannies. I had another couple. They were there as well with their children, and we had lunch on the island,” Lutnick said during a Senate Appropriations subcommittee hearing on Tuesday. Lutnick said he and his family spent roughly an hour on the island and then left. “We were on family vacation,” he added.

Lutnick said he searched for his name through the files released by the Justice Department from its Epstein investigation, and among the millions of pages found about 10 emails connecting him with the disgraced financier over a 14-year period. “I didn’t look through the documents with any fear whatsoever, because I know, and my wife knows, that I have done absolutely nothing wrong in any possible regard,” Lutnick said.

“I did not have any relationship with him. I barely had anything to do with that person. OK?” he said.

Sen. Chris Van Hollen (D., Md.), a member of the Senate Appropriations Committee, asked Lutnick if he would share his own personal records related to Epstein with Congress. “I will surely talk about that. I hadn’t thought about that. I have nothing to hide, absolutely nothing,” Lutnick said.

Lutnick, formerly the chairman and CEO of Wall Street firm Cantor Fitzgerald, is facing calls to resign over his relationship with Epstein , which a recent batch of files shows was more extensive than he previously revealed. “Secretary Lutnick remains a very important member of President Trump’s team, and the president fully supports the secretary,” White House press secretary Karoline Leavitt said Tuesday.

The Commerce Department said in a statement that Lutnick and his wife met Epstein in 2005 and had very limited interactions with him over the next 14 years. “This is nothing more than a failing attempt by the legacy media to distract from the administration’s accomplishments including securing trillions of dollars in investment, delivering historic trade deals and fighting for the American worker,” the statement said.

Records recently released by the Justice Department appear to show Lutnick emailing with Epstein, his former Upper East Side neighbor, arranging calls and being scheduled for a drink in 2011. They also show Lutnick planning a trip to Epstein’s private island, Little St. James, during a yacht trip with his family in the Caribbean in December 2012.

“Hi Jeff, We are landing in St. Thomas early Saturday afternoon and planning to head over to St. Bart’s/Anguilla on Monday at some point. Where are you located (what is exact location for my captain)? Does Sunday evening for dinner sound good?” Lutnick wrote to Epstein in an email dated Dec. 19, 2012.

In subsequent emails, Lutnick confirms lunch plans with Epstein on his island. On Christmas Eve, 2012, Epstein’s assistant emailed Lutnick a message from him: “Nice seeing you.” A few days later, on Dec. 28, 2012, the two men signed a document for stakes in a now-defunct technology company, Adfin.

In 2017, Epstein contributed $50,000 to a dinner honoring Lutnick and another investor, which was put on by Jewish philanthropic organization UJA-Federation of New York, documents show. Epstein was offered a table and 10 seats to attend the event but declined, writing in an email to “tell lutnik he can fill them.”

In 2008, Epstein pleaded guilty in Florida state court of procuring a child for prostitution and of soliciting a prostitute. He served almost 13 months in jail. He was arrested again in 2019 by federal authorities and charged with sex-trafficking conspiracy, and died in jail while awaiting trial. His death was ruled a suicide by New York City’s medical examiner.

Lutnick said in an interview last year on the “Pod Force One” podcast that he decided never to be in the same room as Epstein, who lived next door to the Lutnicks, after Epstein gave them a disturbing tour of his home in 2005. Lutnick recalled Epstein showing him and his wife his massage table in the middle of his house.

“My wife and I decided that I will never be in the room with that disgusting person ever again,” Lutnick said. “So I was never in the room with him socially, for business or even philanthropy. That guy was there, I wasn’t going because he’s gross.”

Following the latest batch of emails released from the Justice Department, several lawmakers have called on Lutnick to resign.

Rep. Thomas Massie (R., Ky.), in a CNN interview over the weekend, said Lutnick should “make life easier on the president, frankly, and just resign.”

“It’s now clear that Commerce Secretary Howard Lutnick has been lying about his relationship with Epstein. He said he had no interactions with Epstein after 2005, yet we now know they were in business together. Lutnick must resign or be fired. And he must answer our questions,” Democratic Rep. Robert Garcia, the ranking Democrat on the House Oversight Committee, wrote on X.

Sen. Adam Schiff (D., Calif.) said Lutnick should resign after he “lied to the country about his ties to Epstein.”

Write to Meridith McGraw at Meridith.McGraw@WSJ.com