President Trump’s new plan to wrest control of the Strait of Hormuz by resuming strikes and reimposing the blockade is his third major shift in military strategy as he searches for a way to turn the tables on Iran in the nearly five-month-old war.

Trump has tried air and missile attacks, a naval quarantine and now the calibrated use of firepower to try to coerce Tehran to agree to his terms, in addition to diplomatic inducements.

But Iran has exploited its proximity to a major oil export route to defy Washington and joust for influence across the Middle East.

A stable peace appears elusive, some former officials say, as each side is calculating that it can win the long game in a test of wills that may haunt the rest of Trump’s presidency, including through the midterm elections in November.

“We are now locked in a coercive war of attrition. Both sides are trying to push the other past some unknowable pain threshold,” said Kenneth Pollack, the vice president of the Middle East Institute and a former CIA analyst. “Coercive wars can go on and on.”

As the fighting continues, the course of the war will be shaped by Iran’s attempts to reconstitute its missile force and air defenses and by the U.S. ability to continue to strike targets, making the most of its declining weapons inventories.

The Trump administration wasn’t bargaining for an open-ended conflict when it rolled the dice in late February and joined Israel’s military campaign to eliminate Iran’s leadership and cripple its arsenal of ballistic missiles and launchers.

Trump acclaimed the prospects for a decisive victory on the opening night of the operation when he touted the might of the American military and called on the Iranian security forces to lay down their arms or face “certain death.”

Trump said the brunt of the fighting could be over in four to six weeks as Iranian forces found themselves outclassed by the more technologically advanced American military that was able to penetrate the country’s airspace at will. For analysts, it was Trump’s version of “shock and awe.”

Even so, there were limits on the scope of the Pentagon’s actions. The use of ground forces was never an appealing option for a president who had decried the forever wars in which the U.S. military has spent long years in Afghanistan and Iraq. If Iran was to be defeated or driven to the negotiating table, the battles would need to be won in the air and at sea, U.S. officials have said.

Following five weeks of intensive airstrikes, the White House declared a ceasefire with Iran to begin talks between Vice President JD Vance and top Iranian officials in Islamabad in April. But after those talks failed, Trump shifted to an economic siege: a maritime blockade.

“Effective immediately, the United States Navy, the Finest in the World, will begin the process of BLOCKADING any and all Ships trying to enter, or leave, the Strait of Hormuz,” Trump said in social-media post that month.

That step was intended to pressure Iran to abandon its own efforts to interfere with oil traffic in the Persian Gulf and meet the American demands that Tehran curtail its nuclear program.

But the White House soon discovered that its blockade could be a double-edged sword. Twenty percent of the world’s oil had passed through the Strait of Hormuz. With the waterway closed, oil stockpiles around the world were declining as countries dipped into their reserves.

That raised the possibility of a perfect political storm i n which oil prices would increase as the midterm elections approached in the U.S. and opinion polls indicated many Americans were unhappy about the Iran war.

The memorandum of understanding that the U.S. concluded with Iran in June to reopen the Strait of Hormuz and set the stage for nuclear talks appeared at first to avert those political risks. One of the Trump administration’s goals in forging that agreement, Vance said in a television appearance was to “refill some stocks” before considering further foreign policy steps.

But within days of signing the interim peace agreement, the U.S. and Iran were clashing over how to interpret the memorandum. While Trump officials saw the deal as unlocking oil traffic through the strait, Iran insisted that it gave it control over the waterway. Iran’s Islamic Revolutionary Guard Corps repeatedly fired at commercial ships transiting the strait, triggering U.S. strikes in retaliation.

By Monday, Trump proclaimed yet another strategy shift: stepped-up airstrikes and the reimposition of a U.S. military blockade that would block ships from entering or leaving Iranian ports. He also declared that the U.S. would be a “guardian” of the strait, imposing an extraordinary 20% fee for cargo passing through the waterway.

Trump orders for open-ended strikes went beyond the tit-for-tat blows Tehran and Washington have exchanged in recent weeks, hitting Iranian radars, antiship missiles and drones that Tehran has used to try to control the strait. Those U.S. attacks also include the first American use of unmanned surface vessels to attack the port at Bandar Abbar and a strike against a rail bridge in northern Iran that is used for trade with Russia and China.

So far, those blows are less intense than the opening weeks of the U.S. military campaign, marking a more calibrated use of American military power as the Trump administration seeks to pressure Iran to abandon its goal of controlling the strait. A half a dozen B-52s that had been deployed in Britain for use against Iran have flown back to the U.S.

While the prospects for a nuclear accord have faded, Trump administration officials are keeping open the option to strike against at the supplies of enriched uranium that American B-2s bombers and cruise missiles hit a year ago.

Iran has continued to strike oil tankers and Gulf states in the region, killing and wounding some sailors. But it has refrained from fresh attacks on Israeli or Saudi targets, as Tehran seems to calibrate its policy in the region.

With this new phase of the war, the prospects for a quick win or an enduring peace have faded.

“Both sides are escalating in a dangerous way in order to increase leverage and weaken the other, and eventually return to the negotiating table preferably with a stronger hand,” said Sanam Vakil, a Middle East director at London think tank Chatham House. But the showdown could backfire and spark a bigger conflict if they don’t find an off ramp, she said.

Former military officials say the U.S. can weaken the regime if it is persistent but warn there is a long road ahead.

“The U.S. will need to identify and pursue a longer term strategy that addresses Iran’s focus on controlling the Strait of Hormuz, threatening its neighbors with missiles and drones and the nuclear weapons program,” said Joseph Votel, the retired Army general who led Central Command from 2016 to 2019. “This is going to be a long-term effort.”

Write to Michael R. Gordon at michael.gordon@wsj.com