The financial troubles at St. Michael’s College hit home for biology professor Declan McCabe when he noticed Buckthorn shrubs encroaching on walking trails near the house of the campus president.

Enrollment declines opened the door to maintenance staff layoffs, giving the invasive shrub the upper hand. McCabe—a roll-up-your-sleeves, get-it-done Irishman—taught his students to identify the woody plant and cut it back with handsaws and loppers. He turned the chore into lessons on the environment.

Tenured professors doubling as groundskeepers at a $70,000-a-year private college in New England is another sign of what is shaping up as the bleakest era for America’s smaller private schools.

Consolidation of the nation’s nearly trillion-dollar higher-education sector is driving a new winner-take-all market, benefiting Ivy League campuses, flagship public universities and schools with high-profile sports teams and renowned research institutions. They enjoy high demand and a surplus of full-tuition payers, while lesser-known campuses juggle cost cuts and steep tuition discounts, including at St. Michael’s, to fill seats.

Shrinking enrollment at 442 private nonprofit colleges—out of 1,700 nationwide—is placing them at significant risk of closing or merging in the next decade, according to a forecast by the Huron Consulting Group , which advises schools on operations and mergers. Small and rural colleges, including many that survived the Great Depression, are especially vulnerable.

St. Michael’s, one of the most well-respected institutions in northern New England, is among them. Enrollment is down by 45% over the past 10 years. The 120-year-old college has run recent budget deficits of $12 million and $9.4 million. It has sold property, rented out dorms, trimmed a third of its faculty, cut courses and about doubled its endowment withdrawals. In 2022, Moody’s cut St. Michael’s once-robust bond rating to junk.

Its current struggles contrast with a decade ago, when over-enrollment forced St. Michael’s to rent out a nearby hotel to house students. Now, some campus housing holds refugee families, including from Afghanistan. A private high school uses one dormitory. Another dorm will house students from Vermont State University.

Elias Pike , a sophomore criminology major, lives in an eight-person dorm suite that he shares with only one other student. They each have their own bathroom. In the lounge, where Pike and his roommate host dinner parties, icicle lights hang from the ceiling and cases of Coca-Cola are stored in a corner.

On a recent evening, Pike walked through the half-filled dining hall near his dorm. Students waved and called his name. “People really do care about each other here,” he said. “But you notice it’s small.”

St. Michael’s is shielded by an $87 million endowment, school leaders said. Loyal alumni, brisk fundraising and success placing students in jobs and graduate schools are cause for optimism, they said, despite the challenges.

Yet even the new president, who took office in 2024 to make tough budget choices, acknowledged that St. Michael’s has maybe a five-year window to get its finances on track before the school’s foundation gives way. The school’s survival is an uncomfortable topic among faculty, alumni and students.

“There are two camps,” said Father Michael Carter , the youngest member of the religious order that founded the school. “The optimists who say we are turning the corner, and the people who say, ‘Well maybe if we had done this 10 or 15 years we’d be OK,’ but now, they’re not so sure.”

Glory days

Like many small liberal-arts colleges in New England, St. Michael’s campus resembles a postcard from a more genteel era. Red brick buildings stand alongside towering maple, oak, birch and sycamore trees. The aesthetics signal orderliness and a testament to the decades of social and economic tailwinds that helped to lift the fortunes of U.S. colleges after World War II.

A growing population, the GI Bill, increased enrollment of women and minority students, the clear financial returns of four-year degrees and federal student loans all helped swell college enrollment and the American economy. For much of the last century, colleges prospered with the same playbook: enrolling a larger slice of an ever-growing pool of prospective students, raising tuition and investing in campus amenities.

In 2001, when St. Michael’s wanted to borrow $9 million to build a new dormitory, Moody’s told prospective lenders that the institution was creditworthy. It had a strong enrollment, a respected brand and a healthy cash reserve.

These days, St. Michael’s and other campuses face the so-called demographic cliff, a drop-off in the number of prospective students that is forecast to last years. The largest number of Americans born in a single year entered colleges this past fall. After those students had been born, the 2008-09 financial crisis hit and birthrates plummeted.

New England and the Midwest have the greatest concentration of small private colleges along with the fastest decline in the number of high-school graduates.

The northeast U.S., where St. Michael’s draws most of its students, reported shrinking birthrates ahead of the rest of the country. The region faces a steeper fall ahead—17% fewer high-school graduates by 2041 compared with 2023, according to the Western Interstate Commission for Higher Education.

In addition to demographics, skepticism about the value of a college degree also has reduced the share of Americans choosing college. The decline in confidence is amplified by the tough job market for recent graduates and uncertainty about the role of artificial intelligence in the workplace.

Tuition that outpaced inflation for decades has saddled students and families with high debt and yielded uneven returns on the time and money poured into a four-year degree. The wealthiest universities can offset tuition costs with large endowments, making many of them more affordable than smaller, lesser-known colleges. Frailer schools try to compete by lowering admission standards and giving large discounts on tuition, which generally carries a sticker price on par with big-name universities.

Like a lot of small colleges, St. Michael’s draws most of its revenue from tuition, fees and room and board. If the school misses its enrollment mark by even a dozen students, the losses stretch over four years. Fiscal cracks appeared in 2016, when St. Michael’s missed enrollment and fundraising targets, leaving a $1.65 million shortfall.

In response, the college cut supplies, maintenance, travel, meals and offered incentives for employees to leave, the school paper reported. Administrators acknowledged that the loan for a new dorm around that time had been poorly timed.

By 2022, undergraduate enrollment declined to about 1,200, down from around 2,100 in 2014. Campus life also took a hit. A school newspaper article described the student center in 2023 as desolate and deserted. In 2024, enrollment fell to 1,120.

For the 2024-25 academic year, St. Michael’s accepted 85% of applicants; 12% of those enrolled and only a handful paid full price, according to federal data. The most competitive U.S. colleges and universities have single-digit acceptance rates and enroll four-fifths of admitted students

As the pandemic ended, rumors spread that St. Michael’s would close or be absorbed by the University of Vermont, located nearby. To make ends meet, St. Michael’s dug into its roughly $90 million endowment. Instead of taking a standard annual disbursement of 5%—about $4.5 million—St. Michael’s drew nearly $20 million over two years.

Last semester, the college suspended its campus newspaper, the Defender, which published for 80 years, in part because not enough students had taken the course required to work on the paper.

“There were maybe five of us trying to do everything,” recalled former executive editor Norah Beckwith, a 21-year-old senior. “On some production nights there were only two or three of us in the room.” Ideally, Beckwith said, there would have been 10 staffers.

Turnaround try

About two years ago, St. Michael’s hired Richard Plumb as president. The 66-year-old engineer at one time helped design radar and sonar systems for the U.S. military. His mission as head of the college was to stabilize finances. He called an all-hands meeting for faculty and staff and said the school needed to reorganize and reduce staffing.

Supervisors filed reports to justify each employee and included the number of students that their program served. Plumb and his faculty leaders weighed the cost to run courses against the enrollment and value for students. Then he reorganized 20 academic departments and consolidated majors. Information science, computer science, data science and applied statistics became analytics. Separate theater and music departments became performing arts.

Plumb also sought to leverage campus attractions. For decades, St. Michael’s had its own fire department and ambulance department, staffed largely by student volunteers and serving nearby towns. Plumb championed a new emergency services major that has drawn wide student interest.

More than 40% of students play a team sport, and St. Michael’s plowed part of a $38 million loan into new locker rooms in the athletic facility, as well as IT upgrades on campus. The debt obligation adds about $3 million a year to the budget, but athletes are among the most likely to stay at the campus through graduation.

Plumb believes St. Michael’s future hinges on getting the word out about its successes: medical schools accept 80% of its pre-med students on their first try, higher than the national average; the school consistently ranks near the top of rankings for NCAA student-athlete graduation rates; about three-quarters of students earn a diploma within four years; and nearly all of them are working full time or are enrolled in graduate school within six months of graduation.

Plumb said he has made all the budget cuts needed so far. Reduced personnel expenses allow the school to operate with as few as 1,200 students, he said, and enrollment in this year’s freshman class ticked up. Last year, the deficit shrank to $2.6 million.

“People believe in this place,” he said. “My biggest fear is that people would give up hope. Hope is a Catholic virtue.”

The reorganization made more students eligible to work at the Defender and the paper is publishing again.

Plumb launched a recruiting campaign with billboards around Boston and a pledge to match the in-state tuition of any state flagship university. In September, St. Michael’s rolled out a new social-media campaign on Instagram featuring Emma MacDonald, a senior who hopes to go into broadcast journalism. In “Try it with Emma Mac,” MacDonald goes ice climbing and snowboarding.

MacDonald said she enrolled in St. Michael’s in part because her parents are alumni, and she wanted to play lacrosse for the school. She raved about her professors, but also expressed frustration that the school doesn’t have a news station. At a TV internship in Boston last summer, she said she was the only one who wasn’t experienced reading from a teleprompter.

Still, MacDonald says she feels a strong loyalty to her tightknit campus community. She doesn’t want St. Michael’s to close and is eager to attract new students.

Her mother also is pulling for the school. Maryanne MacDonald fondly recalled her years there—hiking, classes and campus social life. She said she was struck by how much smaller and quieter the school has become since she graduated in 1993.

“You have this little pain in your stomach,” she said. “It’s like, ‘Oh God, I hope they turn it around.’”

Write to Douglas Belkin at Doug.Belkin@wsj.com