FOR A FIVE-DAY TRIP to Mexico with her college-age daughter, Arlana Keller of Calgary, Alberta, insisted they book a hotel rated four stars or up. In her experience, that threshold meant a nice room, good service and tasty food.

So when Keller reserved a 4.5-star resort in Cancún through WestJet Vacations , which sells package trips, she expected a dreamy stay.

The wake-up call was harsh. “The bed was clean. The sheets were clean,” Keller, 50 years old, conceded, before listing her woes. “The service was awful. I didn’t feel welcomed at all,” she said, adding that it felt “more like a two-star.” The hotel is no longer listed on WestJet’s website. (A spokesperson for the company said it has a dedicated team that decides star ratings and that hotels can be removed “for a variety of reasons,” including “recurring constructive feedback.”)

Kaushik Vardharajan, who spent 20 years consulting for hotels and is now Boston University’s real estate program director, calls the unregulated hotel star system the bane of the travel industry’s existence. “Each website has its own methodology,” he said. “Nobody really talks about it openly because it is quite vague and opaque in many cases.”

Not only do hotel managers often decide how many stars to slap on their marketing materials, the sheer number of review sites further complicates objectivity.

Most travelers start with a Google search , which lets them filter by stars: from two (“just the basics”) to five (“top service”). But a number of travel agencies and hotel groups, like Virtuoso and the Set Collection , muddy the waters with their own classification systems, with assessments like “small and luxurious” or “independent and unique.” Add to that a bottomless well of user-generated reviews on sites like Tripadvisor, Expedia and Google, and who knows what you’re getting into when you hit the “book” button?

In 2009, some industry players proposed World Hotel Rating , a network that would develop a standardized system to rate hotels, but it never came to fruition. Instead, we’re left with stars that don’t always align with a typical traveler’s experience.

Hotels introduced the star-rating system to communicate to guests the level of service they could expect, explains Ceridwyn King, a professor at Purdue University’s School of Hospitality and Tourism Management. A five-star stay traditionally indicates a full-service experience with bellhops and amenities galore . Three stars, says King, might mean the only food service offered is breakfast.

These days a three-star hotel experience won’t necessarily be of lesser quality than a five-star, says King. But you might need to eat out more or forgo a business center.

Opting for a big name can limit surprises. If the hotel is part of a big brand like Marriott or Hilton , says King, that can help guarantee it will meet reasonable expectations.

While Vardharajan is “all for independent hotels,” he agrees name recognition can offer travelers some assurance—even if that means sacrificing character. Vardharajan says booking with a familiar chain can ensure a hotel will “meet those minimum standards.”

But small hotels bring their own charm, and that’s when a stamp of approval from a credible organization like Leading Hotels of the World or Small Luxury Hotels of the World goes a long way. Jessica Parker, founder of the travel advisory Trip Whisperer , says she often checks if a hotel is affiliated with these portfolios featuring elite properties. That can help travelers discern whether a place has “been vetted through a proper channel,” Parker said.

Another reliable source: the independent rating system of Forbes Travel Guide , which started as Mobil Travel Guide in 1958. This guide sends incognito inspectors to review hotels using set criteria. “If I know [the stars] are coming from the Forbes Travel Guide or…from another source that I respect, then I have a little bit more comfort in that,” said Vardharajan.

King says that besides checking the star ratings’ origins and consulting multiple sources, you might try what’s perhaps the best vetting method: an old-fashioned vibe check. Reach out to the hotel directly, ask them a few questions about their facilities and see how they react. “If they’re responsive and give you feedback, that’s a good sign that they are engaged,” said King.

The Wall Street Journal is not compensated by retailers listed in its articles as outlets for products. Listed retailers frequently are not the sole retail outlets.