I interviewed for a company once where I got a real dataset from their business and just a vague assignment to analyze the data and find something that could be optimized better. So I came up with a suggestion on how to improve factor A, but they told me later that what they were looking for was for the candidates to come up with suggestions on how to improve factor B.

Why factor B was deemed more important than factor A wasn’t something that possibly could have been understood from the dataset alone.

They were just screening out unlucky people.

That can be done by shuffling the resumes and dumping half of them in the garbage. Saves everyone time as well.

They are applying a Leaking Bucket Algorithm.

I believe most recruiters already do this.

My solution is to produce a design in my head, and figure out its limitations. Then I ask for each limitation if this limitation will be a problem. If it is, then I figure out how to iterate the design to eliminate that limitation. Once I don't know of any unverified limits, then I start explaining my design, and explain how I know it will perform to spec.

This is also a good way to approach system design in the real world. Don't simply go with the first solution that looks like it might work. Also don't add on nice looking features on general principle. Instead sketch out the simplest thing which works in your head, figure out the limitations, check whether each is OK, and then improve my design if it doesn't work yet.

Only if the interviewer cooperates with candidate. In my past cases, some interviewers are just nuh-uh or "let's continue" or just keep saying the original ambiguous requirement without stating additional requirements or giving any feedback.

Some interviewers like to move the goalposts. If I encounter those, I'll go ahead and give the simplest solution and state a general idea of what its limits are. Then when the goalposts are moved on me, I can iterate on the design for the new rules.

If an interviewer cannot tell my design skills from that, then I figure that the company failed, not me.

Lately Ive been going the other way as an interviewer. I try to get the candidate to deeply explain their system design; starting point, requirements, conflicting priorities, compromises, etc. Alternatively we can pivot slightly to solving the same class of problem with my system.

I can almost always find a relevant overlap between my background and theirs, which helps. And (I think) its beneficial for them to talk about their experience and existing decision making as opposed to hypotheticals. As a bonus you can also gauge the ability to communicate complex or nuanced ideas in a succinct manner.

This has been my experience as well. These interviews should be open-ended with no right answer as long as you can reasonably support your design and discuss its tradeoffs. Unfortunately, many interviewers have a set solution / structure / technology already in mind.

It takes a skilled interviewer to deal with and evaluate that level of nuance on the fly. It's much easier to scale an interview process with checklists.