The answer to this question is to use a generator. If you don't know this, then both people are going to wish that they had that half hour of their life back.
Is this a good question? If you want somebody with a lot of Python expertise, then knowing about generators is a good thing.
If I was interviewing a candidate, and they didn't know this, then I would just find a pleasant way to move on after a minute or two. If this is the only interviewing question, then that makes me happy, because this company has a flawed process, and is leaving some great talent for me to hire.
It's not a big deal if you didn't know this at the time. You might not have known what a generator was, and the person interviewing probably wasn't very good at interviewing people.
> The answer to this question is to use a generator. If you don't know this, then both people are going to wish that they had that half hour of their life back.
That's funny...I've implemented more than my share of such things in a long career, and my first intuition for an answer to the question, as phrased, was to implement stream sampling. It never even occurred to me that the answer might be a syntactical feature of the language (and yes, I know what generators are).
According to you, I'd be so wrong that I'm a waste of time.
Do you see the problem yet? It isn't theoretical. I've hit this kind of situation with so many interviewers that I'm actually more nervous when I'm sitting across from someone who is clearly on their first or second job out of school. They're asking (unclear) questions in the "python syntax" category, and I'm thinking about the engineering consequences of consuming an endless stream of data...
This is a...youthful...industry, and a lot of junior people (with "senior" titles!) are out there doing interviews. It takes experience to become a good interviewer, and if you've never had that experience, you don't know what you don't know.
I don't think so, because you actually know about generators. If you were hiring for people who had a lot of Python experience, but didn't know about generators, that would be a bad sign. You would probably want to ask a some other questions, so you would have a more complete picture of what the candidate knows.
My main point is that the company's interview process is flawed.
Sure, but will the OP give me a chance to demonstrate, or simply flip the idiot bit as soon as I don't give the expected response? In the context of a phone interview, I probably wouldn't get anywhere near discussion of python generators in an answer to this question. I'd probably say something like:
"For an infinite stream, my main concern is system resources, which are never infinite...I'd want to use something like stream sampling to calculate the desired metrics, but it's hard to know the right solution without more information about the problem."
Far too many interviewers hear that as weasel-wording, and simply disengage. It's even worse when the interviewer thinks they're asking a clear question about python syntax. It takes a high level of self-awareness to know that you might be the problem when you're the interviewer.
An experienced interviewer might engage with that, but many folks don't have the patience to listen to an unexpected response, nor the respect for the candidate to give them the benefit of the doubt. As I said, this pattern isn't theoretical. I've been on the interviewer end of this kind of pattern many times before.
> My main point is that the company's interview process is flawed.
Agreed on this.
It surely matters if the data source is push or poll or something else, matters if it's a lot of data or a trickle, matters if you need all of it or if you can miss some (and how much), if the source buffers it or doesn't. I'd expect discussions about message busses, queueing, buffering (maybe ring buffers), data stores (maybe SQL or No-SQL), as well as concurrency or parallelism expecting the question could move to "and what if there was 1000x more data?". "Process stock market data" needs more than "read a temperature every minute".
Or at least for a candidate to ask about some of them and be told "this isn't so complicated", not "it's up to you". If it's only asking whether you can loop over a generator instead of trying to read all the data up front, the question could surely be more focused?
I still answered the question in my head on a whim, half-expecting it to be false, it's a really vague formulation. Sounds like the interviewer was randomly browsing language features and cutting and pasting random descriptions from whatever tutorial. Or maybe the company was using python generators in a really specific "data pulling" niche that the answer seemed extremely obvious to the technical team writing the questions and they forgot that outsiders are not wired to think of the same patterns.
I think that's only applicable if you are reading from a socket. Although I don't know too much about asyncio streams.py, so I'm not going to get a job at this company.
Wrong answer.
The heap data structure would be a right way to go about representing such data.
>>If you don't know this, then both people are going to wish that they had that half hour of their life back.
Same in your case now.
"How would you pull in data from an arbitrarily long -- maybe never-ending -- source of data in Python?"
me : "What kind of data? What is it used for? How should it be represented afterwards?"
Silence on the other end , followed by "Well that's up to you."
I then fumble out a few ways to do it (verbally), to a recruiter that doesn't seem to understand anything that i'm saying -- he's probably reading a question from a script and pattern-matching what I say to the supposedly correct answer given to him in the same script.
Finally it ends with "Oh, that's interesting. ...We'll get back to you." , and the interviews that go this direction -- they never get back to you; you've already lost.
I'm not spouting gibberish, i've been at this for a long time; i'm just not spouting out whatever tech-jargon that their company supports and is needed to conform to his script, and quite honestly the person on the other end of the phone call seems to be universally under-equipped to judge anything remotely CS related.
It's a really weird/ephemeral/surreal experience. Imagine a surgeon being appointed to a position at a hospital because the secretary quizzed him on some flash cards (flash cards provided by Acme Sharp Scalpels), and the surgeon -- by chance -- prefers Acme Sharp Scalpels, and so mentions that brand. Instant hire according to the flash cards.
That's a lot like what it's like when a technically-accurate developer gets passed over for one that blithely mentions popular software packages (pandas/tensorflow/whatever is popular this week) without any real technical knowledge on how to achieve the goals wanted by the hiring party.