News for hackers. Stuff that matters.
Re:If software engineering is in demand, why is it so hard t... by mateo411 on Monday August 30, 2021 @12:41PM (#28357597)
> "How would you pull in data from an arbitrarily long -- maybe never-ending -- source of data in Python?"

The answer to this question is to use a generator. If you don't know this, then both people are going to wish that they had that half hour of their life back.

Is this a good question? If you want somebody with a lot of Python expertise, then knowing about generators is a good thing.

If I was interviewing a candidate, and they didn't know this, then I would just find a pleasant way to move on after a minute or two. If this is the only interviewing question, then that makes me happy, because this company has a flawed process, and is leaving some great talent for me to hire.

It's not a big deal if you didn't know this at the time. You might not have known what a generator was, and the person interviewing probably wasn't very good at interviewing people.

Re:If software engineering is in demand, why is it so hard t... by timr on Monday August 30, 2021 @01:12PM (#28358095)
> > "How would you pull in data from an arbitrarily long -- maybe never-ending - source of data in Python?"

> The answer to this question is to use a generator. If you don't know this, then both people are going to wish that they had that half hour of their life back.

That's funny...I've implemented more than my share of such things in a long career, and my first intuition for an answer to the question, as phrased, was to implement stream sampling. It never even occurred to me that the answer might be a syntactical feature of the language (and yes, I know what generators are).

According to you, I'd be so wrong that I'm a waste of time.

Do you see the problem yet? It isn't theoretical. I've hit this kind of situation with so many interviewers that I'm actually more nervous when I'm sitting across from someone who is clearly on their first or second job out of school. They're asking (unclear) questions in the "python syntax" category, and I'm thinking about the engineering consequences of consuming an endless stream of data...

Re:If software engineering is in demand, why is it so hard t... by cratermoon on Monday August 30, 2021 @01:39PM (#28358500)
This is a sign that the interview questions aren't there to actually assess technical ability, but are geared towards excluding anyone who hasn't travelled in the right circles. The employer isn't looking for someone skilled, they are looking for someone who is pre-filtered to think a certain way and know certain things.
Re:If software engineering is in demand, why is it so hard t... by timr on Monday August 30, 2021 @02:22PM (#28359074)
Maybe. I tend to think that's a cynical take. Never assume malice when incompetence will suffice.

This is a...youthful...industry, and a lot of junior people (with "senior" titles!) are out there doing interviews. It takes experience to become a good interviewer, and if you've never had that experience, you don't know what you don't know.

Re:If software engineering is in demand, why is it so hard t... by cratermoon on Monday August 30, 2021 @11:13PM (#28363569)
No malice is needed. Just incompetence at hiring and defaulting to the kinds of measures that don't actually determine if the person has any aptitude.
Re:If software engineering is in demand, why is it so hard t... by mateo411 on Monday August 30, 2021 @01:25PM (#28358298)
> According to you, I'd be so wrong that I'm a waste of time.

I don't think so, because you actually know about generators. If you were hiring for people who had a lot of Python experience, but didn't know about generators, that would be a bad sign. You would probably want to ask a some other questions, so you would have a more complete picture of what the candidate knows.

My main point is that the company's interview process is flawed.

Re:If software engineering is in demand, why is it so hard t... by timr on Monday August 30, 2021 @02:11PM (#28358916)
> I don't think so, because you actually know about generators.

Sure, but will the OP give me a chance to demonstrate, or simply flip the idiot bit as soon as I don't give the expected response? In the context of a phone interview, I probably wouldn't get anywhere near discussion of python generators in an answer to this question. I'd probably say something like:

"For an infinite stream, my main concern is system resources, which are never infinite...I'd want to use something like stream sampling to calculate the desired metrics, but it's hard to know the right solution without more information about the problem."

Far too many interviewers hear that as weasel-wording, and simply disengage. It's even worse when the interviewer thinks they're asking a clear question about python syntax. It takes a high level of self-awareness to know that you might be the problem when you're the interviewer.

An experienced interviewer might engage with that, but many folks don't have the patience to listen to an unexpected response, nor the respect for the candidate to give them the benefit of the doubt. As I said, this pattern isn't theoretical. I've been on the interviewer end of this kind of pattern many times before.

> My main point is that the company's interview process is flawed.

Agreed on this.

Re:If software engineering is in demand, why is it so hard t... by jodrellblank on Monday August 30, 2021 @02:57PM (#28359489)
How does "use a generator" answer this question? A generator can yield an endless amount of data, but if they ask "our customers buy things 24/7 on our website, how would you pull the endless orders into Python?" and you said "a generator", I'd think you were trying to guess the teacher's password ( https://www.lesswrong.com/posts/NMoLJuDJEms7Ku9XS/guessing-t... )

It surely matters if the data source is push or poll or something else, matters if it's a lot of data or a trickle, matters if you need all of it or if you can miss some (and how much), if the source buffers it or doesn't. I'd expect discussions about message busses, queueing, buffering (maybe ring buffers), data stores (maybe SQL or No-SQL), as well as concurrency or parallelism expecting the question could move to "and what if there was 1000x more data?". "Process stock market data" needs more than "read a temperature every minute".

Or at least for a candidate to ask about some of them and be told "this isn't so complicated", not "it's up to you". If it's only asking whether you can loop over a generator instead of trying to read all the data up front, the question could surely be more focused?

Re:If software engineering is in demand, why is it so hard t... by Banana699 on Monday August 30, 2021 @03:53PM (#28360138)
The question is garbage as phrased to be sure, but I think the auxiliary context matters a lot in this type of questions. Mentioning "python", when databases are language agnostic, heavily implies the answer is a language feature. Furthermore the way the question is asked (reading from a script) also implies there is a specific non-open-ended answer.

I still answered the question in my head on a whim, half-expecting it to be false, it's a really vague formulation. Sounds like the interviewer was randomly browsing language features and cutting and pasting random descriptions from whatever tutorial. Or maybe the company was using python generators in a really specific "data pulling" niche that the answer seemed extremely obvious to the technical team writing the questions and they forgot that outsiders are not wired to think of the same patterns.

Re:If software engineering is in demand, why is it so hard t... by jph on Monday August 30, 2021 @12:52PM (#28357763)
How about asyncio streams.py?
Re:If software engineering is in demand, why is it so hard t... by mateo411 on Monday August 30, 2021 @01:27PM (#28358326)
That seems like a good answer too.

I think that's only applicable if you are reading from a socket. Although I don't know too much about asyncio streams.py, so I'm not going to get a job at this company.

Re:If software engineering is in demand, why is it so hard t... by kamaal on Tuesday August 31, 2021 @08:01AM (#28366799)
>>The answer to this question is to use a generator.

Wrong answer.

The heap data structure would be a right way to go about representing such data.

>>If you don't know this, then both people are going to wish that they had that half hour of their life back.

Same in your case now.