News for hackers. Stuff that matters.
Re:Why do we need Flask, Celery, and Redis? (2019) by pinkbeanz on Saturday April 18, 2020 @09:13AM (#22907475)
I think a hard part with lots of these “what do I use x for” examples is it starts with the tool and the discusses the problem that it solves. I find it more helpful to start with a problem, and discuss the various tools that address it, in different ways.

Forget email, say you have a app that scans links in comments for maliciousness. You rely on an internal api for checking against a known blacklist, which follows shortened links first, and an external api from a third party. You want the comment to appear to submit instantly to the poster but are comfortable waiting for it to appear for everyone else. What are your options?

You could certainly use message queues and workers. If you’re cloud native maybe you leverage lambdas. Maybe you spin up an independent service that does the processing and inserting into the database in the background, and all you need to do is send a simple HTTP request on an internal network.

Your solution depends on your throughout requirements, the size of your team and their engineering capabilities, what existing solutions you have in place. Everything has its pros and cons. Pretending that celery/redis is useless and would be solved if everyone just used Java ignores the fact that celery and redis are widely popular and drive many successful applications and use cases.

Re:Why do we need Flask, Celery, and Redis? (2019) by lmeyerov on Saturday April 18, 2020 @06:05PM (#22911497)
We've been experimenting with nice compromises for using pydata on-the-fly compute with caching. Basic tension is indiv requests get blocked by their blocking APIs, while caches (eg, IP-pinned) are best when same-python-app-thread.

Right now, we do hypercorn multiproc -> per-proc quart/asyncio/aiohttp IP-pinned event loop -> Apache arrow in-app cache -> on-gpu rapids.ai cache.

But not happy w event loop due to pandas/rapids blocking if heavy datasets by concurrent users. (Taking us back to celery, redis, etc, which we don't want due to extra data movement..) Maybe we can get immutable arrow buffs shared across python proc threads..

Ideas welcome!

Re:Why do we need Flask, Celery, and Redis? (2019) by StavrosK on Saturday April 18, 2020 @04:41PM (#22910922)
While I agree with the rest of your comment, the sentence "if you’re cloud native maybe you leverage lambdas" made me irrationally angry.
Re:Why do we need Flask, Celery, and Redis? (2019) by CalRobert on Saturday April 18, 2020 @05:50PM (#22911399)
Can you explain why? I use lambdas often and they seem to solve the problems they're meant for well.
Re:Why do we need Flask, Celery, and Redis? (2019) by StavrosK on Saturday April 18, 2020 @05:53PM (#22911421)
It wasn't the lambdas, it was the combination of "cloud-native", which is a very salesmany term, and "leverage", which is my pet hate word. It's exactly as useful as "use", only much more pretentious. I'm just easily triggered with language :P

More off-topic (or, rather, on-topic), I find lambdas great for things like a static website that needs a few functions. I especially like how Netlify uses them, they seem to fit that purpose exactly.

Re:Why do we need Flask, Celery, and Redis? (2019) by normalnorm on Sunday April 19, 2020 @02:46AM (#22914057)
> I'm just easily triggered with language :P

Me too! It makes me irrationally angry when people regurgitate linguistic clichés. I was already mad with:

"python does not have a go-like concurrency story"

when it would be enough (and 1000x less cringe) to say:

"python does not have go-like concurrency"

I think these mindless clichés make language really ugly and dysfunctional, and even worse they are thought-stoppers, because they make the reader/listener feel like something smart is being said, because they recognize the "in-group" lingo. In my experience, people get really offended when you point this out. It's kind of an HN taboo to discuss this. Which is also interesting in itself.

Going forward we should pay more attention to our communication use cases. Btw: I wonder if we can stack several of these clichés. For example: "leverage" + "use case" = "leverage case".

Re:Why do we need Flask, Celery, and Redis? (2019) by StavrosK on Sunday April 19, 2020 @06:24AM (#22914795)
I agree, I think Orwell's "Politics and the English Language" is spot on here. I try to use simpler language whenever possible, I agree that people think that using longer words makes them sound smart but is just worse for communication.

I've found it's a taboo to discuss anything even slightly personal. People are averse to feeling bad, so criticism needs to be extremely subtle in order to not offend.

> Btw: I wonder if we can stack several of these clichés. For example: "leverage" + "use case" = "leverage case".

I hate you for even thinking of this.

Re:Why do we need Flask, Celery, and Redis? (2019) by rumanator on Sunday April 19, 2020 @08:13AM (#22915351)
> I've found it's a taboo to discuss anything even slightly personal. People are averse to feeling bad, so criticism needs to be extremely subtle in order to not offend.

The personal association you made between "discussing anything even slightly personal" and "criticism needs to be extremely subtled" makes it sound that your problem isn't language or Orwellian discourse but the way you subconsciously link discussing personal matters with harshly criticising those you speak with for no good reason.

If your personal conversations boil down to appease your own personal need to criticise others then I'm sorry to break it to you but your problem isn't language.

Re:Why do we need Flask, Celery, and Redis? (2019) by stavros on Sunday April 19, 2020 @08:30AM (#22915458)
You just misconstrued my saying "personal" and clearly meaning "personal criticism" as meaning personal things in general and then criticized me on that straw man. I don't hold that opinion at all.

You also went to "Orwellian discourse", which has a specific meaning, from a text by Orwell I mentioned. It seems to me like you got personally offended, interpreted my comment in the most uncharitable way, and chose to lash out at me instead, and I'm not sure why. I wasn't even talking about anyone specifically.

Re:Why do we need Flask, Celery, and Redis? (2019) by rumanator on Sunday April 19, 2020 @08:36AM (#22915492)
You were the one associating discussing remotely personal stuff with criticising others, and if that was not bad enough your personal take was that you felt the need to keep criticising others but resorting to subtlety just to keep shoveling criticism without sparking the reactions you're getting for doing the thing you want to do to others.

I repeat, your problem is not language. Your problem is that you manifest a need to criticize others. That problem is all on you.

Re:Why do we need Flask, Celery, and Redis? (2019) by heyoni on Monday April 20, 2020 @01:31AM (#22921383)
That is such quite a stretch of the imagination and I sure didn’t read it that way. I may be wrong but here’s one fun example from this comment section that I wanted to “respond” to and demand some clarification on.

https://news.ycombinator.com/item?id=22911497

PS: I work with like 65-70% of that stack daily

Re:Why do we need Flask, Celery, and Redis? (2019) by StavrosK on Sunday April 19, 2020 @08:42AM (#22915520)
Ah, okay. If I had said "any remotely personal criticism", like I meant, you'd have an entirely different conclusion, but I guess that doesn't matter when you just want to jump to conclusions.

This shows from the fact that you're saying "criticize" like it's a bad thing.