macobo

In practice product requirements can get in the way of technically ideal solutions. One example of this is that analytics products allow users to pass in and analyze arbitrary number of user properties to do analysis on - more than even a columnar database can handle. The current solution of storing JSON as a column indeed has a very significant performance trade-off, but it's also needed to power queries users need to run. This is also why we're really excited about the new Object data type that landed in 22.3 as it handles these cases gracefully by creating dynamic subcolumns.

On JOINs - again, requirements bite us in different ways. Product analytics tool data ingestion pipelines can get quite complicated due to needing to handle merging anonymous and signed in users and user properties changing over time. Handling that via JOINs is as a go-to-market helps avoid that upfront cost by centralising the logic in SQL, but indeed does come with a significant cost in scalability. Delaying in turn allows you to be building tools users need. That said every loan needs to be paid at some point and we're currently knee deep in re-architecting everything to avoid these joins.

Also note that JOINs don't work the way you described from our experience - rather the right hand side of the join gets loaded into memory. The bottleneck there is memory pressure rather than I/O with a good ORDER BY on the table.

All that said, what a great summary of all the different things to keep an eye on. Thanks for reading and sharing your thoughts!

> I'm convinced the only people who have good things to say about using linux (or BSD for that matter, been there done that, no thanks) on a laptop are the kind of people who keep their "laptops" on the same desk, plugged in to ethernet, and are effectively using a desktop with poor thermals

Good for you for making your own decisions, but don't be a condecending arschloch. Personally I prefer linux because it works fine and consider Apple is overpriced piece of spyware and many their users smug idiot hipsters.

Happy holidays!

Examples, suggestions?

Why take 60% of the rating space up by negative ratings? It seems like what you really care about is degrees of goodness.

An alternative approach:

1 - I disliked it. 2 - It's OK 3 - This is good 4 - This is great 5 - This is a must-read

https://fishshell.com/ for sure

Just having sensible defaults on a shell works wonders on my day-to-day productivity.

Add a couple of aliases for productivity and off you go.

  abbr --add s "git status"
  abbr --add gap "git add --patch"
  abbr --add gco "git checkout"
  abbr --add gd "git diff"

  alias recent="git for-each-ref --sort=-committerdate refs/heads/ --format='%(color:yellow)%(refname:short)|%(color:bold green)%(committerdate:relative)|%(color:blue)%(subject)%(color:reset)' | column -ts'|' | tac"
  alias r="recent"

In europe, https://eagronom.com/ is taking an interesting approach - rather than throw all existing farming practices out with the bathwater, try to build on top of them and modernize to achieve sustainable farming.

This smells a lot like meteor. Would love to see an actual semi-large app built with this approach.

How so? The code still documents what is happening, the commits however lay out the whys.

The point is that these two are separate questions, and that trying to use comments as a crutch to join the two religiously is a headache. It's impossible to keep everything in sync and I don't want to read needless or worse misleading information.

What's worse, in comments we often omit the important details such as why was the change made, what other choices were considered, how was the thing benchmarked, etcetc.

That said, comments still have a place. Just not everywhere for everything and especially not for documenting history.

There's an infinite amount of detail that's impossible to capture in a comment and which invariably changes over time and doesn't hold in the future.

For my team, the solution has been writing longer commit messages detailing not only what has changed, but also the why and other considerations, potential pitfalls and so forth.

So in this case, a good commit message might read like:

``` Created square root approximation function

This is needed for rendering new polygons in renderer Foo in an efficient way as those don't need high degree of accuracy.

The algorithm used was Newton-Raphson approximation, accuracy was chosen by initial testing:

[[Test code here showing why a thing was chosen]]

Potential pitfalls here include foo and bar. X and Y were also considered, but left out due to unclear benefit over the simpler algorithm. ```

With an editor with good `git blame` support (or using github to dig through the layers) this gives me a lot of confidence about reading code as I can go back in time and read what the author was thinking about originally. This way I can evaluate properly if conditions have changed, rather than worry about the next Chthulu comment that does not apply.

You need a queue in front of your database irregardless of write latency. Otherwise you tie your availability to database availability as downtime (even for upgrades) is often unavoidable and network problems are common.

Dan gave a pretty good talk about the high-level details of how/why postgres a couple of years back: https://www.youtube.com/watch?v=NVl9_6J1G60.

One reason of why pg is that SQL is really powerful for building complex queries.

Note that working remotely takes some getting used to just as with working at an office. Yes, you can dump someone into unfamiliar conditions and draw conclusions from that, but that's just confirming your prior assumptions without actually trying.

But you just said this:

> I've looked at large articles I contributed to a few years ago and they are now disasters.

Examples would be welcome.

Would you mind elaborating on what sort of remote positions yelp offers. Here's a few common ones:

- It's okay to work from home a few/most days of the week. - Anywhere, same working hours for everyone - Flexible working hours, limitations (i.e. must have N hours of overlap and be in US/CA) - Globally distributed organization

Remote work is an option. :)

Regarding a), a sharded postgresql (i.e. with citus data) can easily accomodate that workflow with just a tiny bit of extra overhead.

Re c) this strongly depends on the usecase, I've seen companies use a) to avoid split-brain problems and having to manage two data pipelines to great success at similar scales. You might find https://www.youtube.com/watch?v=NVl9_6J1G60 interesting.

You can always reach out to companies yourself.

I graduated from a second-tier university in a small eastern-european university and did find internship opportunities in SV so I can assert with confidence: your options are what you make of them with your own skills, aptitude and attitude.

Please keep in mind I was responding only to the original comment, which I quoted in its full glory. It had nothing to do with what you assert above, just a mistaken assertion that US is keeping (now was implicit in what you said) nukes in the Baltic States.

Given Russias interest in toying and more with its neighbors, misinformation like this goes a long way of "normalizing" those conflicts. It prepares whomever is reading your comment to say, "huh, the both sides here are shades of gray" and just accept that conflict as normal.

So yeah. There are no US nukes in Estonia. If there are, please back up your sources.

Original comment was:

> And the US isn’t putting nukes into Estonia?

Honest question: Where is this coming from? I'd hate to call this war-mongering and spreading misinformation, but this is a great way to polarize a conversation in one fell scoop.

You're missing a cultural reference. http://www.thelittleprince.com/work/the-story/

This is great, I can see myself using this.

However, this suffers from being a SPA:

- URLs are not persistent, meaning if I cannot bookmark searches I care about.

- Outbound links are buttons for some reason instead of anchor tags. This means that ctrl-click does not work and will bring directly to the website.

Other things:

- I don't think it's easy to filter conferences which are within an acceptable distance. A map-based search a'la yelp would go a long way here.

Note that the project is quite new and has a really active dev base - if you use those features, feel free to submit a PR (or add an issue) and it's pretty sure to get accepted.

De-identification has its limits and information can still be learned even from anonymized datasets. An alternative to this is something like Sharemind [1][2] where sound cryptography is used to make secure multi-party computation possible.

[1]: http://sharemind.cyber.ee/

[2]: https://www.youtube.com/watch?v=bAp_aZgX3B0

Out of curiosity, have you looked at Sikuli [1]?

[1]: http://www.sikuli.org/

A good intro is `Competitive Programming` by Steven Halim [1].

It covers the types tasks typically used in these kinds of competitions, with over 1000 categorized problems. I came away with a through understanding of the intuition behind many of the algorithms and why they work.

[1]: http://cpbook.net/

This is a genuinely scary scenario.

Have there been precedents of profiteering by hiding _something_ invisible to the end-user but with an ulterior purpose into normal consumer electronics?