scottrogowski

The main bottleneck as an engineer is no longer writing or testing code. It is how long it takes to understand complex systems. This is a really nice approach that, if you have the tokens and the patience, feels like I great way to learn something and I think we'll see more and more stuff like this.

I had a similar realization a few months back and am working on a tool that generates "mermaid walkthroughs". It is 1000% less pretty but it is fast and is pretty good at explaining how services work or what a code review does or just as a way for your agent to explain some decision to you.

https://github.com/scottrogowski/ariel

This sort of thing has not been at all uncommon for Google over the past few years. I’m looking forward to the day when they no longer have this level of power to make or break tiny companies. With any luck, they will either stumble with LLMs and become irrelevant or emerge as only one of several companies and be forced out of their monopoly.

I think this is a more compelling idea than people in the comments are giving it credit for

> The problem is that you can easily come up with 100 different hypotheses for what’s going on. Ok, so you run 100 different studies to test each one. But studies take a long time to run — let’s say 6 months per study. Congratulations, you’ve just locked yourself into 50 years

This is a major problem with science whenever you have less of a theoretical foundation. Compared to physics or chemistry, we know very little about nutrition or sports science. Because of this, the search space is very large. One could argue that given the number of surprising results (and difficulty reproducing those results), medicine and psychology also fall into this category.

> A riff trial takes advantage of the power of parallel search. Some riffs will work better than others (or at least differently), and parallel search helps you find these differences faster, especially if the differences are big.

What if we did more to encourage people to track and report their personal experiments? If even 10% of everyone on a diet (any diet) just tracked what they ate, what exercise they did, and how much weight they lost, and reported it to a centralized database, scientists could then look for patterns in that data and do formal studies based on suspected patterns.

We could do similar things with longevity/happiness. Look at the "Harvard Study of Adult Development" but imagine it was spread out over 10s of thousands of diverse people instead of just 300 upper-class American men? The data quality wouldn't matter much if all you are doing is searching for patterns to do follow-up studies.

Would suggest checking out pyquery. It uses JQuery-like syntax. It's been around a long time and in my opinion, it's way easier to use https://pypi.org/project/pyquery/.

This is actually really helpful. I'd be concerned that deleting the browser would result in other apps breaking but that's probably a risk worth taking.

I'm currently reading Digital Minimalism from Cal Newton and have started preparing for a 30-day digital declutter. So I was really excited to read this title and immediately disappointed after clicking - the Cat phone has a web browser. If you have access to the entirety of the internet in your pocket, it is hard to limit yourself and no little trick or hack is going to actually help - in my experience.

I'll go ahead and take advantage of this post to ask, what do other digital minimalists on HN use for their phone? I'm looking for something with messages, maps, a camera, and WhatsApp with no access to a browser or an app store. The most difficult item on that list is definitely WhatsApp. Unfortunately, if you travel a lot (which I do) it's not really optional. Outside of North America, EVERYTHING happens on WhatsApp. I've started looking into custom Android ROMS but that feels like an extreme step for what must be a common problem?

The two are very similar. I wrote the first version of this almost 10 years ago and only became aware of the existence of Sourcetrail while doing the rewrite a few months back.

On the surface - there is a difference in languages. Sourcetrail explicitly supports C/C++/Java/Python while Code2flow supports Python/JS/PHP/Ruby.

I would love to give code2flow the capability of Sourcetrail but would need to feel very good about it making money before I devoted that time to it.

Wow. Seeing your open source project while scrolling through hacker news is something that really brightens your day. I’m happy to answer any questions people have.

If we are judging by one completely objective measure (high school graduation rates) California is currently the lowest-ranked state in the country: https://worldpopulationreview.com/state-rankings/high-school...

I maintain a similar (but less popular) open-source CLI that generates flow charts for dynamic languages (Python, JS, Ruby, PHP) - https://github.com/scottrogowski/code2flow/

The thought has crossed my mind to try to actually sell it as a SaaS. But by itself, call graph visualizers are the sort of thing that you would need only once or only every once in a while. That sort of software is very difficult to monetize - as 10,000 failed productivity app companies could tell you. I don't know the details of what happened with Sourcetrail but I assume it was similar - developers might have found it useful occasionally but it's not something most developers would use daily or even monthly - so it's hard to justify a subscription.

Yes. This is a "Very Online taxonomy".

While I think it's useful to think about mechanisms of collapse, the author seems to have a clear bias towards the belief that collapse will happen which feels very non-objective.

In my experience, the vast majority of people I interact with (maybe 90%+) fall into the author's categorization of "radical denier" or "partial denier".

I don't think there's anything "radical" about the belief that while things will change, society will go on. But calling the belief radical and suggesting that this is a belief only shared by Very Evil People (e.g. Koch Brothers) is a disingenuous straw man.

For the record, if I had to put myself into this unscientific taxonomy, I might be somewhere between a technological and economic optimist.

Oh man this brings back memories.

I must have been 6 years old. My dad came home with a computer that they were getting rid of at his work. He set it up that night and, having never seen a computer, I wanted to play with it. Well, for some reason, it had Kid Pix installed. I remember drawing something with a couple of green lines and a few rabbit stamps for good measure. As I recall, I then insisted that we had to print it and wouldn’t take no for an answer.

Ah. I think I understand a little better. Knowing nothing else about what you're doing, the way I would structure that would be to run multiple fastmap tasks with crawler + processor code and then pull down just the processed data. So basically, have fastmap as the most upstream node.

Whether or not that's possible with the realities of your application is another question of course :).

If you're curious, I'd be happy to provide some compute time (say 100 vCPU-hours) in exchange for feedback. Email me if that's at all interesting.

Hey, a couple of thoughts:

I think you identified the tradeoff correctly. What I'm building is designed for when compute needs outweigh the data transfer overhead. This works for some applications and not others. In particular, I think my approach will work especially well for ML model training and web scrapers.

That being said, the fewer bytes transferred the better. Apart from colocating servers, there's not much I can do for the data. No matter what, it has to go over an ethernet cable. It's possible to cache source code though and is something I'm working on doing over multiple layers.

In your example, for preprocessing pipelines, Fastmap probably wouldn't make sense - or at least that's my instinct. In my experience, it's rare to see pipeline steps where the compute significantly outweighs the data transfer. I'd be curious to hear your problem though. I might have a blind spot?

ONSITE - Location TBD

I'm a solo founder and am interviewing next week in the final stage of a somewhat well-known tech incubator.

The open source framework / SaaS I'm developing is called Fastmap https://fastmap.io. The one-liner is: "Fastmap offloads and parallelizes arbitrary Python functions on the cloud" but I'm also considering the more colloquial, "Fastmap is a server in your pocket". My short term strategy is to market to data scientists. My long term strategy is to offer a smarter way to do backend infrastructure.

As far as skill set, I am looking for someone with a deep level of expertise in distributed systems and/or data engineering. I would also be open to a non-technical co-founder with business experience in SaaS.

Building a co-founder relationship at a late stage is inherently tricky. For what it's worth, I believe that relationships work best when they are equitable and that 95% of the work of Fastmap is in the future.

Interested? Email me! Let's see if we can find a way to test whether we click: [email protected]

Not a biologist but I understand that isolation is an important factor of diversity and by default, this simulation wouldn't have that. So it makes sense to evolve in different areas and put them back into the same area.

Very cool. Interesting to hear that they actually managed to evolve. Would be curious to see what happens when they can eat each-other. Though I recognize that might be significantly more complicated

This is amazing. My question is whether there are emergent structures in a long-running sandbox environment? The videos that were posted appeared to have quite complex structures but it was unclear whether they were designed or if they "evolved" from earlier more-basic structures. Would be curious to get the author's take.

Interesting concept! Personally, I'm not seeing much in terms of a practical application here but possibly a recreational one? I'm imagining a game of clue where you follow the notes to some sort of geocache location.

"scientists like me and my colleagues are on the threshold of gathering data that may be relevant to the existence of intelligent extraterrestrial life. But this evidence involves subtle findings about phenomena far away in the galaxy"

I'm very curious about this. It sounds like more than the standard spectroscopy that looks for organic compounds and other life signatures. Could anyone provide some more context?

I really like the duck-typing analogy for addiction. Addiction is a complex set of causes, symptoms, and consequences and every addiction is different.

It's nice but it's no Mongita ;) https://github.com/scottrogowski/mongita

In all seriousness, it seems like there's a lot of demand for embedded Mongo-like databases. Congrats on a successful Show HN post!

This should be the top comment. Social media companies can either be monopolies or they can put limits on the speech of their platform... but they can't do both.

While not against the law of freedom of speech, political moderation processes are certainly against the intent. Every once in a while we get a case like this where an unpopular fringe opinion becomes mainstream and underlines the point. But this isn't just a matter of Facebook / Twitter / YouTube needing a better moderation process - this is more fundamental. No person or organization - no matter how benevolent or wise - should have the power to declare truth in a society.

Fringe opinions need constitutional protection - regardless of the era or the technology.

My argument was not about one person being disappointed by expecting value and getting none - it was a broader point about the cultural encouragement - prevalent in engineering circles - of working for free.

To be specific - I think a good argument could be made for free development in the service of the underprivileged or other altruistic causes. However, a large portion or most development ends up benefitting large public corporations who make large profits. But because, as a culture, we encourage this work, we have basically set ourselves up to be exploited.

I recognize that this is an unpopular opinion and is difficult to make well, in a thread, without being downvoted. But I think there is always value at questioning our underlying cultural assumptions.

As someone who has fallen into the idealism trap, I couldn't agree more.

Here's another opinion I've been mulling - the emphasis on FOSS is actively damaging to our community and we need to take steps to end it. We're highly trained professionals who often sacraficed a lot to get to where we are. Having this culture of giving away our professional work for free - often to people and organizations who can readily afford to pay for it - borders on masochism.

Instead of constantly singing the praises of open source, we should be encouraging a deep level of self-esteem in younger engineers by encouraging them to "know their worth".

There are of course clear and obvious benefits to FOSS - like no licensing friction and better security - but we have to find a sustainable way to reward the developers. Little donations here and there aren't going to do it either.

This is a manifestation of the general rule, "don't do things that scale" - at least if you're not the absolute best in the world at what you do.

Whenever something has high fixed and low marginal costs, this sort of winner-take-all distribution results. It's the exact same for indiegames, smartphone apps, or digital music.

Often lost underneath the privacy debates is a problem that is just as pernicious - social media is an addictive and harmful product that is being sold to us for money.

For both factors, addictiveness and harm, there are large piles of scientific studies.

Apart from magnitude of harm, I don't think it's a stretch to think that social media should be grouped together with tobacco as something we want to limit as a society. There might even be an argument on magnitude given that social media causes loneliness and loneliness causes early death https://press.aarp.org/2016-12-07-AARP-Foundation-Draws-Atte.... While I'm not saying that using facebook reduces your lifespan by 15 years, I think there is cause to assume a real physiological impact.

Sure, social media has had its positive impact - every new technology is mixed - but certainly, we need to find common-sense ways to regulate social media to reduce addictiveness and harm.