karamazov

The most counterintuitive thing we found doing this is how often jobs (especially old jobs) are undersized. You can run faster and cheaper if you're running out of resources (especially memory).

Espresso AI | Staff ML, Staff Infra, FDE | Brooklyn or San Francisco | Full time

We're using LLMs to build neural optimizers, neural scheduling systems, and neural workload tuners. (If you're ex-Google, you can think of it like Borg powered by LLMs.)

Today we use ML to make data warehouses and spark jobs more efficient. We're hiring staff ML engineers to train models that can understand how much compute a job needs, how it scales to larger machines, whether a machine can run more jobs, and so on; and staff infra engineers to take those models and deploy them on real-world production systems.

We're also looking for FDEs who can help us talk to users and run pilots. This is a pretty technical role (you need to be able to do data analysis and debug in prod) that's also user-facing - it should be a good fit for a former (or future) technical founder.

If this sounds cool, please email me: ben [at] espresso [dot] ai

Espresso AI | Staff ML & Staff Infra Engineers | Brooklyn or San Francisco | Full time

We're using LLMs to build neural optimizers, neural scheduling systems, and neural workload tuners. (If you're ex-Google, you can think of it like Borg powered by LLMs.)

Today we use ML to make data warehouses and spark jobs more efficient. We're hiring staff ML engineers to train models that can understand how much compute a job needs, how it scales to larger machines, whether a machine can run more jobs, and so on; and staff infra engineers to take those models and deploy them on real-world production systems.

If this sounds cool, please email me: ben [at] espresso [dot] ai

Espresso AI | Staff Engineers | NYC ONSITE | Full Tim

We use ML to make data warehouses and spark jobs more efficient. We're hiring staff ML engineers to train models that can understand how much compute a job needs, how it scales to larger machines, whether a machine can run more jobs, and so on; and staff infra engineers to take those models and deploy them on real-world production systems.

If this sounds cool, please email me an intro and a resume: ben [at] espresso [dot] ai

Using ML to make SQL queries more efficient.

This is what my company does (https://espresso.ai/), I'm taking advantage of the end of year quiet time to hack on some more R&D-style projects we have.

Chiming in, I'm one of the founders of Espresso AI - we do both query optimization and warehouse optimization, both of which are hands-off. In particular we're beta-testing a fully-automated solution for query optimization (it's taken a lot of engineering!).

Based on the responses here I think we're a superset of where baselit is today, but I could be wrong.

We have better tech. For our customers, this translates directly into more savings.

We also have less setup and overhead than most of the other companies in the space. many of them come in with recommendations for system changes that you need to implement, and which they then charge you for; we take about ten minutes to set up and then generate savings automatically.

The immediate next thing is expanding to Databricks, followed by BigQuery and other data warehouses.

In the long run we plan to optimize source code (e.g. C++).

Espresso AI | https://espresso.ai/careers | Founding Engineer | NYC Onsite| Full-Time Espresso AI is hiring founding engineers to automate performance engineering, starting with Snowflake data warehouses. Our team worked on ML and performance engineering in Google Search and Google Cloud, and we're applying our expertise to build the world's first neural optimizer.

We're well-funded with paying users, but early enough for you to have significant ownership and impact. Reach out to me directly: [email protected].

there's not a lot to demo - you just turn it on and your bill goes down. I'll try to think of something we can put up though, thanks for the note.

For now I'm hoping 15 minutes is a reasonable investment to save $10k or $100k.

Yes, for right now. We have waiting lists for Databricks, BigQuery, and Redshift; we're hoping to get there later this year.

Shoot me an email if you're interested in one of those: [email protected].

Espresso AI | https://espresso.ai/careers | Founding Engineer | NYC Onsite| Full-Time

Espresso AI is hiring founding engineers to automate performance engineering, starting with Snowflake data warehouses. Our team worked on ML and performance engineering in Google Search and Google Cloud, and we're applying our expertise to build the world's first neural optimizer.

We're well-funded with paying users, but early enough for you to have significant ownership and impact. Reach out to me directly: [email protected].

How's your health? The brain fog, in particular, jumps out as something that may have other causes. In particular:

* Are you exercising?

* Sleeping well? Sleeping consistently?

* Eating well?

* Getting enough vitamins? Vitamin D is a common, easily fixed deficiency that can cause trouble concentrating; you can get your doctor to test it with a blood draw.

* Any chance you have long covid?

If you physically don't feel good on a daily basis, I would absolutely dial back your work and focus on getting in shape for, say, 2 months. 70+ hours per week clearly isn't getting you where you want, so aim for 40 and put in a hard cap at 50, and get used to the idea that some stuff won't get done. Once you're feeling better, continue keeping reasonable hours and resume studying then.

Even if everything else is fine, you might just be working too much. I think the vast majority of people would have trouble studying after a month straight of oncall and 12-hour days.

First, a reality check - it’s great that you can ship defect-free code, but that’s table stakes for a good senior engineer. You’re locally a 10x engineer because you wrote most of the code, so you’re naturally going to be a lot more effective than the other people on your team. This probably won’t translate to new projects.

If you joined a new project where someone else had written 80% of it, it would take you years to catch up to their productivity; if they were controlling and continued to write 80% of everything, it would be impossible. The next step in becoming a better engineer for this project is figuring out what you need to do differently for everyone else to be more productive; for example, if other people are pushing bugs, you need to add tests to make that impossible. If people take a long time to ramp up, you need to refactor the code so that someone doesn’t need to understand -all- of it to start contributing; you might also need onboarding docs.

The next step in becoming a better engineer generally is to switch teams and learn new skills. If you enjoy hard technical work, miss feeling challenged, and genuinely feel like you’re a much better dev than average, try switching to a hard field that’s new to you: distributed systems, ml, performance engineering, etc.