News for hackers. Stuff that matters.

alandu

Re:Show HN: DaLMatian – Text2sql that works on Saturday March 23, 2024 @03:07PM (#39802329)
An example question in that eval set is "How many publications were published between 2019 and 2021?". That's something GPT without any context can understand how to answer from a schema (which I assume has a column called publications). An example question that I'd get in my previous role at an ecommerce fraud detection company could be something like "what's the chargeback rate on the ATO segment". Neither chargeback rate nor ATO segment are defined in the database schema. Not only did they have different definitions depending on the context (e.g. which customer), the definition also change over time within the same context.
Re:Show HN: DaLMatian – Text2sql that works on Saturday March 23, 2024 @03:03PM (#39802304)
The bottleneck was mostly in writing the SQL query, which took a lot of time due to the messiness/complexity of the data
Re:Show HN: DaLMatian – Text2sql that works on Friday March 22, 2024 @02:59AM (#39788036)
Thanks we will rethink how the product is presented on our website!
Re:Show HN: DaLMatian – Text2sql that works on Friday March 22, 2024 @02:57AM (#39788031)
Right, we are not open source, but the IDE is free to use. The Slack integration and other product offerings that are in the works will be in the 'premium' version of the product that's sold to enterprise
Re:Show HN: DaLMatian – Text2sql that works on Friday March 22, 2024 @02:53AM (#39788020)
There are other product additions in the works, like hooking it up to your locally opened Slack. The plugin would be limiting
Re:Show HN: DaLMatian – Text2sql that works on Friday March 22, 2024 @02:51AM (#39788004)
If you open a .sql file into the workspace, the queries in that file will be auto parsed and used as context for Q&A. If you're willing, would love to help debug - could you email [email protected]
Re:Show HN: DaLMatian – Text2sql that works on Friday March 22, 2024 @02:43AM (#39787979)
We have not come across any benchmark dataset that's actually worth evaluating on because the questions are not representative of real world enterprise problems. They don't reflect the degree of context needed to answer domain/business-specific questions accurately.
Re:Show HN: DaLMatian – Text2sql that works on Friday March 22, 2024 @02:31AM (#39787925)
I used to get slammed with so many requests that my boss had to tell the sales team to reduce the number of questions and only ask highest priority ones. Analytics teams serve a lot of different teams in an org, and the requests can really pile up. I was basically a bottleneck, which was a lose-lose for me since I was slammed with work and for business stakeholders too since they had to either wait a long time for responses or were limited in what they could even ask.
Re:Show HN: DaLMatian – Text2sql that works on Friday March 22, 2024 @02:24AM (#39787895)
1 - yes our current solution does require you to be allowed to use ChatGPT/OpenAI. Unfortunately the accuracy using smaller models (even GPT-3.5) is poor. We don't see a local model (which will be much worse than GPT-3.5) even with fine tuning being anywhere close to good enough (would also require a really large number of queries). So we are relying on GPT-4 for now.

2 - agreed the background isn't why anyone should adopt a tool, just wanted to share our story. I would add that creating a good wrapper can actually be quite challenging, need to synthesize many pieces under constraints like memory, compute, speed, accuracy.

Re:Show HN: DaLMatian – Text2sql that works on Friday March 22, 2024 @02:12AM (#39787856)
Just want to clarify that OpenAI does not train on the query code and schema info we send via API. It's equivalent to using https://chat.openai.com/ setting "Improve the model for everyone" (previously "Chat history & training") to off in Data Controls.

Show HN: DaLMatian – Text2sql that works

https://www.dalmatian.ai/download

Hey HN, we've built DaLMatian, a text2sql product that meets the needs of data analysts working with enterprise data. We built this app because as a data analyst at an enterprise I could not find a text2sql product that was (1) actually useful for my day-to-day and (2) easy to set up on my computer. Existing products either fall apart when tested on gnarly enterprise data/queries or require...

( Read More... | 28 comments | 19 points | on HN )