No datas are sent to any 3rd party.YoBulk is self hosted.Your personal information is stored on your database.Feel free to ask any data security related question.

How are you running GPT locally? OpenAI is a third party.

I understand that this product does use OpenAI’s API, but I just want to stress that OpenAI doesn’t own GPT, it did not create GPT, and GPT is not something usable exclusively through OpenAI.

A priori, there’s no reason they couldn’t include their own GPT model that also lives on your own server.

According to Wikipedia, OpenAI did create GPT[0] and the GPT-3 model[1], which this tool depends on, is exclusively available through the OpenAI API. It's true that some open alternatives to GPT-3 have been popping up but I believe they are still behind in terms of quality.

[0] https://en.wikipedia.org/wiki/Generative_pre-trained_transfo...

[1] https://en.wikipedia.org/wiki/GPT-3

Thank you for adding a correction. I shouldn’t have been so flippant: OpenAI did not invent transformers, GPT-3 is an iteration of the BERT architecture, which is an encoder, whereas GPT is a decoder model.

Regardless, a GPT model is not a proprietary technology you must get from OpenAI. You’re right that this tool uses OpenAI’s API, but this isn’t necessarily implied by the use of GPT.

> Thank you for adding a correction. I shouldn’t have been so flippant: OpenAI did not invent transformers

BTW you answer structure is very similar to ChatGPT’s when you point it to a mistake.

Well to be fair GPT-3 was trained on internet comments!

@l33t233372 you are absolutely spot on.

We are using OpenAI to only to create schema, column matching and Regex generation.No CSV data are sent to open AI..

So you ask OpenAI to generate a schema and computer code that will clean input data to conform to that schema, and then run the user's data through that program? Is it possible for users to obtain and audit the generated code for correctness, performance, etc.? How do you prevent things like the AI from generating catastrophically-backtracking regex, O(N!) algorithms, or outright mistakes?

@nerdponx Open AI only generate a validation schema.We have our own schema generation engine for any custom validation which can be used where Open AI is not able to understand or generate correct schema.Yes you are absolutely right.Open AIs output is not always right.We have integrated a JSON parser which validate Open AI output and currently developing a regex parser which validates Open AIs output.Hope it answers your query..Happy to understand your pain point more.

Interesting, thanks for explaining how it works. Would it also be possible to construct one of these JSON schemas by hand, without OpenAI? The core data cleaning system sounds like just as interesting a piece of technology as the AI schema generation.

@nerdponx Yes YoBulk provides a way to write JSON schemas by hand..We have added some custom keyword like validate which is not defined in JSON standard..But our validation engine can understand that..You can pass a javascript function through validate keyword..It's a game changer.. "first_name":{ "type":"string", "format": "first_name", "validate": "(x) => {\r\n let regex = new RegExp(\"([a-z][a-zA-Z]*)\");\r\n return regex.test(x);\r\n }" },

This is a very interesting combination of technologies. Thank you again for explaining how they work! I tend to prefer open-source solutions for my own work, but I can see this being highly useful and valuable for many businesses.

Thanks keep following YoBulk..We will keep on doing some path breaking work in CSV importing space.

So what exactly is sent to OpenAI?

@justeleblanc we do auto column matching through open AI..Ex:If you have defined a template with column name Date of Joining and the CSV data donor upload a CSV with a field DOJ,then your validation engine will skip the importing as it does not match with expected column names..Here Open AI comes handy..It gets the context and smartly identify that DOJ is same as Date of Joining and does the importing..You can go through https://doc.yobulk.dev/YoBulk%20AI/AI%20usecases to understand more on AI usecases of YoBulk.

Sorry for the offtopicness but can you please email me at [email protected]?

Cheers to OP for whatever magic this interaction has unlocked!

I resort to comments like that when I don't have another way of contacting the user. If it happens in a Show HN thread, it's probably that I want to ask if they've considered applying to YC with the project. People often underestimate what YC might be interested in. Open-source startups are a big part of what gets funded these days. I particularly love it when startups make it in to YC through HN—it's great for both YC and HN, and if a big success ever came out of it, it would easily fund HN for another century :)

If I post a please-email-me outside of a Show HN thread, it might be anything! but the most common reason is that I want to send them a repost invite for some cool article they posted long ago. Invited reposts make it into the second-chance pool (see https://news.ycombinator.com/pool and https://news.ycombinator.com/item?id=26998308).

it looks like the only thing openai does is generate a schema from a specific hand written prompt from cursory overview.

There are multiple use cases where we are using Open AI Example:uploaded CSV's column to template column string matching

That makes sense. It would be good to add this information to the README.

Sure We will add it in README..we captured it in our documentation.Please have a look. https://doc.yobulk.dev/YoBulk%20AI/AI%20usecases