I mean it does send a screenshot of your screen off to a 3rd party, and that screenshot will most likely be used in future AI training sets.

So... beware when you use it.

OpenAI claims that data sent via the API (as opposed to chatGPT) will not be used in training. Whether or not you believe them is a separate question, but that's the claim.

Beware of it seeing a screenshot of my music set? OpenAI will start copying my song structure?

You can turn it on and off. Not necessary to turn it on when editing confidential documents.

You never enable screen-sharing in videoconferencing software?

I completely agree. A huge business with a singular focus isn’t going to pivot into the music business (or any of the myriad use cases the general public throws at it). And if they did use someone’s info, it’s more likely an unethical employee than a genuine business tactic.

Besides, the parent program uses the API, which allows opting out of training or retaining that data.

Yes this makes perfect sense. As we know, businesses definitely do not treat data as a commodity and engage in selling/buying data sets on the open market as a "genuine business tactic". Therefore, since the company in question doesn't have a clear business case for data collection currently, we can be sure this data will never be used against our interests by any company.

So glad when I saw this, thanks for sharing this! It was exactly music production in Ableton was the spark that lit this idea in my head the other week. I tried to explain to a friend that don't use GPT much that with Vision, you can speed up your music production and learn how to use advanced tools like Ableton more quickly. He didn't believe me. So I grabbed a Ableton screenshot off Google and used ChatGPT -- then I felt there had to be a better way, I realized that I have my own use-cases, and it all evolved into this.

I sent him your video, hopefully he'll believe me now :)

You may be interested in two proof of concepts I've been working on. I work with generative AI and music at a company.

MidiJourney: ChatGPT integrated into Ableton Live to create MIDI clips from prompts. https://github.com/korus-labs/MIDIjourney

I have some work on a branch that makes ChatGPT a lot better at generating symbolic music (a better prompt and music notation).

LayerMosaic allows you to allow MusicGen text-to-music loops with the music library of our company. https://layermosaic.pixelynx-ai.com/

Oooh. Yes, very interested in MusicGen. I played with MusicGen for the first time the other week and created a little script that uses GPT to create the prompt and params which is stored to a text file along with the output. Let it loop for a few hours to get a few 100 output files that allowed me to learn a bit more about what kind of prompts that gave reasonable output (it was all bad, lol!)

Oh LayerMosaic is dope. I'm not entirely sure how it works, but the sounds coming out of it is good -- so you have me intrigued! Can I read more about it somewhere, I might have a crazy idea I'd like to use this for.

My brain read midjourney until I clicked on the GH link. What a great name, MIDIjourney!

Is it just me or is it incredibly useless?

"Here's a list of effects. Here's a list of things that make a song. Is it good? Yes. What about my drum effects? Yes here's the name of the two effects you are using on your drum channel"

None of this is really helpful and I can't get over how much it sounds like Eliza.

I made that video right at the start but since then I've asked it for example what kind of compression parameters would fit with a certain track and it could explain to me how to find an expert function which I would have had to consult a manual for otherwise.

Yeah I thought the same. Ultra generic advice and no evidence it has actually parsed anything unique or useful from the user’s actual composition.

I made another one: https://www.youtube.com/watch?v=zyMmurtCkHI

In the one I posted I was just so amazed how well it worked and didn't really try anything useful. In this video you can see it giving me quite good advice on how to make a bassline dubby and how to carve frequencies out of the kick drum to make space for the bass.

It also looks at spectrograms and gives feedback / takes them into account. I'm pretty amazed.

Did you change the GPT Vision system prompt at all? I wonder if changing it to state getting help with specifically Ableton, and maybe some guidelines around what kind of help you want could make it better?

No. But I found it good enough as it is

I just made a video where I test it with a proper use case. It helps me find effects to make a bassline more dubby and helps carve out frequencies in the kick drum to make space for the bass.

https://www.youtube.com/watch?v=zyMmurtCkHI