News for hackers. Stuff that matters.
Re:Ask HN: How are you improving your use of LLMs in product... by GPUboy on Wednesday July 19, 2023 @12:31PM (#36789112)
Cheat layer has thousands of global users running our agents, and we started building agents over a year ago. We were the first to get approved by openAI to sell GPT-3 for code generation in summer 2021. We're growing MAU 40% MOM and MRR 50% MOM. We won #1 product of the day 4/1 on Product hunt, and users can pay to access our no-code agents that uses a unique project management interface to build actually useful agents for businesses.

Some improvements we've made: 1) Agents use a global supabase to store "successful" code from tasks so new users don't have to go through regenerations to find the same solution.

2) Each user has their own supabase to grow valuable business data, and our autonomous sales agents can now maintain conversations across email, sms, and voice calls. This is our defensive moat vs Microsoft, since they can't hire overnight to copy this data and can't sell it back to customers. This allows agents to implicitly learn things like "high converting copy" and our new users don't have to start from 0 when starting a new business. We even had a customer reply to one agent 5+ times: https://twitter.com/CheatLayer/status/1676959310562349059?s=...

3) We designed a new 'guidelines' framework that dynamically constructs the system prompt based on context, which allowed us to push the limits in specific use cases.

4) A library to divide up tasks and solve them as a hierarchy https://www.youtube.com/watch?v=FFb59WmQoFU

5) Most exciting recent update was adding voice synthesis to our autonomous sales agents, who can continue the conversations across email, sms, and now voice. https://www.youtube.com/watch?v=2s4iQ_joToc

6) We setup a testing framework for the guidelines I mentioned above to make sure we constantly iterate. In this way we can also prove the old GPT-4 model is definitely worse and some automations don't work at all if you take full advantage of the new model.

Check out our weekly live streams for all the updates, since there's been a lot since we've been working on this since 2021: https://www.youtube.com/@CheatLayer/streams

Re:Ask HN: How are you improving your use of LLMs in product... by VSerge on Wednesday July 19, 2023 @12:51PM (#36789465)
Hi, from the tweet exchange in 2, it seems like your agent bought something, as opposed to selling it. Am I missing something?
Re:Ask HN: How are you improving your use of LLMs in product... by mpeg on Wednesday July 19, 2023 @01:03PM (#36789685)
You're not missing anything, if anything that's a great example of why you shouldn't use an LLM to handle your whole sales pitch without supervision, the person being pitched to is clearly not interested in the product but asks if they want to make a video with them (probably as a form of paid content) and at one point the LLM starts to reply like it is the one selling the video...

Awful example to use, in the end the LLM here would have ended up buying a video for $80

Re:Ask HN: How are you improving your use of LLMs in product... by GPUboy on Wednesday July 19, 2023 @01:58PM (#36790528)
Actually we've been using automated GPT-4 powered responders in production to get demos on auto-pilot for months, and we have the data to prove it. It's significantly better than Apollo.io because we actually used it in conjunction with Apollo before we switch to completely personalized outbound.

We have lists of demos and sales closed with this already I can happily show anyone who wants to see over a call.

Personalized agents are the UPGRADE to the automation our customers have been using for months: https://drive.google.com/file/d/1DPPjnMvRJI0N__PrrbBFi3UDsD8...

Seems odd that people would get irritated by this since all startups could use this

Re:Ask HN: How are you improving your use of LLMs in product... by mpeg on Wednesday July 19, 2023 @03:58PM (#36792298)
Look, I don't mind at all when I get a cold email that is thoughtful and well targeted. But LLM written emails are very obvious, and irritate me to no end because I feel like this is the future of marketing – endless waves of automatically written spam, generative voice phone calls, and who knows what else

I think LLMs have a huge potential for changing the way we work, and creating hybrid human/AI interfaces where the AI does the boring parts and the human becomes more productive as a result – but with these agent systems it feels you're aiming to completely replace the human and instead go for quantity over quality

At Salesforce and Experian I worked with some exceptional sales people, who would be very likeable and turn a cold email into a $1m/yr contract. These people would be very smart and leverage their network, background and expertise. Maybe AI can completely replace them one day, but in the meantime I think the real money in this area is building tools that one of these professionals can use to 2x, 3x, 10x their workflow.

Re:Ask HN: How are you improving your use of LLMs in product... by GPUboy on Wednesday July 19, 2023 @04:32PM (#36792803)
We had a sales team who did exactly this before these agents, and they got demos on auto-pilot without touching apollo.io or salesforce, because we used GPT-4 to respond like they would using their history of responses. You can set this up yourself using the webinar I linked above. We setup a calendly link with a round-robin scheduler to the whole sales team, and they constantly get demos on auto-pilot.

Parker Harris was actually one of my early mentors, because my first co-founder was the first investor in Salesforce(Halsey Minor). A lot of his advice went into building this.

We're not selling to the CEO, we're selling to the VP of sales who can now a/b test this, or the startup founder who can't afford a sales team yet, and I'm very sure a large percent of reps will fall below the line vs the auto-responders. We increased the number of demos our sales team saw with just GPT-4 auto-responding, then we doubled the total reply rate with personalized outbound. It makes sense that it wouldn't have the exact same reply rate. I'd be happy to give you access to get your insight on the direction we should go if you're interested.

Re:Ask HN: How are you improving your use of LLMs in product... by GPUboy on Wednesday July 19, 2023 @12:56PM (#36789554)
Hi yes this is correct, the lead list 1500 youtube partners to form partnerships and sponsorships, and it actually formed over a dozen deals. You can see the setup here: https://www.youtube.com/watch?v=uj-gH4f6RUM

This agent was built for influencer partnerships to help us growth hack

We're also using it to sell Cheat Layer, and getting 10% reply rates vs the cold static outbound in Apollo.io