Your GPU infra is a local server?

not local, it's colocated, but I specced and built it specifically to do inference at scale

Can you describe your GPU setup? I have been super interested to get some coloc space, but have a ton of questions.

Nothing fancy, just bought a motherboard from SM, and a 4U case. I have a couple of boxes with 6 NVIDIA 3080s and recently upgraded to a used super micro with 8 A100s

Colocation wise, you pretty much buy the space, show up with the server and rack it yourself into a locked cabinet, not much else

That is sick, I wanna build out something of a similar size (sans gpus). How much does colocation cost in your area?

I don't mean this in a critical way to you but, man, this makes me so sad to think about. Models making money from lonely people so desperate for a connection that they pay to chat with someone and then they don't even end up chatting with a human at all.

25% of US college students on antidepressants. Up 64% from 2020. People staring at screens 3/4 of their life. Probably nothing. Technology making the world a better place.

It's not really that sad. Young guys with a bit of disposable income could do a lot worse than subscribing to someone's onlyfans and chatting with a bot. I don't charge subscribers directly and don't do any pay-per-message scheme, it's free for the user (provided they're subscribed to my client's page).

The conversations aren't even overtly sexual in nature, mostly just guys sending "hey hope you're having a good day" during work hours. Llama is good, but honestly I think anyone would probably know they're talking to a bot after a few days, but they still keep talking to it anyways.

> The conversations aren't even overtly sexual in nature, mostly just guys sending "hey hope you're having a good day" during work hours.

I actually think that's worse.

Ed: but props to you for filling a technology niche.

I think it's worse too.

They aren't even getting any sexual gratification from it. Just lonely little unrequited missives. Hoping and failing to make a connection and not even realizing they haven't.

How many parameters LLama do you use? What framework do you use to launch it?

How much are you saving running your own GPU infra? What are the total monthly costs for the service?

Running costs are about 30% of what an equivalent AWS GPU setup would be

Interesting! How it works? I am a bit confused.

I am always in awe of people that simply solve a problem (instead of having to get all esoteric about it) but I cannot understand what problem chat solves when the subject is more interesting to look at than speak to. :)

[edit] 20 jpegs into a 150MB LoRA

For chat, I'd say 70% of the messages are non-sexual in nature. People just want to have someone to talk to during their work day. Not all, but quite a few models have been open about how their chat during the day is AI generated.

Shame on you.

You can moralize and pearl clutch as much as you'd like

I could see why'd you say that, but I think it's a case of hate the game, not the player. We have crafted a society that has a dearth of compassion and companionship, that's the unfortunate reality.

There are lots of sad, lonely guys out there and that's not GaryNumanVevo's fault, he's just catering to them.

Willing players are what create the game.

Whether or not GaryNumanVevo runs his service there are going to be lonely men out there. I don't know how we'd even begin to fix that.

I do feel like "Don't hate the player, hate the game" is used to shield folks from criticism about actively participating in a societal ill in exchange for money. I realize that sex work and its presence in society goes all the way back to antiquity, and technology is only going to make it more prevalent. Some folks with compartmentalized morality may as well think "Life is short. Why work a crappy 9-5 when I can make a ton of money doing this other stuff instead?" I get the appeal.

That said, if shame were brought back into the mix on a large scale, a lot of this profiteering would evaporate away, and push this industry underground.

Please forgive my ignorance, but what are the obvious reasons for you not having a website?

Never really needed one, I started the business because a few friends asked if a solution existed and all the marketing has been word of mouth. We send out a weekly analytics email to clients breaking down subscriber usage and some topics analysis, but nothing too complicated.

How is that you promote Gary Numan.