News for hackers. Stuff that matters.
Re:How to run Llama 13B with a 6GB graphics card by postalrat on Sunday May 14, 2023 @01:09PM (#35939438)
I doubt you've ever worked with people if you think that's unusable slow
Re:How to run Llama 13B with a 6GB graphics card by bcrosby95 on Sunday May 14, 2023 @01:28PM (#35939608)
The computer doesn't ask for annoying things like a paycheck or benefits either.
Re:How to run Llama 13B with a 6GB graphics card by mejutoco on Sunday May 14, 2023 @04:18PM (#35941059)
Money upfront and a small salary in the form of electricity bills.
Re:How to run Llama 13B with a 6GB graphics card by nsteel on Monday May 15, 2023 @06:08AM (#35945877)
My Windows computer is always demanding updates and reboots. And when it goes to sleep it sometimes doesn't wake up. It's quite annoying.
Re:How to run Llama 13B with a 6GB graphics card by ineedasername on Monday May 15, 2023 @05:05PM (#35954475)
Meh, universal healthcare for PC's can easily be avoided by denying maintenance until they die and a new crop is purchased. At least at scale. For any individual user the friction of switching hardware may still incentivize some minimal checkups, i.e., keep windows update & defender services running and let them reboot the system no more than once every two two months.

Figure the local router port-forwarding will protect against the most obvious threats and otherwise hope your personal BS filter doesn't trojan in some ransomware. If it does & it's a person pc then wipe (more likely buy) a new machine, lose some stuff, and move on. If it's a corporate pc, CYA & get your resume together.

As my own CYA: These are not my own recommended best practices and I don't advocate them to anyone else as either computer, legal, or financial advice.

Re:How to run Llama 13B with a 6GB graphics card by ineedasername on Monday May 15, 2023 @04:57PM (#35954373)
True enough. I wonder how many poems (or whatever) per hour Hallmark expects of its human workers to be close to production-ready pending editorial review?

Is 10 reasonable, with maybe 1 or 2 truly viable after further review? That would be roughly 5 mid-range laptops of my type churning them out for 8 hours a day. Maybe 2 if they're run 24/7. Forget about min/maxing price & efficiency & scaling, that's something an IT major-- not even Comp-Sci focused-- could setup right now fresh out of their graduation ceremony with a fairly small mixture of curiosity, ambition, and google (well, now, maybe Bing) searching.

There are countless boutique & small business marketing firms catering to local businesses that could have their "IT Person" spend a few days duct taping something together that could spit out enough material to winnow wheat from chaff to produce something better-- in the same period of time-- than human or AI could produce alone.

I have a focus in a comp-ling background (truly ancient by today's standards especially) enough that I see the best min/max of resources as being equivalent to-- in the the translation world-- as "computer-aided human translation" as a best practice. Much better than an average human alone, and far cheaper than the best possible that can be provided by a small dozens of humans.