Ask HN: What do you use for ML Hosting

Re:Ask HN: What do you use for ML Hosting by smoldesu on Tuesday May 02, 2023 @03:10PM (#35792622)

I'm currently running a Discord bot with a 7B model off a free Oracle Ampere instance with their Pytorch Accelerated[0] image. It's not terribly fast, but totally usable for group chats that want to interrogate an AI. If you're doing some sort of offline processing or non-time-imperative operation, something like this might be worth looking into.

[0] https://cloudmarketplace.oracle.com/marketplace/en_US/adf.ta...

[ Parent | on HN ]

Re:Ask HN: What do you use for ML Hosting by password4321 on Tuesday May 02, 2023 @03:25PM (#35792823)

Does that use all 4 OCPUs / 24GB memory?

[ Parent | on HN ]

Re:Ask HN: What do you use for ML Hosting by smoldesu on Tuesday May 02, 2023 @03:28PM (#35792858)

It can! I'm using 2 cores per request though, and I've got memory to spare.

[ Parent | on HN ]

Re:Ask HN: What do you use for ML Hosting by aditya on Tuesday May 02, 2023 @03:34PM (#35792938)

what discord bot? :)

[ Parent | on HN ]

Re:Ask HN: What do you use for ML Hosting by bootsmann on Tuesday May 02, 2023 @03:47PM (#35793117)

Oracles free forever tier is so underrated. They just throw half a startup at you at no cost.

[ Parent | on HN ]

Re:Ask HN: What do you use for ML Hosting by mathewvp on Wednesday May 03, 2023 @04:19AM (#35799536)

They also shutdown your instances after a while and ask you to upgrade to paid tier.

[ Parent | on HN ]