News for hackers. Stuff that matters.
Re:Cost Per 1M tokens Of Self Hosting Llama-3 by Melatonic on Friday June 14, 2024 @07:10PM (#40686034)
Read the other comment and also immediately thought of Cloudflare tunnel instead - is there a reason you chose that? Wondering if I should do the same with my old Titan XP (probably slower than your 3070 but it does have 12gb of vram)
Re:Cost Per 1M tokens Of Self Hosting Llama-3 by czhu12 on Saturday June 15, 2024 @10:23AM (#40690007)
No reason other than that cloudflare is quite a nice reverse proxy and I already use it for managing my DNS's. Thus far, I've not noticed any issues with it, other than when my home wifi goes down.