News for hackers. Stuff that matters.
Re:Building your own deep learning computer is 10x cheaper t... by unbearded on Tuesday September 25, 2018 @12:26PM (#18067766)
Unfortunately, the fight goes even further than that when you go against the cloud.

Last week I was in an event with the CTOs of many of the hottest startups in America. It was shocking how much money is wasted on the cloud because inefficiencies and they simply don't care how much it costs.

I guess since they are not wasting their own money, they can always come up with the same excuse: developers are more expensive than infrastructure. Well... that argument starts to fall apart very quickly when a company spends six figures every month on AWS.

I'm on the other extreme. I run my company stuff on ten $300 servers I bought on eBay in 2012 and put inside a soundproof rack in my office in NJ, with a 300 Mb FIOS connection using Cloudflare as a proxy / CDN. The servers run Proxmox for private cloud and CEPH for storage. They all have SSDs and some have Optane storage. In 6 years, there were only 3 outages that weren't my fault. All at the cost of office rent ($1000) + FIOS ($359) + Cloudflare costs and S3 for images and backups.

With my infrastructure, I can run 6k requests per minute on the main Rails app (+ Scala backend) with a 40ms response time with plenty of resources to spare.

Re:Building your own deep learning computer is 10x cheaper t... by spullara on Tuesday September 25, 2018 @01:23PM (#18068374)
Only a Rails developer would think 6k requests per minute with 40ms latency is reasonable with all that hardware. If you rewrote it you probably only need 1 server but you will probably make an argument about how developer time is more valuable :)
Re:Building your own deep learning computer is 10x cheaper t... by unbearded on Wednesday September 26, 2018 @03:35PM (#18079476)
The 6k/req with 40ms is just at the front door.

I'm talking about a real application here. With 100s of database and API calls on each web page load. I could make the whole thing in Golang or Scala and that would be at least one order of magnitude faster. But then I would have to throw away all the business knowledge that was added to the Rails app.

For instance, the slowest API call on the 40 ms is one that hits an ElasticSearch cluster with over 1 billion documents and is made on a Scala backend using Apache Thrift. There's a lot of caching but still, long tail and customization will kill caching at the top level.

Re:Building your own deep learning computer is 10x cheaper t... by spullara on Thursday September 27, 2018 @03:17AM (#18082589)
It sounds very similar to the Rails frontend I helped replace at Twitter — no business logic was thrown away, it was done without any loss in application fidelity. We did get approximately 10x fewer servers with 10x less latency after porting it to the JVM as a result. However, without the latency improvement, I don't think we should have done it. Fewer servers just isn't as important as the developer time necessary to do the change as you just pointed out. Just as using the cloud to simplify things and reduce the amount of effort spent on infrastructure is the main driver of adoption. There is clearly a cross-over point though where you decide it is worth it. The CTOs you are speaking of are making that choice and it probably isn't a silly excuse.
Re:Building your own deep learning computer is 10x cheaper t... by ghostbrainalpha on Tuesday September 25, 2018 @01:49PM (#18068682)
I get Rails people are crazy biased, but I still don't understand your argument.

What type of business that someone would basically run out of an office closet like this would need to service more than 6k requests per minute?

Re:Building your own deep learning computer is 10x cheaper t... by spullara on Thursday September 27, 2018 @08:13PM (#18089981)
I was talking about reducing the number of servers to serve 6k/minute. No idea if that was a synthetic benchmark or the current load on the service.
Re:Building your own deep learning computer is 10x cheaper t... by captain_crabs on Tuesday September 25, 2018 @01:36PM (#18068536)
It's a matter of dosage. You might be talking about how another tech stack would perform better at this metric, but the price of that is the company would have been simply unable to ship anything in the time they could afford.

Swinging the conversation beyond the dosages of either side doesn't produce interesting insight

Re:Building your own deep learning computer is 10x cheaper t... by unbearded on Wednesday September 26, 2018 @03:37PM (#18079486)
Exactly. I'm pretty sure if I developed the whole thing in C, it would be really fast. But I would be the slowest part of the development process.
Re:Building your own deep learning computer is 10x cheaper t... by therealdrag0 on Wednesday September 26, 2018 @03:38PM (#18079498)
10 servers does seem a bit much. But who knows what their hardware specs are.
Re:Building your own deep learning computer is 10x cheaper t... by treis on Wednesday September 26, 2018 @12:23PM (#18077551)
>In 6 years, there were only 3 outages that weren't my fault.

How many where there that were your fault? And of those, how many would have been avoided by using Heroku?

>All at the cost of office rent ($1000) + FIOS ($359) + Cloudflare costs and S3 for images and backups.

What about the time spent creating and maintaining this infrastructure?

Re:Building your own deep learning computer is 10x cheaper t... by unbearded on Wednesday September 26, 2018 @04:00PM (#18079692)
I'm not sure how many outages I could avoid using Heroku, but I guess at least a few.

One time I was using Docker for a 2 TB MongoDB and it messed the iptables rules. I notice everything slow for a few days until the database disappeared and when I logged in to check, there was a ransom note.

I flew from Boca Raton to NJ to recover the backup and audit if that was the only breach. That was the longest outage.

Like Rome, this infrastructure was not created in one day. Adding Optane storage is something more recent, for example. Or adding a remote KVM to make easier to manage than dealing with multiple DRACS, which I did after a moved to Florida.

But I'm not against using the cloud. I'm actually very in favor. What I'm against is waste.

In my case, being very conservative with my costs and still have a lot of resources available allowed me to try and keep trying many different ideas in the search for product/market fit.

Re:Building your own deep learning computer is 10x cheaper t... by Rafuino on Tuesday October 09, 2018 @02:44PM (#18178700)
Are you using the Optane SSDs for Ceph as you mentioned in your original comment? Curious what benefit you're seeing if not for Ceph, and if for something else, would you mind commenting? We're looking to share best practices with the community we're building over at acceleratewithoptane.com on how to take advantage of Optane SSDs.
Re:Building your own deep learning computer is 10x cheaper t... by marme on Tuesday September 25, 2018 @04:07PM (#18070029)
If you buy equipment you can only deduct its depreciation costs every year. If you pay for services you can deduct the full cost from your taxes.
Re:Building your own deep learning computer is 10x cheaper t... by mangymutt on Tuesday September 25, 2018 @04:27PM (#18070211)
Anyone who thinks spending $10 instead of $1, so that you can book the expense (presumably for a tax write-off which might save you 30%... MAYBE?) needs to stay away from Finances.
Re:Building your own deep learning computer is 10x cheaper t... by anoncoward111 on Wednesday September 26, 2018 @06:10AM (#18074441)
Correct, unless the org doesn't have the money to make all the capex upfront
Re:Building your own deep learning computer is 10x cheaper t... by karmajunkie on Tuesday September 25, 2018 @04:16PM (#18070115)
You can typically take the full depreciation in the year you purchased the equipment under IRS Section 179, up to a limit which varies depending on which way the wind is blowing in Congress. For 2018 the limit is $1MM. Whether or not its more beneficial to you to take the depreciation over time is a question for your accountant—technically if you later sell the equipment you're supposed to recapture the revenue from the sale for tax purposes.