News for hackers. Stuff that matters.
Re:Will Amazon Kill FedEx? by graycat on Thursday September 01, 2016 @06:39AM (#12404220)
Gee, for a while I was Director of Operations Research at FedEx. My office was next to that of FedEx founder, COB, CEO F. Smith.

Poor Smith: One night at about midnight, I was in my office getting in a little violin practice before heading back to my rented room for some sleep (my wife was still in her Ph.D. program at Johns Hopkins in MD). As I left to get some sleep, I saw Smith in his office. He had had to listen to my violin practice of scales, intervals, etc.!

The first good thing I did for Smith was write some software for scheduling the fleet. The BoD was seriously asking if such scheduling could be done at all. One evening R. Frock (he has a book on FedEx) and I used my software and produced a schedule for the full, planned operation. Smith's remark was "Solves the most important problem". Our two BoD representatives of BoD Member General Dynamics said "It's a little tight in a few places, but it's flyable". The BoD was happy, and crucial funding was enabled.

But by the night of the violin practice, I was working on better scheduling. Right, I was encountering the question of P = NP. So, I had chatted with G. Nemhauser about integer linear programming set covering.

Later I went for my Ph.D. in applied math and, in the studies, kept in mind the problems I'd seen at FedEx. So, e.g., model various cases of arrivals as a Poisson process -- from an axiomatic derivation of that process or just from the renewal theorem. Use some stochastic optimal control for some of the operational issues. Have some much more accurate cost estimating of the airplane operations and include those in the integer linear programming, etc.

Gee, for the last 10 years, I didn't know that Amazon was doing so much in such logistics. No way can they be avoiding some optimization problems for their logistics without wasting significant bucks. For a lot they were doing, I'd been there, done that, went to grad school for it, written software for it, etc. Could have helped them!

Scattering warehouses, sort centers, fulfillment centers, ships, airplanes, trucks, bicycles, drones, etc. all around and changing all that ASAP over time has to encounter some severe challenges in applied math and corresponding software, especially having those two keep up with the operational changes.

I know; I know; if assume that the operations will be big enough, then there are some approaches to those discrete optimization problems that use just some continuous techniques that are much easier. In that case, maybe for a while, during the rapid growth period, go ahead and don't try very hard on the optimization, scatter around, and anticipate that, when reach the target size, nearly all those warehouses, sort centers, etc. will be close enough to something optimal.

Then I wonder: What is Wal-Mart doing in response? They have a lot of warehouses, trucks, and stores and a Web site for on-line ordering? Right, they bought Jet.com. But anything else?

Fundamentally I begin to wonder about Amazon and airplanes: For FedEx, sure airplanes are crucial. But for Amazon? I wonder.

Why? What's the difference?

For FedEx, hour by hour, a huge fraction of what FedEx receives and ships is different, literally never done before, that is, could not have been stocked in a warehouse. E.g., FedEx may receive a legal brief, some medical samples, a custom 3D printout, etc. That was the first and last time those particular items were ever shipped. The variety of what FedEx ships is beyond belief -- stand in the sort center in Memphis and watch the stuff go by and have to give up on any simple overview.

For Amazon, all they ship is what's for sale on their Web site. So far, next to none of that stuff is perishable. And, for nearly all of it, the time, hours, days, weeks, from when it is created by the original manufacturer to when it is needed in an Amazon warehouse doesn't much matter.

So, in comparison with FedEx, the variety is much less. So Amazon has an easier time just stocking nearly all their reasonably popular items at each of their local warehouses and delivering to those warehouses by the cheapest way possible, typically 18 wheel trucks.

Okay, maybe Amazon is selling some item, expects to sell only 15 of them in the next month, but has 60 local warehouses. So, can't stock the item at each warehouse. So, for slow moving items, have to stock in some central warehouse and ship from there via airplane. So, have 60 planes, and each day ship from the central warehouse for slow moving items to each of the 60 local warehouses all the slow moving items that warehouse needs.

Right, maybe have fewer than 60 planes, often have one plane serve more than one warehouse, on a given day maybe have some planes make more than one trip from the central warehouse, and each day, given the loads to be moved from the central warehouse to each of the 60 local ones, decide which planes go to which warehouses in what order -- back to integer linear programming set covering again. Okay.

How does that set covering work? Okay, just enumerate all reasonably efficient trips a single airplane might do. If have several airplane types, have to do that separately for each type. Assume that if an airplane stops at a warehouse, then it brings everything that warehouse needs (one warehouse that needs more than one airplane can carry would be a special case, easier in some ways). Then take those candidate airplane trips and pick the ones that move all the loads, within the performance limitations of the airplanes, in the times required and that give the least total operating cost. That last part is the optimization and 0-1 integer linear programming.

But, still, the variety for Amazon is much less than that for FedEx.

Actually, FedEx has long had a warehouse in Memphis. So, shipping customers would use trucks to put their stock in that warehouse, and FedEx would pick from that stock and ship via FedEx planes the orders that had arrived and accumulated from end-user customers. So, in that case, a truck was used for the trip to Memphis and an airplane, for the trip from Memphis.

Airplanes are strange beasts: Generally, if can keep an airplane flying nearly full lots of hours each day, then can have a license to print money. But if have an airplane sit on the ground or, worse, flying with not much load, then have a path to going broke. How much usage FedEx can get from an airplane is well understood, at least by Smith. But I question if Amazon can get enough usage out of airplanes.

Of course, maybe by now, beyond what was in the OP, Amazon has plenty of data to know that they can put a big warehouse there in southern Ohio for their slow moving items and ship those items to local warehouses and distribution centers as orders arrive from the Amazon Web site. Maybe. But, again, for their fast moving items, just stock the local warehouses and distribution centers by 18 wheel trucks -- no expensive airplanes involved.

Gee, in my Ph.D. program, I took courses in optimization, stochastic processes, facility location, multi-objective optimization, etc. from world-class people. I never got a call from an Amazon recruiter! I wasn't that difficult to find: The guy who taught the course in multi-objective optimization and was one of my Ph.D. dissertation advisers was, during a lot of those years of Amazon's growth, President at CMU. The guy who was Chair of the committee that approved my dissertation research was a world-class guy in transportation.

Lesson 1: Go get a Ph.D. for some ways to get approximately optimal solutions to some really complicated NP-complete problems, maybe also with some random elements, and even if a big business has a lot of need for such applied math maybe still won't get an opportunity to apply what studied.

Lesson 2: If want to make money with some applied math and, then, some corresponding computing, then pick own practical problem to solve, do the math to solve the problem (yes, pick the problem so that can use some math to get an advantage in getting a good solution to the problem -- that is, pick the problem and the math together, as a good pair), write the software, and do own startup where are already the CEO. That is, for own future applying math and computing, don't depend on some other CEO. And that's what I'm doing -- my own startup.

Re:Will Amazon Kill FedEx? by uptown on Thursday September 01, 2016 @07:02AM (#12404304)
Can you elaborate?
Re:Will Amazon Kill FedEx? by graycat on Thursday September 01, 2016 @07:12AM (#12404343)
What I wrote is about 8859 bytes and, thus, already "elaborate". What do you want elaborated?
Re:Will Amazon Kill FedEx? by uptown on Thursday September 01, 2016 @07:18AM (#12404366)
I'd be curious to read more about your startup you mentioned at the end. What problem are you solving?
Re:Will Amazon Kill FedEx? by graycat on Thursday September 01, 2016 @08:15AM (#12404641)
Assumption: There is a lot of content on the Internet, and generally people have interests and want appropriate content. The most effective current means are based on keywords/phrases; these means can work really well in some cases but are at best poor for about 2/3rds of content on the Internet, queries people want to do, and content they want to find.

For more: Generally, finding stuff has long been part of the field of information retrieval. That field long ago divided the whole problem into pieces. One big division was between (A) content a person knew existed and had some information about, e.g., keywords/phrases and (B) content that person did not know existed. The old library card catalog subject index is close to case (A) and so are the Internet search engines based on keywords/phrases. For case (B), that is closer to what might be called discovery, recommendation, curation, notification, subscription.

My startup is for being the best for case (B). For case (A), that will remain. What I am doing for case (B) is very different from what has worked well for case (A). Very different: E.g., there is no use of keywords/phrases.

So, my work brings together in a coordinated way (i) the unmet part of the user need to find stuff on the Internet, (ii) a new and very different UI/UX, (iii) some important new data, and (iv), for processing the data to get the results for the users, some new applied math (complete with theorems and proofs, based on some advanced pure math prerequisites) that is the crucial core of the work, that is, the means of getting especially good results for the users.

Some of the efforts seem to have taken some simple techniques and simple, old data sources for recommendation engines and bent the real problem to what these techniques could do; my view is that this is not very promising.

E.g., I believe that my work will do much better for letting people look for, say, movies than what Netflix got from their Netflix Challenge contests. Why? I believe that Netflix formulated the problem poorly, in too narrow a way. What I have done does not fit what Netflix assumed in their problem formulation.

Yes, my work is quite new but, still, supposed to be really easy and intuitive for users. E.g., I hope that a child in a third world country who knows no English will be able to get good with my work in a few minutes alone or sooner watching one example of usage. Right, I might put such an example on YouTube and have my Web site Help page link to that.

There is part of the math that is surprising: One would guess that no such good thing could be true, but it is. Of course, users will not be aware of anything mathematical.

To be a little more clear, my math is original and has nothing to do with anything I've seen in artificial intelligence (AI) or machine learning (ML). To be a little more clear, the AI and ML I have seen do contain some applied math but which is apparently quite narrow. In contrast, on the shelves of the QA sections of the research libraries, there is enormously greater variety, and my work is not on those shelves. So, instead of AI or ML, I just derived some applied (applicable) math.

Then I wrote the corresponding software. Currently there is about 100,000 lines of typing, for both the programming language statements and the documentation in comments in the code. There is much more documentation outside the code. And there is the math, typed into D. Knuth's TeX.

All the code appears to run as intended. The code appears to be essentially all that is needed for at least early production. Currently the code is in alpha test. I'm a solo founder, 100% owner.

The next steps are to complete the alpha test to let me be rock solidly sure the code is correct and ready for production, load some more initial data, do a beta test and get back comments, maybe tweak some parts of the code, say, for the UI/UX, maybe load some more initial data, and, then, just go live in a routine way, get publicity, users, ads, and revenue. Simple.

Right, my math has nothing to do with the social graph, the interest graph, image processing, semantic processing, the semantic graph, speech recognition, natural language processing, singular value decomposition, AI, ML, principal components analysis, support vector machines, collaborative filtering, cluster analysis, intuitive heuristics, neural nets, classification and regression trees (L. Breiman's CART), random forests, gradient descent, etc. Instead, I just derived some new math; I'm supposed to be able to do that; I've done it before on other practical problems of wide variety. It's just some new math -- it has no name like ML. Some aspects of the math, some provable good properties, give me confidence the results for the users will be good.

At first, I will concentrate on users and advertisers just in the US. Then later I may "go international".

If people like my work, then I should have a nice business. Why? From my understanding and assumptions, I have by far the best solution for the problem I am solving; due if only to the math there is essentially no chance that anyone else will do anything at all comparable or competitive; the problem I am solving is so far at best poorly solved; the problem is important to essentially every user of the Internet in the world.

But, right, mostly people don't see the importance of the problem or a solution and are not screaming out for a solution. This is to be expected.

Supposedly Henry Ford said "If I had asked people what they wanted, they would have said a faster horse.". That is, even if people have a big problem, usually they don't envision a solution until they can see the possibility. In Ford's case, that solution involved some cast iron and some gasoline, both quite foreign to people with horses and the problem.

People don't envision my solution now because they don't see the means or the possibility. They will easily envision, and I hope embrace, my solution once they see it. That's what happened when horse owners saw Ford's Model T. It's also what happened with FedEx. And now, Amazon. And for that matter, essentially all of the commercial Internet, e-mail, PCs, laptops, smartphones, etc. For something new, for people to like it, first they have to see it.

Venture capital? Apparently, and I have a lot of evidence, there is nothing in common between my startup and venture capital. That is, across the table, we won't be able to shake hands, ever.

The venture capital people would look at the work I have done to date and say that it and a dime won't cover a 10 cent cup of coffee. They also won't like that I am a solo founder -- formulate your own reasons why. They will hate that there is a role for some original applied math with some advanced prerequisites; they seem to prefer technology that is just routine software. In the patterns they want to use, they have not seen a successful startup like mine. Right, they need to find startups that are exceptional, and to do that they want to use patterns based on successful startups of the past.

They might get interested once my startup has traction (users and/or revenue) significant and growing rapidly, but by then, with me as a solo founder with meager burn rate, I will be nicely profitable, already have a lifestyle business, and have plenty of cash for more servers, etc. So, we can't shake hands across the table because now they believe I am too early and, by the time they are ready to shake hands, I will believe that they are too late.

Is that the elaboration you had in mind?

Re:Will Amazon Kill FedEx? by ufmace on Wednesday September 14, 2016 @05:33PM (#12501377)
Sounds cool, I'd like to check it out when it's ready too.

You probably already know this, but VCs and investors won't like your work now because there's no proof of value. You should avoid them at this point, because you'd only get deals on terrible terms, like pocket change for a majority share to a no-name VC.

Don't talk to investors until you have enough traction and proof of value for them to want to invest in you, and draw up your own business plan for exactly what you intend to do with the money and how much it will expand the business. Then you can have investors coming to you and only accept deals on your terms.

Re:Will Amazon Kill FedEx? by catbird on Thursday September 01, 2016 @01:21PM (#12407134)
This sounds really cool. I am struggling trying to wrap my head around how a recommendation system could be accomplished without some kind of graph connecting things together.

I'll be really interested to see how it turns out!

Re:Will Amazon Kill FedEx? by uptown on Thursday September 01, 2016 @08:22AM (#12404680)
Thanks for sharing more details. Sounds interesting. I'd love to get a look and provide feedback when you're ready to open it up to more users. My contact info is in my profile.
Re:Will Amazon Kill FedEx? by graycat on Thursday September 01, 2016 @08:45AM (#12404804)
You are now on the beta test list! Thanks for your interest!
Re:Will Amazon Kill FedEx? by asimuvPR on Thursday September 01, 2016 @09:26AM (#12405026)
Id like to be on the beta list too if possible. :)

pr @ asimuv . com

Re:Will Amazon Kill FedEx? by graycat on Thursday September 01, 2016 @09:52AM (#12405229)
Same for all who so post here. I have the URLs and when ready for beta testers will return and contact everyone here.

Many thanks to all for your interest.

Re:Will Amazon Kill FedEx? by jmeister on Friday September 09, 2016 @12:17PM (#12463602)
Please add me to the beta test list too! This sounds awesome.
Re:Will Amazon Kill FedEx? by FiatLuxDave on Thursday September 01, 2016 @10:37AM (#12405520)
Also interested here. I hope you are ready to post soon; I'm looking forward to trying it. Good Luck!
Re:Will Amazon Kill FedEx? by CPLX on Thursday September 01, 2016 @01:03PM (#12406973)
Interested as well. Email is "nick" at the domain listed in my profile.
Re:Will Amazon Kill FedEx? by wincy on Thursday September 01, 2016 @04:37PM (#12408685)
This sounds fascinating. I'd also like to be in the beta. Thanks.
Re:Will Amazon Kill FedEx? by smokinn on Friday September 02, 2016 @03:21PM (#12415550)
Also interested. My email is in my profile. Thank you!
Re:Will Amazon Kill FedEx? by jbgreer on Friday September 02, 2016 @04:39PM (#12416103)
Interested. Contact info in profile. Thanks.
Re:Will Amazon Kill FedEx? by deegles on Thursday September 01, 2016 @05:36PM (#12409100)
Me too please. Contact in profile!
Re:Will Amazon Kill FedEx? by Lambdanaut on Thursday September 01, 2016 @02:58PM (#12407942)
Interested. Contact in profile
Re:Will Amazon Kill FedEx? by jhkim on Friday September 02, 2016 @04:36PM (#12416080)
Same here, thanks.
Re:Will Amazon Kill FedEx? by Argher on Friday September 02, 2016 @02:00PM (#12414881)
Interested.
Re:Will Amazon Kill FedEx? by asimuvPR on Thursday September 01, 2016 @11:41AM (#12406159)
Thank you!
Re:Will Amazon Kill FedEx? by steejk on Friday September 02, 2016 @10:19AM (#12413199)
Wow, this is something I've been thinking about recently. Would be very interested in seeing how this progresses.
Re:Will Amazon Kill FedEx? by Kevin_S on Thursday September 01, 2016 @09:12AM (#12404949)
Can I also get on the beta list? This is fascinating.
Re:Will Amazon Kill FedEx? by kevincox on Saturday September 03, 2016 @05:19PM (#12421259)
Did all the "you"s get deleted from your post? It was a little awkward to read.
Re:Will Amazon Kill FedEx? by graycat on Saturday September 03, 2016 @06:24PM (#12421538)
There is always a question about that: When the subject of the sentence is clear, omitting that subject can result in more compact writing , but whether the writing is easier to read or not is a question. I thought that omitting some obvious subjects would help, but thanks for your feedback.