News for hackers. Stuff that matters.
Re:Things I Learned Building The Most Powerful Language Proc... by akavlie on Friday April 19, 2013 @08:52PM (#5579367)
One of the Stremor devs here. I'm helping to build Liquid Helium, and wrote the cited README. Never expected that it would make its way to a public blog post :-).

Yes, language processing is hard. There are two challenges here:

1) Understanding the ambiguities of language, when every word in the sentence can be 2+ parts of speech.

2) Making it fast, and making it fit within the relatively constrained RAM limits of App Engine instances.

We're wrestling with both while we greatly expand on what Liquid Helium can do. It's not easy, but some of the things we're able to do with it are pretty magical.

Re:Things I Learned Building The Most Powerful Language Proc... by drakaal on Friday April 19, 2013 @09:21PM (#5579439)
To akavlie's Ram limit comment: We run on Google AppEngine. It gives us great scalability, but it limits us to about 512 Megs of ram.

We made this choice because with a small team it gave us the most freedom to focus on the code, rather than the scalability and the infrastructure. We don't manage servers, or routers, or load balancers.

Re:Things I Learned Building The Most Powerful Language Proc... by aidos on Saturday April 20, 2013 @04:06AM (#5580738)
I was going to ask about that, seemed strange to feel that a self imposed constraint was a challenge you had to work around.

There are other options outside of app engine that could help you achieve the same thing. I outsource most of my processing to picloud (specifically because I need more ram than I have).

I don't work in your field but I think it's a really interesting problem you're working on.

Re:Things I Learned Building The Most Powerful Language Proc... by drakaal on Saturday April 20, 2013 @04:14AM (#5580756)
If you are going to compete with Google you might as well step in to the arena running on the same hardware. :-)

Also the best targets for acquisition put Google high on the list so building things the way they would makes sense.

Re:Things I Learned Building The Most Powerful Language Proc... by akavlie on Friday April 19, 2013 @09:54PM (#5579584)
Well, technically we are now allowed a limit of 1GB per instance, as of a couple of months ago:

http://googleappengine.blogspot.com/2013/02/app-engine-175-r...

Re:Things I Learned Building The Most Powerful Language Proc... by drakaal on Friday April 19, 2013 @09:37PM (#5579511)
To the ambiguity comment:

Can is really annoying. Can is a verb. Can is a Noun. It is part of a Noun Adjunct.

The rules to determine "The chicken soup can help when you are sick" and "The chicken soup can landed on my foot" and "We can peaches to eat later" is very hard. More so if you put them in a complex compound sentence.