dlwh

Stanford CRFM | Bay Area | Full-Time | On-Site, Hybrid

Foundation models like ChatGPT, PaLM, and Stable Diffusion are transforming the world around us. The Stanford Center for Research on Foundation Models (CRFM; https://crfm.stanford.edu/), which is part of Stanford HAI, is an interdisciplinary initiative that aims to make foundation models more reliable, transparent and accessible to the world. We take on ambitious projects that seek to rigorously evaluate existing foundation models and to build new ones.

We are currently seeking a research engineer to join our engineering team. This is an unique opportunity to work with seasoned engineers who have spent many years in industry as well as PhD students, post-docs, and faculty at CRFM. You will contribute to cutting-edge research, publish papers, gain access to the latest foundation models, and be immersed in the vibrant CRFM community.

You will work on our open source software projects, including:

  - HELM, a framework for holistic evaluation of large language models 
  - Levanter, a framework for transparent and accessible large-scale language model training 

For more information or to apply, please go to https://careersearch.stanford.edu/jobs/research-engineer-213...

I highly recommend the book 1491, which goes pretty deeply into what we can infer about the pre-Columbian Americas.

Thanks for the questions.

I am very unlikely to take on visualization. I don't acutely need it for what I do, and I am some-but-not-nearly-enough interested in visualization for its own sake. I started to read about the grammar of graphics stuff at one point and decided it was too far down the rabbit hole.

I have looked more into gpu stuff, and agree specifying a compute graph (and then implicitly optimizing it) is more likely to be the future. FWIW, this is basically what XLA (from TensorFlow) and whatever it was FB announced on Friday are doing.

I wrote my thoughts up recently on the Breeze mailing list here: https://groups.google.com/forum/#!topic/scala-breeze/_hEFpnI...

I'm starting to think it through but I'm not sure I have time for that either :(. A 4-month old and a startup take up a lot of time.

Thanks for the kind words.

Breeze does a large chunk of (dense) compute via netlib-java, which calls out to "real" lapack if you set it up. Are things really faster than that? Or are you referring to the non BLAS/non Lapack things?

Main author here.

Breeze has breeze-viz, which is very basic but at the time there wasn't anything else. I highly endorse using something else. I personally like http://sameersingh.org/scalaplot/

They're under the same aegis basically because they're all mine. ScalaNLP started out as really being just NLP, but it scope-crept. That said, Epic is a library for structured prediction first and foremost, and one of the main applications of structured prediction is NLP.

Breeze is basically like SciPy and large chunks of it power Epic. It's really the only thing that doesn't belong in the namespace.

So, this is (largely) my project.

Not sure why this is on the front page of HN, but I'm happy to answer any questions.

I'm not really giving these libraries the love they need these days. I mostly started them in grad school before the deep learning revolution really hit my subfield (NLP), and I haven't had time to modernize them. They still have their uses, especially Breeze, which is used in Spark's MLLib and directly by a number of companies.

Sure, but the FP side of Scala is definitely firmly in the advanced static type system camp.

Semantic Machines | Software Engineers and Machine Learning Engineers | Boston, MA and SF Bay Area, CA (Berkeley) | http://www.semanticmachines.com/careers/

Semantic Machines is developing technology to power the next generation of conversational artificial intelligence: AIs that you can actually have a conversation with. Think Google Assistant or Alexa or Siri, but without having to carefully craft commands like you're talking to a Bash shell.

Our team has built much of the core technology underlying Siri and Google Now, and our founders (including both the former Chief Speech Scientist for Siri and the head of UC Berkeley's Natural Language Processing group) have multiple >$100 million exits under their belt.

We're looking to hire a few talented software engineers and machine learning engineers to help build out technology, by expanding our core NLP infrastructure, data processing pipelines, neural net clusters, and backend services.

Experience with natural language processing systems is a plus, as is experience with the JVM (especially Scala), but we're mainly interested in passionate engineers who can learn quickly and work effectively in complex systems.

Please reach out to me directly or email [email protected]. Thanks!

Not a neuroscientist, but I do do NLP, and I only lightly skimmed the paper.

This doesn't really speak to UG.

First, you can believe in the structures they purport to show without accepting the existence of UG, by appealing to the existence of general mechanisms in the brain for assembling hierarchical structures, which is equally validated by this experiment.

Second, they looked at two languages with sentences of up to ~7 syllables each with at most two constituents (Noun Phrase Verb Phrase). You can't show any evidence for any hierarchy of interest in 7 syllables. They demonstrated that phrases exist and phrase boundaries exist, but it's entirely possible to have "flat' grammars without interesting hierarchy, especially in simple sentences. If they want to show interesting hierarchy, they should conduct experiments with more interesting structure (say, some internal PPs and some limited center embedding) and show something that correlates with multiple levels of the "stack" getting popped, or something.

It's still interesting work, but as usual oversold by the university press office.

Prosopagnosia[1] is a real condition that affects ≈2.5% of people. You might check out "The Man Who Mistook His Wife for a Hat" if you're interested.

[1] https://en.wikipedia.org/wiki/Prosopagnosia

to add another:

    System.out.println(0.0 == -0.0); // true
    System.out.println(java.util.Arrays.equals(new double[] {0.0}, new double[] {-0.0})); // false

(It's documented in the contract for Arrays.equals, but still kind of ridiculous.)

statically checked, but still runtime-determined, matrix sizes can be done in Scala. Doing specialized implementations would require bytecode magic, though.

  case class Dim(size: Int)

  trait Matrix[X <: Dim with Singleton, Y <: Dim with Singleton] {
    def *[Z <: Dim with Singleton](other: Matrix[Y, Z]):Matrix[X, Z] = ???
  }

  object Matrix {
    def apply[X <: Dim with Singleton, Y <: Dim with Singleton](x: X, y: Y):Matrix[X,Y] = ???
  }

  val x = Dim(100)
  val y = Dim(readFromFile(...))
  val z = Dim(37)

  val A = Matrix[x.type, y.type](x, y)
  val B = Matrix[y.type, z.type](y, z)

  A * A // compile error
  A * B // ok

(I'm one of the authors of Overmind.)

First, it's important to know that AI is a very broad term. Roughly, there's "modern" AI (machine learning and statistical modeling) and "classical" AI (mostly logic and search). Game AI (as I understand it) falls more into the "classical" AI camp, and classes these days are focusing more and more on modern non-game techniques.

kaybe's link is a good starter class for modern AI. Norvig quite literally (co)wrote the text book on AI, and Sebastian Thrun. The PI on Overmind (Dan Klein) co-taught a class on edX (https://courses.edx.org/courses/BerkeleyX/CS188.1x/2012_Fall...) that you might also want to check out. They're all based on the same underlying book, just different focuses.

Overmind itself is a collection of some reasonably sophisticated ML/modern AI, classical AI (which is more common in games), and a lot of horrible hacks. Really, tons of them.

If you want, I can try to dig up the lecture slides for the "advanced" AI class we taught once that specifically focused on the kinds of techniques one would want to write StarCraft bot AIs.

Machine translation is no where near solved.

Machine translation is still one of the most funded areas in NLP, and the quality is still incredibly bad for many language pairs. DARPA has been running one machine translation grant program or another for more than a decade.

It is true that some of the best systems take a long time to translate sentences (on the order of 1 cpu-minute or more). But their quality is still not anywhere near human level for most languages.

This article is basically nonsense. The "Deep" here doesn't mean "deep learning"/neural networks. They just mean non-superficial understanding of text (beyond keywords, named entity recognition, and the like.)

For example, the Illinois NLP group has a grant under this project, and their pubs are largely not using "deep learning." http://cogcomp.cs.illinois.edu/page/funding_view/43

> Yes it does. AFAICT it's working well. There are 5 releases in the 2.10.x line and I haven't had a compatibility issue between any of them.

This isn't quite true. I have a library (Breeze) that works with 2.10.3 but not 2.10.2. Something in name mangling changed, I think.

Make the prior art fee scale up by number of patent applications submitted by that entity, then. First one's free. Then $100, then start growing at some small-ish exponent.

Actually, NNs (specifically auto-encoders) are more like a nonlinear PCA. PCA can be seen as an "information bottleneck" where you try to represent an N dimensional space as a K < N dimensional space such that your ability to reconstruct the original space is maximized (that is, the "reconstruction error" is minimized.)

Autoencoders are precisely the same idea, except that you use nonlinear transformations (e.g. softmax or tanh) in addition to linear transformations (matrix multiplications). You can also use multiple layers in an autoencoder, just like any neural net. (The linearness of PCA makes multiple layers useless. You can always replace multiple layers with a single layer.)

They're not disposable. They reuse them. You get a rather sizable discount (50%) for giving an empty one back when you get a new one.

So, this is a good post and everyone should be aware of these. But kernels ("non-linear" svms) are not a panacea. If you use kernels, then your model will need to remember some non-trivial number of the data points (the "support vectors" that define the decision surface). SVMs are nice in that they need a relatively small number of points compared to the size of the dataset. Moreover, classification is now linear in the number of points you kept around, and training is potentially quadratic. If you use a straight-forward linear classifier, you only need a single weights vector.

Second, SVMs---especially with a non-linear kernel---are much trickier to tune than logistic. SVMs are very sensitive to choice of hyper-parameters (user specified tuning weights), which means repeatedly retraining. And if you use too powerful of a kernel, you will overfit your data very easily.

For these reasons, my advice is to start with logistic, and then if you're not satisfied, switch to SVMs.

(As an aside, it's totally possible to kernelize logistic regression. Without suitable regularization, it'll be even worse with regards to the number of datapoints you need though.)

Factorie's a nice piece of work, I highly recommend it. The team at UMass has done a great job.

Huh, I was wondering why I had a bunch of stars on github in the last few hours. (Sad there's no analytics for Github projects.)

I'm the lead author of ScalaNLP, and I'm happy to answer questions. It's mostly a scaffolding library for other systems. There's not much that works "out of the box."

I am currently working on releasing Epic (hopefully tomorrow), which is my parser/structured prediction framework, that will be a little more useful to non-specialists.

Aren't the ram chips usually soldered to the mobo on ultrabooks? Crucial indicates they can't upgrade ram on the XPS13. And a little googling says soldered RAM... Nothing on the Dell website suggests they can upgrade past 4.

I suspect this might have at least a little to do with the Google+ rebranding. Remember how they removed the + operator from search?

My girlfriend's organization is dedicated to doing this. www.maplight.org

Here's SOPA aka HR 3261: http://maplight.org/us-congress/bill/112-hr-3261/1019110/tot...

They're a very careful organization, so they're very cautious in stating a congressperson's vote before the actual vote. But it's a start. The contribution ratio for my representative definitely prompted me to call her today.

I didn't downvote but, pretty much every academic in the CS/physics/stats/math I've met uses LaTeX. Even a bunch of linguists I know use LaTeX.

It's definitely not perfect--I really wish there were something better--but for writing technical academic papers, it seems like far and away the best.

I'm somewhat familiar with this work. (My advisor talked to the authors some. I could be misrepresenting it a little, but not nearly as much as the article.)

It's not learning to play the whole game. It's learning to cheese (in gaming parlance) the opponent. The strategy it learns is to build a warrior as fast as possible and go and attack the enemy's city. If that fails, it almost always loses. The manual gave it some hint in that direction.

Option can occasionally be a pain to handle, I admit. But typically it works really well for me.

The standard Haskell-ish approach is to use for-comprehensions:

val maybeWebpage = for(foo <- maybeFoo) yield { // do things with foo, yielding some resulting webpage snippet or whatever }

As he points out, this can cause Options to propagate, but it does allow you to defer dealing with the exception until later. You can reasonably argue that it leads to better modularity, since you can dump all of your error handling in the same function, and different callers can decide how they want to process errors.

Nevertheless, the OP wants a getOrThrow method (which he could of course add via implicits), but you can also just use getOrElse, which usually returns some other expression in the case of None, but since throw is just an expression in Scala, you can use it there:

scala> val foo = Some("foo").getOrElse(throw new Exception("whatever")) foo: java.lang.String = foo

scala> val foo = None.getOrElse(throw new Exception("whatever")) java.lang.Exception: whatever at $anonfun$1.apply(<console>:5) at $anonfun$1.apply(<console>:5) at scala.Option.getOrElse(Option.scala:104) at .<init>(<console>:5)

Just to help sell it some more: autojump basically watches where you cd and builds a model (of some sort) that guesses where you want to go based on a few characters.

For instance, I have a project called "structured-prediction" deep in some folder hierarchy, and I can just type "j stru" or "j pred" or even "j uct" and it goes to the directory.

The best part is that it figures this out automatically. You don't have to remember to bookmark anything.