dbfclark

I think your per-axis std normalization is likely doing a big pile of the work —- it’s fairly well-known that “wrong” PCA, setting sigma=Id or just taking a square root, gives better embeddings than the un-normalized version. It would be worth showing a comparison to similarly-normalized PCA I think, if it’s not too hard?

I did a goodly chunk of vibe coding over the summer and I found that the best language for me was Rust implementations with Python bindings for interface. A few reasons:

- Everything about rust enforcing correctness catches lots of bugs

- Using a high-level API means I can easily hand-check things in a repl

- In addition to tests, I required a full “demo notebook” with any PR — I should be able to read through it and confirm that all the functionality I wanted has actually been implemented

If the philosophy is (and it should be) “loc is free”, it’s worth thinking about how we can make LLMs produce more loc to give us additional comfort with correctness. Language choice is very much a way.

Another completely viable solution (other than adding extra starch) I’ve found is to sprinkle a bit of sodium citrate (the sodium salt of citric acid, a common food additive and cheap on Amazon) over the cheese before adding to the pan. This improves the melting qualities of the cheese and avoids the starch issue altogether. You’re basically using pecorino velveeta.

A good way to understand why cosine similarity is so common in NLP is to think in terms of a keyword search. A bag-of-words vector represents a document as a sparse vector of its word counts; counting the number of occurrences of some set of query words is the dot product of the query vector with the document vector; normalizing for length gives you cosine similarity. If you have word embedding vectors instead of discrete words, you can think of the same game, just now the “count” of a word with another word is the similarity of the word embeddings instead of a 0/1. Finally, LLMs give sentence embeddings as weighted sums of contextual word vectors, so it’s all just fuzzy word counting again.

Convergent sequences are always Cauchy; for metric spaces, compactness and sequential compactness are the same.

The key question here isn’t so much whether GPT-4 beats the actual human decks as much as what it had to fabricate to do so. The humans are probably constrained by things like “reality” and “what their business has done in it” while GPT-4 could make up anything it wanted. A fair comparison would be to humans given the same prompt and told to invent whatever facts they wanted.

Very let down by the title -- why no ancient Roman abstract algebra?

Not my company, but Zocdoc has the best values I’ve ever seen exactly because they all have anti-values:

Patients First

Important, not Immediate

Learners before Masters

Together, not Alone

Progress before Perfection

Adaptable, not Comfortable

Sure. Moonshine I know a lot about, relative to physics at least. Given that I now work somewhere that specializes in term embeddings, it kind of feels like I've spent half my life injecting stuff into vector spaces...

Hey, I'm a mathematician, not a physicist. We tend only to get as far as "oh yeah, here's all this pretty math we care about. I hear it has some applications to physics..."

Presumably the extra credit is that the binary Golay code is closely related to the Leech Lattice, and thus to the entire moonshine situation, which gets you to string theory. See:

https://en.wikipedia.org/wiki/Leech_lattice https://en.wikipedia.org/wiki/Monstrous_moonshine http://motls.blogspot.com/2015/03/umbral-moonshine-and-golay...

Very company dependent. At Luminoso (http://luminoso.com), we live in Python, with varying doses of Javascript for frontend, very little R (but some of us know it), Java and C++ are pluses (for working with other people's software), and we have at least some Haskell in production (mostly for preprocessing). And plenty of ops tools for deployment. We have used a few neural networks frameworks and machine learning packages; definitely the Python machine learning ecosystem is big for us but we haven't found our One True neural network framework yet. Other companies (say in image recognition, Ditto Labs for instance) are definitely more CNN-oriented, yet others are certainly doing work in R.

If you have a company that is doing genuinely interesting AI work, it's likely that they are somehow on the forefront of research, pushing existing tools to do things that they can only barely do. If you find yourself in an AI role (typically only some roles actually do AI, of course -- systems need to stay up, great UIs need to be built, etc.), I would guess that you'll need to familiarize yourself with their particular toolsets and methods rather than assuming that there are universally correct things to go learn.

Indeed -- it's struck me recently how Wes was probably among the first (if not necessarily number one, if only because of ill-definedness) people to make computers smaller. Looking back at the sixty years since he began, it's been quite something.

Quite. Minsky was actually advisor to my company, which is based in part on his notion of common sense computing. There is much to mourn recently.

He'd talk about computers for fun, at least. But try to convince him that the internet actually worked and you'd have a whole evening's conversation on your hands...

It's a reference to a story about Ramanujan, who once (remarkably) told G. H. Hardy that 1729 wasn't a boring number because it was the smallest number that could be expressed as the sum of two cubes in more than one way. Sometimes trivial but always erudite in the best way, that was Wes.

Everyone has stories, most better than me. I find my most vivid memory of experiencing him as an adult was that every time you spoke with him he'd have had some completely flabbergasting new life experience you'd never imagined. He was a guest on something like the second trip to China after Nixon, took seminars with Oppenheimer at Berkeley (he tended to call him Oppy), and may have been invited to help create an early ATM. The list goes on and on.

One Christmas, we were talking about how difficult it is to assess the accuracy of an algorithm when you only get a few tries because running it is expensive; he promptly told us about his experience with this problem, which he encountered driving plutonium nuggets across Richland, Washington. After the video from the ACM conference on personal workstations got published here, I learned for the first time he'd been fired from MIT three times for insubordination, and I never got a chance to ask him what the story there was...

Oh! And he actually made a physical turing machine good enough to do fairly sophisticated computations on. It ended up in my father's intro CS class at Princeton for many years.

And HN will of course appreciate one of his favorite one-liners:

Did you know that 49 is the lowest number that can be expressed as the square of 7 in only one way?

I'm his grandson, not his colleague, so far be it from me to say he was a great man, but he was a great grandfather and an inspiration. He taught me an enormous amount, from logic to just how hard it can be to get people to adopt new technologies -- he eschewed word processing, writing every document directly in PostScript even up to the present -- but most importantly he showed me how important it is to love what you do, and for that I owe him a ton of my happiness and career.

Fare you the hell well, Gramps.

I couldn't possibly agree more in general, though of course I'd quibble with many of the specifics...

As to stodginess, I'd say there's a big difference between personal/small commercial lines and the big ticket enterprise-type stuff -- the underwriting process goes from something data-driven to something relationship-driven very quickly indeed. The bigger the commercial line, the more likely it's all about who throws the best yacht parties (reinsurance in particular suffers from this massively).

Crowdsourced insurance is indeed a terrible idea, though you could imagine 'web of trust' insurance that almost made sense -- say my ten thousand best friends and I know that we're all great actuarial risks, perhaps because we have some kind of information on which it's illegal to select (that we're all in the same gym, say). We could then try to write ourselves health insurance for cheap, because our plan would select only us gym members. You can sort of make it work, as long as you're prepared to make the regulators hate you.

Which is the real problem, of course -- most people buy insurance because they have to, not because they want to. Auto insurance that wouldn't pass muster with the police, or home insurance that wouldn't satisfy the bank holding your mortgage, doesn't solve the problem.

Do good problems exist? Sure. A web-based Managing General Agency, for instance, could do very well for itself, but the expressed ideas in this section are pretty terrible.

This is an impact of the default Unicode normalization, which is set to NFKC. This normalization is lossy for things like the ordinal indicator and trademark symbol; if you'd like to keep the ordinal indicator unchanged, use NFC normalization:

  >>> print ftfy.fix_text(u'ordinal indicator º to o in addresses.')
  ordinal indicator o to o in addresses.

  >>> print ftfy.fix_text(u'ordinal indicator º to o in addresses.',normalization='NFC')
  ordinal indicator º to o in addresses.

Luminoso cofounder here (we make ftfy, among other things). Our use case is fairly specific: a customer uploads text documents, often as a spreadsheet originally exported from someone else's tool, but doesn't think hard about the encodings involved, so in order to serve them well we have to fix whatever happened. Accordingly, we put the most effort into solving problems that happen for our English-centric, US-centric customer base; they're the most important problems for us (though Russian has also gotten some love, as you can see from the commit history). On top of that, making Asian language exports from other tools work correctly usually requires enough encoding-awareness to mitigate a bunch of the problem[1], so we see those problems less frequently.

That said, if you have examples where ftfy fails in any language, please submit them! We want this tool to work well, because anything that we can't fix will cause us to have egg on our faces with a customer someday...

[1]No peeking: how many format options in Excel's "Save As" dialog, excluding Excel formats, produce a document from which Unicode can be recreated accurately?

One meaningful point: we're early-stage and we bought a booth and we seem to be getting great value out of it, but we have a very specific target audience not mentioned by the original poster: digital marketers. At least the interactive part of SXSW is now way more a marketing conference than a tech conference and if you regard it in this light you can do very well. If you need to pitch social media managers, it's awesome, if you just want to talk about something consumer-y, not so much.