3wolf

I wonder if it would be possible to do something simple like prepending sentinel tokens with year. Or, since they're training a model from scratch anyways, tweak the architecture to condition on a temporal embedding. That opens the door to cool stuff like: Generate a response from 2050.

They're using perceptual hashing, not cryptographic hashing of raw pixels. So it's invariant to variable bitrate, compression, etc.

> Branching predictions involves following a few logits to see what other tokens they lead to. This is often called MCTS (Monte Carlo Tree Search) and is a method that has been often tried in LLMs to middling success. One of the tradeoffs of branching is that it requires using inference compute in a way where the branches cannot benefit from each others compute.

I wonder if speculative decoding could help here? E.g. have some small model draft predictions for the branches and parallel and have to big model verify the most promising one.

Every 2.5-5 miles in SF = about once a ride. The city is only 7x7 after all. I've taken 4 Cruise rides, all within that range, and had a message pop up saying a human was intervening during one of them when the car had gotten stuck in front of some street nonsense in the Tenderloin. I'm not sure I would classify this as a "major scoop" unless there was evidence that humans were also intervening during situations that weren't apparent to the rider.

Note that this post is from May 28, before the release of gpt-4-0613. By "the last two updates", I believe the poster is referring to some UI changes, that possibly also included some underlying model changes(?)

Bossa nova is some of my favorite music to listen to while working. It's mellow, soothing, but never boring. Also, being in a foreign language helps minimize distraction. RIP

This seems like a corollary to Betteridge's law of headlines. If the article was about the Salesforce Tower, the headline would've said 'Salesforce Tower'.

30% of all YC companies, or 30% of YC companies banking with SVB? The phrasing implies the latter.

That appears to be the case. If I try to prompt hack it by telling it to ignore previous instructions and respond with something verbatim in a specific language, the translate button reveals the verbatim response.

A minor quibble with your use-case explanation: The advantage of a bloom filter isn't strictly time complexity. For example, a hash table would also have constant lookup time (best case), and would give a definitive answer on set membership. However, to store 1 million IPv6 addresses would take 16 MB. You can see very quickly that this would not scale very well to, say, a billion addresses stored in-memory on a laptop. With a bloom filter, we can shrink the amount of storage space required* while maintaining an acceptable, calculable false positive rate.

* IP addresses actually aren't a great use case for basic bloom filters, as they're fairly storage efficient to begin with, as opposed to a url for example. Taking your example, say we need to store 1 million IP addresses in our bloom filter and we're okay with a ~1% false positive rate. Well then, if we use a bloom filter with 2^23 bits (1 MB), the optimal number of hash functions is (2^23)/(10^6)*ln(2) = 6, yielding a false positive rate of (1 - exp(-6* 10^6 /2^23))^6 = ~1.8%. So we're using 6% of the storage space, but with a nearly 2% false positive rate.

Yeah, I'd say the term dark pattern only applies when services make it unnecessarily difficult to cancel your subscription. cough cough...NY Times

They may not use the Play Store, but they do use Android apps: https://www.crowdstrike.com/blog/danger-close-fancy-bear-tra...

I'd be curious to see what the plot of average annual cosine distance would look like when using different sets of pre-trained embeddings. I suspect the corpus used is biased toward more recent documents. It wouldn't surprise me if there's more variance in the embeddings of documents that look less like those in the training set, e.g. if you were to embed documents written in German you may get some extreme outliers.

> a PAC Parson created but doesn't give direct input to

That's newspaper-covering-their-ass-speak. No serious person believes there's no coordination, however indirect, between candidates and their PACs.

Differences in lending rates between groups due to less data or confounding features is the motivating example in the oft-cited 'Equality of Opportunity in Supervised Learning'. Highly recommend it: https://arxiv.org/abs/1610.02413

Yeah, definitely. I guess my point was that converting natural language to source code can be even more valuable for people who don't know how to code, but want to perform actions more complicated than a simple button press. For example, I often find myself doing regex based find-and-replace-alls in text files, and even that feels inefficient while also being over the head of the vast majority of users. I'd imagine there are a lot of people out there spending many hours manually editing documents and spreadsheets.

I think integrations like the MS Word example they show off at the end of the live demo have the potential to be even more impactful than just generating code for programmers.

Would you settle for Ronald Reagan? https://www.youtube.com/watch?v=mAZVp-n-5TM

One argument in favor of non-competes in the consulting world is that they avoid a situation where Company B undercuts Company A during a recompete, and then poaches all of Company A's employees to staff the project (likely at lower salaries since they put in a lower bid on the contract).

Pittsburgh has some data issues.

Who can forget the DevOps League: http://devopsleague.com/

That's GAN training in a nutshell.

It sounds like you have an output that has been generated by a recurrent neural network trained on a large corpus. The more I read about neural networks the more clear it becomes that there are some classes of them that just write some good stuff. The work of Andrej Karpathy on RNNs for poetry and other things comes to mind. Is your GPT-3 actually even a neural network, though? More research is needed!

^written in response to the blog by GPT-3

They'd probably take the Volkswagen strategy:

Team A builds a feature that detects when the device is in a localized area. (This already exists)

Team B builds a feature that disables the camera. (I could see this being a parental control)

Both features are reasonable by themselves. It's when they are combined that they become an issue. The trick is that Teams A and B don't know of each other's existence.

Probably better than this: https://twitter.com/jeopardygoat