youngprogrammer

Atlantic and New Yorker my two favorite publications for long form articles. Although the Atlantic has got a bit too political for my tastes

At a 3 minute shot, I’d rather use the same time to do a pour over

You might as well play "who can memorize the most openings and lines"

It can get complicated quickly if you're actually using it in a production system. At my prev enterprise saas company we had feature flags that could be turned on per customer / per environment (dev, staging, prod) with permission + logging model such that our support team could also toggle flags with history of who turned on what. We also had "per user" feature flags for certain test users at companies and had DSL rules to evaluate the features

this is essentially how bindcraft works for drug discovery: https://www.nature.com/articles/s41586-025-09429-6 (minus the accumulation step)

feedback from af2 folding confidence + structural scoring

Fish sauce is delicious but had to stop using it since it's high in histamine (gives me a stuffy nose) and potentially carcinogenic due to its high levels of nitrosamines

industrialized overfitting is basically what ML researchers do

should go a bit earlier with word2vec, NMT, seq2seq, attention, self attention

Little late to this thread but from my list:

LLM (foundational papers)

* Attention is all you need - transformers + self attention

* BERT - first masked LM using transformers + self attention

* GPT3 - big LLM decoder (Basis of gpt4 and most LLM)

* Instruct GPT or TKInstruct (instruction tuning enables improved zero shot learning)

* Chain of Thought (improve performance via prompting)

some other papers which are become trendy depending on your interest

* RLHF - RL using human feedback

* Lora - make models smaller

* MoE - kind of ensembling

* self instruct - self label data

* constitutional ai - self alignment

* tree of thought - like CoT but a tree

* FastAttention,Longformer - optimized attention mechanisms

* React - agents

It seems like most of these imperceptible changes could be addressed by something like ascii folding (https://www.elastic.co/guide/en/elasticsearch/reference/curr...) but this might not apply for non-english use cases.

If you're interested in adversarial NLP, I also recommend reading this blog post on adversarial attacks on GPT2 with universal triggers (e.g. adding "nobody" as prefix for all inputs causes all entailments to be predicted as contradiction).

They probably didn't have a Gantt chart to help them figure out the dependencies to properly plan it on their roadmap

You could do something similar to how they trained a ML model to find antibiotics compounds: https://www.cell.com/action/showPdf?pii=S0092-8674%2820%2930.... First, train a deep learning model to learn a representation of molecules from their molecule structures. Then feed in the thousand or so known compounds that produce pleasant or unpleasant smells as training data with some score of "pleasantness". We can then use this model to quickly score millions of compounds and select candidates to test.

Anecdote: Our caterer in silicon valley said there was a supply issue for tofu.

This is nothing like the sharpshooter fallacy. The analysis determined the average performance of the analysts with >100 stocks rated and 10 analysts out of 16 did better than the rest.

How do you measure top ten if you don't have a ranking system?

Top 10 performers out of 16 or so analysts in the analysis is not survivor bias.

Outliers were removed to get a better measure of the "accuracy" of the price targets.

10 day windows were used to reduce the amount of volatility/noise in a time frame

Return horizons for 1 years was used because price targets are for one year.

Theres only 15 or so analysts I looked at.

I was doing this as an exploratory data analysis and didn't want to pull out my old stats textbook.

Cutoffs were chosen to reduce volatility of measurements since I was looking at percentages. A stock going from $1.5 to $2.0 is a 33% increase whereas the movement of $100 to $133 is significantly more impactful. Stock with lower market cap have more volatility. The minimum analyst rating was chosen to eliminate analysts with very small number of ratings as they would be unreliable.

I will agree that the methodology is not as rigorous as it could be but where can you prove it is "wrong"?

My blogpost shows that stock price predictions also show a terrible track record. They are wildly off and on average higher than actual results.

I probably should have included the outliers when analyzing overall performance but if I recall correctly they did not have a significant effect.

The top analysts were determined by the average performance from one year after their ratings have been made. This isn't the top analysts out of 50,000 it's the top out of 50 or so analyst-rating pairs. There were only 16 or so analysts in total that I looked at. This isn't an instance of survivor bias as your example states. If I were to be more rigorous I could give a statistical test for this.

You're correct that I should have accounted for outliers when measuring performance of this strategy. If I recall correctly, even with the outliers, they did not significantly affect the average performance of analyst ratings. I would have to rerun the numbers though.

The analysis was more about measuring the performance of analysts which is why the price data for before and after the recommendation. For practical purposes of using this strategy, you are right that the price data from days after release would be better.

If the top 10 stocks you picked beat the market and have consistent earnings and dividends over a period of time, would this not be a repeatable strategy?

Hey Matthew, thanks for running your website! It was very helpful for obtaining the data I needed for my blogpost.

In my analysis, I do hypothesize that analyst opinions become a self-fulfilling prophecy as you described. However, I would like to believe that analysts do some sort of sophisticated breakdown and analysis of a company's financial statements and market outlook when releasing a justifiable rating.

1. Removing outliers was for making the data easier to analyze as some outliers were skewing the average. Of course when investing you cannot ignore outliers, but you could possibly curb them with stop losses/stop limits.

2. A more in-depth analysis could be done on analyst releases' effect on prices but assuming that this does occur, then the performances are understated and provides further evidence to the conclusion that outperform ratings can do better than the market.

3. Not sure how narrowing down the top analysts is a flaw here.

This blogpost is probably not as mathematically rigorous as it could be as I just wrote it as an exploratory analysis for fun and out of curiosity.

Unfortunately, the website I used to scrape the data only had 2 years of data. But each datapoint is looking at each individual analyst rating and price target for a stock and then comparing it to the price in 1 year. However, you are correct that datapoints during a downturn would be quite different.

How to travel like a millionaire? Ask the millionaire