poomer

As someone who's spent a lot of years in an academic social science field and in analytics/modeling more generally, I feel strongly that most educated people have way too much trust in academic literature.

Outright generated data like this is rare and dumb, instead it's very common to play around with data cleaning/variable selection/modeling assumptions to get whatever result you want. I've seen it from grad students, tenured academics, DS/MLE in big tech, basically anywhere where you mix high pressure & neurotic personalities with analytics work.

I'm very skeptical that this will succeed. As with other extremely hyped social medias (anyone remember Clubhouse?) it's already filled with influencers and e-celebs trying to get in early, but these people always poison the well with insincere, low-quality, monetization-oriented content. AFAIK, every successful social media platform grew more or less organically and only had these folks latch on later (usually causing the degradation of the platform when they did).

Having worked in analytics at various tech companies including Fb, the reason is almost always just that app users have way higher usage rates and ad clickthrough rates. Of course if you know basic stats you will realize this is spurious correlation (social media addicts prefer apps, using an app doesn't make one a social media addict), but somehow management never cares.

The paper does not claim that firms have increased profit margins: "the limited available data does not point to a widespread increase in markups"

https://www.imf.org/en/Publications/WP/Issues/2023/06/23/Eur...

I find this an extremely odd and uninterpretable article for this reason. When people say that "x drives y" they usually mean that "x causes y". And inflation is by definition price hikes. So article seems to be saying that "corporate profits causes price hikes", which is meaningless since the causal direction should be in the reverse (price changes cause profit changes).

The fact that labor costs (which are part of a company's profit calculation) are considered a separate "driver" makes this even more confusing.

There is a stop sign in front of my house and probably around 95% of cars stop, and the amount of cars that completely ignore it (ie don't even slow down) is easily <1%. I think this is just more of an issue with local cultural norms.

This class of startup, "build domain specific LLMS using your own data", is extremely crowded right now but I am not optimistic about their future. For large companies, the actual modeling work for this is already easy for any ML team, thanks to existing FOSS work on stuff like PEFT and LoRA. The hard part is figuring out what data goes into the fine tuning process and how to get this data in a usable form, but this is very business specific and can't be automated in a SaaS process.

For SMBs, the value would be in using the LLM to generate responses to customer Q&A/search queries. But these companies aren't going to integrate some external third party service, they'll only use it if it's already baked into their CMS - Wordpress/Shopify/Wix/etc. I just don't see who the final consumer for this product would be.

Thank you, I appreciate your transparency with this work.

Most time series models assume you've already deseasonalized your data in advance. Typically, seasonality is obvious to the human doing the modeling (e.g. sales being up near Christmas), so it's usually preferable for the human to deseasonalize the data in advance using a separate model that bakes in some of their human knowledge of how the world works. Forcing the model to learn seasonal trends fully on its own adds another layer of estimation error.

Prophet is popular because it works off the shelf with non-deseasonalized data and mixed frequency data, which makes it great for quick forecasting exercises. But IMO it is never the ideal model if you have a lot of time and expertise to work with.

These shape matching questions are the problems asked in the study:

https://www.researchgate.net/profile/Jordan-Barbone-2/public...

I would bet that quite a few people just guess when they find this type of question too difficult.

At work we were facing this dilemna. Our team is working on a model to detect fraud/scam messages, in production it needs to label ~500k messages a day at low cost. We wanted to train a basic gbt/BERT model to run locally but we considered using GPT-4 as an label source instead of our usual human labelers.

For us human labeling is suprisingly cheap, the main advantage of GPT-4 would be that it would be much faster, since scams are always changing we could general new labels regularly and be continuously retraining our model.

In the end we didn't go down that route, there were several problems:

- GPT-4 accuracy wasn't as good as human labelers. I believe this is because scam messages are intentionally tricky, and require a much more general understanding of the world compared to the datasets used in this article which feature simpler labeling problems. Also, I don't trust that there was no funny business going on in generating the results for this blog, since there is clear conflict of interest with the business that owns it.

- GPT-4 would be consistently fooled by certain types of scams whereas human annotators work off a consensus procedure. This could probably be solved in the future when there's a larger pool of other high-quality LLMs available, and we can pool them for consensus.

- Concern that some PII information gets accidentally sent to OpenAI, of course nobody trusts that those guys will treat our customers data with any level of appropriate ethics.

You could make a nice living covertly selling the AI "startups" to wealthy folk who just want bragging rights about being founders.

Great article. This part especially rings true to me: "If researchers have an ideological bent, a meta-analytic null may just be an expression of the typical sentiments of researchers".

When I was in academia, it was increasingly the case that my peers thought of research less as a way to determine the truth, but just as a method to influence policy and public opinion. If we thought something was 80% likely to be true, there was pressure to "close ranks" and pretend as though it was 100% true, and to avoid publishing anything that contradicted it. It is also well known that papers that support certain "sides" tend to be easier to publish (and in higher ranked journals), plus can yield more media attention. See for example, this fraud in sociology - https://en.wikipedia.org/wiki/When_contact_changes_minds.

This may be better in the natural sciences, but in social science you should not trust any paper unless you read through and fully understand the methodology. Any non experimental results has so much wiggle room in the modeling methodology that it's easy to generate any result you want. The actual percentage of papers with credible results is very low, much lower than laypeople think.

I do not remember the specific metrics, but there was a big focus on the fact that their CPC/CPM/etc was lower than alternative social media, meaning advertisers put less value on clicks/views from Reddit than other platforms. And they believed this was because their ads were not targeted enough since there was (is?) no fancy ML models behind it, advertisers could just chose some basic rules for what subreddits they want to target.

To solve this they were building out a huge ad relevancy team to target ads at users using posting history, similar to Meta/LinkedIn/etc.

The 42% stat is not correct, the poll in question has that 42% of US has not heard of chatGPT, only 14% of US has used it.

https://www.pewresearch.org/short-reads/2023/05/24/a-majorit...

I had a job interview with Reddit last year for a modeling related position and it was one of the strangest and most user-hostile interviews I've ever had, even as someone who's spent many years working for SV adtech companies. All product interviews were laser focused on maximizing a few specific advertiser revenue metrics, anytime I brought up effects on the consumer it would immediately get dismissed and I'd be asked to refocus on advertiser effects. My guess is their leadership is pressuring the company hard to boost their numbers, no matter the long term cost.

Quite a few do - here is databricks https://www.databricks.com/

I am skeptical that this is due to any positives on the NYC side, but more a story of the Bay Area's decline.

Of the 796 US cities with 50k+ population, San Francisco ranks 796/796 in population growth % over the last 2 years (-7.5%) while NYC ranks 793/796 (-5.3%).

https://www.census.gov/data/tables/time-series/demo/popest/2...

I find this article to be too high-minded. Most Americans don't own cars or support car-friendly policies due to some notion of car=freedom or some other culture wars nonsense.

Americans own cars because most of them live in single-family houses on large plots of land, and that doesn't make public transit for daily commuting a realistic possibility. In Paris car ownership is very low, maybe 1/3 of adults, but in rural France the car ownership rate is easily 95%+. I haven't seen a single developed area in the world that has violated the rule that low density = high car ownership and vice versa.

The other rule that I have never seen violated is that the large majority of middle and upper income people do not want to live near low income people, due to crime or other reasons. In Europe, poor people live in the suburbs, so the middle income live in the city with high density housing. In the US and some other places (south asia), low income people live near the business center, so the middle income live in low density housing in the suburbs. These are for historical reasons and cannot be easily changed.

It does feel like there are more and more startups every YC batch that are just cynical cash grabs. For example, all the recent LLM-related startups that are just UI wrappers around APIs - its hard to imagine that the founders built them for any reason other than to make money, or solely for the reason of wanting to be in YC.

Unfortunately, this is a tech industry problem, not just a YC problem. Tech skills and tech jobs are now seen as a status symbol, and increasingly new entrants in the field are the types of folks who would've gone into law or finance in the past.