hcarlens

Interesting, thanks for sharing!

That was true in the first Makridakis competition ("M1") in 1982, and possibly until M4 in 2018, but both M5 and M6 were won by what would generally be considered relatively sophisticated methods (e.g. LightGBM).

The Wikipedia article doesn't have that much detail on M5 or M6, but the M5 papers are in the International Journal of Forecasting[1] and M6 should be published later this year (there's already a preprint on arxiv [2]).

I recently spent some time looking into the history and results of the M competitions and had a chance to speak to Professor Makridakis about them, as well as the winners of each of the M6 competition tracks [3]. While the methods have become more sophisticated, some conclusions from M1 still seem to hold: in particular, that there is no overall "best" method, and that the winning method tends to be different for different types of data, time horizons, and evaluation metrics.

[1]: https://www.sciencedirect.com/science/article/pii/S016920702... [2]: https://arxiv.org/abs/2310.13357 [3]: https://mlcontests.com/state-of-competitive-machine-learning...

XGBoost, LightGBM, and Catboost are all used quite frequently in competitions. LightGBM is actually marginally more popular than the other two now, but it's pretty close. In the M5 forecasting competition a few years back, many of the top solutions used primarily LightGBM.

Yeah, this book was incredible and the tech in it has aged extremely well. Have you tried any of Ted Chiang's books? They're also great hard sci-fi. Another one that plays with similar ideas to Permutation City is the Bobiverse series by Dennis E. Taylor.

Also relevant: https://aimoprize.com/

($10m prize for models that can perform well at IMO)

Agreed, and not only do they not compare their model to Phi-2 directly, the benchmarks they report don't overlap with the ones in the Phi-2 post[1], making it hard for a third party to compare without running benchmarks themselves.

(In turn, in the Phi-2 post they compare Phi-2 to Llama-2 instead of CodeLlama, making it even harder)

[1]: https://www.microsoft.com/en-us/research/blog/phi-2-the-surp...

There have been quite a few interesting Kaggle competitions in recent years, as well as other interesting ML/data science competitions on other platforms.

Platforms like Kaggle, DrivenData, Zindi, AIcrowd, CodaLab and others are running dozens-hundreds of competitions a year in total, including ones linked to top academic conferences. One interesting recent one is this one on LLM efficiency - trying to see to what extent people can fine-tune an LLM with just 1GPU and 24h: https://llm-efficiency-challenge.github.io/

Or the Makridakis series of challenges, running since the 80s, which are a great testbed for time-series models (the 6th one finished just last year): https://mofc.unic.ac.cy/the-m6-competition/

An interesting approach I came across at NeurIPS a few weeks ago is called "ML with Requirements"[1]: https://arxiv.org/abs/2304.03674

My basic understanding is that it combines "standard" supervised learning techniques (neural nets + SGD) with a set of logical requirements (e.g. in the case of annotating autonomous driving data, things like "a traffic light cannot be red and green at the same time"). The logical requirements not only make the solution more practically useful, but can also help it learn the "right" solution with less labelled data.

[1] I don't know if they had a NeurIPS paper about this; I was talking to the authors about the NeurIPS competition they were running related to this approach: https://sites.google.com/view/road-r/home

I had a chance to chat to them again today, and wrote some more details here: https://mlcontests.com/neurips-2023/tutorials/#exhibit-hall

Also as the other comment mentioned, https://positron.ai seems to be live now.

I built a slightly less detailed version of this, which also also lists free credits: https://cloud-gpus.com/

Open to any feedback/suggestions! Will be adding 4090/H100 shortly.

Having your own GPU is nice, but it's power hungry, heats the room, and only really makes sense if you use it a lot.

For a list of cloud GPU providers with rough price comparisons, I created this page: https://cloud-gpus.com

If anyone is interested in a GPU-specific comparison across these three and other providers, I created this page for that: https://cloud-gpus.com

Happy to get any feedback on this!

Good try, and there are indeed different types of infinities, but your examples both represent the same "countable" infinity of the natural numbers! You can create a bijection between these two sets. I can't see anything helpful about this example for kids, and in fact I think it teaches them the wrong intuition.

If you wanted to show different infinities, there might be a way to make Cantor's diagonal argument more accessible to children, to show them countable vs uncountable infinities, but that's a lot less intuitive!

What I mean by a bijection: in your example, you can map the nth dog to the nth paw, and each dog will always have a unique corresponding paw (albeit not its own paw, except for the first dog). Showing them different infinities would mean showing two sets where it's not possible to have this kind of mapping.

For example, an integer numbering to the set of all possible dog names (of unbounded length)

Hi! I started mlcontests.com because I wanted to be able to have a simple overview of live Kaggle-style competitions, and I've been running it as a side project for almost two years now. Would love to hear your thoughts/suggestions!

Source is at https://github.com/mlcontests/mlcontests.github.io, and I'm always happy to accept contributions :)

Yeah it's a shame. I don't own the IP, my previous employer does, and I think when they got it published in the journal of trading they had to agree to some level of exclusivity which meant it can't just be uploaded elsewhere.

I just linked to it in case someone here had a subscription.

The ones that are currently live tend to have features that try to negate this (randomised uncrossing times, price collars, order priority based on arrival time or size)

Haha yeah! Though times for these auctions are double-digit milliseconds, a lifetime for your fpga strategies! And it's still fairly niche, these are complementing CLOBs/dark pools rather than replacing them. Where did you move to from hft?

When they were first introduced, each of the fba books had slightly different mechanics (matching priority, timing, price determination), so each book needs a slightly different approach. I guess you could see it as gamesmanship, but in equity markets dealing with market mechanics properly is just part of the job.

Actually, it got implemented by most major exchanges in Europe in 2017/2018 and cboe seems to be bringing this to the US now: https://www.cboe.com/us/equities/trading/offerings/periodic_...