Applied Machine Learning for Tabular Data

Re:Applied Machine Learning for Tabular Data by levocardia on Thursday July 25, 2024 @10:27PM (#41075359)

My experience with tabular data textbooks is that they introduce many different techniques, in a relatively un-opinionated way, which doesn't help give you intuitions on what your strategy for real-world problems should be. In practice, almost all tabular data problems I've encountered boil down to:

1. Make sure you don't have any impossible nonsense or data leakage in your features or target

2. Split your data in an intelligent way (temporally for time-series, group-wise for hierarchical data)

3. Try a really simple linear regression / logistic regression model to get a "dumb" baseline for your accuracy/error metric and make sure it is reasonable

4. If you need interpretability, consider a GAM, else, throw it into XGBoost and you'll get state of the art results

[ Parent | on HN ]

Re:Applied Machine Learning for Tabular Data by fbdab103 on Thursday July 25, 2024 @11:25PM (#41075543)

For 4, I would always start with RF. You have fewer knobs to turn (aka degrees of freedom with which to hang yourself), but still get within spitting distance of XGBoost.

[ Parent | on HN ]

Re:Applied Machine Learning for Tabular Data by durraniu on Thursday July 25, 2024 @11:24PM (#41075538)

What do you think about the mechanics of machine learning book by Jeremy Howard? It seems to have a systematic approach to fit models and improve them.

[ Parent | on HN ]

Re:Applied Machine Learning for Tabular Data by code_biologist on Friday July 26, 2024 @03:13AM (#41076476)

At risk of being excessively cheeky, I think you should throw it into xgboost and not worry about it.

I love Jeremy Howard, but if you're doing applied ML on tabular data, tweaking specifics of model construction probably won't get you anything beyond stock xgboost. The concerns highlighted by your sibling commenter jamesblonde end up eating most of your time in deployed ML products, not model construction.

If you're looking to learn about ML model construction I'm sure it's a great book.

[ Parent | on HN ]

Re:Applied Machine Learning for Tabular Data by jamesblonde on Friday July 26, 2024 @01:20AM (#41075956)

I am writing a tabular data textbook for O'Reilly - on building ML systems with a feature store.

I try to be opinionated about modelling - XGBoost is all you really need, but the challenges are more like you say - how to prevent data leakage (ASOF LEFT JOIN (or use a feature store)), separating model-independent data transformations from model-specific data transformations, APIs for things like time-series splits, logging, monitoring, building and operating the pipelines. All pretty standard software engineering in Python nowadays.

Free chapters:

https://www.hopsworks.ai/lp/oreilly-book-building-ml-systems...

[ Parent | on HN ]

Re:Applied Machine Learning for Tabular Data by teleforce on Friday July 26, 2024 @05:20AM (#41077081)

Personally I am very interested on building big data pipeline for machine learning with initially batch and then real-time data of ECG and seismic, for CVDs screening/early detection and earthquakes early detection/prediction respectively. Any idea when the completed book will be available?

Just wondering what is the main difference between your book and this book, Architecting Data and Machine Learning Platforms also from O'Reilly:

https://www.oreilly.com/library/view/architecting-data-and/9...

[ Parent | on HN ]

Re:Applied Machine Learning for Tabular Data by jamesblonde on Friday July 26, 2024 @09:36AM (#41078531)

My book is a hands-on book, where you build AI systems. The first 4 chapters are already out.

I have run a course at KTH for years, here are the AI systems they built in 2024 over a 2-3 week period. There was an earthquake project amongst them!

https://id2223kth.github.io/assignments/2024/ID2223Projects2...

[ Parent | on HN ]

Re:Applied Machine Learning for Tabular Data by usgroup on Friday July 26, 2024 @02:19AM (#41076209)

xgb.cv(X,y) all-the-things.

Its far from a principled statistical approach to what is supposed to be at least an observational study in every case —- and have all the caveats and preparations thereof —- but it is very easy to explain to devs, and one cannot stop the tide unfortunately.

[ Parent | on HN ]

Re:Applied Machine Learning for Tabular Data by wongarsu on Friday July 26, 2024 @12:14PM (#41079821)

I'd replace XGBoost with either LightGBM or CatBoost. Though all three are capable of producing excellent results, and XGBoost has the advantage of not being maintained by an organization you might disagree with.

[ Parent | on HN ]