LawrenceHecht

Building or running data infrastructure is an important part of 55% of 372 data engineers’ jobs, according to the “2020 Kaggle Machine Learning & Data Science Survey.” ...Check out three charts I created showing differences between data scientists, data engineers and machine learning engineers: https://thenewstack.io/software-engineers-use-spreadsheets-d...

Monoliths are "part of the future". Data shows that there is dissatisfaction with using both monolithic and microservices architectures at the same time. In 2017, 66% were using hybrid approach to approach, but only 54% in 2019. See the interactive graphic at the bottom of https://thenewstack.io/observability-and-elk/.

Yes, this was a big job -- one of the reasons I stopped working on something similar 1.5 years ago was that I wasn't capturing the non-GitHub activity. Of course that back then I also wasn't controlling for bots.

Hey, you worked on the project as a freelancer, right? Would you be interested in chatting with me about how to do this again in the future?

Just checked, the # of data engineers rose to 9,246 (42%) in the last six months. So, the shortage is at least being addressed by people changing their job titles on LinkedIn.

I'm not a regular on Hacker News, but thought my tweet would be worthy here too:

Based on our analysis of LinkedIn, there are 1.7 developers for each SW engineer. Raw data here: bit.ly/1QOobbN

the interesting bit is looking at how users of containers are behaving, not the % or marketshare