Re:Burnout can exacerbate work stress, further promoting a v...
by nikonyrh on Wednesday November 11, 2020 @05:38PM
(#25063868)
Well in addition to all that Data Science stuff I have become quite knowledgeable about several related fields. At least these topics come to mind:
- Data Engineering: various DB technologies' pros and cons (AWS Athena, Snowflake, MS Sql Server, Elasticsearch, Redis), Airflow, AWS S3
- ML Engineering: AWS autoscaling clusters, EC2, Spark cluster set-up & management, IO/CPU bottle neck identification, optimizing workloads (like how large chunks, how many Spark worker processes, how many threads on Python's libraries, ...)
- Umm Web Engineering?: HTTP API design, load balancing, Docker & Kubernetes, partial results caching for real-time responses
I have understood that on larger organizations each of these could have a dedicated team behind it. Anyway I've been lucky to find this company & role and gotten an opportunitu to learn and apply so many technologies, but I must admit it is getting a bit tricky to keep (shall I say pickle) all that in my brain :D
- Data Engineering: various DB technologies' pros and cons (AWS Athena, Snowflake, MS Sql Server, Elasticsearch, Redis), Airflow, AWS S3
- ML Engineering: AWS autoscaling clusters, EC2, Spark cluster set-up & management, IO/CPU bottle neck identification, optimizing workloads (like how large chunks, how many Spark worker processes, how many threads on Python's libraries, ...)
- Umm Web Engineering?: HTTP API design, load balancing, Docker & Kubernetes, partial results caching for real-time responses
I have understood that on larger organizations each of these could have a dedicated team behind it. Anyway I've been lucky to find this company & role and gotten an opportunitu to learn and apply so many technologies, but I must admit it is getting a bit tricky to keep (shall I say pickle) all that in my brain :D