kevin948
Sure, the poster's feelings may stem from a 'wider set of technology/tooling', but that doesn't necessarily take from the point. People are sensing that LLM technology is being used as an accelerant for further alienation, whether attributed perfectly to the specific technology or not.
Your point stands that many people like physical labor. Whether they want to artisanally craft something, or desire being outside/doing physical or even menial labor more than sitting in an office. True, but that doesn't solve the above issue, just like it didn't in reverse. Telling miners to learn to code was... not great. And from my perspective neither is outsourcing our thinking en masse to AI.
I'm extremely skeptical of the argument that this will end up creating jobs just like other technological advances did. I'm sure that will happen around the edges, but this is the first time thinking itself is being commodified, even if it's rudimentary in its current state. It feels very different from automating physical labor: most folks don't dream of working on an assembly line. But I'm not sure what's left if white collar work and creative work are automated en masse for "efficiency's" sake. Most folks like feeling like they're contributing towards something, despite some people who would rather do nothing.
To me it is clear that this is going to have negative effects on SWE and DS labor, and I'm unsure if I'll have a career in 5 years despite being a senior with a great track record. So, agreed. Save what you can.
Location: Seattle, WA
Remote: Yes, hybrid ok
Willing to relocate: No
Contracts: No, full-time W2 only
Technologies: Python (including the data stack: NumPy, SciPy, Pandas, Polars, Scikit, XGBoost, Matplotlib, Seaborn, Tensorflow, PyTorch, SHAP), SQL, Map/Reduce, Git, Dagster, PowerBI, Databricks, Big Data Systems, Windows, Linux
Skills: Classification, Regression, Clustering, NLP, Relevance Systems, Recommendation Engines, Semantic Search, Unsupervised Learning, Deep Learning, Embeddings Learning, Graph Analytics, Retention/Churn Analysis, ETL, Data Pipeline Orchestration, Visualization, Business Intelligence
Résumé/CV: https://drive.google.com/file/d/1dXulCDsIkQH_7LbKNCz1YZTPSz2...
Profile: https://www.linkedin.com/in/kevin-field-82061368/
Email: kevin948 [at] gmail [dot] com
Hi, I'm Kevin. I've had experience over 11+ years as a software engineer, data engineer, and data scientist that give me a unique perspective across the stack and makes me a self-sufficient and motivated contributor. With a natural curiosity and passion for learning and analysis, I strive to gain a deep understanding of the challenges at hand. I pride myself on my ability to work across a diverse set of disciplines ranging from NLP to ETL to predictive analytics to graph theory.
My extensive experience in data analysis and machine learning give me the skills and expertise necessary to excel in most DS roles (though I'm open to and have experience data engineering as well). I've applied a wide range of algorithms, from simple linear regression to complex neural networks to extract meaningful insights from data. I tailor the complexity of these algorithms to meet the specific needs and objectives of the business, ensuring optimal results and actionable recommendations. I'm especially excited about embeddings, unsupervised and self-supervised learning, clustering, and relevance algorithms.
Most recently at Microsoft, I identified and addressed a critical need to provide segmentation for millions of businesses that use Microsoft's Office/Cloud suite. By creating playbooks based on businesses' behaviors, we were able to engage tens of thousands of companies simultaneously. This improved field representatives' effectiveness by enabling them to anticipate behaviors and avoid incompatible clients. As the sole Data Scientist leading this project, I conducted extensive research, designed suitable segmentation algorithms, and collaborated closely with various teams across the organization.
In addition to my technical expertise, I excel in communicating complex data analysis to diverse audiences in a clear and concise manner. I thrive in collaborative environments and have a proven track record of collaborating across different organizations and disciplines to drive meaningful business impact.
I look forward to discussing how these skillsets can contribute to the success of your company!
I was wondering if there was a reasonable way to store raw data next to the embeddings such that: 1. Analysts can run queries to filter down to a space they understand (the raw data). 2. Nearest neighbors can be run on top of their selection on the embedding space.
Our main use case is segmentation, so giving analysts access to the raw feature space is very important.
I've had Nash's infinitely collapsible sphere stuck in my head for some time: https://www.quantamagazine.org/mathematicians-identify-thres...
For purposes of nearest neighbors this seems like an incredibly interesting shape to inscribe into: The sphere, despite having spherical properties also maintains linear properties due to the corrugation. To me that means we can try to inscribe orthogonal properties into both of the spaces.
My understanding of these geometries isn't complex enough to make the connections, so my question is this: Do you think its feasible to use shapes with this 'corrugated' property to make better nearest neighbor compression? My intuition tells me that you can use the shape's linear nature to push apart independent components and inscribe the rest of the details into the spherical components. Or perhaps the opposite way.
Hopefully that made sense!
You can do similarity search and all the sorts of things you do for word embeddings on embeddings generated for other scenarios.
As more NN methods become viable, some more savvy data scientists complain to me "this NN is just approximating SVD/PCA/POD/etc!" Wonderful, that's explicitly the point! The network we're applying to this problem compares/combines multiple approaches to dimension reduction. The network created a latent space that makes way more semantic sense than just PCA or SVD for this problem (No Free Lunch). It still takes effort and understanding, but the value I've personally gotten over just applying PCA for my problem-sets has been incredible. In fact I'm certain it has made my career. Turns out diagonalizing covariance matrices aren't the only dimension reduction game in town!
If you wanna get Black Mirror-esque, perhaps a Soma-like medication from Brave New World (essentially pacifies/zombifies you by creating endless bliss) could be made. Or the "bliss" drug episode of Doctor Who.