News for hackers. Stuff that matters.

gjreda

The recent changes have gotten absurd
Grab some baked goods at Hewn (in Evanston), visit the Bahai temple, and then walk across the street to Gillson Park to wander the Lake Michigan shore and eat your goods from Hewn.

The easternmost portion of Northwestern's campus also has a nice walking/biking path along the lakeshore with a great view looking back towards the Chicago skyline.

Re:Ask HN: Who wants to be hired? (February 2024) on Thursday February 01, 2024 @11:56AM (#39218107)

    Location: Chicago, IL
    Remote: Yes
    Willing to relocate: No
    Technologies: python, machine learning (ML), AI, LLMs, HuggingFace, scikit-learn, pydata stack (pandas, numpy, scipy, matplotlib, etc), FastAPI, Pydantic, SQL, Airflow, Spark, Celery, AWS
    Résumé/CV: https://docs.google.com/document/d/1hJpdKFyDNzT4Hs2Ml7XjIBzmSDFSZTT_cFNYp4sPB3w/edit?usp=sharing
    Email: [email protected]
    Website: https://gregreda.com/
For the past ~15 years, I've been an AI/ML engineer and data science leader at high growth tech companies like Instacart, Sprout Social, and Grubhub.

I've had a blast working on the earliest data science problems at Grubhub, to building and leading the data science org at Sprout Social, to scaling real-time logistics systems during the rapid, covid-fueled growth days of Instacart.

After taking a health-related career break [1], I'm ready to get back to work. I’m looking for an individual contributor role in engineering and AI/ML and am particularly excited about roles that have a platform or infrastructure component, allowing me to build systems and tools for researchers, engineers, and scientists.

1. https://gregreda.com/2022/11/30/this-ones-for-me/

Re:Ask HN: Could you share your personal blog here? on Thursday July 06, 2023 @10:13AM (#36616099)
https://gregreda.com/

I've mostly written technical, code-centric posts on Python, ML, and data science. Some of my early posts (2013) were wildly popular at the time and hit the top of HN and various subreddits.

I haven't written much recently, but I've been trying to branch outside of technical posts as I felt like my profession had started to become too much of my identity.

The post I'm most proud of:

- https://gregreda.com/2022/11/30/this-ones-for-me/ - Feeling pride and catharsis after years of bad health luck (leukemia, bad bike crash, cardiac arrest).

My most popular posts:

- https://gregreda.com/2013/03/03/web-scraping-101-with-python... - Web scraping tutorial using Python and beautifulsoup

- http://www.gregreda.com/2015/02/15/web-scraping-finding-the-... - Another web scraping tutorial with Python, but this time for sites that dynamically load content

- https://gregreda.com/2013/10/26/intro-to-pandas-data-structu... - The start of a series of posts on Python's pandas library

- https://gregreda.com/2013/07/15/unix-commands-for-data-scien... - Some useful unix commands for data processing

- https://gregreda.com/2015/08/23/cohort-analysis-with-python/ - Tutorial on doing cohort analysis using Python and pandas

- https://gregreda.com/2017/01/07/freelance-data-science-exper... - My experience as a freelance data scientist

- https://gregreda.com/2018/02/04/hiring-data-scientists/ - My approach to hiring data scientists (though my thoughts on this have evolved over the last five years).

I initially built this same "chat with PDFs" prototype with LangChain and qdrant. I then rebuilt it from scratch for the sake of learning and comparison.

Some context: I've been a jack-of-all-trades data scientist / machine learning engineer for the past 15 years (officially titled as an MLE the last four years).

I share that only because I think it plays a role in how I'm typically accustomed to working.

1. I found LangChain to be overkill for this use-case. While it might allow some to move more quickly when building, I found it to be cumbersome. My suspicion is this is largely because of my background - I understand how to build much of what's "under the hood" in LangChain. Because of this, I think it felt overly abstracted and I found the docs difficult to navigate and sometimes incomplete.

2. I used Qdrant via their docker image and it was simple to setup and start using. I didn't try to push the limits with it, so I can't say anything about performance. Because Qdrant runs as an http service, I found that it didn't fit well into my workflow - I'm accustomed to being able to visually inspect my data inside the interpreter, debugging, trying out commands, interacting and experimenting with my results, etc. Again, my suspicion is this is my own bias in how I typically work. Qdrant otherwise seemed very nice.

3. LanceDB felt powerful yet lightweight, and fit well into my workflow. It was far more intuitive for me. It was as if sqlite, the python data ecosystem, and a vector database had a child and named it LanceDB. Under the hood, it's built on Apache Arrow and integrates nicely with pandas, allowing me to seamlessly go from LanceDB table on disk, to pandas dataframe, and into some analysis or investigation of my LanceDB query results. This line [1] is a great example of why I liked it. This feels nicer to me than the world of API params and HTTP requests.

1. https://github.com/gjreda/scratch-pdf-bot/blob/main/gpt_pdf_...

Re:Falcon 40B LLM (which beats Llama) now Apache 2.0 on Wednesday May 31, 2023 @10:16PM (#36146891)
Not specific to this model, but beyond the large players (OpenAI, Cohere, etc) are there any free hosted versions of the open(ish) LLMs? Even the smaller 7B parameter ones? I'm prototyping out a project and using OpenAI for now, but it feels like there has to be a hosted alternative somewhere.

I spent some time today exploring HuggingFace's Inference API but if the model is sufficiently large (> 10gb), HF requires you to use their commercial offerings.

I recently prototyped out a "chat over PDF documents" project.[1] I opted to use LanceDB for vector (embeddings) storage and retrieval and found it really nice to use.

I'm working on using it in a large project now.

[1] - https://github.com/gjreda/scratch-pdf-bot

> 5/1000 colonoscopy patients have complications (some fatal) which is way higher than the base rate for colon cancer.

Can you provide a source for this?

At least for Twitter, I think this still happens if you move everyone into a list and only view the list.
Re:Ask HN: Fed Rate Hike today – 50, 75 or 100 BPS? on Wednesday June 15, 2022 @10:17AM (#31753278)
Expectations are overwhelmingly for 75bps. Prior to yesterday, 50bps was expected.

- https://www.cmegroup.com/trading/interest-rates/countdown-to...

Imatinib (Gleevac) revolutionized treatment for patients with chronic myeloid leukemia (CML). Prior to the drug’s discovery, CML patients generally had seven years to live (possibly less depending on how advanced the cancer was). Now their lifespan mirrors the general population.

I’d highly recommend the book The Philadelphia Chromosome if you’re interested in learning more.

Recently wrote some code to scrape a friend's reviews and ratings from Goodreads. Maybe it'll be useful to folks here: https://gregreda.com/2020/11/17/scraping-pages-behind-login-...
Re:Grubhub sued for listing restaurants without permission on Thursday October 29, 2020 @03:29PM (#24934590)
Here's an excerpt from their October 2019 letter to shareholders. TL;DR - they're a public company and the markets told them they needed to keep growing in order to survive.

> For restaurant inventory, we will rapidly expand our recent pilots of putting non-partnered restaurants on the platform. For reasons we’ve discussed many times, we believe non-partnered options are the wrong long-term answer for diners, restaurants and shareholders. It is expensive for everyone, a suboptimal diner experience and rife with operational challenges. With that said, it is extremely efficient and cheap to add non-partnered inventory to our platform and it can at least ensure that all of our current and potential new diners have the option to order from any of their favorite restaurants now, even if it’s not the best solution. By leveraging non-partnered options, we believe we can more than double the number of restaurants on our platform by the end of 2020.

> At the same time, because we know that partnered relationships are critical to the long-term success of this business, we will be investing aggressively in our independent restaurant sales organization to support converting as many of these non-partnered restaurants to partnered relationships as quickly as possible and to take advantage of other innovations in the restaurant space, like virtual restaurants.

https://s2.q4cdn.com/772508021/files/doc_financials/2019/q3/...

This was GrubHub's original model. Restaurant's don't want to pay a flat fee when they don't see enough orders they feel is necessary to justify said fee.

[1] https://youtu.be/a2oTps7tKS4?t=893

Re:National Park Typeface on Tuesday June 04, 2019 @02:38PM (#20097926)
Having recently road tripped across the country for a move to SF, I found myself admiring the design of national park signage. It feels very timeless and iconic.

https://i.pinimg.com/originals/13/76/e2/1376e2e5690719701689...

Re:Uber Revenue Slows as Quarterly Loss Surges to $1.1B on Wednesday November 14, 2018 @05:41PM (#18454882)
Does this change when the two options are perfect substitutes?

As a rider, there's no "cost" to having both apps, so I'll just check both.

Similarly, as a driver, I'll just sit with both apps open and see which one I get a rider on.

Re:A cure for cancer: how to kill a killer on Wednesday November 07, 2018 @12:54PM (#18400808)
> I guess I don't really have a reason to post any of this, other than it helps to get it out there

I know that feeling well. Best of luck to you.

Re:A Gentle Visual Intro to Data Analysis in Python Using Pandas on Thursday November 01, 2018 @11:19AM (#18354752)
I agree. Hadley Wickham (a very prolific author of important R libraries) wrote a great paper about this method using one of his libraries. I'm a Python + pandas user, but his paper really helped me understand the approach better: https://vita.had.co.nz/papers/plyr.pdf
Re:Ask HN: Who is hiring? (May 2018) on Tuesday May 01, 2018 @11:59AM (#16968195)
Sprout Social | http://www.sproutsocial.com | Engineering & Non-Engineering | Chicago, IL & Seattle, WA | ONSITE | Full-time

Sprout Social builds social media management tools for businesses of all sizes. We are built on the idea that the world is better when businesses and customers communicate freely. We exist to help streamline and enhance those conversations — with customers, prospects and enthusiasts.

Some openings:

    * Front-End Engineer (Seattle, WA)
    * Senior Front-End Engineer (Seattle, WA)
    * Senior Software Engineer - Platform (Chicago, IL)
    * Staff Software Engineer - Platform (Chicago, IL)
From an engineering perspective, we do not operate as lone wolves, cowboy coders, or "10x devs." Instead, we're building diverse, collaborative teams that get the best results sustainably. We follow Spotify's engineering model with squads being made up of platform, front-end, QA, design, and product managers, all working together to drive our product initiatives to successful outcomes.

Our platform team uses Java, Python, MySQL, and NSQ, while our front-end team uses React, Redux, Ember, ImmutableJS, and Gulp, all to build highly scalable software that is used by more than 20,000 organizations around the world. Companies like Dropbox, Zendesk, Fender, Brooks Running, Seattle Cancer Care Alliance, and Evernote rely on our products to create stronger relationships with their customers.

If you're a creative, highly motivated, and inquisitive learner, we'd love for you to come build great software with us.

https://sproutsocial.com/careers/

Clickbait post title leaves off that this is an estimate from a research firm. Title makes it seem like a sure thing.
Re:Bonobo – A data processing toolkit for Python 3.5+ on Saturday April 22, 2017 @07:08PM (#14175402)
In case you're looking for more, this tutorial series hit the front page of HN a few years ago: http://www.gregreda.com/2013/10/26/intro-to-pandas-data-stru...

Modern pandas is a bit more idiomatic now though: https://tomaugspurger.github.io/modern-1.html

Re:Ask HN: Who is hiring? (April 2017) on Monday April 03, 2017 @12:44PM (#14024398)
Sprout Social | Senior Data Scientist | Chicago, IL | ONSITE | https://sproutsocial.com

Our team uses Python and its data stack (pandas, numpy, scikit-learn) along with a little Java to drive decisions and power software that is used by more than 17,000 brands around the world. Companies like Microsoft, Zipcar, Hyatt, Google, and Zendesk rely on Sprout to create stronger relationships with their customers through social media.

We’re looking for curious, analytical, and creative people to help utilize the vast amount of data we have. If you love finding ways of using data to build better products and solve problems, we’d love to talk with you.

Responsibilities:

    - Build predictive models in support of product and business goals
    - Conduct research to surface new opportunities and understanding
    - Build tooling for all of the above
    - Mentoring, teaching, and helping to level up the team
Requirements:

    - You have professional experience working on data-driven problems. Examples might be classification, recommendations, churn prediction, topic modeling, fraud detection, etc.
    - You’ve built production machine learning pipelines in Python or Java.
    - You have a strong understanding of statistics (probability, experimental design, bayesian methods).
    - You enjoy thinking about business and product, and how data can be used to inform or improve both.
    - You understand, appreciate, and can explain the tradeoffs between simple methods and complex models.
    - You probably studied a technical, quantitative, or analytical field throughout school, but also might have taken a different route. We’re looking for at least a Bachelor’s degree or equivalent.
Nice to haves:

    - Experience working with text data and using natural language processing methods
    - Experience with Apache Airflow, Spark, or AWS Data Products (EMR, Redshift, Kinesis)
    - Experience with deep learning toolkits, such as Keras, Theano, or TensorFlow
Re:Ask HN: Who is hiring? (December 2016) on Thursday December 01, 2016 @02:01PM (#13082332)
Sprout Social | Sr. Data Analyst | Chicago, IL | http://sproutsocial.com/

Sprout Social builds social media software that is used by more than 17,000 brands around the world. Companies like Adobe, Pivotal Labs, Google, Marvel, and Evernote rely on Sprout to create stronger relationships with their customers through social media.

    About the role
    --------------
    - The role is cross-functional and will work extensively with our Data Science, Marketing, and Optimization teams.
    - Our Data Science team uses Python and its data stack (pandas, numpy, scikit-learn, airflow, etc.), but we like R users too.

    Responsibilities
    ----------------
    - Use rigorous analysis, statistics, and a good deal of consideration to help us think about our funnel.
    - Plan, analyze, and report out on split testing experiments.
    - Use data to inform and influence product roadmaps and departmental goals.
    - Build tools to help us make better decisions and automate analyses.
    - Conduct research to surface new opportunities and understanding.
We're also hiring for a variety of other engineering and non-engineering roles: http://sproutsocial.com/careers/open-positions
Re:Add Reactions to Pull Requests, Issues, and Comments on Thursday March 10, 2016 @05:23PM (#11262973)
This is a welcome addition. I've run into bugs in projects before and wanted to "+1" a thread, but it always felt like spamming the maintainers.

It'd be cool if they added a way to search through your list of reactions. This would allow you to effectively comment on an issue in an OS project, while simultaneously bookmarking it, so that you can go back and commit a fix when you have a free moment.

Re:Swirl: Learn R, in R on Friday February 12, 2016 @01:13PM (#11089154)
For anyone interested in data analysis in R or Python (specifically Wickham's dplyr or McKinney's pandas), I can't recommend reading Wickham's paper on the split-apply-combine methodology enough. It really helped me understand working without looping.

The paper's examples might be slightly dated as they use dplyr's predecessor, plyr. I imagine they're pretty similar though.

http://vita.had.co.nz/papers/plyr.html

Re:Advice for companies with less than 1 year of runway on Friday January 22, 2016 @06:01PM (#10956132)
Are drivers really under Uber's control when they can easily have both Uber, Lyft, and any similar app open at the same time?

I think in major cities, both Uber and Lyft are becoming commodities - both of them have ample enough supply that I just choose the cheaper option.

Re:Amazon Kind of Sucks and We’ve All Just Come to Accept It on Tuesday November 10, 2015 @12:44PM (#10540883)
Conspiracy theory: Amazon's sorting is intentionally bad so that you have to look through other items you might purchase, much like grocery stores organize so that you have to work your way through them.

I don't actually believe that, but I do find myself thinking of other things I've needed/wanted when browsing Amazon.

Older »