wbarber

What's to say this isn't just a demonstration of memorization capabilities? For example, rephrasing the logic of the question or even just simple randomizing the order of the multiple choice answers to these questions often dramatically impacts performance. For example, every model in the Claude 3 family repeats the memorized solution to the lion, goat, wolf riddle regardless of how I modify the riddle.

Looks like we have the same question.

Plasmic.app works with vue - why not just use that?

Very cool. Are you familiar with plasmic.app? I'm expecting to integrate it into my next site build and would be curious for your take having built something in the same general category.

I'm attempting to write a function that splits a long document into shorter segments of text, splitting the text into the topics discussed as a step in a data processing pipeline prior to embedding the shorter segments of text for vector search.

I'm attempting to use v1.0 of the pomegranate python library as I get the impression it will be more performant than some of the other common options I looked at. Below is my code. I'm a self-taught developer just trying to solve a niche problem that's of interest to me so I've not used any of these libraries before or attempted to build a hidden markov model before so be gentle and many thanks for the help.

You can see my current attempt and the error I'm getting at the link I provided. As well as here on stack overflow if you want some internet karma for your kindness: https://stackoverflow.com/questions/76409619/hidden-markov-m...

Thanks for sharing! Looks great, care to share your thoughts on the model decisions in the models.py file. I assume you have a take on the speed vs storage costs vs relevance quality (and/or licensing) of these models vs others here: https://huggingface.co/spaces/mteb/leaderboard and here: https://www.sbert.net/docs/pretrained_models.html:

bi_encoder = SentenceTransformer('multi-qa-MiniLM-L6-cos-v1')

cross_encoder_small = CrossEncoder('cross-encoder/ms-marco-TinyBERT-L-2-v2') cross_encoder_large = CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')

qa_model = pipeline('question-answering', model='deepset/roberta-base-squad2')

I'll follow up via email. Thanks Paul.

Depends a bit what your goals are. But I think generalist are undervalued on average. Range: Why Generalists Triumph in a Specialized World by David Epstein is all about this and he says it better than I can so it’s easiest to just point you there.

There seem to be a few "long rang transformer options" (https://huggingface.co/blog/long-range-transformers). They're slower (not an issue for my use case though). I've heard one talk on this issue mention that there are some "tricks" you can use with web docs (html) like hrefs, to get creative. But more commonly from what I've read, people try different approaches for segmenting the documents, then embed those segments. Heard good things about using a Hidden Markov Model for the segmenting approach.

I hadn't seen that approach to evaluating search engines. But looking at the github repo I'm also not quite following how I would use this/what the standard approach is for scoring relevance ranking approaches, and how this approach differs from that standard approach. If it's not too much trouble a tldr on that would be a really useful intro.

I do intend to have the UX be setup in a way for the user base to sort of re-rank or at least provide some feedback on which results were irrelevant to help with re-training the model over time. For certain applications where very limited domain knowledge is required (for example, is this a hardware product or a sofware product? - or is this a product or a service?), I can also use mechanical turk or similar to label data and I fully intend to do that.

I'm a minimally technical entrepreneur that could use some guidance on piecing together an MVP for a search product.

Some quick background on the idea (also in the linked document):

What Users Want: Ability to quickly generate a list of company websites that match their custom industry definition/query.

MVP Goal: Complex enough to properly validate/reject the idea, simple enough to not be prohibitively expensive or too many man hours to reach validation/rejection step. Ideally, I can add features/complexity to the implementation post validation without completely changing the architecture (for example, adding vector search post initial validation with something like BM25)

Example user input queries: software for catering businesses crane inspection service laboratory reagent suppliers

Output for each case would be a list of relevant businesses they can further filter by relevant criteria like employee count, location, etc.

Some key questions I could use some help answering are: a) At present, how much value will vector search add beyond BM25/BM25F or similar? b) Given the recent rate of progress in LLMs, I'm expecting embeddings for search to improve at a similar rate, and therefore assuming I should expect to be implementing vector based search in the near future even if it's not part of the MVP.

I've share some of my research so far in the linked document. Would really appreciate some feedback on it. How would you build this MVP if you were trying to do it bootstrapped/solo?

This is awesome. Will definitely be playing with this tomorrow.

Unfortunately 90%+ of people need to continue taking the drug to keep the weight off.

It’s an expensive drug but there is increasing competition in the drug category which will be good for consumers in the long run.

Patients do indeed need the drug to keep the weight off. They have proven this. But that is not an issue at all. Individuals who have gained a significant amount of weight do not produce leptin, grehlin, and other related hormones in the same way that people at a normal weight do. Once you’ve put on a significant amount of weight you can’t just expand the size of your fat cells you make new cells entirely. Even if you remove those new cells via liposuction, the body recreates them as a homeostatic response. The result is that something like 10% of obese people that lose weight have kept it off two years later. The people who keep it off have been studied extensively as a group and generally they weigh themselves daily and many of them count calories meticulously. As soon as they stop doing that, they regain the weight. It’s a path dependent outcome. Once you’ve walked the path to obesity, your body fights you walking back towards healthy. This drug adjusts some of the same hormones that get out of whack when you’ve been overweight before. Unfortunately not all of them can be safely played with (giving people leptin and leptin like hormones tends to cause cancer for example). But these seem to be both safe and effective.

There really shouldn’t be a stigma associated with taking drugs like these for the rest of your life.

Very likely so. I know of at least one study in which they are interested in determining if weight loss from semaglutide results in a similar decrease in cancer rates that they see in bariatric surgery patients.

The drugs cause you to eat less and the mechanism of the drug may, in and of itself have some cardio protective effects. To get a drug approved for diabetics, you have to demonstrate it doesn’t cause cardiovascular issues because the comorbidity among diabetics is so high, and other drugs have a history of helping with blood sugars but causing cardiovascular issues. On those measures, iirc, semaglutide specifically showed my that the treatment group saw a significant reduction in cardiovascular events. I’m on mobile so can’t provide links at the moment.

Curious to hear PM's thoughts on scenarios like this when it might make sense to change x or use a specific technology in the first place to keep developers happy or more easily recruit devs.

Anyone know of a company other than Paddle and PayPro Global that act as the merchant of record and remit taxes for you globally?

I'm a solo founder not interested in dealing with either the complexities of these billing issues nor with the taxes issue. But my product would easily be of interest globally and the API's I've built have more interenational customers than global customers so I suspect the same may be true for the product I'm building out of those apis...

Googled for confirmation that the Google CSE index is different from "the real Google Index" and haven't been able to confirm that.

I've tinkered with Google CSE a lot and it does have its uses. The knowledge map options are fun to tinker with even if a bit of a black box.

As for the xml - when you download the xml that's intended to be your starting point for an engine, it doesn't include parts of the definition of the CSE like sites you've listed to exclude. So that functionality seems to be a little broken, but I've confirmed you can add and remove elements of the knowledge graph at least via the xml file.

Lurker MBA student interested in PM roles here. Would be very interested to hear more about what you liked about Kepner-Tregoe, and how you learned it.