You don't have to mine the site at all. There's a schema for the data that basically all cooking sites support if they want SEO https://developers.google.com/search/docs/advanced/structure...

Perhaps I'm dim, but I don't understand how that means you don't have to mine the site. Doesn't that simply mean you could perform the process all of the URLs in the database here by making the http request, grabbing those`recipeIngredient` sections, converting units and such, and then populating your simpler database from that?

That seems like pretty textbook (albeit very simplistic) mining to me.

Interestingly, someone built a recipe app with this narrative and received a ton of backlash online: https://www.bbc.com/news/technology-56241653

I love how they say the content that users dislike is "relevant to the recipe", rather than admitting its SEO fodder.

It's actually not SEO fodder (necessarily).

Recipes are not copyrightable but if you pair a recipe with a story, the entire deliverable is copyrightable [0] because the recipe becomes a work of expression rather than a rote list.

> In other words, a bare recipe, without literary expression, is not copyrightable

[0]: https://www.lexology.com/library/detail.aspx?g=3741a5c0-f146....

Interesting point, I hadn't considered this.

I don't think its SEO fodder I think its to space out the page so you have to scroll through multiple advertisements so they get their revenue.

Either way its total garbage.

Yeah. Screw those guys trying to make a living!

I mean, yeah trying to make a living by annoying the snot out of people is a thing that people don't like. It's the "business-model-belle of the ball" at the moment, but that doesn't mean we folks subjected to it have to like it, nor is it somehow "morally wrong" to object to being inconvenienced.

You can try to diminish the real-world confusion and inconvenience of said business model by saying "cmon, it's not that bad, what's the big deal, why not let them make some money, why you gotta be such a stick in the mud?" That does nothing to reduce my or other peoples annoyance and lost time caused by said business model though. It's real, it does waste my time, and your time, and everyone else's time, it does inconvenience us, and it does make it harder to get the information we want.

Because of that, I'm glad to see these other websites arrive to replace blogspam recipe mills. I plan to use websites like http://www.cookingforengineers.com and https://stovetop.app/ search for most all my recipes going forward. Thanks OP for stovetop and thanks to other commenters here for posting more resources for us to use.

I've automated my meal planning using a command line app I wrote in python (https://github.com/steven-p-walsh/menuplannercmd) The app tries to estimate the best menu based on what I like, what I have, what i've made in the past, and something I call slots, which let me give myself more or less time to cook. I also add some randomness to keep things fresh.

As i've built the app, I realized I almost never care about anything more than a rough estimate of the ingredients needed. Even then, I really only care about a few key ingredients. Here is an example (https://github.com/steven-p-walsh/menuplannercmd/blob/master...)

In addition to the other suggestions, a Firefox extension that does this:

https://addons.mozilla.org/en-US/firefox/addon/recipe-filter

I wrote recipe-filter thanks to a comment here on HN years ago!

https://github.com/sean-public/RecipeFilter

https://plainoldrecipe.com/ does this!

It didn´t work with this link: https://www.bakerita.com/oatmeal-fig-bars-gluten-free-vegan/

I've personally found mysaffronapp.com to be way better at this: It has been able to process recipes from every website I've shot at it. I think the developer contributes here to HN, as i learned about the site from a post she/he made.

Man, what a disappointing thread.. the BBC site is great except it’s all weird British ingredients and units (how do I convert grams of British flour into cups of American flour without a scale?)

The OP’s site is a really nice search engine, but it dumps you into the recipes’ unreadable SEO trash pit websites instead of parsing out the data

PlainOldRecipe isn’t working to strip recipes down for me

"it’s all weird British ingredients and units (how do I convert grams of British flour into cups of American flour without a scale?)"

dear God please let this be satire, and make it satire if it isn't

Volume measurement for flour is inconsistent and annoying.

Metric and by weight is by far the best format for recipes and best way to measure ingredients.

In professional kitchens it's all we ever used (I used to work in some top restaurants, talking world's 50 best back in the day).

Oh, that's good to know. Maybe it's time to invest in a kitchen scale. (And I'll buy an actual cookbook while I'm at it, so I won't have to deal with this SEO disaster any more..)

EDIT: Hey, wait, I actually found a good website with recipes: http://online-cookbook.com/goto/cook/rpage/000DDF

and a good search engine: https://search.marginalia.nu/search?query=pancake+recipe&pro...

A scale makes for less dishes too. Just reset the scale and pour next ingredient into the bowl. No need to use another spoon to dig from the flour or sugar bag.

The reason why volume measurements developed in the US rather mass measurements can be traced back to Frankie Farmer’s The Boston Cooking-School Cook Book of 1896. The first cookbook with accurate and repeatable recipes. Volume measurements were a practical matter. Every kitchen had measuring cups. No one had a scale.

https://en.wikipedia.org/wiki/The_Boston_Cooking-School_Cook...

And now you can buy a digital scale for like $20. It's the 21st century, weight measurements are way more accurate, easier too for small quantities.

It's almost as if legacy recipes don't exist, nor are very important to people, nor have working knowledge built around these units.

And? Legacy units can be converted.

I've got cookbooks that date back to the 17th century. Lots of recipes from the 19th century. And old family recipes too. Doesn't mean I won't use metric and weights when I create new recipes, write recipes down, etc... And all my professional cookbooks are metric and by weight.

While this is true and adds effort, it is relatively easy to convert them too.

Interesting, the 1860 historical farm I toured as a kid had a balance and they were careful to not have volume measures at all in the house. Most general store exhibits of that time have kitchen scales and no volume measures.

Not sure if calling the metric system weird is actually serious or not but the solution is to get a scale. Modern cooking highly utilizes the scale and probe thermometers. They seem unimportant to the new cook but I end up using one or both for nearly all my meals now. Dry ingredients especially should be done by weight.

If you’re baking, sure measure by weight for best results. If you’re cooking basic, non-technical meals it’s more about proportions than anything else.

> how do I convert grams of British flour into cups of American flour without a scale?

A web search will turn up dozens of helpful sites that will do the conversion for you. Or the US could try joining the rest of the world.

To be fair, the UK went metric many decades ago yet we still use MPH on the roads; and most people would struggle to tell you how tall they are or how much they weigh using only metric measurements.

Haha, yeah. I wanted to turn my flour, milk, and baking powder into pancakes, but it’s surprisingly nightmarish to get the two floating point numbers I need in order to get the ratios right

“Why not simply…”

I assure you that the optimal way to turn flour, milk and baking powder [no eggs? -Ed.] into pancakes is to keep trying different ratios until it works properly.

There's a LOT of variance in flour, milk and baking powder that make attempts at measurement based recipes a fool's errand.

No baking powder would be a start. The rest is just 3-6-3. That is 3 eggs, 6 dl milk, 3 dl flour and a pint of salt and some melted butter or oil.

The road to my hometown is called 363 so easy to remember. Just have to figure out which of them was the milk now again.

haha,

if somehow you could combine the search of OPs site, plus the garbage stripping convenience of plainoldrecipe, plug in some really good unit conversion system, & save my preferences while you're at it, you might have the ultimate recipe tool.

This. The amount of ads/js on most cooking/recipe sites is terrible.

> it dumps you into the recipes’ unreadable SEO trash pit websites instead of parsing out the data

This is the BS that I as hoping would be removed, but it was left intact unfortunately.

As someone who lived in the US for 8 years... How much is a cup?

A cup is exactly three tenths of a half of a metre wide/long divided by the inverse square of a millilitre, provided the initial measurement was in cubic zlotys at half a pint over Pi.

Let me know then next time you pay for a kilogram of petrol at the pump. Some things make more sense to measure by volume, others by weight. This is disjoint from imperial vs metric.

Fuel would make more sense to measure by mass, because the volume and energy density changes with temperature. It just happens to be easier to measure by volume. I believe many locales mandate pumps which correct for temperature and dispense slightly more volume on a hot day.

Yes, it makes more sense to measure something fluid by volume when it doesn't fit on a scale.

Also, the fact that gasoline pumps are pressurised and standardise the volume based on a certain temperature should be a clue to one pitfall of volume-based measurements.

Precise recipes, i.e. anything related to baking, where ambient temperatures matter - everything is measured in grams, even water. I'm so used to this by now that measuring anything by volume sounds disappointing, let alone dark age units like cups and spoons. I get it, not everyone is serious about cooking enough to own ingredient scales but are these users still the majority of recipe consumers?

I don't bake much but for all my other cooking, I can mess with most ingredients by 100% with little delta in the experience. Sure it tastes a little different but I'm not a factory, this is home cooking. This also makes you much more resilient to missing an ingredient.

Making Coq-au-Vin, and use double the chicken? Probably not a problem. Add carrots, 0 carrots, totally fine.

Barbeque? You can do all sorts of things with the spice mix, it'll likely come out fine.

Making mac and cheese? 4x the cheese, totally fine, just keep adding it till it melts. Too thick? Add cream. Not tasty enough, add some bbq sauce. Measure? Why!

I commonly use a Pinch and a Dash of seasoning. Really it's tasting it and knowing what's off and being able to balance.

Because here's a real truth: every batch of a given ingredient likely has different strength. Dried, fresh, been in the jar for 2 years, brand new from the store, McCormick vs Store Brand vs Different part of the country/world... Also your vegetables will taste different depending on time of year, location, variety, as will your meats, dairy, cheeses, etc, literally everything.

Yes. I cook based on recipes a lot and anecdotally so do a lot of people I know and we still use cups, table/teaspoons and so forth.

The rest of the world uses litres instead of gallons. Much easier to then convert to weight by using molar mass of petrol and volume.

> As someone who lived in the US for 8 years... How much is a cup?

While its not quite right (its a little smaller), at the level of doing a recipe, 250ml [0] (but for flour in baking, you should use weight, not volume, anyway.)

[0] 4 cups = 2 pints = 1qt ≈ 1l

I love the cookbook app for this, you can point it at a URL and it parses the important bits, ingredients, method, pictures etc. and saves the recipe. But it also saves a link to the original incase you want that story.

I primarily use it because it was the only app I could find with a good OCR mode for importing recipes from physical cook books.

A lot of sites have a "Jump to Recipe" link. If this linked directly to that section, it'd be okay.

Cooking sites were pretty good with micro formats a few years ago, so it wasn’t that hard to scrape them

This is already mentioned upstream, but the Paprika app does exactly this. It makes online recipes useful and understandable again.

https://www.paprikaapp.com