adam

Every forecast question we ask has a verifiable outcome. We then score it based on what actually happens in reality and create a track record for all of it. In typical analysis, nothing is scored, and no one goes back and checks if it was right or wrong, but we do - whether we're scoring humans in human forecasting exercises, or seeing how AI does.

Not exactly - all the forecast questions get scored so we know how accurate the AI is and how calibrated it is. It's no different than asking a large crowd of humans on Good Judgment Open or Metaculus these same types of questions. Everything we ask eventually happens or it doesn't, and we can score it all.

In this model the probabilities were measured by an ensemble of LLM's. We give them context and ask them to follow a specific methodology for making a forecast, and then aggregate their forecasts into a "crowd" forecast. We can get input from humans as well to compare/combine them further, but in this particular instance I just used our AI Forecaster.

You're right that a human analyst - especially a very experienced one - is going to be able to still do a better job of analysis most of the time, but the AI is useful for having loops to re-examine the question, consume and curate new information, etc. We've built the system with that assumption: let humans do what they're best at, let AI do what it's best at...

Congratulations on the launch. We sell B2B crowd forecasting software, a fairly niche market, so something like this could be very interesting for us. We sell mostly into governments, think tanks, and other types of research institutions, with the occasional commercial application. A couple thoughts:

Perhaps I didn't do something correctly, but I ended up with a list of companies. That saves me a little time, but I want to be able to then get to something role-based. Is this behind the upgrade or do I never see this information?

Related, I want role to be one of my primary filtering mechanisms. Find me all Chief Risk Officers in x,y,z industry for companies larger than 500 people, for example. Then let me review their backgrounds, get their contact info, and contact them.

I think this is how many sales people / founders operate to try and start to sell. What am I missing?

We run a public crowdsourced forecasting site for the government here: https://inferpub.com. For our various stakeholders, we're using ChatGPT to summarize forecaster's rationales for their probabilistic forecast. We ask it to both summarize the arguments for the event happening, and against. We still need a human in the loop given who the information it's going to, but it's been an excellent starting point.

Somewhat similar conversation going on in the /r/chicago subreddit. Top comment from that thread[1]:

Anthony Bourdain is a far superior wordsmith than myself, so I will leave this here:

Here are Bourdain’s most memorable quotes about Chicago.

“I’ve done shows in LA, but LA’s a fantastic sprawl. San Francisco? A great town. New Orleans? A state of mind. Chicago? Chicago is a city.” — No Reservations, 2009

“Chicago is big — not just any kind of big — I’m talking major metropolis big. I love this city. In my opinion, it’s the only other real metropolis in America.” — No Reservations, 2009.

“You wake up in Chicago, pull back the curtain, and you KNOW where you are. You could be nowhere else. You are in a big, brash, muscular, broad shouldered motherfuckin’ city. A metropolis, completely non-neurotic, ever-moving, big hearted but cold blooded machine with millions of moving parts — a beast that will, if disrespected or not taken seriously, roll over you without remorse.” — Medium essay for Parts Unknown, 2016.

“It is, also, as I like to point out frequently, one of America’s last great NO BULLSHIT zones. Pomposity, pretentiousness, putting on airs of any kind, douchery and lack of a sense of humor will not get you far in Chicago. It is a trait shared with Glasgow — another city I love with a similar working class ethos and history.” — Medium essay, 2016

“Chicago is a town, a city that doesn’t ever have to measure itself against any other city. Other places have to measure themselves against it. It’s big, it’s outgoing, it’s tough, it’s opinionated, and everybody’s got a story.” — Parts Unknown, 2016

[1] https://www.reddit.com/r/chicago/comments/vlxv9m/comment/idx...

2021 Tesla Y owner here. Midwest location. Took a 150 mile trip yesterday. Temps were 38F. Left my house fully charged at 330mi range, battery topped off. Arrived with 100mi estimated range left.

On the way home later that evening, temps were 28F. Left again almost fully topped off with estimated range of 310mi after visiting a supercharger. Arrived home with estimated 70mi of range left.

Many lament the downfall of traditional media then crucify them when they creatively explore new opportunities for themselves.

Should the "Gray Lady" have just stubbornly continued to print its papers until it was bought up by the same PE vipers who have destroyed countless other newspaper companies?

I applaud them for establishing other opportunities for themselves to survive and thrive. IMHO, the NY Times is not the enemy here.

We run the Cosmic Bazaar forecasting service (Economist article: https://www.economist.com/science-and-technology/2021/04/15/...) on their PaaS and their services and people have been excellent to work with all around. Friendly, professional shop.

Also relevant: replacing 4-way's with roundabouts in Carmel, IN: https://www.nytimes.com/2021/11/20/climate/roundabouts-clima...

Having worked in a consulting firm for years, I spent very little time in the office anyway. Instead I was constantly deployed to client sites. I wonder how this will impact those who are working for clients who have different policies. Will the consultants still be remote, or will they have to adhere to their client's policies, some of whom inevitably will require being in the office. In that case, this isn't as sweet a deal as it may seem.

I hope people don't read your comment and choose not to read the article. I was very moved by how Bobby's death impacted his family and fiancee in different ways. It's an extremely well written piece and doesn't deserve (IMHO) the flippant "tl/dr" treatment given its subject matter and quality.

I would encourage anyone with a little extra time to read about the PLATO system. Truly ahead of its time, invented by an engineer at University of Illinois named Don Bitzer https://en.wikipedia.org/wiki/Donald_Bitzer

I was fortunate to have a parent who worked for Dr. Bitzer and we had a PLATO terminal at home, hardwired in to the university mainframe at 1200 Bd. Played most of the games mentioned in the article along with Empire, a Star Trek-inspired multiplayer team game. Great memories.

I'm curious if you allow the project to enter actuals after the fact, then you can compare them to individuals/aggregate estimates so you know how biased your model/individuals/the organization is on these types of estimation efforts. You could give that as feedback as an additional service: "You're constantly overconfident on X type tasks..."

I immediately thought of business travel as well. Project us all in to a virtual conference room and give us a suite of collaboration tools to use while we're all there.

The only thing missing is the after meeting drinks and dinner, but there will inevitably be services to put us all in a restaurant/bar environment, pipe in some bar white noise, have food sent from a local restaurant, etc. for an "in-person" virtual happy hour...

Reminded of this Twitter thread from a couple months ago about how this will all go down: https://twitter.com/ShaanVP/status/1371972261004070913

There's a forecast question on Foretell (a crowd forecasting site run by a think tank at Georgetown) about the likelihood of a successful acquisition if anyone is interested. Current crowd consensus is 41% likelihood, down 2% in the last day: https://www.cset-foretell.com/questions/102-will-nvidia-acqu...

I'm reminded of this article from 2011 about Silicon Valley executives who send their kids to their local Waldorf School...Where tech is forbidden. https://www.nytimes.com/2011/10/23/technology/at-waldorf-sch...