I think there's a big advantage to be had for folks brining "old school" ML approaches to LLMs. We've been spending a lot of time looking at the expert systems from the 90s.

Another one we've been looking at is applying some query planning approaches to these systems to see if we can pull responses from cache instead of invoking the model again.

Obviously there's a lot of complexity to identifying where we could apply some smaller ML models or cache-- but it's been a really fun exploration.