brandonmrose

That's definitely the goal! Look at arbitrary datasets and try to understand what's in them. It's particularly important for climate data where aligning different datasets geospatially and temporally can be a challenge.

That's actually a great idea. We use schemas for the categories it detects but could tie them to wikidata entries which is far more generalizable. Will look into this!

We're also aiming to explore using large language models to support/improve this.

Cartwright is a data profiler that identifies and categorizes spatial and temporal features. Cartwright uses deep learning, natural language processing, and a variety of heuristics to determine whether a column in a dataset contains spatial or temporal information and, if so, what is specifically contained.

Cartwright was built to automate complex data pipelines for heterogenous climate and geopolitical data that are generally oriented around geospatial and temporal features (think maps and time series). The challenge that Cartwright solves is automatically detecting those features so they can be parsed and normalized. This problem turns out to be quite tricky, but Cartwright makes it simple.