Re:Parsing city of origin and destination city from a string
by PaulHoule on Monday July 06, 2020 @12:49PM
(#23749747)
I think the top answer is right philosophically in terms of putting entity recognition first and predicate recognition later.
Trying to do it with grammar is exactly why you get into the rabbit hole where you get mixed up about Old York and New York. There is that "New York City" that daddy doesn't live in no more, and the "New York State" I live in where you frequently find dirt roads that look passable on GPS and paper maps but aren't.
It is an ontological problem, not a grammatical problem.
You are right that "air traffic level" is key in curating your city and/or airport database. You would start with a list like
and work down from the top. At some point you can put together a "proof" that it works for the top 20% of the airports that get 80% of the traffic and work up from there to cover 95% of the traffic.
For any project like this you need to build a test set if not a training or evaluation set. If you are not willing to look at 1000 examples by hand then don't get started.
People are willing to spend six months fighting with everything from procurement to device drivers to run the same models on the same digits that everybody else runs, but it is hard to get people to make evaluation sets.
I can't guarantee that the people who make those sets will succeed, but it is certain that those who don't make them will fail.
Trying to do it with grammar is exactly why you get into the rabbit hole where you get mixed up about Old York and New York. There is that "New York City" that daddy doesn't live in no more, and the "New York State" I live in where you frequently find dirt roads that look passable on GPS and paper maps but aren't.
It is an ontological problem, not a grammatical problem.
You are right that "air traffic level" is key in curating your city and/or airport database. You would start with a list like
https://en.wikipedia.org/wiki/List_of_busiest_airports_by_pa...
and work down from the top. At some point you can put together a "proof" that it works for the top 20% of the airports that get 80% of the traffic and work up from there to cover 95% of the traffic.
For any project like this you need to build a test set if not a training or evaluation set. If you are not willing to look at 1000 examples by hand then don't get started.
People are willing to spend six months fighting with everything from procurement to device drivers to run the same models on the same digits that everybody else runs, but it is hard to get people to make evaluation sets.
I can't guarantee that the people who make those sets will succeed, but it is certain that those who don't make them will fail.