mm/citybert
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

details in readme

Browse Source
This commit is contained in:
mm 2023-05-04 10:22:17 +00:00
parent fab8952d59
commit e9adbed41a
1 changed files with 10 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
README.md
View File

@ -1,8 +1,8 @@
# city-transformers
# citybert
Generates dataset of cities (US only for now) and their geodesic distances.
Uses that dataset to fine-tune a neural-net to understand that cities closer to one another are more similar.
Distances become `labels` through the formula `1 - distance/MAX_DISTANCE`, where `MAX_DISTANCE=20_037.5 # km` represents half of the Earth's circumfrence.
1. Generates dataset of cities (US only for now) and their pair-wise geodesic distances.
2. Uses that dataset to fine-tune a neural-net to understand that cities closer to one another are more similar.
3. Distances become `labels` through the formula `1 - distance/MAX_DISTANCE`, where `MAX_DISTANCE=20_037.5 # km` represents half of the Earth's circumfrence.
There are other factors that can make cities that are "close together" on the globe "far apart" in reality, due to political borders.
Factors like this are not considered in this model, it is only considering geography.
@ -12,5 +12,10 @@ However, for use-cases that involve different measures of distances (perhaps jus
A particularly useful addition to the dataset here:
- airports: they (more/less) have unique codes, and this semantic understanding would be helpful for search engines.
- aliases for cities: the dataset used for city data (lat/lon) contains a pretty exhaustive list of aliases for the cities. It would be good to generate examples of these with a distance of 0 and train the model on this knowledge.
- time-zones: encode difference in hours (relative to worst-possible-case) as labels associated with the time-zone formatted-strings.
see `Makefile` for instructions.
# notes
- see `Makefile` for instructions.
- Generating the data took about 13 minutes (for 3269 US cities) on 8-cores (Intel 9700K), yielding 2,720,278 records (combinations of cities).
- Training on an Nvidia 3090 FE takes about an hour per epoch with an 80/20 test/train split. Batch size is 16, so there were 136,014 steps per epoch
- **TODO**`**: Need to add training / validation examples that involve city names in the context of sentences. _It is unclear how the model performs on sentences, as it was trained only on word-pairs.