update plots to reflect epochs used

This commit is contained in:
mm 2023-05-05 02:01:51 +00:00
parent 03313d3904
commit 8bd3e6f55e
5 changed files with 8 additions and 14 deletions

View File

@ -29,13 +29,10 @@ clean:
@rm -rf output/ @rm -rf output/
@rm -rf checkpoints/ @rm -rf checkpoints/
compress: plots/progress_35845_sm.png plots/progress_680065_sm.png compress: plots/progress_136013_sm.png
plots/progress_35845_sm.png: plots/progress_35845.png plots/progress_136013_sm.png: plots/progress_136013.png
@convert -resize 33% plots/progress_35845.png plots/progress_35845_sm.png @convert -resize 33% plots/progress_136013.png plots/progress_136013_sm.png
plots/progress_680065_sm.png: plots/progress_680065.png
@convert -resize 33% plots/progress_680065.png plots/progress_680065_sm.png
install: install:
pip install -r requirements.txt pip install -r requirements.txt

View File

@ -59,14 +59,11 @@ The approach demonstrated can be extended to other metrics or features beyond ge
After training, the model should be able to understand the similarity between cities based on their geodesic distances. After training, the model should be able to understand the similarity between cities based on their geodesic distances.
You can inspect the evaluation plots generated by the `eval.py` script to see the improvement in similarity scores before and after training. You can inspect the evaluation plots generated by the `eval.py` script to see the improvement in similarity scores before and after training.
After five epochs, the model no longer treats the terms as unrelated: After one epoch, we can see the model has learned to correlate our desired quantities:
![Evaluation plot](./plots/progress_35845_sm.png)
After ten epochs, we can see the model has learned to correlate our desired quantities: ![Evaluation plot](./plots/progress_136013_sm.png)
![Evaluation plot](./plots/progress_680065_sm.png)
*The above plot is an example showing the relationship between geodesic distance and the similarity between the embedded vectors (1 = more similar), for 10,000 randomly selected pairs of US cities (re-sampled for each image).*
*The above plots are examples showing the relationship between geodesic distance and the similarity between the embedded vectors (1 = more similar), for 10,000 randomly selected pairs of US cities (re-sampled for each image).*
*Note the (vertical) "gap" we see in the image, corresponding to the size of the continental United States (~5,000 km)* *Note the (vertical) "gap" we see in the image, corresponding to the size of the continental United States (~5,000 km)*
@ -86,6 +83,6 @@ There are several potential improvements and extensions to the current model:
# Notes # Notes
- Generating the data took about 13 minutes (for 3269 US cities) on 8-cores (Intel 9700K), yielding 2,720,278 records (combinations of cities). - Generating the data took about 13 minutes (for 3269 US cities) on 8-cores (Intel 9700K), yielding 2,720,278 records (combinations of cities).
- Training on an Nvidia 3090 FE takes about an hour per epoch with an 80/20 test/train split. Batch size is 16, so there were 136,014 steps per epoch - Training on an Nvidia 3090 FE takes about an hour per epoch with an 80/20 test/train split and batch size 16, so there were 136,014 steps per epoch. At batch size 16 times larger, each epoch took about 14 minutes.
- Evaluation on the above hardware took about 15 minutes for 20 epochs at 10k samples each. - Evaluation (generating plots) on the above hardware took about 15 minutes for 20 epochs at 10k samples each.
- **WARNING**: _It is unclear how the model performs on sentences, as it was trained and evaluated only on word-pairs._ See improvement (5) above. - **WARNING**: _It is unclear how the model performs on sentences, as it was trained and evaluated only on word-pairs._ See improvement (5) above.

Binary file not shown.

After

Width:  |  Height:  |  Size: 230 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 255 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 224 KiB