Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

If anyone else was curious, their sample OSM derived PMTile file (protomaps-basemap-opensource-20230408.pmtiles) is 103GB.

https://docs.protomaps.com/basemaps/downloads



It's 107 GB now.

It would be great to be able to download only diffs and apply them to a daily, weekly, monthly baseline.


I wonder if the compressed Hilbert ordering inside the tiles make them hard to diff efficiently.


The ordering applies across tiles, and not within - it should be efficient to diff two archives because there is a single, well-defined ordering over the entire archive, and writing algorithms over the tileset is all 1D operations instead of 2D (a single zoom level plane) or 3D (multiple planes)


Hmm I was just going off the Firefox estimate, maybe there's some browser difference?


The newest major version has moved to a daily build channel (the last major version was last updated in April)

Today 10/23 is 107GB.

https://maps.protomaps.com/builds/

I'll update the documentation page today.



Exactly the kind of thing that could be downloaded using Bittorrent!


I assume that would require BitTorrent v2 (and I think more specifically something along BEP-46 - Updating Torrents Via DHT Mutable Items) or your map is gonna be out-of-sync and sad after a while.


are the torrents in the wild that uses this? and are there bittorrent clients that implement this? how do they surface the mutable aspect to the user?

(is it like, your torrent client will silently update files, and if you dont want to update anymore you need to pause? can you still seed if you dont want to update?)


I've been testing a setup that automatically generates OpenStreetMap/OpenMapTiles formatted pmtiles and mbtiles, then makes them available by torrent. The torrents get placed in a rss feed at https://planetgen.wifidb.net/

Basically what I did was set up 'qbittorent' client to watch the openstreetmap rss weekly torrent feed. when it downloads a new pbf file, it runs this script I made at https://github.com/acalcutt/PlanetilerTorrent , which was based of the osm torrent creation process.

The script creates pmtiles and mbtiles using planetiler, then makes torrent and starts seeding them in qbittorent.

In qbittorent I have options like how long I want to seed for (30 days), what do do when done seeding (Delete the files) and also speed controls so I can limit bandwidth during my working hours.

I have this running on a old laptop with 2TB nvme/64GB memory. It seeems to work pretty well so far. It would be nice if my internet speeds were a little better for initial seeding, but at least the torrents share the load with other people who are downloading/


That's interesting. But it doesn't use BEP-46, right?

https://www.bittorrent.org/beps/bep_0046.html (discussion on HN at the time, https://news.ycombinator.com/item?id=12282601)


I don't think so because it is not updating the same torrent it is making new ones. I am transmitting the torrent over Bittorrent/DHT like they mention, not http, but without it being the same file I'm not sure that is the same.


You can't modify files, because torrent file itself is just sequence of chechsums. But there is RSS extension that allpo to pipe updates of new files.


But with BEP-46 you can, right? (BEP-46 apparently doesn't use RSS)

I was asking like, how widely is BEP-46 adopted in practice. It's a spec from 2016!

https://www.bittorrent.org/beps/bep_0046.html (discussion on HN at the time, https://news.ycombinator.com/item?id=12282601)


I use it in anacrolix/btlink. There's support in anacrolix/dht.


That's very cool!

Does this serve decentralized "web app" through bittorrent?

Do I need your client to seed it, or can something like qbittorrent seed it?

Will I seed only the last version of the page or all versions? Can someone download an earlier version or list all past versions?


What are they using to compress tiles? Could they get even smaller with AVIF? Or AVIF+SVG overlaid for text?


Yes thank you I was wondering exactly this. Quite reasonable size one might say.


thanks for the info, weird that this isn't answered in the FAQ




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: