Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I'm wondering if torrents as such are actually useful for this. I'd figure some kind of virtual file system (perhaps based on BitTorrent) would be very useful. You'd simply pass a file path to an open() routine in your scientific code and data would get opened transparently. You currently have this with URLs and HTTP but there's no useful caching or data distribution.


You might like https://tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/docs/ab... , https://urbit.org/docs/using/filesystem/ , or https://ipfs.io

BTSync and SyncThing are also tools to do this, and I'm sure there are FUSE things to work with BT and block chains ("bittorrent fuse" google results look promising).


With the same principles, http://infinit.sh/ adds security and editability to the mix.


Lots of interesting things to read, thank you! (To the others as well.)


IPFS (https://ipfs.io) does that.


Magnet links from torrents already do this. You just need to have the SHA1 reference of the torrent, working effectively as a url, to load the metadata, and then the files described by the metadata payload.

The P2P nature of the network then help the descentralization of sources, populating several clones of the dataset.


If you have the torrent file, you can use one of the things that mount them as virtual file systems, like btfs (https://github.com/johang/btfs).


Filesystems have distribution problems. You don't always have permissions to install a new filesystem.




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: