[WIP] bittorrent dht crawler
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
Anton Luka Šijanec 872a765eee initial commit, UNTESTED bencoding parser 1 week ago
src initial commit, UNTESTED bencoding parser 1 week ago
.gitignore initial commit, UNTESTED bencoding parser 1 week ago
Makefile initial commit, UNTESTED bencoding parser 1 week ago
README.md initial commit, UNTESTED bencoding parser 1 week ago

README.md

travnik - NOT IMPLEMENTED YET, COME BACK LATER!

... is a tool that connects to the bittorent dht network and waits for infohashes of torrents, fetches their metadata, stores it in a database and indexes them via a web-interface. It's meant to be a lighter and simpler alternative to btdig.com's erlang crawler.

travnik operates single-threadedly, including the BEP-5 (DHT), BEP-9 (metadata exchange) and HTTP client.

travnik implements BEP-3 (bencoding), BEP-5 (DHT) and BEP-9 (metadata exchange) itself, other things (mysql client and http server) are handled by libraries.

requirements

build-essential, libmicrohttpd-dev, default-libmysqlclient-dev

installation

debian and derivatives users add my package repository (prog.sijanec.eu)[https://prog.sijanec.eu] and apt install travnik.

other users compile from source with make.

usage

debian users get a systemd service installed and can start travnik with service travnik start. logs are written to the default log (journalctl -xe or service travnik status). travnik is started as the user nobody.

other users can execute ./travnik; the command hangs and reports logs to stderr. it's important to start travnik as a user with the least amount of priviledges, nobody for example.

travnik then starts crawling and opens up a web interface on port 8728, open http://localhost:8728 for searching and exporting the database. 8782 can be remembered by picturing letters TRAV on a mobile phone keyboard.

prebuilt binaries

... are available for my CI/CD server. check the badge below if the latest release was built successfully.

Build Status