Coding a new BitTorrent tracker?

01 06 | 2011

Coding a new BitTorrent tracker?

Written by Tanguy

The BitTorrent protocol works with trackers, that track the clients or peers that share a given file. This is one specific design of a peer-to-peer protocol, very appropriate for official distribution: for instance, Debian runs a tracker for distributing the installation images.

The problem

There exists many BitTorrent clients, but, for what I have seen, few BitTorrent tracker. Within Debian, I think the only trackers available are the original BitTorrent's one, and its fork BitTornado. Both suffer from serious drawbacks:

they have been unmaintained (upstream) for years;
they provide both a tracker and a client in the same package, whereas a BitTorrent client and a tracker do not have much in common, and most people are only interested in the client, so the package maintainer had to completely disable the tracker daemon, which complicates the task to get a working tracker;
the tracker's logging simply sucks: they had the brilliant idea to bufferize their log, with what seems no buffer control at all, so you get half-written lines, that get complete half an hour later, and so on.

Proposed solution

Now that I have criticized a lot, let us be more constructive. All these issues could be fixed by modifying the code and the package (well, by adopting it, for the first one). I think it is not worth doing so, exactly because they implement both a client and a tracker: there are many good BitTorrent clients so there is no real need for them as clients, and because a tracker's job is quite different, and easier than a client's, it would be useless to maintain a client with the tracker.

So, I am considering coding a BitTorrent tracker. Even though the tracker protocol is based on HTTP, it is completely stateful so it would be a standalone daemon rather than a CGI-style program. I would code it in Python because this is the language I currently prefer, and I may call it tbtt (the BitTorrent tracker, or Tanguy's BitTorrent tracker, or transport blindé tout terrain (all terrain armoured transport)).

If you have any information or suggestion, please comment! Perhaps I missed some existing tracker that would do the job instead of coding a new one?

Tags : BitTorrent, BitTornado, tracker, daemon, tbtt, python

20 comments

wednesday 01 june 2011 à 23:51 Bob said : #1

It's not written in python, but there are some trackers out there :
http://code.google.com/p/torrentpier/source/checkout?repo=xbtt
http://code.google.com/p/torrentpier/source/checkout?repo=ocelot

There are written to power some private trackers, so they include some code for computing stats.

Maybe this will help...

wednesday 01 june 2011 à 23:52 stbuehler said : #2

http://erdgeist.org/arts/software/opentracker/
and you may want to provide your comment form with english labels.

wednesday 01 june 2011 à 23:52 Jochen said : #3

There's Opentracker as well:

http://erdgeist.org/arts/software/opentracker/

It doesn't appear to be *that* well maintained (last update in, *cough*, CVS, *cough* was five months ago), but I assume Erdgeist is still maintaining it.

wednesday 01 june 2011 à 23:58 Tanguy said : #4

I already had a look to opentracker. It looks like it is coded by someone that does not care at all about license, so the only thing it has that looks like a license is three like that talk about beer and say nothing about using, modifying or whatever. This is not a free license, alas, this is default copyright, meaning: you have the right to do nothing.

wednesday 01 june 2011 à 23:59 Tanguy said : #5

@Bob: It appear that I forgot to mention something I expect from a BitTorrent tracker: to use no relational database. :-)

thursday 02 june 2011 à 00:17 Anonymous said : #6

Transmission seems to work too.

thursday 02 june 2011 à 01:15 Bob said : #7

@Tanguy: the database must be for the per user stats. IMO, it can be easily removed from the code...
FYI, Ocelot is powering What.cd, and it's working quite well!

thursday 02 june 2011 à 03:42 Anonymous said : #8

The license is Beerware http://en.wikipedia.org/wiki/Beerware, and should be fine, Debian for example has plenty of code with such license already.

And for a list containing other existing free software implementations see http://en.wikipedia.org/wiki/BitTorrent_tracker_software

thursday 02 june 2011 à 14:49 Cùran said : #9

There was a very good introduction to opentracker at 24C3, what I remember most, was the amount of load opentracker could handle on really small hardware setups, so in case you want to watch it too, go to [0] (I know, the video is in German, but maybe you understand it or know somebody who can translate it for you).

Cheers,
Cùran

[0] http://events.ccc.de/congress/2007/Fahrplan/events/2355.en.html

thursday 02 june 2011 à 16:21 nion said : #10

use opentracker, no reason to code yet another one
xbtt is crap, ocelot is okish but is more focusing private trackers with ratio rules.

sunday 05 june 2011 à 19:06 Tanguy said : #11

@nion: Thank you for this information. I shall try and contact opentracker's author to see if he can make its license more clear, because it cannot be considered as free software with what is currenctly written.

wednesday 08 june 2011 à 10:19 Tanguy said : #12

Some news about opentracker. Although it is intended to be free software it suffers from a license bug: its license statement is not explicit enough be considered free by distributors such as the Debian project. The author does not seem to be willing to change that, so unfortunately opentracker will probably never enter Debian.

I regret that but it will give me the opportunity to try and code a tracker, although I do not expect produce anything as good as opentracker. :-/

friday 10 june 2011 à 15:33 Jeff Schroeder said : #13

What about DHT? Major bittorrent sites like the pirate bay long ago moved away from the centralized bittorrent "tracker" model to the Distributed Hash Table model.

Very brief overview: http://wiki.bitcomet.com/using_dht_tracker

monday 13 june 2011 à 11:39 Tanguy said : #14

@Jeff Schroeder: Excellent, I shall test that. If it can seamlessly replace tracking, then it is probably the best solution indeed, so the only thing I would then have to code is a metainfo (aka torrent) file generator, which is rather easy.

thursday 13 september 2012 à 16:12 erdgeist said : #15

Sorry for the late reply to that thread. No one here ever really contacted me about the licence issues, so I never had the chance to get stuff straight.

If anyone ever would have asked, I would have provided a special permission to re-distribute opentracker under every brain-fart-licencing scheme is the flavour of the year now. Hand written, if necessary. I'm astonished what the community nowaday demands from programmers to graciously accept the software given away for free.

Take it or leave it.

thursday 13 september 2012 à 17:14 Tanguy said : #16

@erdgeist : Well, I clearly remember contacting you about that. Your answer was something like: “I am not interested in licensing issues and I do not want to spend time on that; I chose beerware to avoid such questions and I will not change the license, take it as it is or leave it.”

At that time, I was looking for a clean way to distribute files, and given the rotten state of the traditional simple tracker (the original BitTorrent one and its fork BitTornado), I was considering OpenTracker for that goal so I was ready to package it for my convenience, and hopefully for other users or potential users too. Your refusal of clarifying the license issue made me abandon that idea.

Since that time, I realized that a tracker was no longer necessary to distribute files on BitTorrent, thanks to the DHT system. For that reason I coded myself a modern torrent generator, which I am using with the DHT now. If you think having OpenTracker in Debian could be beneficial for users, I can still package it though, provided that you license it under a free license, preferably a well-known one.

tuesday 18 december 2012 à 05:18 Ben said : #17

I, too, am interested in seeing opentracker added to Debian (and Ubuntu).

We use opentracker at http://indietorrent.org and it has served us very well. It would be nice to see opentracker's development continue and for the software to be documented thoroughly.

The prospect of packaging opentracker for distribution in Debian is already being discussed at: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=685575 . (I have brought this post to the package maintainers' attention.)

Its inclusion in Ubuntu is also being discussed: https://bugs.launchpad.net/ubuntu/+bug/518711 .

Thanks for taking the time to put the "official word" into the record, erdgeist!

tuesday 18 december 2012 à 09:55 Tanguy said : #18

@Ben : Well, currently I have no specific interest in packaging OpenTracker anymore. Because of that licensing bug, and as my discussion with the author was not successful in clarifying that point, I looked for other solutions and decided to look at the DHT that allows for trackerless torrents. So far, I am satisfied with that.

Now, I do not share your optimism, because in my private discussion with the author, it appeared he was willing to allow anyone to redistribute OT but not to relicense it, whatever that could mean. Also, he did not want to simply clarify this by making the license explicit in the source package itself, which could have been done by either copying the entire text of the license (three lines only!) or giving a link to it. In fact he appeared to be quite annoyed by the discussion so I was not able to continue it much further. So basically, I agree that this program is intended to be free, but that was never properly explicited, so to me this “license bug” is still there, preventing OT to enter Debian or Ubuntu. Of course, if it is packaged, the decision of including it into Debian will ultimately be the FTP masters'.

sunday 06 january 2013 à 18:44 gnuzer said : #19

Sorry to dig this up, but this discussion is ridiculous. Beerware *is* a clear enough license and is a free software license for sure.

Fedora considers Beerware free ans GPL-compatible : https://fedoraproject.org/wiki/Licensing/Beerware

I don't know if debian includes beerware-licensed software, but they already include software under similar licenses such as WTFPL.

Legitimity of copyright relies on the wish of the author, on the contract established between the author and the rest of the world. And the author has given his word : you can do anything you want, period. If the word of the author doesn't count, then why should we respect copyright ?

Some people might think that a statement without proper formatting and ugly dislaimers written in caps has no moral value. But what I read in the comments of erdgeist (and in the words of Poul-Henning Kamp which are behind the spirit of Beerware) is much more a proof of good faith to me than any juridical bullshit.

PS: I upvote the idea of packaging opentracker for debian repos.

sunday 06 january 2013 à 21:45 Tanguy said : #20

@gnuzer : A licensing statement is acceptable if it is clear. For some reason, when I thought about packaging OpenTracker, I thought the plain sentence “ this is beerware” was not clear enough, so I asked the author to clarify it, by adding a link to a page where that license is written. His reply was a clear refusal so I just gave up. Now that I have found another way to do what I wanted to do, which is distributing files with BitTorrent, I no longer have any reason to package OpenTracker. If you think it is acceptable for Debian (in other words, if you think the FTP masters will accept it) and if you have time to package it, please go ahead. I will not.

Rss feed of the article's comments

Tanguy Ortolo