Scene Releases

BitTorrent (BT) is a peer-to-peer (P2P) communications protocol for file sharing. The protocol was designed in April 2001, implemented and first released July 2, 2001 by programmer Bram Cohen, and is now maintained by BitTorrent, Inc.

BitTorrent is a method of distributing large amounts of data widely without the original distributor incurring the entire costs of hardware, hosting and bandwidth resources. Instead, when data is distributed using the BitTorrent protocol, recipients each supply data to newer recipients, reducing the cost and burden on any given individual source, providing redundancy against system problems, and reducing dependence upon the original distributor.

Usage of the protocol accounts for significant traffic on the Internet, but the precise amount has proven difficult to measure.

There are numerous compatible BitTorrent clients, written in a variety of programming languages, and running on a variety of computing platforms.

OPERATION

A BitTorrent client is any program which implements the BitTorrent protocol. Each client is capable of preparing, requesting, and transmitting any type of computer file over a network, using the protocol. A peer is any computer running an instance of a client.

To share a file or group of files, a peer first creates a “torrent.” This is a small file which contains metadata about the files to be shared, and about the tracker, the computer that coordinates the file distribution. Peers that want to download the file first obtain a torrent file for it, and connect to the specified tracker which tells them from which other peers to download the pieces of the file.

Though both ultimately transfer files over a network, a BitTorrent download differs from a classic full-file HTTP request in several fundamental ways:

* BitTorrent makes many small P2P requests over different TCP sockets, while web-browsers typically make a single HTTP GET request over a single TCP socket.
* BitTorrent downloads in a random or “rarest-first” approach that ensures high availability, while HTTP downloads in a contiguous manner.

Taken together, BitTorrent achieves much lower cost, much higher redundancy, and much greater resistance to abuse or “flash crowds” than a regular HTTP server. However, this protection comes at a cost: downloads take time to ramp up to full speed because these many peer connections take time to establish, and it takes time for a node to get sufficient data to become an effective uploader. As such, a typical BitTorrent download will gradually ramp up to very high speeds, and then slowly ramp back down toward the end of the download. This contrasts with an HTTP server that, while more vulnerable to overload and abuse, ramps up to full speed very quickly and maintains this speed throughout.

In general, BitTorrent’s non-contiguous download methods prevented it from supporting “progressive downloads” or “streaming playback”. But recent comments by Bram Cohen and new developments by Red Swoosh suggest that streaming torrent downloads will soon be commonplace.

CREATING AND PUBLISHING TORRENTS

The peer distributing a data file treats it as a number of identically-sized pieces, typically between 64 kB and 1 MB each. A piece size of greater than 512 kB will reduce the size of a torrent file for a very large payload, but will reduce the efficiency of the protocol. The peer creates a checksum for each piece, using a hashing algorithm, and records it in the torrent file. When another peer later receives that piece, its checksum is compared to the recorded checksum to test that it is error-free. Peers that provide a complete file are called seeders, and the peer providing the initial copy is called the initial seeder.

The exact information contained in the torrent file depends on the version of the BitTorrent protocol. By convention, the name of a torrent file has the suffix .torrent. Torrent files contain an “announce” section, which specifies the URL of the tracker, and an “info” section which contains (suggested) names for the files, their lengths, the piece length used, and a SHA-1 hash code for each piece, which clients should use to verify the integrity of the data they receive.

Completed torrent files are typically published on websites or elsewhere, and registered with a tracker. The tracker maintains lists of the clients currently participating in the torrent. Alternatively, in a trackerless system (decentralized tracking) every peer acts as a tracker. This is implemented by the BitTorrent, µTorrent, BitComet and KTorrent clients through the distributed hash table (DHT) method. Azureus also supports a trackerless method which is incompatible (as of April 2007) with the DHT offered by all other supporting clients.

In November 2006, BitTorrent Inc. introduced its “Publish Torrent” service which creates and hosts a torrent file (seeded from an existing web-hosted media file), and tracks downloads. The service (http://www.bittorrent.com/publish) requires a client which supports web-seeding (currently only the official client, Azureus, and μTorrent).

DOWNLOADING TORRENTS AND SHARING FILES

Users browse the web to find a torrent of interest, download it, and open it with a BitTorrent client. The client connects to the tracker specified in the torrent file, from which it receives a list of peers currently transferring pieces of the file(s) specified in the torrent. The client connects to those peers to obtain the various pieces. Such a group of peers connected to each other to share a torrent is called a swarm. If the swarm contains only the initial seeder, the client connects directly to it and begins to request pieces. As peers enter the swarm, they begin to trade pieces with one another, instead of downloading directly from the seeder.

Clients incorporate mechanisms to optimize their download and upload rates; for example they download pieces in a random order, to increase the opportunity to exchange data, which is only possible if two peers have different pieces of the file.

The effectiveness of this data exchange depends largely on the policies that clients use to determine to whom to send data. Clients may prefer to send data to peers that send data back to them (a tit for tat scheme), which encourages fair trading. But strict policies often result in suboptimal situations, where newly joined peers are unable to receive any data (because they don’t have any pieces yet to trade themselves) and two peers with a good connection between them do not exchange data simply because neither of them wants to take the initiative. To counter these effects, the official BitTorrent client program uses a mechanism called “optimistic unchoking,” where the client reserves a portion of its available bandwidth for sending pieces to random peers (not necessarily known-good partners, so called preferred peers), in hopes of discovering even better partners and to ensure that newcomers get a chance to join the swarm.

TECHNOLOGIES BUILT ON BITTORRENT

The BitTorrent protocol is still under development and therefore may still acquire new features and other enhancements such as improved efficiency.

Trackerless torrents

In May 2005, Bram Cohen released a new beta version of BitTorrent that eliminated the need for web site hosting of centralized servers known as “trackers.” It is now possible to have a torrent up in minutes, with a file, a website, and no understanding of how it works. Cohen explained that the “trackerless” feature is part of his ongoing effort to make publishing files online “painless and disruptively cheap”. The move is only one of several designed to remove BitTorrent’s dependence on centralized trackers. In June 2005, software version 4.2.0 was released, supporting “trackerless” torrents, featured a DHT implementation that allows the client to download torrents that have been created without using a BitTorrent tracker. BitTorrent Mainline DHT: BitTorrent client (4.1.0+), µTorrent (1.2+), BitComet (0.59+), and BitSpirit (3.0+) all share DHT which is based on an implementation of the Kademlia DHT , for trackerless torrents.

This change is said to cause some trouble in the legal efforts to shut down illegal file sharing. However, Tarun Sawney, BSA Asia anti-copyright infringement director, said BitTorrent files could still be identified, since with or without the tracker sites, actual users still host the infringing files.

Search

In addition, Cohen launched a BitTorrent search engine on, which searches the web for .torrent files, including those on popular BitTorrent trackers. BitTorrent has also licensed content which it co-mingles with search results on its website at bittorrent.com.

Opera Software ASA now incorporates BitTorrent downloads through its popular browser software.

Indexing

The BitTorrent protocol provides no way to index torrent files. As a result, a comparatively small number of websites have hosted the large majority of torrents linking to (possibly) copyrighted material, rendering those sites especially vulnerable to lawsuits. In response, some developers have sought ways to make publishing of files more anonymous while still retaining BitTorrent’s speed advantage. The Shareaza client supports three alternative protocols to BitTorrent: eDonkey2000, Gnutella, and Shareaza’s native network, Gnutella2(G2). If the tracker is down, it can finish the file using the other protocols, and/or find new (Shareaza) peers over G2. The use of distributed trackers is also one of the goals for Azureus 2.3.0.2 and BitTorrent 4.1.2. Another interesting idea that has surfaced recently in Azureus is virtual torrent. This idea is based on the distributed tracker approach and is used to describe some web resource. Right now, it is used for instant messaging. It is implemented using a special messaging protocol and requires an appropriate plugin. Anatomic P2P is another approach, which uses a decentralized network of nodes that route traffic to dynamic trackers.

Web seeding

One recently (2006) implemented feature of BitTorrent is web seeding. The advantage of this feature is that a site may distribute a torrent for a particular file or batch of files and make those files available for download from that same web server application; this can simplify seeding and load balancing greatly once support for this feature is implemented in the various BitTorrent clients. In theory, this would make using BitTorrent almost as easy for a web publisher as simply creating a direct download while allowing some of the upload bandwidth demands to be placed upon the downloaders (who normally use only a very small portion of their upload bandwidth capacity). This feature was created by TheSHAD0W, who created BitTornado. From version 5.0 onward the Mainline BitTorrent client also supports web seeds and the BitTorrent web site has a simple publishing tool that creates web seeded torrents. The latest version of the popular download manager GetRight supports downloading a file from both HTTP/FTP protocols and using BitTorrent.

Broadcatching

Another proposed feature combines RSS with the BitTorrent protocol to create a content delivery system, further simplifying and automating content distribution. Steve Gillmor explained the concept in a column for Ziff-Davis in December, 2003. The discussion spread quickly among bloggers (Techdirt, Ernest Miller, Chris Pirillo, etc.). In an article entitled Broadcatching with BitTorrent, Scott Raymond explained:

“I want RSS feeds of BitTorrent files. A script would periodically check the feed for new items, and use them to start the download. Then, I could find a trusted publisher of an Alias RSS feed, and ’subscribe’ to all new episodes of the show, which would then start downloading automatically — like the ’season pass’ feature of the TiVo.”

The RSS feed will track the content, while BitTorrent ensures content integrity with cryptographic hashing of all data, so subscribers to a feed receive uncorrupted content.

An early implementor of this approach is the IPTV show mariposaHD, which uses BitTorrent to distribute large (2-4 GB) WMVHD files of high-definition video.

One of the first software clients ( free and open source) for broadcatching is Democracy Player. Other free software clients such as PenguinTV and KatchTV are also now supporting broadcatching.

APIs

The BitTorrent web-service MoveDigital has the ability to make torrents available to any web application capable of parsing XML through its standard Representational State Transfer (REST) based interface. Additionally, Torrenthut is developing a similar torrent API which will provide the same features, as well as further intuition to help bring the torrent community to Web 2.0 standards. Alongside this release is a first PHP application built using the API called PEP which will parse any Really Simple Syndication (RSS 2.0) feed and automatically create and seed a torrent for each enclosure found in that feed.

Encryption

Protocol header encrypt (PHE) and Message stream encryption/Protocol encryption (MSE/PE) are features of some BitTorrent clients that attempt to make BitTorrent hard to throttle. At the moment Azureus, Bitcomet, BitTyrant, BitTornado, kTorrent and µTorrent support MSE/PE encryption.

Some ISPs throttle BitTorrent traffic because it makes up a large proportion of total traffic and the ISPs don’t want to spend money purchasing extra capacity. Encryption makes BitTorrent traffic harder to detect and therefore harder to throttle. Recently, ISPs have announced possible future hardware upgrades in order to minimize BitTorrent traffic.

Several universities have already taken these steps, including the University of Maryland at College Park, Emory University, Brigham Young University, ASU, UTC, University of Washington, Washington University in St. Louis, and WPI.

Tor workaround

It is possible to route BitTorrent traffic into the .onion domain via Tor, although some consider the use of BitTorrent over TOR to be counter to the goals of the TOR project.

* However, client and server support for this is nearly non-existent, and there is generally a loss in network efficiency when doing so.
* Future versions of BitTorrent may support Tor, but this may take two or three years to happen.

Peer exchange

Peer exchange (PEX) is another method to gather peers for BitTorrent in addition to trackers and DHT. Peer exchange checks with known peers to see if they know of any other peers.

Multitracker

Another unofficial feature is an extension to the BitTorrent metadata format proposed by John Hoffman and implemented by several indexing websites. It allows the use of multiple trackers per file, so if one tracker fails, others can continue supporting file transfer. It is implemented in several clients, such as BitTornado, KTorrent and µTorrent. Trackers are placed in groups, or tiers, with a tracker randomly chosen from the top tier and tried, moving to the next tier if all the trackers in the top tier fail.
Torrents with multiple trackers, called MultiTorrents by indexing website myBittorrent.com, can decrease the time it takes to download a file, but also has a few consequences:

* Users have to contact more trackers, leading to more overhead-traffic.
* Torrents from closed trackers suddenly become downloadable by non-members, as they can connect to a seed via a open tracker.

Source: Wikipedia