Torrent download automation
By using an RSS feed for .torrent files, a RSS feed downloader and sorter such as Flexget and a Bittorrent client such as Transmisssion, you have a nice combination to automate your content desires. In this configuration, Ubuntu 11.10 is used with two open source programs: Flexget and Transmission.
For who not yet knows what RDF Site Summary (or Really Simple Syndication) is: A way to open up or enable content to be used in other ways than browsing in a traditional form. This blog has an RSS feed that shows the same content as the weblog posts, but in a stripped down format, so other things can make sense of it, such as Google Reader or your iGoogle start page. Firefox also allows RSS feeds to be “Live Bookmarks”.
RSS uses XML (eXtensible Markup Language) that shows that <text> is text and <image> is an image for example. The fun part is that it can be used to show text, but also images (often used by Tumblr users), audio files (used with PodCasts) and much more. A website that has the RSS feature, can be called website feed. In this case, we are using a RSS feed of a torrent file indexing service. Please make sure if you are legally allowed to surf to such sites. Legislation may differ around the world, but technically a .torrent file cannot be harmful. The data interchange with use of a Bittorrent client in uploading mode can be harmful, so please inform yourself before you start.
So we have a nice RSS feed with .torrent files. And then? We use Flexget. This open source piece of software aids us in sifting through the multitude of various relevant and irrelevant content provided in the RSS feed. This is done by accessing the feed, downloading the appropriate content, decide what content should be kept, save the appropriate .torrent file and register the download (to avoid doubles!). All this is done by the script, regardless how the file is named.
With the use of a nice configuration file, you are set to get whatever you would like to get from the RSS feed. To give a practical example: If you would like to download a .torrent file of your favourite singer talent show, you can add the show name to the configuration file, put it in the appropriate position to ensure the video quality, set the download directory for the .torrent file and save your configuration. If you run Flexget, it will retrieve the found results.
The design of Napster has created a whole surge of Peer2Peer packet distribution programs. With the redesign of the P2P protocol to the torrent environment, the file distribution business has been changed forever, allowing for high speed file transfer worldwide using multiple sources (your peers as seeds) and receivers (fellow leechers). An open source variation on Bittorrent (Transmission) does a great job with some nice advanced features. The basic function is that a larger file is divided into smaller packets and you get packages from many people. Just like a sort of Lego, where you can build a file using the designed pattern, but you need to ask friends if you can borrow their brick to duplicate the brick characteristics.
The use of Bittorrent is restricted in some countries. Please check if you legally are allowed to use this kind of software. Transmission is harmless in itself, just like a gun without bullets. However, if you have the gun, the bullets AND pull the trigger (read: enable upload) you are likely to break a law, so be careful! Before adding .torrent files always make sure that the settings are not making you do illegal things.
Okay, from this point on, you are considered warned. Now for the good stuff: you can add .torrent files to your Transmission program to download whatever your heart desires, as long as there are peers that seed the file. Please take into account that the popularity of a file decreases over time. This means that it will be harder to obtain an old file, and sometimes impossible. Furthermore, you will have to be patient, as not always everyone is available 24 hours, 7 days a week to supply you with your desired packages.
This is where it becomes really neat. If you set up Flexget to cover your media need, you could call the program on a regular interval and have the program provide you with a lot of .torrent files that you would like to use with Transmission. But hey, Transmission is capable of monitoring a folder on your harddrive. So if you would push the Flexget .torrent downloads to this monitored folder, Transmission will act on it own…
So, we set up the script, schedule a nice repeat action (every hour) with the use of Cron and let Transmission do its magic.
Edit your config.yml of Flexget for example with gedit using superuser rights. You can find the file in your user directory: ~/user/.flexget/config.yml
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
presets: tv-series: series: settings: 720p: timeframe: 12 hours 720p: - TV Series 1 - TV Series 2 normal: - TV Series 3 - TV Series 4 download: ~/path/to/download/dir/ feeds: feed_1: rss: http://yourserver.com/feed/ preset: tv-series feed_2: rss: http://yoursecondserver.com/feeds/all.rss preset: tv-series download: ~/other/download/path/
Save the file and now you are ready to test your script by calling flexget in your terminal
Flexget will produce some output similar to this:
1 2 3 4 5 6 7 8 9 10 11
2011-12-01 13:06 VERBOSE details feed_1 Produced 30 entries. 2011-12-01 13:06 VERBOSE feed feed_1 REJECTED: `TV SHOW 1` by seen plugin because entry with title `TV SHOW 1` is already seen 2011-12-01 13:06 VERBOSE feed feed_1 REJECTED: `TV SHOW 2` by seen plugin because entry with title `TV SHOW 2` is already seen 2011-12-01 13:06 VERBOSE details feed_1 Summary - Accepted: 0 (Rejected: 4 Undecided: 26 Failed: 0) 2011-12-01 13:06 VERBOSE details feed_2 Produced 50 entries. 2011-12-01 13:06 VERBOSE feed feed_2 REJECTED: TV SHOW 4` by seen plugin because entry with title `TV SHOW 4` is already seen 2011-12-01 13:06 VERBOSE feed feed_2 REJECTED: `HD 720p: TV SHOW 3` by seen plugin because entry with title `HD 720p: TV SHOW 3` is already seen ... 2011-12-01 13:06 VERBOSE details feed_2 Summary - Accepted: 0 (Rejected: 16 Undecided: 34 Failed: 0)
In the above feedback, the ‘seen’ means that Flexget already has downloaded the file or tv show .torrent details earlier and decided upon it to download or not.
If you have never used an automated action before, please consider this manual for Crontab in ubuntu.
Add a cronjob on the user level using with the following command in the terminal:
Please add to your user crontab configuration file the following job:
@hourly /usr/local/bin/flexget --cron
Open the Transmission Preferences to set up automatic adding of .torrent files in a directory: Edit > Preferences. Under the Transmision tab in the section Adding, select the Automatically add torrents from: option and browse to the directory you have defined in the configuration file of Flexget.
Congratulations, now your automated, scheduled downloads are coming in as long as you have your computer running, an internet connection and a RSS feed supplier that works.