资源说明:A Ruby on Rails bittorrent tracker and community application.
This project aims to provide the basic functionalities required by a private bittorrent tracker and community. It is built with the Ruby language and the Ruby on Rails framework. - Live Demo: http://mogok.morphexchange.com (thanks to Morph Labs: http://mor.ph) - Demo Video: http://www.youtube.com/watch?v=ep_dUaTb1GU (watch in HD) - Screenshots + http://bayimg.com/image/oaamaaacg.jpg + http://bayimg.com/image/oaamcaacg.jpg + http://bayimg.com/image/oaamhaacg.jpg - Current State: Beta Testing - Licence: see doc/MIT_LICENCE - History/Changelog: see doc/HISTORY - Contact: mogoktracker[at]gmail ABOUT SOME OF THE FEATURES - Torrent categorization Torrents are categorized by Type, Category, Format and Tags. Such structure allows to organize the torrents in specific or generic forms. Check below examples on both scenarios (note that users can't choose or even see the torrent type, this is done implicitly when a category is selected). + A community specialized in music: - this structure is configured as the example in the create_data migration file type[audio] formats[MP3, OGG, FLAC] category[Rock] tags[acid, rockabilly, indie] category[Blues] tags[east-coast, british, st-louis] category[Jazz] tags[swing, cool, electric] type[video] formats[AVI, OGG, MPG] category[Master Class] tags[popular, classical, theory] category[Music Video] tags[rock, pop, arthouse] type[ebook] formats[PDF, DOC, LIT] category[Score] tags[guitar, violin, big-band] category[Theory] tags[popular, classical] type[software] formats[EXE, ZIP, RAR] category[Software] tags[windows, linux, mac] + A community with more generic content: type[music] formats[MP3, OGG, FLAC] category[Music] tags[rock, blues, jazz] type[movie] formats[AVI, OGG, MPG] category[Movie] tags[podcast, documentary] type[ebook] formats[PDF, PPS, DOC] category[eBook] tags[literature, manual, self-improvement] - User Roles Users are classified by roles, which define their rights to perform certain operations. The colors of the links in the interface indicate the minimum role that has access to it. There are a set of reserved roles required by the application (check create_default_data migration file): system, owner, admin, mod, user and defective. The system role is supposed to be used by just one user (must have id = 1) and is used to identify automatic messages (system user won't accept messages). Users with roles system, owner and admin can edit the profiles of users with lower ranking. Roles and users also have tickets that grant them rights, those currently in use are: + inviter: grants the right to send invitations + staff: grants right to see and edit staff info + perpetual: will not be inactivated or removed if absent from the site + wisher: allows user to request torrents - Ratio policy The application will apply ratio enforcement rules and promote users based on their ratio and upload credit, check the background tasks item below for more on that. Cheat detection scripts are not implemented though. - Bonus You can specify bonus rules for users who upload data in certain conditions. Currently the application will give a 50% bonus for the data uploaded when seeding a torrent, check app_config.yml and lib/bittorrent/tracker.rb. - Mail The application can send emails for events like password recovery and invitation. It comes with a test configuration that uses a regular Gmail account (just add a valid username and password to config/initializers/action_mailer.rb). The plugin to access Gmail is action_mailer_optional_tls, it is in vendor/plugins. As configured in action_mailer.rb, mailer won't throw errors in development mode. - Logs Several types of logs are provided so you can check what is going on with the app: + error logs + tracker announce logs (can be turned off) + activity logs + background tasks logs - Error logs The application will alert you when an exception is thrown by the application, you can see what happened in the error logs page. In production mode only unexpected errors will be logged. - Application params Some configuration parameters are stored in the database so they can be changed without restarting the application, although they have to be queried every time they are used. Values can be anything loadable by YAML. - Login Block After a limited number of login attempts (currently 5) the used IP will remain blocked by a period of time (currently 4 hours), but if the user changes its password using a password recovery email the block is cancelled. - Signup After a successful signup the used IP will be blocked for a period of time (currently 1 day). This is a simple measure to prevent automated signups and a more sophisticated mechanism should be implemented if necessary. Currently the application won't send an activation email to a new user. - Announce URLs - are used by the bittorrent clients to send requests to the tracker - they contain a special value called announce passkey, that identifies the user who is sending the request (e.g. http://localhost/tracker/USER_ANNOUNCE_PASSKEY/announce) - when a user downloads a torrent file, this special url is included in it. this means that if a torrent is stolen, the thief will be able to download using the user's credits. to minimize the damage the application generates the announce passkeys combining the user personal passkey and the torrent announce key (using OpenSSL::HMAC), so if a user has one torrent stolen it will compromise only that particular torrent - users have to be oriented to reset their passkeys if they notice any suspicious activity in their accounts, like peers with strange IPs or unusual ratio variation - Torrent Upload - the application will accept any torrent, even if the announce url is from another tracker or empty (it will be replaced by the user's personal announce url) - if the torrent is not set to private the application's torrent parser will add a private flag to it (such flag indicates that the torrent belongs to a private tracker, so the bittorrent clients will block DHT and only peers registered in the tracker will be allowed to join that torrent). - Requests This feature allows users to request torrents that are not uploaded yet. After a request is created users can add bounties to it as a way to reward who fills that request. After a request is filled it will stay in pending status until a moderator approves or rejects the filling. Only after the approval the bounty will be granted to the filler. Note that internally 'requests' are called 'wishes' to prevent naming conflicts with Rails' requests. - Rewards Users can reward torrent uploaders by adding rewards to torrents. This will transfer upload credit from the rewarder to the torrent uploader. - Reseed Request Users can ask for other users to reseed a dead torrent, that will cost the requester upload credit (define the amount in the file app_config.yml). Once the request is made, the user who created the torrent and some (also configurable) of those who snatched it will receive a message. - Background Tasks These are tasks that run periodically to keep the site tidy. They clean stale data, update things, etc. - feature components: + BgTasks module: a dispatcher to invoke tasks and utility methods to load the tasks (lib/bg_tasks/) + BgTask model: contain all the business logic of the tasks (app/models/ and app/models/bg_task/) + BgTaskParam model: holds the task params in yaml format + configuration file: define the tasks and their behavior (config/bg_tasks.yml) - to add a new task create an entry in bg_tasks.yml, add a new bg_tasks concern file (just follow the pattern in files bg_task.rb and in the concern files in app/models/bg_task/) and reload the tasks - tasks also can be executed individually in the tasks page (that won't alter the schedule) - there is a log page where the tasks report their execution status - if next_exec_at is empty the task won't be executed, just scheduled - if interval is empty the task will be ignored - Implemented tasks (see app/models/bg_tasks/) + cleanup delete or inactivate records in various tables. + cleanup_peers delete inactive peers based in a inactivity limit period. + refresh_stats gather stats about users, torrents, peers and so on. + promote_demote assign roles to users based on their ratio and uploaded data amount. + ratio_watch require users to maintain a minimum ratio based on how much data they've downloaded. when users violate the ratio requirements their roles are set to 'defective' and a date is defined as a limit for them to improve their ratio. after this date the system will check if the defectives' ratio is ok and set their roles back to 'user' or invalidate/remove them. - tasks are supposed to run as a cron job, but there is a mock for testing + running the mock - also works in Windows - will run in 'development' mode by default - run with the commands: $ ruby script/bg_tasks_mock.rb $ ruby script/bg_tasks_mock.rb production + running as a cron job - the 'tasks' page has a link (can be disabled in app_config.yml) to include cron jobs in your system automatically (it invoke a system call to the 'whenever' gem) - to include the cron jobs, configure the scheduling in the file 'config/schedule.rb' (it is already configured to run every hour) and click 'update' in the 'tasks' page, you should see the new cron jobs in the cron jobs table (note that if the app is deployed in more than one machine you'll have set the cron tabs manually in the specific one). - IMPORTANT: file script/runner must be executable (check if you're not deploying with Capistrano) - to manually edit the cron jobs use the commands below: - using the 'whenever' gem: $ whenever --update-crontab mogok (same command called by the application as mentioned above) - using the crontab tool: $ crontab -l (list cron jobs) $ crontab -e (edit cron jobs) $ crontab -r (remove all) - example (runs every hour at minute 5): 5 * * * * /var/vhosts/mogok/current/script/runner -e production "BgTasks::Dispatcher.exec" # SOME TECHNICAL THINGS - Rails version 2.3.2 - Database The application requires MySQL as it uses database specific column types and features. - Search engine It uses the MySQL's fulltext indexation, see migration files for torrent_fulltexts and topic_fulltexts. The information was duplicated in these tables because fulltext indexes require the table to be of type MyISAM. - Debugging To make the application flow more understandable, the code is filled with debugging logs that are printed by the Rails framework (won't be printed in 'production' mode). For a friendlier view, the logs start with smilies: :-) - everything is ok :-o - something out of ordinary happened, but most likely it is the user's fault :-( - an unexpected error occurred - Authentication It uses the restful_authentication plugin, but with some changes, check lib/authenticated_system.rb. The modifications don't affect the AuthenticatedSystem module interface or the plugin code, so if you want the original behavior just replace the authenticated_system file by the one originally generated by the plugin, no other changes in the application code are required. IMPORTANT: don't forget to change the restful_authentication site key (see below) - Pagination The application extends the functionalities of the will_paginate plugin. Check vendor/plugin/mogok_will_paginate/README for more about it. Note that the will_paginate plugin is also in the plugin directory to prevent the extension from become broken. - Share Nothing This application follows the share nothing principle, which means that all the application data is stored in the database, including the torrent files data (caching is used to minimize database operations, see below). - Sessions Currently is cookie based (the Rails default). - Testing Two strategies are used to test the application functionalities: Cucumber features for integration tests and RSpec for unit testing. The implemented tests main focus are in the features used by the regular users, which means that admistrative features are not extensively covered. To run the tests you must create the 'test' database instance (check appendix B). IMPORTANT: caching is enabled by default during the tests but it is recommended to run them also with the cache disabled (change the flag in 'config/environment.rb') as the test frameworks bypass cache_money synchronization. + Cucumber Features Are in the directory 'features'. Note that those tests rely on messages in the 'en.yml' file and on the content of the default english views, so changing them may break some of the tests. IMPORTANT: tests involving torrent files may be problematic on Windows (files may get corrupted). - running mogok$ rake features (run all) mogok$ cucumber features/signup.feature (run only the signup feature) mogok$ cucumber features --name "Some scenario" (run only scenarios matching the given name) + RSpec Tests Are in the directory 'spec' and focus on the business logic implemented in the models - running mogok$ rake spec (run all) mogok$ spec spec/models/forum_spec.rb spec (run only spec tests for model 'forum') - Caching The application uses memcached as the cache store. Caching is disabled by default in development mode, but you can enable it by changing the flag in the file 'config/environment.rb' to ensure that the cache is interacting well with the application and to get a better logging of the cache activities. - currently the application uses three caching strategies: + cache_money (http://github.com/nkallen/cache-money/tree/master) - very easy to use, it caches records every time a query for a specific record, like User.find(1) or user.role, is made - the way it is being used, list queries, like User.find(:all) or user.messages won't be cached - note that cache_money is very recent and there are some reports of problems when using it with Passenger - to quit using it just remove the 'index' calls from the models (search for 'cache_money') and delete its initialization code in the file 'config/initializers/caching.rb' + custom cache - caches lists of records that are frequently used, mainly domain values like categories and formats - the implementation is inside some of the models and use the methods in 'lib/caching_methods.rb' - are reset by sweepers (app/sweepers) + fragment caching - used to cache portions of pages, currently the torrent browse page (the most performance critical page in the application) and the application menu. - the torrent browse page will be cached only when browsed with the default options, which means no search parameters and ordered by 'created_at DESC'. if a user searches for something or order the table by another column the application will retrieve the information from the database and won't cache it. also won't be cached for admins and mods as they can see things that users can't. - the torrent page cache will be reset (expired) if a torrent is uploaded/removed/inactivated or by expiration of the period configured in the app_config file (expiration time should not be too long because informations like seeders/leechers are updated all the time) - the fragment expiration time is made possible by the mogok_timed_fragment_cache plugin which is a rewriting of the timed_fragment_cache plugin - Tracker request A tracker request is treated just like any regular Rails request. Under very heavy load this could be a problem, so a faster mechanism (like an external tracker) might be necessary on such scenario. - Bittorrent The implementation is based in the specification found at: http://wiki.theory.org/BitTorrentSpecification The module Bittorrent is in: lib/bittorrent/ - Internationalization Although the application does not support multiple languages, it is structured in a way that makes it easy to translate. So, if you want to translate, let's say to Spanish, this is what you have to do: + create a file named 'es.yml' in config/locales using the existing 'config/locales/en.yml' as the base for your translation. this file will contain localized formats for date, time and numbers and translations for the messages generated by controllers, models and some modules in the lib directory. + change the default locale in the file app_config.yml to 'es' + translate the views. note that you can let the original english views intact and just add the translated ones following this name pattern: my_view.html.erb -> my_view.es.html.erb. + IMPORTANT: changing or deleting the default 'en.yml' file and the default english views may break some of the features tests. - Configuration Lays in the files below. The files come with an 'example' extension, just create a copy of each file without the extension (see rake task below) and change the configuration when needed. Note that the resulting configuration files are listed in the file '.gitignore', preventing them from being committed to the repository and possibly exposing sensitive information about your system. - to generate the config files copies execute the command below $ rake app_config:generate_files - config files list > app config + config/app_config.yml + config/bg_tasks.yml + config/database.yml + config/environment.rb + config/memcached.yml + config/schedule.rb > initializers + config/initializers/action_mailer.rb + config/initializers/session_store.rb (change the secret!) + config/initializers/site_keys.rb (holds the restful_authentication key, change it!) > custom migrations + db/migrate/099_create_data.rb (configure the app initial data, like roles, categories and so on) > install and deploy + setup/backup.rb + setup/deploy.rb + setup/passenger_stack.rb + setup/server_setup.rb # DEPLOYMENT If you are new to Rails and use Windows, an easy way to put the application running is to use InstantRails (install the correct version of the rails gem). For a real scenario, a good option is to deploy on Ubuntu using Passenger (modrails). A nice way to have a real world remote server scenario is to emulate a production setup using VirtualBox, which is a free OS emulator by Sun Microsystems (check Appendix D on how to configure it). IMPORTANT: if you are running on Windows, change the name of the file 'lib/mysql.rb.windows' to 'lib/mysql.rb' if the application can't find your mysql gem. - Deploying on Ubuntu with Passenger (tested for Ubuntu 9.04 with Passenger 2.2.2) - the deployment can be done in two ways, manually or using the 'capistrano' and 'sprinkle' recipes + recipes - will automatically install all required software and gems in your server and deploy your application, the only exception in the database setup, which has to be done manually - during the installation and server setup phase, the recipes will ask you to provide the credentials of a sudo user so they can have the necessary permissions to execute properly (this user tipically is the one you create when installing the system) - if during installation the recipe stops (usually because a software repository site is down), just execute it again and it will resume its work - note that the install recipe is based in the 'passenger-stack', which installs Ruby Enterprise Edition instead of the regular Ruby interpreter - the server setup recipe will create a user (without sudo privileges) to be used to deploy and run the application (its username will the the application name and you will be asked for a password) - finally, all the deployment will be done using the application user created above + manual deployment - check below to a complete list of the steps necessary to do a manual deployment, basically the steps executed by the recipes > Steps for the first deployment, using recipes - all commands are called in the application root directory of your local machine, unless when specified - local code repository - currently the deploy recipe will look for the code in the git repository of your local machine (more convenient as you don't have to push a particular branch to the remote repository just to be able to deploy it), but if you prefer to deploy from the remote repository then check 'setup/deploy.rb' on how to do that and ignore the three steps below + install git in your local machine + if Windows is your local system, install Tar + download tar at http://gnuwin32.sourceforge.net/packages/libarchive.htm + extract the files in C:\Program Files\GnuWin32\bin + create a copy of bsdtar.exe and rename it into tar.exe + add C:\Program Files\GnuWin32\bin to PATH + make a clone of the repository in your local machine $ git clone git://github.com/pepeb/mogok.git - virtual box + if you are emulating a server with VirtualBox, perform the required SSH port forward (check Appendix D) and change the port to 2222 (or the one you choose) in the recipes or they won't be able to connect to the server - server installation + make sure that your server accepts ssh connections + install the 'capistrano' and 'sprinkle' gems $ sudo gem install capistrano $ sudo gem install crafterm-sprinkle --source http://gems.github.com/ + run the command below (will install all required software and gems, it will take some time) $ sprinkle -v -c -s setup/passenger_stack.rb - server setup (creates app user, app dir, passenger app config and starts memcached): + run the command below $ cap -f setup/server_setup.rb server_setup - database setup (this step has to be done manually) + log in on your server via ssh $ ssh -l-p 22 localhost + log in on mysql (password should be empty as the installer doesn't set it) $ mysql -u root -p + set the root's password mysql> SET PASSWORD FOR root@localhost=PASSWORD('root_password_here'); + create mysql> create database mogok_production; + create database user for the app mysql> GRANT ALL PRIVILEGES ON mogok_production.* TO 'user_username_here'@localhost IDENTIFIED BY 'user_password_here'; + add the username and password to your local file 'config/database.yml' (production instance) - deployment setup (creates deploy dirs and uploads your local config files and custom migrations to deploy dir) + be certain that all the configuration files are prepared (see item Configuration above) + run the command below $ cap deploy:setup - deployment (uploads the application code, runs the database migrations and starts app) + run the command below $ cap deploy:cold - now the application should be running, use owner/owner or system/system to log in > Redeploying with recipes (tasks to submit code and config changes to the server) - adding a new release of your code to the server $ cap deploy - rollback to the previous code release (do not touch the config files) $ cap deploy:rollback - replace the config files in the server with your local ones and restart app (see 'config/deploy.rb' for the affected files) $ cap deploy:config - replace a particular config file in the server with your local one and restart app $ cap deploy:config:set_file file=config_file.yml $ cap deploy:config:set_file file=initializers/config_file.yml - panic tasks (use only if users passwords or sessions are compromised!) - replace 'config/initializers/site_keys.rb' in the server and restart (all passwords will get stale if keys changed!!!) $ cap deploy:config:replace_site_keys - replace 'config/initializers/session_store.rb' in the server and restart (all current sessions will get stale if keys changed!!!) $ cap deploy:config:replace_session_store - check 'setup/deploy.rb', 'setup/setup.rb' and 'setup/backup.rb' for other useful tasks - IMPORTANT: + because the deployed code is being transferred from your local repository to the server and not from the remote repository (Github in this case) to the server, you MUST commit any change or it won't be deployed + note that the config files (those that came with an 'example' extension) are ignored by git (see .gitignore) and are uploaded directly from your local machine to the server, so no need to commit when they change > Manual deployment - server software installation + check Appendix A for a complete list of the required software - server environment setup + check Appendix B for a complete list of the steps - application code and database deployment + copy the application code to the server + run the migrations (make sure that the Memcached server is running or it will fail) $ rake db:migrate # Appendix A: Server Installation Steps + build-essential $ sudo apt-get install build-essential + apache $ sudo apt-get install apache2 $ sudo apt-get install apache2-prefork-dev + mysql server $ sudo apt-get install mysql-server $ sudo apt-get install libmysqlclient15-dev $ sudo apt-get install libdbd-mysql-ruby + memcached (there is also a Windows version, just look for it) $ sudo apt-get install memcached + ruby (consider also the Ruby Enterprise Edition option) $ sudo apt-get install ruby1.8 ruby1.8-dev rdoc1.8 ri1.8 irb libopenssl-ruby + install RubyGems (version 1.3.1+ required) - look on the net about how to install it properly, it changes all the time - add github gems repository $ sudo gem1.8 sources -a http://gems.github.com - update RubyGems if necessary $ sudo gem1.8 update --system + rails $ sudo gem1.8 install rails -v 2.3.2 + mysql gem $ sudo gem1.8 install mysql + gems required to run the cucumber and rspec tests $ sudo gem1.8 install cucumber -v 0.3.3 $ sudo gem1.8 install notahat-machinist -v 0.3.1 $ sudo gem1.8 install rspec -v 1.2.6 $ sudo gem1.8 install rspec-rails -v 1.2.6 $ sudo gem1.8 install ruby-debug -v 0.10.3 $ sudo apt-get install libxml2 libxml2-dev libxslt1.1 libxslt1-dev libxml-ruby libxslt-ruby $ sudo gem1.8 install webrat -v 0.4.4 + gems required for caching $ sudo gem1.8 install memcache-client -v 1.7.2 $ sudo gem1.8 install nkallen-cache-money -v 0.2.5 $ sudo gem1.8 install SystemTimer -v 1.1.1 + gems required for automatic crontab update (ignore if you are setting it manually) $ sudo gem1.8 install javan-whenever -v 0.3.0 + passenger $ sudo gem1.8 install passenger -v 2.2.2 $ sudo /usr/bin/passenger-install-apache2-module - passenger will ask you to edit your apache 'httpd.conf' file, instead create the file '/etc/apache2/extras/passenger.conf' and include passenger's configuration in it (don't include your application's VirtualHost configuration, that will be done in another file): LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2 PassengerRuby /usr/bin/ruby1.8 # Appendix B: Server Setup - app user (for security reasons create a non-sudo user to run the app) + create $ sudo adduser mogok + add user to Apache user group (may be another if not on Ubuntu) $ sudo adduser mogok www-data - app directory (where the app will be deployed) + create $ sudo mkdir /var/vhosts $ sudo mkdir /var/vhosts/mogok + change ownership $ sudo chown mogok:www-data -R /var/vhosts/mogok + change permissions $ sudo chmod 775 -R /var/vhosts/mogok - database + log in on your server via ssh $ ssh -l -p 22 localhost + log in on mysql $ mysql -u root -p + set the root's password if not set mysql> SET PASSWORD FOR root@localhost=PASSWORD('root_password_here'); + create mysql> create database mogok_production; + create database user for the app mysql> GRANT ALL PRIVILEGES ON mogok_production.* TO 'user_username_here'@localhost IDENTIFIED BY 'user_password_here'; + add the username and password to your local file 'config/database.yml' (production instance) - passenger + create a file called 'mogok' in '/etc/apache2/sites-available' (sudo nano /etc/apache2/sites-available/mogok), containing your application's passenger configuration: ServerName localhost (or your domain name) DocumentRoot "/var/vhosts/mogok/current/public" + note that this config is the minimal required to run the app, check passenger docs on how to add further parameters + change file ownership to 'root' if not set $ sudo chown root:root /etc/apache2/sites-available/mogok + enable it $ sudo a2ensite mogok + reload apache $ sudo /etc/init.d/apache2 reload - memcached - start (it MUST be started before the deployment or the migrations will fail) see Appendix C (will start automatically when server is restarted) # Appendix C: Controlling Memcached - options -d = run as a daemon -l = ip address -p = port -m = Mb of memory for the cache -P = where to put the pid file -vv = verbose - starting as daemon $ sudo memcached -d -l 127.0.0.1 -p 11211 -m 256 -P /tmp/memcached.pid - starting in verbose mode (so you can see what is being stored and read from it) $ sudo memcached -vv -l 127.0.0.1 -p 11211 -m 256 -P /tmp/memcached.pid - restarting $ sudo /etc/init.d/memcached restart # Appendix D: VirtualBox Setup > Port Forward (so you can access the guest server from your machine or externally) - browsing the application (port 80) + execute the commands below VBoxManage setextradata "ubuntu_box_name" "VBoxInternal/Devices/pcnet/0/LUN#0/Config/apache/Protocol" TCP VBoxManage setextradata "ubuntu_box_name" "VBoxInternal/Devices/pcnet/0/LUN#0/Config/apache/GuestPort" 80 VBoxManage setextradata "ubuntu_box_name" "VBoxInternal/Devices/pcnet/0/LUN#0/Config/apache/HostPort" 8080 + restart the guest system if running + open the browse at http://localhost:8080 on your system and it will be forwarded to port 80 on VirtualBox - connecting via SSH (port 22) + execute the commands below VBoxManage setextradata "ubuntu_box_name" "VBoxInternal/Devices/pcnet/0/LUN#0/Config/apache/Protocol" TCP VBoxManage setextradata "ubuntu_box_name" "VBoxInternal/Devices/pcnet/0/LUN#0/Config/apache/GuestPort" 22 VBoxManage setextradata "ubuntu_box_name" "VBoxInternal/Devices/pcnet/0/LUN#0/Config/apache/HostPort" 2222 + restart the guest system if running + connect via SSH on port 2222 of your system and it will be forwarded to port 22 on VirtualBox: $ ssh -l-p 2222 localhost
本源码包内暂不包含可直接显示的源代码文件,请下载源码包。