English
资源说明：A prototype iPhone based remote control for the Firefox browser
------------------------------------------------------------
fRemote: iPhone to Firefox Remote - Public Prototype Release
------------------------------------------------------------
The iPhone remote for Firefox, code named fRemote, is a prototype iPhone based remote control
for the Firefox browser. The remote works by having a Firefox Add-On and Safari on the iPhone
open a non-blocking connection to a Tornado server. Then the two communicate back and forth,
as if they were chatting, but sending Javascript commands and responses that make Firefox do
different things.
The goal of this project is to create an interface to "elements" on a webpage using a remote
control. Remotes are easily created and shared among the community, and can be single or
multi-session.
------------------------------------------------------------
Video Demo
------------------------------------------------------------
• http://vimeo.com/8230873
------------------------------------------------------------
Public Prototype Release
------------------------------------------------------------
You can download the code and run this yourself but please be aware that this is just a
prototype and needs a lot of work.
In many cases, you will want to check the code to see what is going on in the various
remotes. For example, custom data is hardcoded - the Google Docs remote points to two of
my test presentations.
If you are not a developer, this project is not at a stage that would be beneficial to you.
You should have knowledge of Javascript and Python if you intend to contribute or hack this.
------------------------------------------------------------
Installing
------------------------------------------------------------
First, get the Tornado server up and running on your computer. You can find out more about
that at:
• http://www.tornadoweb.org/documentation
Now that you have that running, make sure the chat demo that comes with it works for you!
Ok, good, now download the fRemote code at:
• http://github.com/dascgo/fremote/
You have a directory with the following sub-directories:
• server (this is the Tornado app we will be running)
• add-on (this is the Firefox Add-On)
We won't be building an XPI file for the Add-On, rather, we'll just point Firefox to it to
remove the hassle of rebuilding the XPI for every change. To do this, create a text file
called fremote@dascgo.com with the content of the file being the location of the "add-on"
directory. For instance, on my Mac, the content of the file is:
• /Users/daniel/webapps/fremote/add-on/.
Now, move this file to the Firefox extensions directory. For instance, mine is located at:
• /Library/Application Support/Mozilla/Extensions/{ec8030f7-c20a-464f-9b0e-13a3a9e97384}
Details about this (and how to do on other OSes) can be found here:
• http://www.borngeek.com/firefox/toolbar-tutorial/chapter-4/ (scroll down to "How to
Develop Dynamically")
Quit Firefox if it is open but do not relaunch it yet. It will fail if the server is not
running (tries to connect and pull .js file from it). So, let's crank up the server.
Move to the "server" directory in a terminal and start the server with the command:
• ./start.py
start.py is the server and is a modified version of the chat demo that comes with the
Tornado server.
Test that it is running by hitting http://localhost:8888/remote and you should see the app
with the default controls. Good.
------------------------------------------------------------
Starting Up
------------------------------------------------------------
We should be able to start using the remote now. There are a couple of options for doing this.
First:
• Launch Firefox (to be controlled)
Then:
• Launch Safari (or another instance of Firefox) to point to the remote.
• Use an iPhone simulator such as PhoneGap (http://phonegap.com/download) to point to the
remote.
• Actually use your iPhone by replacing localhost in the url with your computers IP
Address.
Which ever you use, point the browser you want to use for your remote to:
• http://localhost:8888/remote
Now, in Firefox, turn on the Add-On by going to:
• Tools > fRemote.
A green bar should appear across the top of Firefox to let you know you are in remote mode.
------------------------------------------------------------
Using
------------------------------------------------------------
Let's test it. The first Remote listed should be Presentation Viewer. Click on that and you
should see a list of two presentations: "pres1" and "pres2".
Click on one of them and Firefox should redirect to the presentation (the presentations are
nothing fancy, just examples). Once Firefox loads the page, the remote should change from a
list of presentations to controls for the currently selected presentation, which consists of
a Next/Back/End button. Try them out and see if the presentation advances.
You can play with the other remotes to get a feel of what they do. I run through each of
them in the video above if you need a reference.
------------------------------------------------------------
Create a New Remote
------------------------------------------------------------
Adding a new remote, in this prototype, is just a matter of a few steps:
• Remote's are JS files, currently stored at /path/to/server/static/dsg_*.js
- The "dsg" prefix is just my initials, denoting remotes i've created.
- Copy one of these and rename it to whatever you want to call it.
• Open /path/to/server/templates/remote.html
- In this file, look for:
-
- You will see the existing remotes called immediately after that. Add your newly
created remote here.
- There is a $(document).ready call on this page that will load the remotes when
the page is ready. Inside that, you'll see an array of remotes being passed to
tools.init. Add your remote id here (info.id - see next section).
------------------------------------------------------------
Coding a Remote
------------------------------------------------------------
Remotes are JSON files. They consist of 4 major blocks: init, screens, buttons, and info:
remote_id = {
info: {},
buttons: {},
screens: {},
init: {}
}
The Info Block:
info: {
id: "dsg_googledocs",
logo: "/static/logos/googledocs.png",
name: "Presentation Viewer",
description: "Control and Share Presentations from Google Docs",
url: "",
shareable: true,
queueable: true,
}
The info block is metadata on your remote. The "name", "description" and "logo" is what is
displayed on the list of remotes screen. "shareable" and "queueable" are options for later
functionality.
The Buttons Block:
buttons: {
next: {
label: "next",
style: { top: "10", left: "10", width: "287", height: "166", color: "#aff4b2" },
mouseup: {
action: function(){
tools.trigger("#ToolbarNext", "MouseEvents", "mousedown");
},
responses: {
success: function(message){ },
failure: function(message){ }
}
}
}
}
The buttons block will contain all the buttons in your remote, regardless of screen. The
formatting for buttons are all generally the same. The label helps to define what button
type and how it's labeled (more on this in a second). The style will place the button in
that location on the screen. Buttons can have mouseup and mousedown blocks that contain
an action and response handlers. This action function is what is called in either of those
states. The responses block for that state will be called once the Add-On in Firefox is
finished with the command it was just sent.
An example of how a buttons mouseevent and response works is as follows:
• a remote has a mousedown action that will tell Firefox to go to a certain webpage
• once that page has finished loading, Firefox will send back that the action has been
completed and call the "success" response for that button's mousedown
• In the success function, you might tell the remote to draw a new screen of buttons
that relate to the loaded page.
The Screens Block:
screens: {
home: ["presentations"],
presentation: ["next", "previous", "end"]
}
A remote can have multiple screens, which is just a collection of buttons. In the above
example, there are two screens. Each item in the screen arrays refer to buttons. So on the
home screen, there is a single button called "presentations" (which will refer to the
remote.buttons.presentations button). The "presentation" screen contains 3 buttons.
The Init Block:
init: function(){
tools.setup_remote();
tools.draw_screen("home");
}
The init will start up your remote. All remotes will need to call tools.setup_remote(); but
you can also do any other setup steps here. In the above example, the only thing we want to
do is draw the initial screen of "home", which, shown above, will draw the one button
"presentations" to the screen.
------------------------------------------------------------
Different Types of Buttons:
------------------------------------------------------------
Right now, there are three types of buttons:
• A normal button with a label.
• An "input button" that takes input from a user (the url field in the Go To URL Remote).
• A "list" button to create a list of clickable items (The list of presentations in the
Presentation Remote).
To identify what type of button you want to create is determined by what you set as the
label. In the above example for the "next" button, the label is a non-empty string (label:
"next"). The engine will take a non-empty string and know that is should create a normal
button with a label in it (the string).
If the label is an empty string, it will know to draw a user input button. Such as the
button for entering a URL in the remote that jumps you to different urls.
If the label is an array then it will draw a list, such as the list of presentations in
the Presentation remote. This actually needs to be an array or arrays. Example:
label: [
["pres1", "http://docs.google.com/present/view?id=dfzbxv77_12dpcdh2dc"],
["pres2", "http://docs.google.com/present/view?id=dfzbxv77_12424mt9gk"]
],
This will create a scrollable list with two items in it. The first item will have a label
of "pres1" and when you click it, the url:
• "http://docs.google.com/present/view?id=dfzbxv77_12dpcdh2dc"
will be passed to the mouseup/down function for that button.
------------------------------------------------------------
Helpers in the Add-On
------------------------------------------------------------
Helpers for the Remote (ways to interact with the HTML loaded in Firefox) are located in:
• /path/to/add-on/chrome/content/application.js
under fremote.commands.[name_of_command](params). All commands take one param which some
split on a pipe character ("|").
• redirect(param): Will redirect the browser to a passed url
• trigger(param): Will trigger a mouse or keyboard event, depending on what is passed in.
The param is split on | to generate up to 4 options:
- [0] The selector to find an element on the page to trigger an event on
- [1] The kind of event this is: MouseEvents or KeyboardEvent
- [2] The type of event this is: mousedown/mouseup, keydown/keyup
- [3] (optional) if keyboard event, this the the keycode to to trigger on.
• flash(param): This is how to send commands to a flash file (such as in the Grooveshark
remote). Params are split into two items:
- [0] id of the flash file
- [1] command to send the flash file
- Calling this, for example, would be tools.flash("gsliteswf","togglePlayback"); which
would find the flash with id "gsliteswf" and send it the command "togglePlayback"
(which in Grooveshark, toggles the pause/play button)
• collect(param): Param is a selector that will be used to select a group of dom objects.
Then, the page as a whole would be grayed out and the first item of this collection will
be highlighted (as seen in the Google search remote).
• collect_next()/collect_prev(): will jump forward/back to the next item in the collection
you gathered with the collect command.
• fullscreen(param): Param will be a selector that will be used to grab an element and make
it take over the whole page (used in the demo to make the flash game fullscreen)
When you call these commands in the JS for the remote, you do not need to combine all
params into a single string at that time. When passed from the remote to the Add-On, the
params will be joined with a pipe character separating them.
The Add-On should contain core functionality, but remote creators can inject their own
helpers if needed, mentioned in the "overall goals" section below.
------------------------------------------------------------
Long Term Goals
------------------------------------------------------------
Now that you are running the demo, you might want to start submitting additions to the
code. That would be awesome, but I would like to throw out my ideas on what I want to
accomplish with this project. By no means is this roadmap set in stone, it's just the
initial direction I am pointed.
Signup:
• There is a web site where a user goes, registers an account.
• Once registered:
- The user will be given a url to point their iPhone to. This page will request a log in and
once in, will show the user remotes they have installed.
- The user will be given the Firefox Add-On to install. On first run of the Add-On, it will
require the user to log in.
• Now, the Add-On and iPhone are associated to a particular user (this is important so
commands know who to go to).
• From the remote, the user can search for remotes to install, such as the Presentation
remote.
• Remotes can accept data for a person and have a limited database schema to store this data.
• The Presentation remote needs to know the user's presentations, so they can add them either
through the iPhone remote interface, or at the website where they registered (you can manage
your remotes there if needed).
• I add some presentations and refresh my remote and find that they are there and can use them.
That describes general use of the service. In a nutshell, here's how remotes are created:
• The web site will have a remote builder interface so that anyone with an account can create
remotes.
• The interface allows "Screens" of "Buttons" to be build with drag/drop functionality to
easily create a remote.
• After you add a button to a screen, you can set the actions that happen on that button.
• The Add-On comes with built in helpers like collecting elements (seen in the Google search
demo), but remote creators can also script additional functionality that the browser will
import when it see the user has selected a remote that needs it.
• Once a remote it built, it can be published (with versioning). Published remotes are available
to all users to install to their account (unpublished just to the creator, for testing or
strictly personal use).
Now, sharing. Examples like the Presentation and Books remote would be perfect to share.
Imagine someone giving a presentation and people in the back of the room can not see the
slideshow. The presenter can turn on "sharing" in a remote, then, anyone wanting to follow
along, can "subscribe" to that username in the Firefox Add-On. So as the presenter advances
through the slide show in his browser, anyone that is subscribed with see their browsers
taking commands directly from the presenter.
That being said, what are you ideas?
------------------------------------------------------------
Known Issues
------------------------------------------------------------
This is a prototype. Hastily written and without long term goals considered. It either
needs massive re-factoring or a rewrite. Here are some known issues:
• Sometimes the server and add-on get tripped up and Firefox and the server should be
restarted.
• Sometimes the selectors seem to fail.
• Sometimes the response from the Add-On back to the Remote does not work.
• Various things are logged to the Firefox Error console so you may want to watch that
(or use fremote.tools.logger("message here") to log your own information).
• The Add-On creates a green bar at the top of the browser with what looks like buttons
and a search field. None of these items are active except the close button.
------------------------------------------------------------
Other Ideas
------------------------------------------------------------
• Have a button in the browser that will trigger recording and it will playback things
you do. Like in google reader, I can record the 'k' button the first time I switch to a
different article. The just sit back and do it from my phone. Easy way to create a remote
• HTML5 video... try to find a site that has free videos that can be used in html5 and then
control it from the remote. find movies from the remote too.
• Hook into the microphone and let people speak what they want to do?
• Would like to see a "mouse" button-type that maybe draws a little "joystick" button on
the screen and how you move your finger over it, it will move the mouse in the browser.
------------------------------------------------------------
Contact
------------------------------------------------------------
I can be reached at dascgo@gmail.com or @dascgo on Twitter.
本源码包内暂不包含可直接显示的源代码文件，请下载源码包。
