...making Linux just a little more fun!

Desktop Bluetooth Remote

By Dr. Volker Ziemann

Controlling the Desktop via S60 Bluetooth Phone

Recently, I bought a new cellular phone (NOKIA E51), which is equipped with Bluetooth. I also had an unused Bluetooth dongle at home, which I'd never had any use for - it had come as a gimmick when I bought my stationary computer. The availability of these gadgets, however, triggered my curiosity and I started wondering whether they could be coerced into doing something useful. This turned out to be easy. In my day job, I frequently give presentations with laptop and beamer, and therefore started to work out how to use the phone as a remote control for my computer.

The first idea was to turn the phone into a Bluetooth mouse, but the phone does not support the required HID Bluetooth profile, so I decided to program a classical server-client application instead. There are other projects that work in a similar way, e.g., the IrDA driver to convert infrared signals to commands, but, as far as I know, it works only with programs that can be remote-controlled. The server on the PC receives commands from the remote, and then itself sends commands via programs like xine-remote to the respective applications (here, the media player xine). I wanted to control the entire desktop, including mouse motion, mouse-click, and cursor or Enter keys. In this way, I could control any application and especially the Acrobat Reader, which does not sport a native remote control. This feature turned out to be the most tricky -- but stay tuned. Just to summarize briefly: I wanted an application on the phone (the client) that reads the phone keys and sends commands via Bluetooth to the PC, where a server is running that receives the messages and causes the mouse to move or click, or causes key-press events.

To keep things orderly, I will describe them in the sequence that the information moves - from your thumb pressing the phone key to the mouse-click on the PC - and start with the application on the phone.

On the Phone

My new cellular phone runs under the Symbian operating system (3rd generation), which is rather common in modern smart-phones. One attractive feature is that it can be programmed in Java and also in Python. The latter is a very attractive language for rapid application development, with a host of powerful built-in functions. Since I had programmed with Python before, I started out my project by installing the PyS60 tools (Python language[1] and script-shell[2]) from the Sourceforge PyS60 Web site to the phone. After doing that, you'll have a working Python environment on your phone, and can test it by running one of the programs (e.g., "ball.py") that comes with the installation. The default installation directory for Python scripts is the folder E:\Python, which resides on the memory card inside the phone. You can now write your own Python scripts, and copy them to the same directory; then you can run them in the same way you ran the example that came with the installation. To give you a feel, here is the typical "hello, world" program. Just put the following line in a file called "hello.py": 

      print "Hi there, World"

and transfer it to the phone, providing that you have the bluez Bluetooth stack and the obex_ftp package installed. You transfer "hello.py" with the obex_ftp program by executing 

      obexftp -b 00:21:08:nn:nn:nn -c E:\\Python -p hello.py

where "00:21:08:nn:nn:nn" is the MAC address of the phone. (You can find this by keying *#bta0# or *#2820# on Nokia phones.) Another way is to turn on the phone and perform a Bluetooth scan with hcitool scan on your PC. The output will tell you the MAC address of your phone's Bluetooth adapter (and probably those of your neighbors' phones, as well). There are very nice pages on the Web about Python programming, and I encourage you to consult them. I found references [4] through [7] particularly useful. A book that I particularly like as a reference for Python is Ref. [8].

[ When scanning for Bluetooth devices, make sure they are visible and "discoverable". Due to frequent scanning "attacks", some vendors have disabled the Bluetooth devices' visibility. I've also seen Bluetooth devices in PCs that stay invisible unless the visible mode is activated. This mode may also be called "inquiry mode". hcitool can change the mode for most devices. -- René ]

Now that we have a working Python environment and a way to transfer Python code to the phone, we can start coding the client program. This will monitor the keys on the phone and send appropriate commands to the server (discussed further below). The entire program is linked here, but we discuss the individual parts one at a time.

First, the Keyboard class that is copied straight from the "ball.py" example in Ref. [6]. This class represents a listener for the keyboard on the phone, and returns the key-codes of the pressed keys to the program. Then we have to open a Bluetooth connection to the server by executing 

     PCBTADDR=00:21:08:nn:nn:nn
     port=4
     btsock = socket(AF_BT, SOCK_STREAM, BTPROTO_RFCOMM)
     target = (PCBTADDR, port)
     btsock.connect(target)

where PCBTADDR is the MAC address of the Bluetooth adapter on the PC. Then, we somewhat arbitrarily define port number 4 for use. (Just make sure no other service on your phone uses the port number you select, by executing "sdptool browse MAC_OF_PHONE" on the PC and checking the port numbers.) In the next line, a Bluetooth socket, btsock, is opened, and the connection to the server is established. This code is modeled after the examples in Ref.[9]. We then enter a loop starting with "while running", where the program waits a short while (0.1 s), and checks whether a key is pressed. If that is the case, a short text is displayed on the screen of the phone, and the corresponding text string is sent via the socket. Here is a snippet from the file "remote.py": 

     running=1
     while running:
       e32.ao_sleep(0.1)
       e32.ao_yield()
       handle_redraw(())
       if keyboard.is_down(EScancodeLeftArrow):
         canvas.text((0,32),u"Left key pressed",fill=0xff0000)
         btsock.send('Left')

          :    # many more 

       if keyboard.is_down(EScancodeHash):
         canvas.text((0,48),u"Hash key pressed",fill=0xff0000)
         btsock.send('Hash')
         running=0

The name of the scan-codes are documented in the PyS60 library reference in Ref. [7]. Please check the file remote.py for the full code listing. Note that apart from the core activities described here, there is also some code to update the user interface on the phone.

This file is all that is needed on the phone. Just copy it via obex_ftp, or any other means, to the Python subdirectory on the phone. Once that is done, we can now progress to discussing the server application on the PC.

On the PC

On the PC, we need a server to receive the commands sent from the phone, and take appropriate action. I decided to write this in Python as well, and needed to install the Python bindings for the bluez stack called PyBluez from Ref.[10]. The server program is then a very simple matter, and the full listing is linked here. The main part of the program opens the socket: 

     server_sock=BluetoothSocket( RFCOMM )
     server_sock.bind(("",port))
     server_sock.listen(backlog)
     client_sock, client_info = server_sock.accept()

and then enters in a runtime loop that receives the package from the phone and takes suitable action. A snippet is shown here: 

     running=1
     while running:
       data = client_sock.recv(10)
       if data=='Hash': running=0
       if data=='Left': os.system("xevent -r -10 0")
           
           :

     client_sock.close()
     server_sock.close()

Note that a hash terminates the runtime loop, after which the last two commands that close the socket are executed. The action to be performed is simply written as a string that describes a Unix command which is passed directly to the operating system via the os.system() function call. This makes the code very adaptable to whatever whims strike the programmer.

Remember the wishlist from the start of this article, where I wanted to control the cursor on the desktop? This feat is actually performed by the program "xevent" that is executed within the runtime loop of the server program. In the above example, it moves the cursor relative to its present position by 10 pixels to the left. How this is done is discussed in detail, in the following section.

Causing X-window events without keyboard or mouse

I wrote the xevent program [11], which is based on code from Ref.[12] and the X Window System library libXtst, to cause X-windows events with a command line program. For example, the command "xevent -a 0 0" will move the mouse cursor to the absolute position pixel (0,0), which is the top left corner of the screen. Please inspect the rather short source of "xevent" for details, but the rough line of execution is as follows: first the display and default screen are assigned and the root window is specified; then, depending on the command-line argument, different actions are taken. If the relative mouse motion (-r) is specified, the libXtst function XTestFakeRelativeMotionEvent() is called with appropriate arguments; if absolute motion is specified (-a), XTestFakeMotionEvent() is called. If a mouse button event is called (-b), the function XTestFakeButtonEvent() is used, either as a button down (is_down=1) event or button up (is_down=0) event. The default action is down and up. Similarly, the libXtst function XTestFakeKeyEvent() is used to simulate a key-press event, either down, up, or both. Please check the manual page for xevent that is available on the Web page specified in Ref.[11]. The libXtst functions are defined in the header file libXtst.h.

Putting It All Together

Once the "remote.py" file resides on the phone, the "xevent" program is installed on the PC, and the "remote_on_pcx.py" is copied to the PC, we can start the server by entering the following in a command window: 

     python remote_on_pc.py

which will wait for the phone to make a connection. Then, we start the client on the phone by running the "remote.py" script there. The phone then makes a connection to the server. The server program on the PC will acknowledge the connection if everything went well, and the screen on the phone will turn white. Then you can use the the big navigation button on the phone to steer the mouse on your PC and the big select button as the left mouse click. If you press it twice in rapid sequence, it will behave as a double-click. The initial assignment of the keys is documented in the following table for easy reference, but this can be easily changed by editing the server program "remote_on_pc.py" and just assigning different events to all keys. I picked just those that are most convenient to steer programs.

Key on the Phone

Action on the PC
Select Key Left mouse click
Left Move mouse left by 10 pixel on PC screen
Right Move mouse right by 10 pixel on PC screen
Up Move mouse up by 10 pixel on PC screen
Down Move mouse down by 10 pixel on PC screen
1 Enter key pressed
2 Middle mouse click
3 Right mouse click
4 Page Up
5 Text cursor up
6 Page Down
7 Text cursor left
8 Text cursor down
9 Text cursor right
Asterisk (*) Tab key pressed
0 Escape key pressed
Hash (#) Exit the program

The code in both the client and server applications was deliberately made rather simple, because that makes it easier to understand and adapt to individual wishes. Turning this skeleton into a user-friendly package will certainly require more work.

A note about copyright: the xevent program is based on the code from Ref.[12], which is GPLed, and therefore xevent is also GPL. The client-server construction is so trivial and commonly known that assigning a copyright seems useless. The keyboard class in "remote.py" is a straight copy from Ref. [6], and is licensed by Nokia for personal use.

That being said: I hope you will have fun playing around with the remote control, and adapt it to whatever fun things you come up with. My next project is a remote control for a USB-controlled foam-rocket launcher.

References


Talkback: Discuss this article with The Answer Gang

[BIO]

Volker lives in Uppsala, Sweden and works as a physicist on particle accelerator based projects at CERN in Switzerland and DESY in Germany. He was forced into using computers during his undergraduate thesis in the mid-eighties and has used them with growing enthusiasm ever since. He is an active Linux user since 1995.


Copyright © 2008, Dr. Volker Ziemann. Released under the Open Publication License unless otherwise noted in the body of the article. Linux Gazette is not produced, sponsored, or endorsed by its prior host, SSC, Inc.

Published in Issue 153 of Linux Gazette, August 2008

Tux