Web development
Slugs and Airports
15/09/09 10:31
I've setup an NSLU2 to serve video to my iPod Touch. Here's what I did.
I have a conflicted position with regard to technology. On one hand I think technology should just work. I have no time for cryptic interfaces and poorly written instruction manuals. This is why I like Apple. I have a Macbook, and Airport Extreme (with printer and hard drives attached) and an iPod Touch. These three devices constitute my home network and I'm pleased to say that it works perfectly.
However, I'm also a geek. When I see the flash of a green LED I can't help but ask myself 'I wonder if...'. Over time my geekish tendencies to fiddle have been tempered by my desire to have elegant solutions, so much so that I know refuse to let a geekish endeavour compromise the integrity a working solution. It turns out that this restriction has, possibly unsurprisingly, caused my geekish side to raise its' game.
A few years ago I bit the bullet and ripped all of my CDs. It took a while, but it was a worthwhile investment. Ever since then I've had my sights set on my DVD collection. Ripping all my DVDs is a harder problem to crack. The sticking point was not the ripping, it was the watching. I have no problem having my music only being accessible from my laptop (or my transferring to my iPod), but that won't do for movies - I don't want to mess with my laptop just to watch a movie. I want a solution that just works;
I want to be able to watch a movies by expending less effort than if I were to put a disc in a dvd player. So how do I do it? First of all lets take a look at the elements that constitute the current solution. Firstly there are the shelves full of DVD discs, secondly there's the DVD player to play them on.
Replacing the DVD player is relatively straight forward. I have a TV-out cable for my iPod Touch which I'm very pleased with. This is the simplest replacement for the DVD player.
Replacing the shelves of DVDs is less straight forward. The first problem is deciding where to store the video. They could be stored on the laptop, but there are problems with this. The laptop only has a 500GB hard drive video files take up a lot of space. Even if the hard drive had nothing except movies on it, it wouldn't take too long before the hard drive was full. Also, if the movies were stored on the laptop then the laptop would have to be turned for the movies to be accessible. My anal-retentive non-geeky side deems this unacceptable as it would require turning the laptop on whenever I wanted to watch a movie. The alternative to storing them on the laptop is to store them on a network drive. With this in mind I bought a 2TB WD RAID drive and attached it to the Airport Extreme. I set up the drive to act in mirror mode to provide a backup if one of the drive fails (it's not possible to backup network drives with Time Machine).
The second problem is how to get the movies from the Airport to the iPod. The movies can be transferred via iTunes or they can be access wirelessly via HTTP or a different protocol, such as uPnP ("there's an app for that"). Obviously transferring movies via iTunes is not an option for the same reasons that storing the movies on the laptop are not an option. That leaves the wireless network option — which raises the question of a server.
The Airport Extreme has only 2 network protocols AFP and SAMBA, neither of which can be accessed via the iPod touch. Game over? No. The solution is to add another network device which can access the Airport Extremes network drives and then make that data available via a protocol that the iPod Touch can work with. The obvious device is another computer. Another computer could easily be setup to act as a server. But there in lies the problem, another computer could easily do a lot more than sever video. Another computer is massive overkill. The wasted energy and computing power of having a computer set up just to act as a web server for one or two clients makes me shudder.
So what else could do the job? The Linksys NSLU2, affectionately known as a slug. The NSLU2 has one LAN port and 2 USB ports. The idea is that you attach USB hard drive to the NSLU2, hook it up to your network and the hard drive become accessible via the network. At least that's how Linksys expected the device to be used, some geeks had different ideas. NSLU2-Linux is the home of the alternate firmware for NSLU2. By installing alternative firmware the NSLU2 becomes capable of much more. There are a few different firmwares to choice from. After a bit of playing with them I decided to go for the Debian Lenny. This provides a full Linux system with a huge selection of software to install.
Martin Michlmayr has detailed instruction for installing Debian on the NSLU2. I used the unofficial image from Slug-Firmware.net and upslug2 for flashing the NSLU2. There isn't a pre-compiled version of upslug2 for OS X. I used a Ubuntu 9.04 virtual machine. There is an upslug2 package in the pre-configure Ubuntu repository. To install it type the following use the following command:
Once Lenny is installed you will need access to edit files. This can be done via SSH and edited in the terminal with vi or nano, but Macfusion offers an alternative. Macfusion makes it possible for arbitrary resources to mounted as part of the local file system, (Macfusion is based on FUSE). FTPFS is such a resource. FTPFS clients can connect to an out of the box Lenny installation. If you install Macfusion you will be able to browse the NSLU2 filesystem with Finder and edit files with the app of your choice.
Once Lenny has installed, login to the NSLU2 via SSH, and install the SAMBA package (we only need the client package):
Create a directory to mount to:
Mount the shared drive:
Check that it worked:
The contents of the root directory of you shared drive should be listed.
Make sure there is a blank line at the end of fstab.
Check that the amendment has worked by rebooting:
After the reboot the contents of the network drive should be available in
Nginx is a very capable alternative to resource-hungry Apache. Nginx is resource-light thus making it well suited to the NSLU2.
Install nginx:
Check that nginx has installed by entering accessing your NSLU2 via a browser.
Organise the web server directory to make it easier to add extra features later:
Create a symlink to the mount point of the shared network drive:
Next, add the mime types for mpeg 4 movies to the Nginx mime type config file. The file is found at
The last step is to configure the Nginx site so that it serves the files from the correct directory and is capable of sending large files. Open the default site config file located at
Change the value of
Finally, restart nginx:
Access your slug via the browser and you will be able to files shared by the Airport. Movies will be accessible from an iPod Touch or iPhone. If everything went as planned the NSLU2 will mount the network drive and start Nginx when it is turned on.
Installing a server-side scripting language gives you greater control over the pages you server. Scripting languages have to run as a fastCGI server for them to be accessible from nginx. I installed PHP. Here's some screen shots of the site my NSLU2 servers up:
I did this by setting the autoindex to a script that is located in the root of public_html. The source code is a bit messy so I won't release it just yet. However, if you'd like the messy version just send me an email.
FireFly is an DAAP server. DAAP is the protocol used by iTunes and AppleTV to access remote media. Unfortunately the version of FireFly in the Lenny repository doesn't serve video correctly. There are other DAAP servers, but I haven't investigated them.
Setting up a dynamic DNS would allow you to access your NSLU2 from anywhere on the internet.
The problem
I have a conflicted position with regard to technology. On one hand I think technology should just work. I have no time for cryptic interfaces and poorly written instruction manuals. This is why I like Apple. I have a Macbook, and Airport Extreme (with printer and hard drives attached) and an iPod Touch. These three devices constitute my home network and I'm pleased to say that it works perfectly.
However, I'm also a geek. When I see the flash of a green LED I can't help but ask myself 'I wonder if...'. Over time my geekish tendencies to fiddle have been tempered by my desire to have elegant solutions, so much so that I know refuse to let a geekish endeavour compromise the integrity a working solution. It turns out that this restriction has, possibly unsurprisingly, caused my geekish side to raise its' game.
A few years ago I bit the bullet and ripped all of my CDs. It took a while, but it was a worthwhile investment. Ever since then I've had my sights set on my DVD collection. Ripping all my DVDs is a harder problem to crack. The sticking point was not the ripping, it was the watching. I have no problem having my music only being accessible from my laptop (or my transferring to my iPod), but that won't do for movies - I don't want to mess with my laptop just to watch a movie. I want a solution that just works;
I want to be able to watch a movies by expending less effort than if I were to put a disc in a dvd player. So how do I do it? First of all lets take a look at the elements that constitute the current solution. Firstly there are the shelves full of DVD discs, secondly there's the DVD player to play them on.
Replacing the DVD player is relatively straight forward. I have a TV-out cable for my iPod Touch which I'm very pleased with. This is the simplest replacement for the DVD player.
Replacing the shelves of DVDs is less straight forward. The first problem is deciding where to store the video. They could be stored on the laptop, but there are problems with this. The laptop only has a 500GB hard drive video files take up a lot of space. Even if the hard drive had nothing except movies on it, it wouldn't take too long before the hard drive was full. Also, if the movies were stored on the laptop then the laptop would have to be turned for the movies to be accessible. My anal-retentive non-geeky side deems this unacceptable as it would require turning the laptop on whenever I wanted to watch a movie. The alternative to storing them on the laptop is to store them on a network drive. With this in mind I bought a 2TB WD RAID drive and attached it to the Airport Extreme. I set up the drive to act in mirror mode to provide a backup if one of the drive fails (it's not possible to backup network drives with Time Machine).
The second problem is how to get the movies from the Airport to the iPod. The movies can be transferred via iTunes or they can be access wirelessly via HTTP or a different protocol, such as uPnP ("there's an app for that"). Obviously transferring movies via iTunes is not an option for the same reasons that storing the movies on the laptop are not an option. That leaves the wireless network option — which raises the question of a server.
The Airport Extreme has only 2 network protocols AFP and SAMBA, neither of which can be accessed via the iPod touch. Game over? No. The solution is to add another network device which can access the Airport Extremes network drives and then make that data available via a protocol that the iPod Touch can work with. The obvious device is another computer. Another computer could easily be setup to act as a server. But there in lies the problem, another computer could easily do a lot more than sever video. Another computer is massive overkill. The wasted energy and computing power of having a computer set up just to act as a web server for one or two clients makes me shudder.
So what else could do the job? The Linksys NSLU2, affectionately known as a slug. The NSLU2 has one LAN port and 2 USB ports. The idea is that you attach USB hard drive to the NSLU2, hook it up to your network and the hard drive become accessible via the network. At least that's how Linksys expected the device to be used, some geeks had different ideas. NSLU2-Linux is the home of the alternate firmware for NSLU2. By installing alternative firmware the NSLU2 becomes capable of much more. There are a few different firmwares to choice from. After a bit of playing with them I decided to go for the Debian Lenny. This provides a full Linux system with a huge selection of software to install.
The solution - How to set up an NSLU2 to server movies
1. Install Debian (Lenny)
Martin Michlmayr has detailed instruction for installing Debian on the NSLU2. I used the unofficial image from Slug-Firmware.net and upslug2 for flashing the NSLU2. There isn't a pre-compiled version of upslug2 for OS X. I used a Ubuntu 9.04 virtual machine. There is an upslug2 package in the pre-configure Ubuntu repository. To install it type the following use the following command:
apt-get install upslug2Macfusion
Once Lenny is installed you will need access to edit files. This can be done via SSH and edited in the terminal with vi or nano, but Macfusion offers an alternative. Macfusion makes it possible for arbitrary resources to mounted as part of the local file system, (Macfusion is based on FUSE). FTPFS is such a resource. FTPFS clients can connect to an out of the box Lenny installation. If you install Macfusion you will be able to browse the NSLU2 filesystem with Finder and edit files with the app of your choice.
2. Install SAMBA
Once Lenny has installed, login to the NSLU2 via SSH, and install the SAMBA package (we only need the client package):
NSLU1:/# apt-get install smbfsCreate a directory to mount to:
NSLU1:/# mkdir /media/MoviesMount the shared drive:
NSLU1:/# mount -t smbfs //AIRPORT-IP-ADDRESS/AIRPORT-VOLUME-NAME /media/Movies -o password=AIRPORT-PASSWORDCheck that it worked:
NSLU1:/# ls /media/MoviesThe contents of the root directory of you shared drive should be listed.
3. Mount SAMBA drive on boot
fstab is used to determine where to mount devices. By adding an entry for the shared drive the NSLU2 will mount the drive at boot. Open /etc/fstab and add the following line://AIRPORT-IP-ADDRESS/AIRPORT-VOLUME-NAME /media/Movies smbfs password=AIRPORT-PASSWORD 0 0Make sure there is a blank line at the end of fstab.
Check that the amendment has worked by rebooting:
NSLU1:/# rebootAfter the reboot the contents of the network drive should be available in
/media/Movies.4. Install a web server (Nginx)
Nginx is a very capable alternative to resource-hungry Apache. Nginx is resource-light thus making it well suited to the NSLU2.
Install nginx:
NSLU1:/# apt-get install nginxCheck that nginx has installed by entering accessing your NSLU2 via a browser.
5. Configure Nginx
Organise the web server directory to make it easier to add extra features later:
NSLU1:/# mkdir /var/www/public_htmlCreate a symlink to the mount point of the shared network drive:
NSLU1:/# ln -s /media/Movies /var/www/public_html/MoviesNext, add the mime types for mpeg 4 movies to the Nginx mime type config file. The file is found at
/etc/nginx/mime.types , open it and add the following lines above the closing curly bracket.video/x-m4v m4v;
video/mp4 mp4; The last step is to configure the Nginx site so that it serves the files from the correct directory and is capable of sending large files. Open the default site config file located at
/etc/nginx/sites-enabled/default (this file is actually a symlink to a file in sites-available).Change the value of
root from /var/www; to /var/www/public_html;. Add the following line into the server section: sendfile off;.Finally, restart nginx:
NSLU1:/# /etc/init.d/nginx restartAccess your slug via the browser and you will be able to files shared by the Airport. Movies will be accessible from an iPod Touch or iPhone. If everything went as planned the NSLU2 will mount the network drive and start Nginx when it is turned on.
Optional extras
Install a scripting language (eg PHP)
Installing a server-side scripting language gives you greater control over the pages you server. Scripting languages have to run as a fastCGI server for them to be accessible from nginx. I installed PHP. Here's some screen shots of the site my NSLU2 servers up:
I did this by setting the autoindex to a script that is located in the root of public_html. The source code is a bit messy so I won't release it just yet. However, if you'd like the messy version just send me an email.
FireFly
FireFly is an DAAP server. DAAP is the protocol used by iTunes and AppleTV to access remote media. Unfortunately the version of FireFly in the Lenny repository doesn't serve video correctly. There are other DAAP servers, but I haven't investigated them.
Access your NSLU2 from anywhere
Setting up a dynamic DNS would allow you to access your NSLU2 from anywhere on the internet.
<select multiple> sucks
13/06/09 14:00
The select element is used to create list of options. In ‘normal’ mode it presents a popup box. In ‘multiple’ mode it presents a list which requires the user to hold a key to select additional items. The native list control in Windows and OS X works exactly the same.
I really don’t like this control. There are no visual clues that the user can select multiple items, which means that most users don’t know that multiple selections are possible. To address this problem websites often add a label to explain how multiple selections is made:
When notes and labels are added to things it’s a huge clue that the thing in question suffers from poor design. Also, the label in the screenshot is inaccurate. It is true in Windows, but not in OS X (and possibly not in true in GTK, QT etc).
The control requires the user to user press a key so that they can make multiple multiple selections - this means that the control is quasi-modal. Modes confuse the user and should be avoid. For such a simple task these failings are inexcusable.
Here’s a better approach:
<div style="overflow-y:scroll;height:6em;width:20em;border:1px solid black;">
<input type="checkbox">Jimmy</input><br />
<input type="checkbox">Jimi</input><br />
<input type="checkbox">Frank</input><br />
<input type="checkbox">Dweezil</input><br />
<input type="checkbox">Jeff</input><br />
<input type="checkbox">Keef</input><br />
<input type="checkbox">John</input><br />
</div>
The above creates a scrolling checkbox list by setting the size and overflow style attributes of the parent block element (in this case a <div>, but it could be applied to the <form> directly). Checkboxes lists are common in OS’s so the user will understand how to use the control.
I really don’t like this control. There are no visual clues that the user can select multiple items, which means that most users don’t know that multiple selections are possible. To address this problem websites often add a label to explain how multiple selections is made:
When notes and labels are added to things it’s a huge clue that the thing in question suffers from poor design. Also, the label in the screenshot is inaccurate. It is true in Windows, but not in OS X (and possibly not in true in GTK, QT etc).
The control requires the user to user press a key so that they can make multiple multiple selections - this means that the control is quasi-modal. Modes confuse the user and should be avoid. For such a simple task these failings are inexcusable.
Here’s a better approach:
Jimmy
Jimi
Frank
Dweezil
Jeff
Keef
John
Jimi
Frank
Dweezil
Jeff
Keef
John
<div style="overflow-y:scroll;height:6em;width:20em;border:1px solid black;">
<input type="checkbox">Jimmy</input><br />
<input type="checkbox">Jimi</input><br />
<input type="checkbox">Frank</input><br />
<input type="checkbox">Dweezil</input><br />
<input type="checkbox">Jeff</input><br />
<input type="checkbox">Keef</input><br />
<input type="checkbox">John</input><br />
</div>
The above creates a scrolling checkbox list by setting the size and overflow style attributes of the parent block element (in this case a <div>, but it could be applied to the <form> directly). Checkboxes lists are common in OS’s so the user will understand how to use the control.
REST + XMLHTTPRequest + 401 != joy
27/05/08 15:38
For the last few months I’ve been working on a web app using PHP that I’m trying to be as RESTful I can. It’s been a learning curve, and has been both satisfying and infuriating. Over the weekend I’ve been working on the authentication. As it is a RESTful app HTTP digest authentication is the obvious (only?) choice. This being 2008 I also want the app to be pretty, like all the other web 2.0 apps (I hate the term “2.0″. If the internet was made of technologies that could be broken in to discrete versions then life would be so much easier. While IE6 is still around the web will be stuck in beta testing).
Unfortunately, after much keyboard thumping and bouts of pseudo-tourette, I’ve reach the conclusion that RESTful apps haven’t got the style to get into Club Web 2.0.
This conclusion is based on the assumption that the standard browser authentication dialog box should be banished to 1997 and has no place in a 2008 web app. I must mention that I have only tested this on Firefox 2, Safari 3 and Opera 9 on OS X 10.4. FF and Safari both suffer from the problem outlined below. (Opera will always display the ugly dialog box even if a username and password were supplied to the xmlhttprequest object).
There are a few tutorials that say that you can use the xmlhttprequest object to suppress the browsers dialog box. It is true that the xmlhttprequest object can do this, but it only works in very select conditions. This is because of two features (bugs?) of the xmlhttprequest:
1. Firstly the XHR only answers the first 401 response per resource. Any subsequent 401 responses will cause the ugly dialog box to pop up. There a quite a few scenarios where the server might respond with a 2nd 401 response. The most common is when the username and password supplied to the XHR object are incorrect. Another common occurrence is that the nonce used in the digest expires causing the server to send a new nonce with the 401.
2. Above I mentioned that “the XHR only answers the first 401 response per resource“. This means that we could re-authenticate using a different resource. The problem with this is that browser send a mix and match of digest information. Here’s an example:
Lets say that www.example.com/restricted/ requires authentication. Once the server has authenticated the client, the client will continues to send the the authentication details to all resources within www.example.com/restricted/. This is the correct behaviour.
Next the client tries to access www.example.com/restricted/secrets.html. When the client sends the request it also sends the authentication headers as before but this time the server response with a 401. The credentials that were valid for the rest of the site are not valid for this one resource. If we this request is made with the XHR object then it counts as the first 401 response from that resource. Therefore the XHR sends the request again using the username and password. So where’s the problem? The problem lies with the fact that the client (Firefox 2 and Safari 3 at least), continue to send the nonce from the very first 401 response. The server may not accept the old nonce for this request and will send another 401 response. This second 401 response will cause the client to show the ugly dialog boxes.
I have developed methods to work around both of these problems so that the XHR could be used. I’m not going to use either of them because they are ugly and complicated. I always strive for simplicity as simple code is easier to understand and there’s less chance for things to go wrong. Introducing unnecessary complication into an authentication system is asking for trouble.
Unfortunately, after much keyboard thumping and bouts of pseudo-tourette, I’ve reach the conclusion that RESTful apps haven’t got the style to get into Club Web 2.0.
This conclusion is based on the assumption that the standard browser authentication dialog box should be banished to 1997 and has no place in a 2008 web app. I must mention that I have only tested this on Firefox 2, Safari 3 and Opera 9 on OS X 10.4. FF and Safari both suffer from the problem outlined below. (Opera will always display the ugly dialog box even if a username and password were supplied to the xmlhttprequest object).
There are a few tutorials that say that you can use the xmlhttprequest object to suppress the browsers dialog box. It is true that the xmlhttprequest object can do this, but it only works in very select conditions. This is because of two features (bugs?) of the xmlhttprequest:
1. Firstly the XHR only answers the first 401 response per resource. Any subsequent 401 responses will cause the ugly dialog box to pop up. There a quite a few scenarios where the server might respond with a 2nd 401 response. The most common is when the username and password supplied to the XHR object are incorrect. Another common occurrence is that the nonce used in the digest expires causing the server to send a new nonce with the 401.
2. Above I mentioned that “the XHR only answers the first 401 response per resource“. This means that we could re-authenticate using a different resource. The problem with this is that browser send a mix and match of digest information. Here’s an example:
Lets say that www.example.com/restricted/ requires authentication. Once the server has authenticated the client, the client will continues to send the the authentication details to all resources within www.example.com/restricted/. This is the correct behaviour.
Next the client tries to access www.example.com/restricted/secrets.html. When the client sends the request it also sends the authentication headers as before but this time the server response with a 401. The credentials that were valid for the rest of the site are not valid for this one resource. If we this request is made with the XHR object then it counts as the first 401 response from that resource. Therefore the XHR sends the request again using the username and password. So where’s the problem? The problem lies with the fact that the client (Firefox 2 and Safari 3 at least), continue to send the nonce from the very first 401 response. The server may not accept the old nonce for this request and will send another 401 response. This second 401 response will cause the client to show the ugly dialog boxes.
I have developed methods to work around both of these problems so that the XHR could be used. I’m not going to use either of them because they are ugly and complicated. I always strive for simplicity as simple code is easier to understand and there’s less chance for things to go wrong. Introducing unnecessary complication into an authentication system is asking for trouble.