theChrisWalker.net

You may notice an Xbox360 related theme in my current posts, as after getting one, most of my technology thought process is involved in something to do with it. Hence the Xbox Live Gamercard API I wrote about before. Now we get on to UPnP.

Universal Plug and Play is used for 2 mains things as far as I can tell.

1. Dynamically opening holes in firewalls/NAT configurations for inbound connections to services. (bad!
2. Media Discover/playback/control on a local network. (good!)

The first is bad in my opinion. I don’t want the ability for some software behind my firewall to allow connections into my network. It might make some software function a little more smoothly, but if it’s that important that connections can be made inbound I would have set up a port-forward myself.

So I am only concerned with the second scenario.

The situation was that the Xbox360 can Stream Music/Pictures and Video from a Windows based PC. Great, except if you haven’t got a windows based PC, or you have all your media on a fileserver running Ubuntu Server and don’t want to have to have another PC on just to stream to your Xbox360. So I did some digging and it turns out that the Xbox360 uses a “standardised” UPnP discovery protocol. I say “standardised” because it’s not quite compliant, but close enough.

I look at the options available and try uShare first. It looks good, but the Xbox won’t see it and it can’t read the ID3 info from my music - rendering the whole thing useless.

Next TwonkyMediaServer. I read somewhere that the linux demo doesn’t expire. I hope not, because it worked beautifully pretty much out of the box. The only gripe is that there’s no nice init scripts and I have to run a shell script every time I want to start/stop/status/restart it. But it read my media and the Xbox360 recognised it and I haven’t looked back.

So I thought, Wow, this UPnP thing is pretty cool. but it must be more than this?. So I booted up a Windows7 PC. I had had trouble with this machine because Windows Media Player didn’t like the fact that all my music was on a Read-Only SMB share. That and I hate the way the Windows won’t let you connect to different shares on the host with different credentials! How rubbish!

Still WMP12 didn’t like my read-only filesystem. However, with Twonky running on the network, all the music just “appeared” like magic in the “Other Libraries” section. nice.

Then I thought of another thing that had been annoying me. Why can’t I stream media from my fileserver to my Android phone over Wifi?

Well, guess what? There’s an App for that!

Andromote can connect to UPnP media servers and play back the content. Also, and this is quite cool, I can specifiy the media server as Twonky, specify the Media Renderer (i.e. what plays it) as my Windows7 PC, and use the Andromote App as the Control Point - allow full media browsing, playback, skipping, seeking and volume control of the PC from my phone. Pretty neat, eh?

The whole time, all the servers and renderers are auto-discovered, no configuration necessary. That’s pretty cool.

So in summary UPnP media is in 3 distinct parts and the best bit is that each part can be on a separate physical machine, or 2 on 1, or 1 each on 3 separate physical devices. Very flexible. The parts:

Media Server — Serves the media - duh. i.e. media is physically stored (or accessed from) here.

Media Renderer — Plays back the media - simples.

Media Control Point — Controls the Media. Reads media info and tells the renderer what to play.

So the most common configuration is the server on one machine and the renderer and control point in another. That’s how the Xbox360 works.

So that was my experience into UPNP. I realise now that it was slightly unecessary as my old Xbox, running XBMC, would have (according to their wiki) worked as the UPNP media server out of the box and is conencted to all my shares, and isn’t a hassle to turn on as it sits right next to my Xbox360. It’ll be my fallback if Twonky does expire…

As my last post said, I’m using ZFS and love the way it works, but am having trouble with the performance of ZFS-FUSE. It’s not unusable, I just expected more.

So I thought, OpenSolaris. It’s the original implementation of the filesystem it must be great, so I export my zpools and boot from the LiveCD (another great reason to try it).

However it wasn’t going to be that easy… When I try to import my zpools in Solaris I get errors stating the vdev’s are corrupt! Something like:

status: One or more devices contains corrupted data.
action: The pool cannot be imported due to damaged devices or data.

Scared I boot back to linux to find they are all fine.

Back to Solaris and they still look corrupt. So I search and find this on the ZFS-FUSE Google Group.

Sun claims best practice is to use an entire disk when assigning disks to vdevs, rather than just partitions. So that’s what I did in linux. However the Linux implementation differs form the Solaris one and whole disks added in Solaris get a single partition first and that is added to the vdev.

Great! Now I have these zpools I can’t use anywhere else. To test I used a pool I set up as a mirror and detached a vdev (one disk). I then repartitioned the disk to have one big partition and also wiped the ZFS pool info from it (easier said than done! Ghost pools seem to be a problem!). Then I re-inserted the new partition into the the zpool and waited for resilver. Then I did the same with the other drive, detach from pool, partition, attach partition and resilver.

Bingo. It works in Solaris! Now I just have to find a spare 750Gb disk so I can do the same for my JBOD style zpool…

I have a fileserver at home. I have done for many years now. The first incarnation was based on Fedora Core 3 (which will tell you *exactly* how long ago that was…) and was very simple. I had an old Xbox HDD (8GB Seagate) and 2 shiny new 200GB Western Digiatl drives for storage.

This was also my first major experience with Linux as a whole, so I was trying to keep things simple. So I did a default type install, with LVM on the 2×200GB disks to give one logical 400GB partition.

All was fine and dandy until I wanted to upgrade a couple of years later and not knowing much about LVM, wiped all my data. Yes, ALL my data. I had some important stuff backed up, but a lot of non-critical but frustrating to lose data (many GBs of painstakingly ripped and encoded CDs and Audiobooks for example).

I was gutted, and vowed not to use LVM again (note my solution to my mistake and lack of knowledge - never use the software again! Well thought out!). So I rebuilt the server using CentOS 4.x ( I can’t remember…) and added another 250GB disk. This time I kept each disk seperate with it’s own filesystem.

This worked fine for ages until the disks were nearly full and I spent all my time moving data from one disk to another to gain a few GBs extra on another disk. Nightmare.

So I decided it was time to start again and did some research into the available filesystems people use for fileservers, how software RAID can be implemented and so on. In this time I came across ZFS and I couldn’t believe what it could do. It seemed to good to be true. I now know this to be false and it can’t do everything, but it can do an awful lot!

So how can I use ZFS? Well, if you use any Solaris variant, then you probably already are. Otherwise chances are slim that you’ve encountered it. I didn’t really want to go down the OpenSolaris route, but instead stick to Linux where I’m more comfortable. This poses issues, as the licencing behind ZFS from Sun doesn’t fit with GPL, so no kernel support for ZFS. Instead someone (I’d love to reference but I forget his name, google will surely help - http://google.com/search?q=ZFS+FUSE) has ported ZFS to FUSE which is great except for the massive memory usage and the poor performance.

Those actually sound like quite big issues! Well, the memory thing is not so bad as it seems. ZFS is not memory instense (well, actually it constantly uses about 256Mb so it’s not light) but uses available memory cache to store data in case it needs it again. A good idea, as the memory is otherwise just doing nothing! The Performance is pretty poor though. It’s enough for my network and to stream music, pictures and video over my network. But single file transfer never hits much more than 8Mb/s read or write.

Now I’m interested in trying some of the other OS options, as ZFS is easy to “export” from one OS and “import” into another. What are the options?

• Option 1: OpenSolaris – Seems like a good idea. The native platform, has a good AMP stack so should be fine for my needs. Have heard some stability issues, but not so much with ZFS, but the OS itself.
• Option 2: Nextenta – Solaris Kernel with GNU userland. Supposed to be a good happy medium, sounds too much like a compromise to me…
• Option 3: BTRFS – different route totally but a new filesystem designed not to mirror ZFS, but to take the best of everything it can. Some aspects very like ZFS. Very promising project, but too young.

Kinda only leaves OpenSolaris… I’ll let you know how I get on.

I had heard about Google Wave, had a quick look at their UI, thought “shiny” and not really paid it much attention. Then I was reminded of it again recently and so I had another look, watched the Keynote Speech that they gave at the Google I/O 2009 and had a bit of a look around the web for related articles.

And I got really excited and really angry.

I got angry, because most people weren’t excited for the right reasons, or were dismissive / “anti” because they didn’t really see what Google has done.

But because I am so excited about Wave (not just Google Wave), I thought I’d post this to help you understand what Google Wave actually is and what it isn’t.

What Google Wave isn’t

Google Wave isn’t Wave.

Wait a second! Why have you dropped the “Google”, what’s “Wave” all about?

Firstly let’s take an analogy, with something you are no doubt familiar with. Consider the statement:

Google Mail is not email.

Imagine that email didn’t exist and Google showcased Google Mail. People might think that email was Google Mail, but we all know that isn’t the case.

The same for Wave. Google have designed a messaging system which can be run by anyone. They have designed the protocols it will use which anyone can implement. This messaging system is not Google Wave, but Wave itself.

What Google Wave is

Google have designed a UI for Wave, which is Google Wave.

Don’t get me wrong, I like their interface. It is written in HTML5 using the Google Web Toolkit and it is super slick.

However, the Wave itself is far more exciting.

What Wave is

Wave is email redesigned for the 21st century. Email was born before the internet and has been flogged and flogged into how we use it now. It is inefficient, single one-way message based, bad with files, incosistent with rich styling, non-collaborative and basically just way past it’s “sell by date”.

With Wave we no longer deal in single messages, but conversations; where content evolves over time and can be “played back” to provide the user with context. Everything email and IM can do and more. They have designed the system to be extensible and so can communicate with other systems. They also designed it to work over a federated networks model, so many people running their own Wave servers can communicate in the same rich way as if they were communicating with their people on their own server.

Amazing really, and provided we can bridge the backwards compatibility gap^* (which we can, remember Wave is extensible!) then I see Wave replacing email altogether.

When Google Wave goes live, Google will also be releasing the server for anyone to run, learn, enjoy and stop using email with. I am looking forward it immensely.

[update 2009-08-06] Further developments have shown that yes “gateways” have been designed in, so Wave can replace email and any other messaging/collaboration protocols/tools if someone creates a “gateway” for it.

Oh my god. I can’t believe how addictive this game is. http://fantasticcontraption.com/. The simplicity of it really appeals to me. The idea is to move a pink block from one area on the course to another, and you do so by building a contraption from wheels and connectors. that’s pretty much it!

Doesn’t sound like much, but if you are like me, you’ll not only want to finish all the levels but also go back and experiment with loads of other different ways to finish in the vain hope of finding the perfect solution. A solution that excels by being so simple, or so elegant. So unusual, or so complex. I could play for hours. But I won’t go on for hours.

I will leave you with some of the best contraptions I have managed to come up with, and the first one you should really vote as amazing, because (and I know this sounds a bit arrogant) is ingenious!

• unusual technique! See if you can guess how it works before you start it!
• Minimalistic elegance, perfect timing.
• I saw a solution with more than one spinny thing, but this seemed a cooler solution (notice the pole to slow down the last ball slightly…)
• A cool way to do “The Ball”

That’s all for the moment, but I’m sure I’ll post more contraptions in the future…