Dark Liquid

Typically things like Vlad and Capistrano make the assumption you already know where you want to run your tasks. They assume you already have some servers somewhere ticking over just fine and you want to do stuff on them according to some predetermined pattern.

However, what happens when you want to write some tasks you can run on any remote server? What happens if you don’t have any servers and you want to build some dynamically and then run some remote tasks on them?

Well, I’ve been asking these questions and I found an answer that helped me out and thought I’d share it here.

Using the dependency system of rake and the lazy role evaluation of vlad along with the fact that it provides some very useful public class methods (so they are presumably safe, rather than hacking into the internals with send or the like) I wrote the following simple test script:

require 'rubygems'
require 'highline'
require 'vlad'
task :server_config do
  ssh_string = HighLine.new.ask 'Enter the SSH details of the server you want to connect to: '
  Rake::RemoteTask.role :server, ssh_string
end

desc 'Run stuff on a server'
remote_task :server, :roles => :server do
  run "echo 'hello i am a server'"
end
task :server => :server_config

What’s going on here is that I have a remote task using a role server which as of yet I haven’t defined. When I run rake server that runs the server task. The server task is dependent on the server_config task which sets up a role ‘server’ with the given string (ideally something like user@server.com). The remote task then executes, evaluating the value of the role, which is now set to the users input, and connects to the server you’ve set at run time. How awesome is that?

Of course, this is a contrived example, you could use a system to setup roles automatically from other things rather than pulling in user input but the lesson to take away here is that is is possible to use vlad tasks on dynamically defined targets easily.

Walking home from through the park on Monday night, we managed to pick up a little follower, a drunken 15-year old who insisted he knew me. He seemed harmless enough and my general attitude to harmless, happy drunks with a sense of over-familiarity is to talk to them and kindly dismiss anything they say regarding knowing me, better to be friendly than to turn them into an angry drunk and have to deal with that.

Generally I find random drunk people sitting alone in the dark at midnight rather odd and vaguely pathetic, I don’t really understand why you would want to do that, but I guess, never having been drunk myself, that I’m beyond understanding such notions. More odd though was the boys attitude and things he was saying. It made me realise just how alien some youngsters seem to me nowadays. Of course, I can hardly base my opinion of all teenagers on an unwanted conversation with one drunk, teenage boy, but nonetheless it was a weird experience.

Essentially, the boy kept insisting he was a ‘grebo’ and that he knew me because I must hang around with other grebo’s, apparently being one myself. This slang a wasn’t familiar with but when I got home a quick search on urban dictionary revealed he meant something along the lines of ‘rocker’. Apparently.

Anyway, after several protestations that I knew neither him, nor any of the people he was referring to, nor any of these other groups of people he kept spouting (emos, goths, chavs, grebos, etc) he just seemed to reset and repeat the questions ad infinitum. It was as if he just couldn’t comprehend that I might not subscribe myself to one of these social castes. It wasn’t as if he disbelieved me, or as if he thought what I was saying was impossible, it was as if not being in some kind of caste was an entirely alien concept his mind and he just couldn’t process and it so just threw any of my protests out of the window as a result. It was a if his mind could not comprehend me as an independent entity and so kept grasping at straws so it could fit me into it’s conceptual world-view.

I found that mind set frightening and disturbing both. When I’m around teenagers of that age, I find I don’t understand them at all, I don’t understand why they do the things they do or ays the things they say. It’s frightening, not in a threatening way but in a more introspective way. When did the world become full of aliens, when did I lose touch with this great swath of humanity that was only 5 years old when I was their age?

I even feel like this to a certain extent with some people only 5 years younger than me or thereabouts, though the effect is lessened somewhat but there is still this strange tint of alienness about them, something unknowable, inconceivable, incomprehensible that I can’t even conceptualise to put into words.

Getting old, I guess.

I’ve been holding off upgrading the firmware on my G1 due to fears about warranties and bricking my primary communications device but my phone was acting up recently, being very, very slow on the stock firmware that I decided, what the hell, lets do it.

My good friend bytey directed me to a useful guide on rooting and flashing your phone. I went a little further than that and so I thought I’d share the process with you.

I first followed the above guide but stopped when it came time to installing a new firmware, instead just following the rooting, recovery and HardSPL instructions. Once that was done, I ran into trouble.

I decided to download and install cyanogen 4.0.4. To use the Apps2SD functionality, you need to partition your microSD card so it has a ext3 partition and a fat32, which I did using fdisk and the mkfs tools under linux. I opted to have the fat32 partition be the first on the disk, followed by the ext3 one. I’d heard that problems happened with large ext3 partitions so I made sure to keep it to 1GB or less.

The install process went along easy enough, I downloaded the firmware from the cyanogen mod site and renamed the update-blah-blah-blah.zip file to update.zip and placed it in the root of the SDcard then powered off the phone. I booted the phone into recovery mode by holding the home button and tapping the power button, did an Alt-L to show the menu, Alt-W to wipe the phone and then Alt-S to flash the rom from the update.zip.

Voila, one reboot later and I’m running cyanogen!

What happened after this resulted in me screwing up my install. I installed the cyanogen updater from the market and tried to install the enoch theme with it. When the updater tried to apply the theme the screwed everything up and I had to reinstall cyanogen.

What I should have actually done was install the cyanogen recovery image first. I did this by downloading it from the cyanogen website, opening a terminal on the phone, su-ing and then typing:

flash_image recovery /sdcard/name-of-cyanogen-recovery-image.img

After installing that the cyanogen updater worked a treat for installing the enoch theme and I was away!

For a while now I’ve been working with both Git and automated server deployment and I was thinking, why not combine the two?

Wouldn’t it be awesome to have a full revision history of provisioned changes to a servers configuration and wouldn’t it be nice if every time you pushed your app it was magically deployed and set up and running? Yes, yes it would.

At SonicIQ we developed a tool similar to gitosis, a tool that uses git to manage git repositories and access control. Building on some of the principals learnt from that I think it would be possible to use a git managed server configuration file that target servers could checkout out and use for configuring themselves, their checkouts happening either periodically or triggered by post-commit hooks somehow. Those servers would need a minimum of software running to be able to parse the config files, determine changes (easy to do with a git repo) and then implement them automatically without manual intervention. These self-deployment servers could also handle requests from app repositories (again via post-commit hooks) and automatically deploy sites when new versions are pushed out.

I can see obvious problems with this approach, such as security concerns (having a server that can basically reconfigure the entire system based on commands sent over a public interface) and management issues due to configuring post-commit hooks and what not, but generally I think the concept is pretty sound and other similar systems already exist such as puppet which are no doubt better and more robust, but I quite like the idea of building a distributed deployment and configuration system on top of git, if only to see if it’s a feasible approach.

As you may well be aware, I’m currently studying computer games development via a study-at-home course. The course has been the victim of a lot of vitriol and doubt from the internet at large as towards it’s authenticity but from my own experiences so far it seems quite good. The course is question is the games developer course offered by train2game in association with TIGA.

It starts off assuming no knowledge of programming whatsoever so for me the early stages are a little pointless, though in fairness they do cover some of the thought processes that you’d need to familiarise yourself with both to be a decent coder and to work on games specifically. I’ve made sure to do everything in the books so far, whether or not I think it’s useless given my existing skillset.

This course is relatively new, something I’ve been stung with before when I did my foundation degree in computer and electronics interfacing, but the course tutors and staff seem extremely responsive and ready, eager even, to give help. The forums also are home to some good community information as well as frequented by a few unaffiliated members of the games industry which provide help and advice of their own accord which is amazing of them and much appreciated.

So far, I’m pretty happy with the course and I’m currently on the early stages of the C++ parts, where you learn the language and how to use it. Going through the material, I’ve been surprised by how much C++ I’ve forgotten over the 8 or 9 years it’s been since I last used it actively. The book has been a great help, as have my tutors, in getting back up to speed.

I’ve had some minor issues with the course material, sometimes the questions in the mini-exams are vaguely worded or terminology used isn’t 100% consistent which can course some confusion but the tutors have always been on hand to answer any questions and are eager to correct any mistakes in the material, putting out errata when needed.

I’m looking forward to breaking out into more complex territory soon as I reach more advanced subjects in the course. In the mean time I’m teaching myself how to use various 3D libraries (my eyes currently set on Ogre3D) and messing around with physics engines (chipmunk for 2D stuff and possibly bullet for 3D).

At this early stage in the course I can’t really lodge any complaints as for the most part I haven’t actually learned anything particularly new to me but as soon as I hit that stage I’ll keep things updated for all of you that are concerned this course might be some kind of scam. So far though, the effort put into it and the general quality of responses from both tutors and the community makes the whole scam scenario seem extremely unlikely. As to whether or not the qualification at the end becomes recognised by the industry remains to be seen.

Okay, I wont deny it. Yes, I’m a miserable bastard. Even my own Dad has told me this on one occasion and rightly so, for it is most certainly true.

Generally I dislike pretty much everything, I enter into things expecting disappointment. I’m a natural pessimist, I guess because I really dislike the feeling of disappointment and so going in with extremely low expectations makes it far less likely I’ll be disappointed (or it will at least take the sting off).

I rarely watch new films nowadays. Why? Because I will be inevitably disappointed. I have a pretty eclectic taste in films and can’t really pin-point what it is about a film I find enjoyable but I know what it is I don’t like. I don’t like so called ‘blockbusters’ most of the time, since generally the idea there seems to be to tie together as many cliques and special effects as possible with just enough narrative to justify calling it a film. Once you’ve seen one set of flashy explosions and special effects you’ve seen them all. Sometimes, I do find those kind’s of films amusing when they bring something new to the table, or a also contain a particular brand of wit I enjoy but this seems to rarely be the case.

I rarely leave the house. There is very little I’m interested in actually doing outside that doesn’t require my own method of transport beyond merely walking to get there. There are places I enjoy going to but don’t because it’s either to difficult or expensive or just takes too damn long, by which time if changed my mind or become interested in something else.

It’s not even really about the money, though I am quite miserly and dislike what essentially amounts to gambling on the entertainment industry in the hopes I might get a payoff of enjoyment when I know the odds are not in my favour.

Consequently, I tend to play it safe, sticking to things I know and own because I have a reliable source of entertainment there at (after the initial investment) essentially zero cost which appeals to the miser in me as well as placating my fear of disappointment. This generally restricts me to reading – something I enjoy immensely, writing, roleplaying, watching films and TV I have already seen before, listening to music, programming, designing electronics hardware and playing computer games.

That’s not to say I don’t like trying new things is just that when I do, I like to do a proper assessment rather than just dive in only to find out I’ve wasted a bunch of my time and money. I consume media having never seen them before based on whether I have enjoyed other products by the same creator in the same genre, whether they are related (sequel/prequel) to things I have enjoyed before, whether I find the concept intriguing enough to take a gamble, whether the cost is low enough to justify such a gamble, whether or not there are already things I’m enjoying that will keep me occupied enough not to need yet another new toy right this instant, opinions from people with proven similar tastes as me on said media when they’ve consumed it and reviews from critics as well as the standard marketing material in the form of trailers, etc that are provided.

That might sound like a lot of work but really it amounts to maybe 30 minutes in total at the very most, usually I can make a snap judgement pretty quickly and then I just happen to change my decision later as new data becomes available and I happen to come in contact with it. It isn’t as if I sit down and exhaustively calculate whether or not I will enjoy something, rather I make a quick judgement on readily available information to me as to whether I’m interested enough at first glance to investigate further, after which (if I am) I then look for some extra info.

Most things don’t get past that initial first check because I have pretty exacting standards and wildly differing standards for exactly interests me. Most actions films like, say, Transformers 2, have very little information about the quality of the actual film instead giving short snippets of action sequences many of which just aren’t entertaining enough for me to want to invest my time in, considering I’m not much of a generic action film fan.

The last film I paid to see at the cinema was Watchmen, I believe and I only did that because of information I received on various news feeds I read, the fact that I like the comics (which I read based on recommendations from friends and based on my own research into comics since I wanted to see if there were any I might enjoy since I’m generally not a comic book fan but wanted to try something relatively new) and that the trailers and associated marketing material appealed to me, as they didn’t all seem to be portraying just another generic action film. Some material made me reticent to see it, such as finding out the ending was different to the comic, which I found somewhat unnecessary, but I felt that the information I’d gathered, plus the fact I could go see it with friends who wanted to see it as well meant that I’d give it a go. I was glad I did as I enjoyed it very much and have preordered it on bluray.

Musically, I like to try new things quite often, mostly because it’s very easy to do so at basically a zero cost in both effort and time (since I can do stuff at the same time as listening to music) via services such as last.fm and spotify. Occasionally I’ll come across music not available but from a band that sound interesting based on how similar they are to stuff I already like and reviews, etc and I might gamble downloading the mp3′s via amazon or occasionally buying an album if it’s cheap enough or I’m intrigued enough.

Books are very easy for me to ‘gamble’ on, they have an extremely low entrance point, since they need no other equipment to use, just my eyes and hands. They are portable and can be enjoyed pretty much anywhere. I also like reading because I like getting inspiration for my own writing and ideas elsewhere. Books generally inspire me, so intriguing me isn’t that hard, I essentially like books by virtue of them being books so, yeah, low entry barrier there. Saying that, there is an awful lot of books out there I will never read and for most of the reasons I’ve stated for other forms of media, being bad marketing material (cover art, blurb) and bad reviews or me simply not being aware of their existence.

So yes, I’m a miserably, miserly bastard because I do anything without weighing it up against several factors, some subconsciously, some consciously. I really do want to enjoy things, but I have such high standards for what qualifies as enjoyable and such a dislike for the feeling of disappointment that I rarely do anything new at all.

I’ve been playing around with Appcelerator’s Titanium system and so far it’s been interesting. I haven’t done very much in the way of app development yet as I am still learning the APIs but I finally managed to get it to run under Linux (Ubunty Jaunty specifically) which is a plus!

How to get the Titanium Installer to run on Ubuntu Jaunty

Basically, I needed to run the following command:

sudo aptitude install libcurl4-openssl-dev

which installs the libcurl4 with OpenSSL library and associated development bits that the Titanium Installer needs to run. The error message that pops up isn’t the most helpful for working this out, complaining about a missing CURL_OPENSSL_3 symbol or some such thing. Perhaps some better explanation would be useful in their installer so it can suggest how you might fix these things.

I just need to install the android SDK under Linux now (I’ve already tested things out under OSX with both the android and iPhone 3.0 SDKs and it all seems good).

I might try working on developing Windows apps (which I need to do as part of my course) under Linux using wine for cross-compilation. That would be awesome.

Okay, so I’ve used Windows 7 a bit on and off and I’m fed up with it. Like all other versions of Windows it’s basically shite.

Why?

Because Microsoft seems to want, for the most part, to not actually be in the computing industry, but more in the ‘computing appliance’ one. Windows 7 I’m sure works great if you have a box built out of 100% certified and Microsoft-approved hardware and you just want to load stuff onto it and use it without a moments thought.

I don’t. I’m running it on a Black Macbook, I want and need to be able to fiddle with various settings and dig under the surface for when things go wrong. I want to understand how my machine is actually doing what it is doing so I can optimise things. I’m a developer, I want and need to access to the tools to tune my system and monitor it, not just from a coding perspective but from a user’s one as well.

Windows 7 just gets in the way, like all the Windows OSes before it, trying to dress things up, hide them away or just refuse to acknowledge they exist so they don’t have to put them in the hands of users. It’s incredibly frustrating.

Yes, Windows 7 is in beta so it can’t be expected to have full driver support for everything but if my linux install can do it, so can Microsoft. I mean seriously, people have to pay for this shit.

Nothing I’ve done has managed to get everything to work on Win7. The performance is abysmal, the sound doesn’t work and the brightness and volume controls don’t work, which is very annoying when suspend turns off your backlight and there is no obvious way to re-enable it. It’s shit like that which I find mindnumbingly irritating. Surely if you could turn it off, you have enough access to the device to turn it on again, why wont you let me?

I’ve tried multiple versions of the boot camp drivers, hacked up drivers, custom drivers. Nothing bloody works and the so called ‘troubleshooting wizard’ just flails around impotently like a blind idiot, flailing at the walls in the hope it might hit something to make things better.

I take back all the nice things I said about Windows 7. It’s nice if you want to buy a computing applicance, not if you want a computer. Then it sucks and it sucks hard.

Well, that was pretty cool, I just managed to recover the data off of a dying 30GB hard drive on my friends laptop. It was surprsingly easy, all I really needed was time.

First of all I downloaded and burnt off the latest beta of SystemRescueCD. After booting from that on the laptop, I set up the networking on it and downloaded dd_rhelp from freshmeat. I then mounted I directory on my main desktop machine via sshfs on the laptop and ran the following command:

./dd_rhelp /dev/sda1 /mnt/ssh/backup.img

That command basically uses the dd_rescue tool on the SystemRescueCD to copy the first partition of the first hard drive into a file on my desktop computer (via the sshfs mount point at, you guessed it, /mnt/ssh). dd_rhelp is nice because it makes the process faster apparently but leaving all the bad sector, I/O error retrying till the end, instead skipping over errors when they happen in order to copy the actually working stuff first. Then it goes in and fills in the blanks. I left this running for about 16 hours after which it had all of the drive except for about 0.5MB which just would not copy. I cancelled the copying at this point and went to work on the backup image on my desktop.

First of all I backed up the backup image. Some of the things I might do to it could be destructive and I don’t want to wait another 16 hours to grab the data again, if the drive will even work. On the backup, I then ran the following command:

fsck.vfat backup.img -rw

This scanned the filesystem in the backup image (in this case FAT32) for errors and asked me to choose a way to fix them. I basically told it to not mess around with anything except broken filenames and incorrect cluster sizes under the assumption that if this could actually see files to mess around with, chances are I didn’t need to play with the actual FAT table on the disk.

After running that and writing the changes to the image, I mounted the file into my system and copied off all the data my friends would most likely want (the entire Documents and Settings folder in this case) and burnt it all onto a DVD for them. I’ll keep the hard drive image around in case they discover there is something else they need in the next few weeks but yep, that was it. Easy.

I must say I was damned impressed with SystemRescueCD, it’s chocablock full of very useful tools and it’s ability to do networking and it’s provision of ssh tools makes it very nice to work with indeed. I could not have done this without it. Well, actually that’s a lie, I was prepared to burn a customised copy of DSL with all the tools I needed compiled and built into it but luckily I found SystemRescueCD before I wasted my time on that.

Fun stuff.