Change...

I've been putting off writing this post for a lot longer than I should have because writing things down often makes them seem more real and final. This is a silly notion, with very little basis in fact, but silly notions are not uncommon to me. Yet here I am writing this down, or rather writing a long and boring introduction to delay the inevitable even further, so bullet bitten here I go.

The 9th of December will be my last day at Simulacra.

A couple of years ago that is a phrase I thought I would never be writing, a year ago it was something that I hoped would not have to come. Finally 6 months ago I realised that my time had come. At the 5 year mark and I had one of the silliest notions I have possibly ever had. I realised that where I was in my life, then and there, was the reality of the answer to the most popular question to ask someone in an interview, "Where do you see yourself in 5 years time?".

Silly notion to be sure, but one that stuck with me, that I couldn't shake. Amongst the many reasons for my leaving the top most is that I needed a change, a new challenge, something quite different.

Just signed up for the new Google Analytics...

Interestingly, it took me a little while to find United Kingdom in the country select list, it was up in the G's so I guess they are ordering by ISO Country Code not the display name. Won't get the first reports for 12 hours so I'll let you know how good it is then. In the mean time sign up your site here.

Geeks love to gossip...

It's true, they are like the proverbial 1950's housewives chatting over the fence , holding forth about the latest rumors, whether Google has sold its soul or if Linux ever will be a viable alternative desktop OS.

The past few weeks us geeks have been in gossip heaven because of the few companies that understand this phenomenon and fuel it to their advantage. First we had Apple's announcement that they would soon be announcing something (that turned out to be the iPod nano), then the joint Google/Sun press conference and finally Apple's "One more thing..." press conference (which turned out to be the new iMac and the Video iPod). Other companies choose to fuel rumor mongering after the announcement by making it so bizzar that no one can quite understand what is going one (eBay's purchase of Skype). Finally there are those companies that just don't get it and make announcements that nobody is interested in (Microsoft/Real, Microsoft/Yahoo).

So looking back on the last few weeks, how did they all do?

Apple (nano announcement)

Rumors built about a new iPod, but few anticipated the storm of media attention that would brew up over the nano. I am constantly amazed at how Apple can keep such a big announcement under wraps and be ready to ship the next day. Unfortunately, while initially this gained them many points they ultimately lost a lot of these amidst problems with easily broken screens on their shiny new nano range. 8/10

Google/Sun

Never have I seen a single press conference announcement trigger such outlandish behavior. The rumors had this about anything from a Wi-Fi infrastructure partnership to GoogleOffice to GoogleOS (that one will never die it would appear). After setting the net alight with wildfire rumors Google's tactic of keeping schtum really backfired with a nothing announcement of a joint marketing and software distribution deal. "Joint marketing" is a phrase that will never get a Geek excited, unless (there you go Craig I didn't use "thanless"!) its Eidos and Playboy.

Interestingly I do wonder if the rushed release of Google Reader (which was the most unready for release Google Beta I've seen) was in part a reaction to the boredom surrounding the Sun joint press conference? 4/10

Apple (iMac and Video iPod announcement)

Rebounding from the iPod nano screen problems Apple offered up their mysterious "One more thing..." press invite. This was designed to really get the rumors flying, and it did. This time there were no disappointments and after telling us for four years that no one wants to watch programmes on a small screen Steve Jobs announced a new iPod "and yes, it plays video". Lots of people saw this coming, they also saw the updated iMac coming however the "One more thing..." of the title was the tie up with Disney (previously, for a short time and while wearing a different hat, sworn enemies of Jobs) to allow the purchase of TV episodes of 5 series, including Desperate Housewives and Lost for $1.99 each.

As always I recommend watching the stream of the announcement as Steve Jobs is still the best at this stuff. 9/10

Microsoft/Real

Apparently Microsoft and Real have settled their dispute and are teaming, in some strange Power Ranger style combo, to defeat the evil Apple and its all destroying iTunes/iPod weapon. No one listened, no one cared. 2/10

Microsoft/Yahoo

To stem the losses on its western flank (Google, Apple is definitely the eastern flank) Microsoft called on old enemies to make a stand with them. M$ and Yahoo announced that in a landmark deal they would allow MSN Messenger and Yahoo "whatever its called" talk to each other. The rest of the world went "woo hoo, I use Gaim ". 1/10

eBay buying Skype

eBay bought Skype for $2.6billion ... no one seems to know why ... but they are still talking about it! 7/10

But the silly season is not yet done. There is another Apple announcement set for the 19th October, could be new PowerMacs, could be a new semi-pro iPhoto who knows...

Google's RSS Reader...

Google Reader is the newest addition to Google Labs. Fantastic AJAX interface, very like GMail, in which you can see the design ideas that could remain consistent all through Google interfaces.

However the most important aspect of this the ability to add any metadata tag/labels to each post as you like. This is very much in the Folksonomy vain. This is only a very small feature of Reader but quite a sea change for Google to include it....of course I cannot see any way in which these tags are used???? Database of Intentions then :)

4th annual 24x24...

This Saturday my friends and I will be doing 24x24 for the 4th year. Watching 24 Season 4 in real time. This year obviously starts at 7am, which is going to be quite hard. The last time we had to start in the morning, Season 2 starting at 8am, we made the mistake of staying up far to late the night before. It was very hard to make it all the way through.

That will be a nice end to a manic few weeks. Yesterday we delivered the most recent piece of work to one of our main clients, MLA (the Museums, Libraries and Archives Council). We've been building an Institutions Server, a central database of institution contact details. It has several Web Services and an OAI Harvesting interface so that the data can be shared between several other MLA websites. The Institutions Server, while a relatively small project, will have quite a reach within this sector so I am very excited to see it finally out there and being used.

We have also started up the developer research days again, now that things are calming down. This is based on Google's model of ring fencing some time for everyone to do work that interests them but that might not prove to be commercial to the company. Each developer is supposed to do one day each month on something, although this tailed off for while during our busy period. Over the last few years we have ended up with some very commercial code out of these days, and it is always nice to take a break from the usual projects for something a little different.

Now that the Institutions Server has been delivered I will finally get round to mine next week. Still firming up what I will do, but looking forward to it a lot.

So that's what I've been up to recently and what's coming. Finally there are a few websites that I wanted to let you all know about.

1) The Clerks 2 Production Diaries

Much like the King Kong and Superman production diary videos, just with a lot more swearing and the wonderful Kevin Smith view on the world.

2) The DARPA Grand Challenge - or the modern Wacky Races

With $2 million up for the autonomous vehicle which completed the 140+ Mile course the quickest the competition this year looks a lot better than last, during which the furthest anyone got was a little over 7 miles. The event itself is on the 8th when the link above will cover everything. Unfortunately my favourite team, the Blue Team, who in a moment of madness decided that they would do it with a motorbike instead of a car, look like they won't make it through he qualifiers. But do go to their website and check out the videos of the riderless bike!

Lost post damnit!!!

I hate it when I loose a post...and Bloggers "Recover post" option didn't work.... damnit!!!

New flat, new broadband, more work...

I have spent the day working from home and I plan to do the same tomorrow. This is a rare occurrence for me, I don't particularly like working from home it feels lonely. But when I have a report to produce and only two days to do it in the lack of distractions is good.

At least I have the option to work at home now that I have finally got broadband back. Not only is it back but it's 8Mb! Makes more of a difference than I thought it would. I've also finally got wireless networking all sorted out, so when I am working here at least I can do it from the comfort of whichever part of the flat takes my fancy.

Aside from all of that, not much else to report, even with my exceedingly long absence from these pages. More soon...

Black coffee with milk, the new trendy London drink...

You may scoff, but this is it. This is the new coffee drink that all the London trendies will be drinking through the fall season, then it will sweep the nation. The big coffee shop chain will, of course, have to give it their own trademarked names. Starbucks will probably call it "Blackuccino" or something equally stupid.

Still doubting me, well try and order a black coffee anywhere in London and 9 times out of 10 you will get the Blackuccino. I have tried asking for a "black coffee, no milk" but I either get a blank stare in response, or the minimum wage flunky behind the counter latches onto the word 'milk' and takes it as a reinforcement of the idea that I couldn't possible have been asking for a black coffee, because no-one drinks those.

At least once the Blackuccino has a name they will know when I am not ordering that!

Takashi Goldberg gig...

I should have posted this much sooner after the event, but I had the distinct pleasure of attending the first ever Takashi Goldberg live performace at Bardens Bar in Dalston. Must fun was had by all and they were brilliant. The highlight, for me at least, was their song "I want to live in New York and be one of the X-Men". They dedicated it to me, probably in an effort to get me to stop pestering them about it.

Really looking forward to the next gig although no one appears quite sure when that will be.

A week training people...

I am spending the better part (depending on your point of view) of the week training other people to use our software. Tomorrow will be a group on non-technical people learning how they will be managing their new website. This is something a little new for me, I have never taken this many people through something before. We spent most of this evening setting up machines for them to use.

Slightly more interesting, and possibly less frustrating, will be Thursday, Friday and Monday when I am taking a group of developer through the technical details of openharmonise. They are going to building a new system on top of our software. Again a little new, because these are probably the most technical people I will have done this with.

So a week of training and new experiences for me. I do enjoy doing this type of work, but I think I am going to be a little tapped out by the end of it.

Also have to find a new flat somewhere in there...

There's no place like home...

It's that time again, time to pack up and move on, time for a new home.

I hate moving, the fact that I have handed my notice in on this flat and am yet to find a new one, effectively having no fixed abode from the end of August. Last time I promised myself that it was the last till I could raise the cash to buy somewhere, but things do change and I am moving in with three beautiful women (two of them may be a little hairy but will always sit on my lap and purr).

So this time I am very exited, it's not just a simple move, its also taking a next step. I'll let you know how it goes, but posts may be a little slim on the ground till towards the end of August, after the move and the insanity that is work right now.

p.s. Fantastic Four = dull, but then you could probably have guess that. Luckily the day was saved when I found out that Van Wilder was on TV :)

Sad to be sick...

I've been trapped at home these last couple of days, unwell. Obviously I am hoping for vast amounts of sympathy from you all.

It is, as always, extremely bad timing. There is loads of work to be done, and worse than that I was due to be going on a nice jolly out to Warwick University for a couple of days for a conference on distributed repositories, but that's out the window now. So I'm stuck here, feeling the headaches returning for merely spending a little while looking at a monitor, which I guess is my queue to stop typing this and have another Lemsip...

London...

I have been a Londoner for five years now. I don't technically live in London, I say this before someone I know points it out, but I do live in the London Borough of Redbridge. I have always worked in the center or City of London and I travel in on the Central Line. What ever people say about where I live, I consider myself a Londoner, and I have never been as proud to be one as I was last Thursday.

My story isn't full of excitement, it's not a tale of close shaves and near misses, it is simply about a very strange day in the office. My girlfriend and I arrive at Liverpool Street station at 9:10am, about fifteen minutes after the first bomb went off. We did notice that the doors to the underground were closed and there was a police cordon around them, but we just assumed it was another scare, or a fire or something. In fact we joked about it, suggesting that there must have been a tiger on the loose in there.

Had we known that a few hundred yards down that tunnel one of our colleagues had been concentrating on reading his book and missed his stop at Liverpool Street. That he had been in the carriage behind the one with the bomb in it and that at that moment he was helping people along the tracks, we might not have made light of the situation. But we didn't know, and wouldn't until we reached work.

Only six of us actually made it all the way into the office on Thursday. We spent the morning tracking down the other members of staff. The guy who was in the Liverpool Street/Aldgate crash got out with only a few cuts and bruises, in fact he was home before we left the office at half four. We had a team of people who were meant to be catching a train from Kings Cross, but they had not got there when it happened and so were fine. Another colleague got off of his train, finding he couldn't take the underground was directed to walk, which he did, straight to Tavistock Place where he was 50 feet from the number 30 when it went up. We heard from him, shaken but okay at just gone 10am.

By midday we had accounted for everyone who we worked with, I had heard from all friends and family. So we all just sat there, watching the web, streaming in News24, listening to BBC London on the radio. Our offices are between Liverpool Street and Old Street stations, it was best that we just stayed put. So we did. We had lots of really sweet phone calls and e-mails from partners, suppliers and clients who knew where we were and wanted to check we were al okay.

My story is not one from the center of what was going on, I didn't see a thing. Instead mine is a story of being stuck and watching events unfold around us. Watching as the London emergency plan, the plan that no one ever wanted to put into action but knew they would, unfolded before us. From my layman's perspective, it seemed to go perfectly. From later accounts the Royal London Hospital was cleared and sealed from the public within 20-30 minutes, by the end of the day they had triaged over 200 people. Within 30 minutes there were 200 ambulances in London. Doctors were flown out to the scenes in the air ambulance to provide onsite care. While it would never be a good thing to have to put emergency plans to the test, the emergency services did seem to do the most amazing job.

Not only the professionals, but also the general public. Apparently all was calm, and most of London was back up and running on Friday morning. As I said, I have never been so proud to be a Londoner. I have been here five years now, and in all that time people knew that this would happen one day, knew that we were a major target. I suspect that since September 11th 2001 and other events, each person has sub-consciously been building there own personal emergency plan, which went into action Thursday morning. I have never heard from so many friends and family on one day. It is not only what people did that is impressive, but the way that they went about it. I lost track of the number of people I spoke to on my mobile, who said something like "well as long as you are okay, I don't want to tie up the phone system any more". People kept their heads, while very few around them lost theirs.

I was going to post a rant about how the London Olympics will never work, and my sudden pride in being a Londoner will not change my feelings about it. But that can wait till next week. Thanks to everyone that phoned to check how we were.

Why does NASA fail in success?...

According to SpaceRef.com NASA are close to ruining any chance of sending the Deep Impact probe onto another comet to continue doing useful work. As always this comes down to supposed lack of funding. From the article on SpaceRef it appears that action needs to be taken within a month to steer the craft towards the new target. This alone will cost a few tens of thousands, but doing it would at least allow NASA the time to be able to fully debate whether or not to continue onto this new target. A failure to act quickly will completely remove this option.

For Deep Impact to continue onto the new target and complete a mission there will cost significantly more, and with NASA currently trying to juggle its budget to be able to cope with the Bush vision for Moon, Mars and Beyond it is understandable that they will need to think hard. But to fail to act now to at least keep that option open is terrible.

NASA have been on a good run for a while now. The machine exploration side of the organisation has been more than keeping their end up while the human exploration side comes to terms with life post-Columbia. In that period we have had the amazing Cassini-Huygens mission to Saturn and Titan, the plucky little Mars rovers that are still going and now Deep Impact. All have generated much public interest, also recently renewed thanks to the exploits of SpaceShipOne and the X-Prize. But consistently NASA seems to be trying to shoot itself in the foot.

Take for example the massive public outcry over the threat to cancel the Hubble repair mission and cut short the life of the greatest telescope ever built. Another example comes from the threat to stop support to the Voyager probes, just when they are passing through the very edges of our solar system, a feat that could not be re-created for decades even if NASA decided to go there tomorrow.

NASA do say that every extended mission takes money away from new or proposed missions, which is a good point. But really we should be questioning why it takes so much money to support these missions. It costs almost $4.2M to support the two Voyager probes, mostly this is to cover the expense of monitoring the signals from the craft using the Deep Space Network. This is a series of large (70m) dishes around the world which monitor all deep space missions. The systems of the DSN grab the signals, process them and route them around the world to the relevant scientists that need them. They do fantastic work, listening for the faintest of signals from the furthest reaches of the solar system.

Understandable though all of this is, each of these potential project cancellations has been announced and responded to in an individual fashion, instead of looking to more long term solutions to these problems.

To start off it appears as if NASA got so used to missions failing over the last couple of decades that they do not regularly plan for the potential of missions being a complete or better success. Why is it that the Deep Impact mission planners know exactly where they would go next, and that the first stage of that would have to happen within a month, yet NASA has not been able to decide ahead of time if it would be willing to fund this?

Shouldn't NASA require that all projects plan for such eventualities, provide cost estimates for extensions, and why is there not a way of grouping some of these costs together? Shouldn't there be a department in NASA which is tasked with caring for aging, but still useful missions?

NASA has lost a fantastic PR opportunity, should they have made the decision on the future of Deep Impact, on the proviso that it worked, then this could ridden the wave of popular press that this mission has had over the last few days. Now that boat has sailed and NASA's indecision is just making them look silly.

Should Deep Impact be allowed to continue on to its next target it would take three and a half years to get there. This would be just enough time for the world's comet studying scientists to analyse all of the data returned from the recent encounter, debate it and plan what aspects to look for in the next encounter. I really hope they get that opportunity.

Just got back from The Wedding Crashers world premier...

So I know that I am going to come off as a complete snob with this post, but I don't give a crap. So stick with it and please understand that I just want to share some experiences with you.

I love going to film premiers. I love walking down that red carpet, surrounded by people. It doesn't matter that they all ignore me, or that the press photographers would rather shoot themselves in the head than shoot a photo or yours truly. I still love it.

It all started back with the release of Attack of the Clones. The wife of one of my friends works for a charity and she organises a lot of the big charity premiers in London. Now don't think that I am taking money away from a charity, I am not. I pay full price for my tickets, it is simply that the lady in question gives me a route through which I can buy these. Attack of the Clones was my first one, and a rather large affair what with being the European premier. We arrived on the start of the red carpet, around the corner from the cinema. We took our time walking the carpet, thinking that this would be the best part of the whole experience. John Williams conducting the Royal Philharmonic live in the square really made it something to remember, but then the best part happened.

While walking slowly round the corner I heard someone scream out "Matt!". Now my first thought was to turn around and look for this Matt person. I was interested because there are so few famous Matts in the world. There wasn't anyone there. The scream came out again. Looking into the crowd I saw a girl that I used to work with, behind the barrier. She was there for a little star spotting, instead she'd seen me. Neither of us had known that the other was going to be there, so we were both quite shocked. We talked for a little while, the people around her wondering who the hell this person was and if I was famous.

That experience was amazing, it really made the whole thing special, but on we went. Just before we reached the doors I heard another scream, "Mr Large, can we have your autograph?". Now I was freaked out.

Turns out that two of my friends, who are teachers and weren't in school that day, had come into London early to get a good place for star spotting and waited for me. They actually made me write an autograph for them, cue even more confused bystanders. Inside the cinema they trooped most of the stars up onto the stage to introduce the film, including the man himself George Lucas.

I was well and truly hooked. Since then I've managed to go to the premiers or preview screenings of Lord of the Rings: The Two Towers (the only one so far to top Star Wars), School of Rock (great intro from Jack Black), Finding Nemo (amazing all digital presentation) and Star Wars: Revenge of the Sith (fun intro from Anthony Daniels).

Tonight was good, but not amazingly special as I went on my own, sad I know. But it was cool in its own way. Vince Vaughn and Owen Wilson were there, as were director David Dobson and co-stars Isla Fisher and Jane Seymore, although only Dobson got up to say anything. As for the film, well if you like films like that (Dodgeball, Road Trip, Old School etc) then you should love this. The whole cinema laughed out loud pretty much throughout, and even spontaneously applauded on occasion. I really enjoyed it so I recommend (again, only if you like similar films) that you go. Also, if you ever have the chance, the red carpet treatment is not to be missed.

So much fun, so little sleep...

Wow, it has been a fantastic weekend. Won lots of money off of my (hopefully not too pissed off) friends at poker, saw War of the Worlds (quite disappointing) and generally had a great time. However I am now here struggling to stay awake, but before I drop my head into that tempting bucket of extra strong coffee here's a little something from Kieron Gillen and Charity Larrison's interview over at Sequential Tart, enjoy;

ST: What advice would you give to a girl who wants to grow up to be in comics?

CL: Make Comics.

KG: Alternatively, have enormous breast implants, add a load of skin tight fabrics to your wardrobe, and hang around experimental gene-therapy labs.

Millstone to milestone...

Today was an important day. We got a project, that has been hanging around for far too long, signed off by the client. This was such good news that I had to send a text message to one of the developers while he is on holiday in Greece. While I am almost certain that he was not thinking about work, he did seem to appreciate knowing that it all went well.

There is a popular misconception that the end of development projects are full of insanely long hours, with the whole team living in the office and eating nothing but pizza. I suppose such stories originate from companies wishing to play on the excitement of it for a big launch. The truth is that while something close to that is occasionally true to make sure we hit a deadline, the end of a project is almost always a long, drawn out process that eats away at your time in ever smaller chunks.

At the start of a project things look good. So far there have been no major mistakes, you are on schedule, at least until the end of the first day. Starts are good. Even in the middle of things you could always catch up. You may in fact hit a deadline with time to spare, thinking that this is the end. But it isn't, because the hardest thing to do in this industry is to get clients to sign off on something.

Thinking about it from their point of view the reasons are simple. The moment they sign it off as complete is the moment that they have to start paying again for any changes. This would not be a problem for clients if they had given you a decent specification in the first place. Invariably they like to see a project as a "collaboration", or an "evolutionary" process. These are wonderful things, and truthfully they are the way almost all developers would wish to work, but at some point, someone has to sign on that dotted line that reads END.

People ask me why projects always overrun so much, and it is very hard to explain to someone who isn't there, and never has been. Mostly projects do not overrun in terms of the number of days worked, it is just that the total length of time from start to finish is much greater than expected.

I know that there will be people reading this who will know exactly the projects I have had in mind while writing this, I only ask that you not name names.

So, one project signed off, several more to go. Can't wait to start the next one...

And the world turns on...

The eagle eyed of you will have noticed that the little JavaBlogs logo that used to be just to the left of this post has gone. My blog is no longer aggregated on that site. I am going to miss having JavaBlogs as an alternative home because I really like what they do there, I like the whole idea of communities of blogs coming together and this is one of the best examples. But they have rules, sensible rules which they are fairly flexible about. One of the most important, and probably obvious, of these is that you should mostly talk about Java. You can, and people do, deviate from this theme quite a bit, but really your main theme should still be Java.

Even the less eagle eyed of you will have noticed that Java and general programming have been a major theme of this blog since it started, however this has to change. I enjoy writing this blog a lot, I enjoy pouring my stupid thoughts into it, but this has always been a little stunted by the thought that I should not be straying too far from the Java theme. Often I have come home and just wanted to rant or share some random things that are nothing at all to do with Java. This has been happening more and more recently so it is time for a change.

Of course I could have moved to another blogging system which would give me categorised posts, one of which could have been Java. But I like being on blogger, my friends are on blogger so here I stay. Maybe we will get categories one day, but I don't really mind.

When I started this blog I thought it was going to be all about getting my name "out there" into the Java community, well I was wrong or at least I wasn't thinking hard enough about why I wanted to do this. Really I just wanted to say stuff. Now I can truly say anything I want to...

...it could be anything...

...wait for it...

...it's going to be good...

... ... ...

Weather terrible, glad I am here...

Occasionally I am reminded about my place in the world, and why I choose to occupy it. I am sitting here watching The Zutons playing at Glastonbury on BBC 3, and I know that much as I love them I would not like to be there. I am definitely not a festival person. Last year my friends and I gave up on a camping trip to Wales after two nights of storms. I would never have been able to take the weather this morning at Glastonbury. But I am happy to enjoy it through the mediums of television, radio and the Internet.

Luckily I have had the perfect opportunity to do just that as I had to work from home today. The underground stopped dead this morning, well at least the line that I should have been using did. Of course I am most glad that I was not one of the unlucky people stuck in a tunnel in soaring temperatures. So back to the flat I came, fired up the laptop and here I've been for most of the day. I ventured out briefly to the local pub for lunch, first time in three years that I have done that. It was full of old men, who ignored each other until they left. At which point the goodbyes were so involved and heartfelt you would think that they had been debating life, the universe and everything for the last hour instead of sitting in silence reading papers and eating soggy ham sandwiches.

This has been a glimpse into the world of people who work from home, and it scares me. For so long I have really wanted to be in a position to do this some day, but now I realise that you need to gear up to it. You need your lifelines, so that you don't go a little nuts. Today my lifeline was MSN Messenger to people in the office, most of whom had not even noticed I wasn't there. You also need to structure your day and ensure that you leave the flat at some point. I didn't have any problem knuckling down and doing the work, it was making myself take breaks that was the problem. In the office I can go to the kitchen to get a coffee, and that can turn into a 15 minute break, what with putting a new pot on and talking to a colleague who is also waiting for something to boil, steam or ping. Lunch today was about 30 minutes, because I went out, ate and came back. Heading out with the team from the office is rarely less than a full hour.

I don't know many people who work from home, but I think I am going to seek them out and find out the true secrets. I still like the idea, but it's 6:30pm and I need to get out for bit because those pesky walls are sneaking in on me again...

Money is there to be spent, right?...

So I went and did it again, I blew a load of money on a new toy. I spent most of last week fighting a battle of wills with the Apple store on Regent Street, here in sunny (for a change) London. It kept on calling to me from clear across the city, and the things in there were all so pretty. When I did cave, I would like to point out, it was limited to a minimum specification iBook.

To be honest with you I had decided that I was going to make the move to Apple, with my next computer, some time ago. I haven't mentioned it till now, partly because I really wasn't planning on getting one so soon. However I also wanted to get that last article out of the way before you all found out that I have switched camps again.

Before I am flamed by Linux advocates everywhere, please understand that I am not turning my back on Linux at all, my other machines will still continue to run Suse. The reason that I moved to Apple is simple, and it is simplicity.

I am a geek and a nerd, everyone who has every met me will testify to that, including my family and my extremely understanding girlfriend. It would be easy to assume that I would be happy to monkey around with Linux, especially being a developer and one who spends most of his time working on an open source project. It is true, but not all the time. There are other things that I want to do. For those things, Linux on the desktop is not quite there. It is there for most standard desktop tasks, it is there for the general office tasks (except Project Management where integration with MS Project really is a must have). But it is not there for video editing or some of the other associated and slightly more esoteric tasks.

By moving to Apple some people will see that as me making a statement of some kind, and I suppose it may well be whether I want it to or not. But really it was all about the right tools for the job, and that job is not what I thought it was going to be when I went down the Linux root. Again I tell you that I am not turning my back on Linux, far from it, nor on open source. One think I am definitely not doing is turning back to Microsoft, they have lost me for a good long while, if not for ever, and for those that know me well, that is a big thing...

Wintel, Lintel now Mactel(?) ... are we in the Post OS era?...

Strangely I haven't yet heard anyone put forward an offering for this, I think Mactel sounds better than Apptel personally. Although we could go a bit all out for something different and use Intac...Perhaps not. Of course I am referring to the worst kept secret in the IT world, which Steve Jobs finally let out of the bag at the Apple Developers conference the other week, that from June 2007 every single computer sold by Apple would be built around Intel chips.

This is truly big news, for many people. Best of all it was delivered in Jobs' perfected nudge, nudge, wink, wink style of delivery. I cannot recommend enough that you watch the whole thing (in QuickTime of course) here. It is a masterpiece of unveiling, starting with the typical catch up on how well Apple has been doing over the last year, then moving into the wonderfulness of Mac OS X 10.4. Jobs focuses a lot on the new Dashboard features, not missing an opportunity to have a dig at Microsoft along the way by showing off the count down calendar, entering 2009 and labeling it Longhorn.

After about half of the presentation he moves to the big build up, talking about the major changes in Apple's history, moving from the 68000 to the PowerPC and then Os9 to OSX and then he simply puts up a slide saying "It's tru_e", the dropped 'e' matching perfectly to the Intel logo. The master stroke though is proving the doubters wrong about how good Mac OS X will be on Intel by revealing that the machine he had been demoing on for the last 40 minutes was in fact an Intel machine...he shoots, he scores and the crowd go wild...

So we come into the new world with three different *tel platforms, what does this mean? Well for a start it will, as many are saying, hurt Apple sales in the short term. This is the reason that they waited and presented a complete and finished solution to the world, and most importantly the developers. So much emphasis was place on the idea of the "Universal Binary" that Jobs was all but screaming "please keep buying the PowerPC machines!!!". The first machines will be on sale by June 2006 and the whole line (by which I presume he means XServe, Mini and laptop machines) will be Intel by June 2007. After that Apple will be in a great position. Not only will they benefit from the greater range of chips that Intel can give them, therefore more powerful laptops which are what people have been crying out for from Apple, but also, and perhaps more importantly, they are no longer tied into a single vendor.

A few people have commented that it is strange that Apple would go for a link up with Intel when many see AMD as having much more momentum. They trumped Intel to the 64bit revolution (Apple and PowerPCs of course having trumped them both), and AMD are seen as being more innovative. Intel must be extremely happy, although perhaps not so much when they are being forced, as part of the launch of this new venture, to describe Apple as the best PC manufacturer and potentially pissing off their many other clients. Anyway, I am digressing from my point, which is that while they have linked up with Intel for the launch there will be nothing stopping Apple from moving away to the AMD camp in the future, or even running multiple lines with different chips.

I find the situation we will be in extremely strange. After all this time and development we are now in a world where there is one single unified consumer computer architecture, and it is the one that we started out on so many years ago. The humble x86 architecture is still in their somewhere and now it will be powering Mac OS X as well as Windows and Linux and Unix. Apple have clearly stated that they will lock OS X so that it will only run on their machines, and not other Intel based machines (presumably using the Trusted Computing Platform architecture). I wonder if that will last. Less than a week after the announcement there was an Intel version of OS X floating around on the net. Sometime soon are we going to be in the position where it really doesn't matter which OS we use, it will become irrelevant. Your choice of OS will simply be personal choice, probably more of a fashion statement (Apple must like the idea of that right now) and incur no technical or functional penalty.

We have been heading this way for years now in the development world. As a Java developer I almost never have to think about the platform(s) that the software will be deployed onto. Now with Mono even Microsoft's own runtime is available on all platforms. Looking at the server market, one which Apple has been storming over the last couple of years with XServe, OS X Server and XSAN. All of the *tel operating systems can claim to be in the running for super computer status, this is in fact the one area where Microsoft is behind. The recent push towards Service Oriented Architectures sitting on top of Web Services and the XML revolution moves us further away from OS lock in. Strangely Microsoft is moving XML Web Services down into the core of Windows with the Indigo project.

Development and the server markets always lead from the front but the consumer market is becoming less and less OS dependent. Microsoft shocked the world when they announced that their holiest of holys, Office, would move towards standards based document formats. We are only a short time away from 100% interoperability between Office and other suites such as OpenOffice. Look also at the resurgence of Web based applications as promoted through Google's efforts with GMail and Google Maps, even more importantly look at the world of blogs which we inhabit, there are people whose whole careers are now dependent on their blog software which they can access from any OS anywhere in the world anytime.

I now have Windows, Linux and Mac based systems at home and it really doesn't bother me which I use for most of the work that I want to do, this is surely a sign of things to come, and I haven't even begun to talk about the impact of portable devices....we truly are approaching Post OS era...long may it reign.

It's been a 5 year stretch so far...

This week I am celebrating(?) five years with Simulacra, and it is causing some serious strolling down memory lane. I joined the company fresh out of university, so fresh that I still had to find out my results and had to specifically request time off during my three month probation to attend graduation.

Five years is a long time to be with your first company, especially in this industry. I remember my father being concerned about my choice of such a traditionally nomadic profession, understandable from his vocational teaching perspective. Yet here I am, wondering if five years really is a long time.

It is not like I've been doing the same job here all that time. I've progressed from being the new coffee boy to heading an amazing team of developers. I have had the chance to work and learn from some really talented people (special mention here for Mark and Mike, you know who you are). The projects, both internal and external have continuously been interesting and challenging. Those five years have flown by, yet I think I am quite a different person to the one who started here.

As part of my nostalgia I have been looking back over old documents from my final year of university. From the comedy acknowledgements of my final year project;

"I would like to thank the following:

Rob Kinmond and Euan Wilson, for putting the fear of God into me once a week, and because they said I'd fail if I didn't thank them.

Katherine Hall and John Large, for their hard work proof reading and the subtle way they asked whether English was, in fact, my first language.

Andrew Hodgetts, for being a shining example of how not to manage a final year project.

My family, for not worrying about me when I was screaming at a monitor at four o'clock in the morning.

The owners and staff of The Star Inn, Copmere, for turning a blind eye to my turning up for shifts on only two hours sleep.

This project was powered by junk food and a vast and regular intake of caffeine."

to my cringe worthy CV;

"I am a problem-solver with the flexibility and awareness to see inter-connections and routes to solutions."

That's going to haunt me for a while.

So with five years down I look to the future, I wonder where I might be five years from now. I am fast approaching 30, which is something scary when you see it written down like that. For some people it carries with it thoughts of pensions and houses, and I won't deny that I have had such thoughts. However foremost in my mind for some time now has been the concept of "career".

It is easy when asked about your career to simply state your job, but is that the truth of the matter. Some people I know who work full time do not consider that they have careers or that the career they foresee for themselves is nothing to do with their current occupation, both valid choices. Five years ago I was almost certain I was about to embark on a career in network administration, yet I dodged that bullet and became a devoted coder. Over those five years I have slowly, and consciously, moved into a more managerial role to where I am today. I have no problem with all the form filling and meetings if they are leading the company to better things. It is fantastic that I have the opportunity to talk to clients so much. One of the major benefits of being in a small company is the number of different hats you can wear should you want to. But as we established earlier, your job isn't really your career.

So we come to the crux of my current preoccupation, what really is my career and how should I be going about it? I don't have the answer, hell I don't really have all the questions yet, but with two years to 30 I guess I had better start working it all out. I'll let you know...

p.s. She-ra, that was one for you :)

Google's fusion initiative...

So Google have finally announced what we've all known for sometime, that they are attempting to pull together all their disparate "products" under one roof in what they are describing as the "fusion initiative". A fancy name for what is basically making Google into a portal.

I suppose that it is good they are taking their time over this, but so far I am not impressed with the results. My problem is partly caused by their tactic of placing all new features into overly long Betas. I now have a GMail account, which I monitor through the Windows notifier and the Firefox plugin, depending on where I am. I can now also monitor this through their new "fusion" homepage, but only if I remember to go there. It doesn't help that I cannot seem to reset the Google Homepage button in the Google IE Toolbar to point to anything except the Google.com homepage.

When setting up my personlised Google homepage it didn't pick up any of my settings from my personlised Google News page that I set up the other week, very annoying. Finally, if I try to load my personlised Google homepage from the google.co.uk domain I get re-routed to the google.com domain. All very patchy.

In each case Google has launched something that I find incredibly useful, but I have to remember how to access each of these things. Perform the special mouse gesture while standing on one leg....okay, not quite that bad, but you get the idea.

I think the "fusion initiative" is fantastic and long overdue, and completely predictable. But they have their work cut out for them to pull back together all the different initiatives launched to date into one cohesive whole. I wish them luck.

Space geekery...

I am a geek in many ways, I am a computer geek, a Sci-Fi geek, a film geek. I have been this way for many years, but longer than all of these I have been a space geek. The thought of everything that is out there, how much we don't know, how much we do and how we found it out always amazes me.

One of the best things about being a space geek is that it sits quite happily with my other geek obsessions. Space exploration is a very technical endeavor and the Internet has been a boon to us space geeks for getting all the latest information, even seeing space exploration as it happens.

People in my office know that if there is a big space event happening during working hours they will probably be able to stand behind me and watch it. We all saw SpaceShipOne's historic flights into space, all three of them. We sat around and watched as the Genesis probe crashed into the desert.

Such visual images of space exploration, whether good or bad, still captivate me, they always have. I remember the first time I saw Viking images from the surface of Mars, especially the ones where you could see Martian frost! I doubt many people could forget the pictures of Earth rise from the surface of the moon. These images still manage to give me butterflies.

The most recent one is perhaps the most geeky one to date. Few people that I have shown this too get quite why it makes me geek out, and it is hard to explain. But I figured that the people who read this blog would instinctively get it. Over the last few months Nasa seems to have been on a Mars hardware hunt. Spotting various Mars landers from orbit, they think they have spotted the doomed Mars Polar Lander.

Cool though those images are, it is another one that amazed me. It is the first clear(ish) picture of a man made probe in orbit of another planet. It is a picture of the Mars Odyssey probe taken from the Mars Global Surveyor. It's not a great image, but damn does it put a big smile on my face.

Enjoy....

I am seeing our code through new eyes...

...in fact they are the eyes of our new developer who started today. I had forgotten how educational it is to listen to someone completely new talk about your code. Someone who has no emotional attachment to it. Someone who has not put blood, sweat and tears into it (honestly each of those at least once over the last 5 years). Someone who obviously doesn't care as much as me....sorry getting carried away there.

But it is true that if you spend as long as we have creating something you do begin to get attached to it. I was in an all day meeting with clients last week, we took them out for lunch. On the walk to Franco's (wonderful pasta place near here) she apologised to me, saying that she knew she could be "picky and awkward". After a moments reflection on this I apologised back to her about being too defensive, remarking that the system she had been commenting on was something that I often though of as my "baby" for want of a better term.

Since starting here in mid 2000 I have seen the code grow from a mere 8 classes to the behemoth that it is now. I lived through the delivery pains of the "Mango" version. The joys of seeing the "Papaya" version in all its' refactored glory. The long nights coaxing the first Swing management console "Guava" out of the door and the horror of starting again from scratch for the "Tangerine" version.

It is often said that most software is no good till at least the third version. Depending on how you count we are there now, and looking back I am amazed and very happy at where we are with the software. I could sit back and bask in the shining light of Open Harmonise.....thank God we have a new developer to point out its' problems and where it could be better.

Lessons Learned #5: Cross-browser Javascript and CSS

Unlike previous "Lessons Learned" articles this one will describe my current project and list some of the little annoyances that I have had to solve. I mentioned in my last post that this project was to build a web based administration system for Open Harmonise that mirrored the Java Swing version as closely as possible. Obviously the Java Swing version was pretty dynamic in nature so this project was going to require a lot of Javascript and funky CSS which would have to work in both of our target browsers (Internet Explorer and Mozilla/Firefox).

I have to admit, that when starting out I really did not think the project would be successful. The Java Swing version of the client includes many custom components to allow the management of complex information. The layout includes Outlook style tabs, a highly customised table view and a tabbed form that is built on the fly from data definitions. These form elements are not simple either, for example a text field is made up of the following components;

• text of the field name.
• button to open a help text window.
• valid/invalid icon (updated in real time, also impacting a similar icon on the tab).
• the text entry field itself.
• a counter which counts down how many characters you have left based on the data definition. This counter turns amber when you get close to the maximum length and red when you go over it.

That is just one of the simple types of form field, we also support number (with or without floating point), date (with or without time) and boolean. Then there are the complex types including vocabulary lookups, relationships to other resources in the CMS and finally compound properties (nested fields like address). Each field in the form can have a min and max number of occurrences and therefore needs to have plus and minus buttons for adding and removing extra fields.

I have ranted on a bit, but you can see why I was a little worried about building this and supporting it in html's request response model. But this was my task so I forged on. Overall it has been nice to discover that building these types of web application is a lot easier now than in days gone by. The latest versions of IE and Firefox interoperate quite well, but there have been many points where I have had to stop and scratch my head for a while. The following are my experiences and solutions, if for any of these you know that I am just being stupid and that there are much easier ways of doing these then please leave a comment.

1) Dynamically adding dynamic content.

Problem;

There are several approaches to adding content to a html page, the easiest of these is to append to the "innerHTML" attribute of an element. In Internet Explorer this doesn't work for forms as any information people have already input to form elements will be lost. Instead you must construct your new elements as html DOM objects and use the "appendChild" method. While this works fine for simple things life becomes complex in Internet Explorer with some special (for want of another word) attributes. These attributes include;

• name on form elements
• value on form elements
• class
• name on iframes
• all Javascript event handlers such as onchange and onclick

The problem with these is that Internet Explorer treats them as plain DOM attributes and does not do the special processing that each requires. The worst offender for these is the Javascript event handlers, they are not parsed by the Javascript engine so they are not actually set as listeners to the events.

Solution;

For most of these it is a simple mater of using the relevant Javascript attribute for the object, such as doing;

element.name='NAME' ;

instead of;

element.setAttribute('name', 'NAME');

However things get a little more complex where the Javascript event handlers are concerned. For these you must again set the relevant Javascript attribute for the object, but you must set it to the method pointer rather than a string. E.g.

inputElement.onchange = elementChangeMethod;

instead of;

inputElement.setAttribute('onchange', 'elementChangeMethod(PARAM1, PARAM2)');

One thing that should be obvious from this is that when having to use the method pointer you cannot provide any parameters to the method. When mocking the interface up in static HTML I was mostly passing "this" into these methods, which is still accessible, but not as a parameter, from a method called using the method pointer. So the rule is to remember that if you are going have to insert a new version of some html that has Javascript event attributes on it, ensure that those methods only deal with the "this" context object and use no other parameters.

2) & in links and window.location

Problem;

Because this is a very dynamic html web application there are many points where I use Javascript to either load a page into a frame/main window or insert an IFRAME in the page. Something that foxed me for quite a while was the fact that "window.location.replace" was working in IE but not in Firefox (and that Firefox was not showing any errors), I would just get a blank page.

Solution;

The solution was embarrassingly simple, but not obvious at all. IE allows you to put "& amp" into the URL for "window.location.replace" and Firefox does not. Mostly this is hard to spot as Firefox doesn't complain, it just shows a blank page.

3) Submitting a form and using window.location

Problem;

I replicated the Java Swing toolbar as a series of buttons in a frame across the top of the page. These submit various forms in other frames to accomplish their tasks. These forms may be visible to the user, for example the "Save Resource" button submits the main visible data entry form. Some are not, the "New Resource" button submits a hidden form. To keep life simple, these forms submit to a page that performs the action and then I redirect the frame back to the previous page which is now in a new state because of the action. However the changes were not happening.

Solution;

After quite a while of fiddling it appeared that the "window.location.replace" was happening before the "form.submit" commands in the Javascript, even though they were in the other order in the source. The evil hacky solution was to put the "window.location.replace" into a delay of a second to allow the "form.submit" to happen, truly horrible but it does work.

4) Comparing the value of a html DOM attribute to a string

Problem;

In many places I am allowing the user to move values from a list into a "select" field. The field has the "multiple" attribute set to "true" so that it appears as a list instead of a drop down, but I want everything in it to be submitted. So I wrote a small Javascript function to find all the elements with a tag name of "SELECT" and a "multiple" attribute with a value of "true". This is a very simple DOM navigating script, so imagine my surprise when it found the elements with the right name but would never match on the attribute value.

Solution;

The html DOM method "getAttribute('ATT_NAME')" returns a "DOMString" object not a "String" object, therefore you cannot do a simple comparison. In the end I did the following;

var attrubuteValue = element.getAttribute('ATT_NAME');
var sValue = '' + attrubuteValue;
if(sValue == 'STRING_TO_COMPARE') {
...
}

I am guessing that I could have compared against the DOMString's "toString()" method, but I was a bit fecked off by the time I thought of that and so haven't gone back in to fix it. But you get the idea. Again, no warnings or errors to be seen.

5) Dynamically inserted IFrame not findable by name

Problem;

Inserting an "IFRAME" element using Javascript works fine, until you come to try and find it by its' name in the "frames[]" array. The name doesn't appear to be available.

Solution;

Another bit of a hack, I found it by comparing the location instead.

Conclusion

So those were the most annoying issues, and the ones that I can remember while writing this. Although these did cause me major headaches, again I must say that the process of building this site has not been anywhere near as bad as I thought it would be. There is a resurgence in the art of building web "applications" instead of websites. Most of this has been the interest surrounding Google's efforts in this area and the fantastic results that can be gained from using asynchronous calls with XMLHttpRequest objects. Of course I would have loved to build this site in that way, but it was a bit too much of a risk given the time and budget allowed. Still I did create the layout structure to allow us to begin replacing various parts in this way over time.

Finally, please to do let me know if I am wrong in anything I have said here. I really do want to find the best way of fixing these issues, and I must thank the Mozilla/Firefox teams for supporting all the non-standard IE ways of doing things so that I could build truly cross-browser Javascript.

So I'm back...at last...

Wow, it has been a long since I last posted. There are many reasons for this. I've been pulling overtime, along with the rest of the team, as we are still a man down. More interviews next week so here's hoping. Also I have a friend staying in my flat at the moment so there isn't much room. Can't really get to my computer late at night, so blogging has been out because of that.

So I guess you might be interested in what I've been doing since the 28th Feb. Well I managed to finish and deliver the project I was working on since early December last year. This was the project that was trying to produce a paper based system. It ended up managing the application of several controlled vocabularies (about 3,600 terms plus other user entered metadata) to almost 18,000 resources. It was truely dissappointing when this resulted in the export of just 5Mb of Zthes exports.

Since then I've been working on producing a web based version of our administration system. The brief was to make it as much like our Swing based version as possible. This scared the crap out of me at first but has eneded being a really nice project. I will be posting some thoughts on the current state of web based applications soon, along with some invaluable hints and tips on cross-browser Javascript/CSS/DHTML.

Finally I wanted to share this most amusing of geek in jokes that I came across the other day on the OASIS site. They are producing a specification of another part of the Web Services stack , this one dealing with asynchronous calls to long running services, i.e. something where time for response is an issue. It's called Asynchronous Service Access Protocol or ASAP for short!! :)

Well it made me laugh anyway.

Oh, before I go I should shout out to my best friend Brada8. He also hasn't heard from me much over the last couple of months. Sorry dude. Promise I'll make that chess move any day now....just considering my winning strategy.

Great XSLT article and update on stuff...

First off, if you ever have to write XSLT, or more importantly teach someone how to write good XSLT, then you should bookmark this article. I especially liked the part where part of his solution to the XSLT problem was to restructure the XML he was transforming. While not always an option it's something I've done many times and it really can help.

You have probably noticed that I've not been posting much of late. Lots going on with the new boss and ongoing recruitment. Over the last few days at work I've had much fun writing some useful code. Of course, as one of my collegues pointed out all of our code is useful, but my point is that over the last three months I've mostly been writing one off Java and XSLT transformation code, probably a couple of thousand lines of XSLT alone.

The last few days I have been re-writing our database creation and default seeding code. Previously we've been using SQLServer specific scripts but it was finally time to bite the bullet and make all of this database independant. Not as big a job as you might think. We have the DataStoreInterface api which should abstracts the SQL into objects. So all I had to do was some up with some XML grammer to represent the table definitions and then to describe the default data to go in them. The only slightly hard part is managing and resolving all the cross-references because of foreign keys. Well it's been a nice diversion from the other work.

Just how sacred is your data?...

The project I am working on at the moment involves putting a lot of information that our clients publish into our system for them to apply metadata to. All of this published information is structured hierarchically, although it isn't obvious from the printed versions what these hierarchies are. So we have been spending the last few weeks moving this information into an XML format that we can load into the CMS.

Some of this work has been done before, so that information was easily exported from another copy of our CMS, however that is only a small part. What we have been finding is that the only machine readable versions of much of this data is in held Quark files and they can only offer us Word document exports from this. There is no semantic markup and the WordML structure involves lots of tables which are used for layout purposes. In the end most of this information has had to be cut and pasted by hand.

I have to admit that I am a little shocked by all of this. The information in question is the life blood of this organisation, in fact it is pretty much the only reason that the department that we are dealing with exists. Having this data locked into a format that is strictly for layout publishing purposes seems absolutely crazy. From what I can see all editing is done directly to this format. I think that the reason this has happened is that our clients have always seen the end product, i.e. the printed versions, as sacred without ever thinking that somewhere in there is pure data which is actually what they should be concerned about.

The true extent of this problem came to light on a related project with the same client and data set. The printed versions have marginal notes which provide cross-references between parts of the information and we needed to know if there were any reciprocal links in there. Our client couldn't tell us this without checking through all the relevant parts of a printed copy!

It is easy for us developers to forget that a client's perspective on something can be very different to our own. We had assumed that there would be a way to get some of this information in an electronic format that we could at least begin to use and transform. I think this has been something of a learning curve for our client, and we are helping them to understand the implications. Of course the great outcome of this project is that they will have these pure data versions in XML. They are now seeing all the possible benefits of this perspective change from thinking that the Quark files are their only precious commodity to thinking about the underlying data as being more valuable.

Reminder:

We have a position open for a Java/XML/XSLT developer at our offices in London (£26-£30K + benefits). If you are interested there is more information available on our website.

Java/XML/XSLT developer position, London...

Sorry that I haven't posted in a little while, but there has been a lot going on. As you will no doubt have guessed from the title, we have a position open at Simulacra for a new developer. All the details are on our website, you will need to fill in an application form, which can be downloaded from that site, and send it in to us with your CV. The closing date is the 23rd Feb.

There are many things that you cannot truly convey with a job advert, and still more that cannot even be posted on the website. This job will not be a free ride for anyone, but I can say that if you get it you will learn stuff, constantly. You will get to work on interesting projects, alongside some very talented people. If you are someone that is willing to take responsibility for your work, and understands how that makes the job better for yourself, then this is definitely for you.

We are a small company and there are pros and cons to that depending on the type of person you are. I think the benefits far out weight the drawbacks. The social side of the company alone would make it worth it.

It is true that I often moan on this blog, but that is mostly because I am writing late at night and need to vent. You don't get the good side as much, this is also because the good side is often covered in NDAs and things, but to give you an idea of some of this...

1) Check out our previous work. Design Council, National Theatre Stagework, QCA National Curriculum Online.

2) Other past and present clients, the work for whom is not as publicly visible, include The British Museum, BBC, Department for Education and Skills, Jane's Information Group, Channel 4, Pearson, Countryside Agency, Museums, Libraries and Archives Council.

3) Check out our open source technology Open Harmonise. See the system you will help continue developing.

4) The quality of our partners, including Convera and Illumina.

I look forward to seeing your CVs and application forms arriving soon.

Hands up if you use Netbeans...no one?...

It would appear that the concentrated effort we put onto sales has paid off and we now have lots of wonderful work, both for old and new clients. This is, of course, wonderful news but also a case of "be careful what you wish for!". I think that it will be a couple more weeks yet before things settle back down into a routine, it propbably wont be untill then that the new projects have been scoped enough to have a complete plan. I like having a plan (no laughing from the people in my office), makes me much more relaxed.

Got the team out of the office for a little while last week to a J2SE 5 presentation at the Sun offices here in London. There wasn't anything in the talk that was a surprise, but I haven't had much of a chance to look in detail at the new language features so this was a welcome walk through.

The highlight of the whole thing though must have been during the introduction where the "Technical Evangelist" from Sun told us exactly what topics he would be covering. One of these topics was Netbeans and, being the interactive kind of a guy that he was, he asked for a quick show of hands from people who used Netbeans.

Thank God Sun's cleaners are good otherwise there would have been tumble weed rolling through that room. Not a single hand went up. His response "Well that'll be a tough sell then..."

The importance of you work environment...

When the new computer arrived this week I immediately got it up and running, of course. I installed Suse Linux, as that it now my OS of choice, I got all the important applications working and that was about all I had time for on that first night. Since then I've been quite busy with my real life, which I am determined to have. That has meant that today was the first day I could really begin to actually use the machine. What I had forgotten was just how important it is to have a work environment that you are happy with.

In preparation for the arrival of the new machine I had moved things around in my living room, and the workstation is in a much better position. I can sit and work and not feel as crowded as before. However, in moving things around and not really trying it out till today I had not realised just how little light I get in this corner. Need to fix that soon.

Your work environment goes much further than that though. I spent a lot of this morning getting my old machine set up next door simply so that I could get a lot of files off of there. Simple things like my collection of desktop ladies (very important to being able to work). I've spent a lot of time today ripping my CD collection. I dug out some old cables to hook this computer up to the Hi-Fi. Things like that also help.

Finally I started to look at my development environment. Previously when moving from one machine to another it has simply been a case of moving some files across. This time though I have gone from Windows to Linux, which means that I have a lot more new stuff to get used to and also it is not simply a matter of a one to one mapping. Simple things like all the little batch scripts that I used to rely on to make my life that bit easier. All of these now need to be recreated as shell scripts.

I am fairly religious about backing up data, I still have experiment write ups from when I was in high school. But stupid me, I didn't back up the Tomcat configurations from my old Windows machine where I was previously developing. Usually not a problem, but I had a very custom configuration in Axis to support the Quintanona code that I was developing. It's been so long since I touched that project that I cannot remember these settings, so I am having to go through my code and reverse engineer them. It won't take me too long, but it is just one more little annoyance.

After a day of all of this I am close to being able to finally do some development, although I still have some work to do on a set of Ant scripts that I used to use. Next week I should be back to development, I cannot wait...

Data interoperability, if you think you have problems...

A large part our work at Simualcra involves trying to establish data interoperability standards across organisations, often involving both the public and private sectors. Most of our work over the last five years has been in the UK education environment, attempting to get the various parties involved in ICT in education working together better. We have helped to create various metadata standards in the UK (both completely new and also creating profiles of international standards), we have helped to develop standard vocabularies and taxonomies for describing education learning materials and we have built one of the core educational websites.

It is easy for us to look at our work and admire how much we have achieved, and it is a lot. But most of the problems we have faced have been of a political nature, the problems of persuading competing companies and different government organisations to talk to each other. The technical challenges have not been so great, not tiny but also not massive. At least they are not massive when you compare them against the work of others in different fields.

If you have read my last post you will know that I am very interested in all things space and astronomy related. Being such a space nut, and a developer involved a lot in data interoperability standards and the practical technology to make this happen, you can understand my interest in the Virtual Observatory concept, well you will once I explain it.

Astronomy used be just people around the world using telescopes and writing to each other about what they have seen, with perhaps a sketch or two. As time went on information was shared more freely in journals and at regular international meetings. Then the use of photography allowed the recording of these observations and these photos could be reprinted. Finally we move into the digital age and CCD images can be shared around the world via the Internet almost as soon as they have been taken.

As well as the more traditional ground based optical telescopes we now have space based observatories (such as Hubble) and people are observing in wavelengths across the complete spectrum. This creates a hell of a lot of data.

While all observations of the heavens have a specific purpose it is reasonable to expect that the images and data produced can be used in different future research. A clear example of this is verifying the orbit of a newly discovered asteroid such as the recent 2004-MN4 which was initially thought to have a chance of hitting earth in 2029. Its orbit was correct when an image of it was found on an old observation which was not intended for asteroid hunting purposes.

This reuse of observation data and also the fact that there is so much of it streaming in constantly from all over the world, and from orbit, led people to suggest the Virtual Observatory concept. This is quite simple really, the idea is that standards for search observatory archives and exchanging data are created. Along with the use of Grid Computing techniques, this could lead to much more automation in the reuse of this data. For example it would be relatively easy to produce image recognition programs to search the worlds observation archives for asteroids or comets (i.e. things in the image that shouldn't be there). This of course is only the tip of the iceberg of possible usages.

While I said the concept is quite simple, the implementation of it isn't. The effort is being led through the International Virtual Observatory Alliance (IVOA), who are bring people from around the world together to help create the required standards. These are complex metadata schemes and interoperability standards that are being produced, you only have to look at a small subset of these to see that. Try the Space-Time Coordinate Metadata standard for example.

Beyond this there are standards or proposals for;

• Observation information (location, telescope information etc)
• Tabular Data (the VOTable spec allows interoperability of tabular data)
• Datatype (for columns in the above tabular spec)
• Resource descriptions
• Archive search queries
• Image retrieval from archives
• Use Cases for all of this
• Global VO architecture

The list goes on and on and none of these are exactly simple tasks.

It isn't healthy to revel in the problems of others, and I don't believe that that is what I am doing, as I really hope that they succeed. However it is really nice to know that there are others in your field with tasks of scales that scare the crap out of you.

Sometimes the grass is greener on your side of the fence...

Off Topic: A Rant on space exploration's lack of reuse...

Those of you that know me will already appreciate that I am somewhat of a space nut. I've been fascinated by it for as long as I can remember. One of my greatest Christmas presents was from my Uncle, it was a copy of "The Illustrated Encyclopedia Of Space Technology (2nd edition)" by Kenneth Gatland. This book amazed me, I memorised almost everything in it, from the complete set of manned launches through to all the details about each of the world's launch sites. I eagerly awaited the, then named, US Space Station Alpha. I looked forward to man's return to the moon. The pictures of other planets of our solar system provided by the Pioneer, Voyager and Viking craft transfixed me. I was 12.

As I've grown up I've had to live through an almost complete lack of solar system exploration, the fact that human exploration has been limited to within low earth orbit. The slow recovery of the Shuttle program after the Challenger disaster. Finally in the late nineties and early 21st century space exploration began to take off again.

Space Station Alpha finally began to take shape, renamed the International Space Station due to the co-operation between US, Russian, European, Japanese, Italian and Canadian space agencies. NASA's Mars program took off in ernest, and despite some spectacular failures has returned amazing results. Our outer solar system understanding has leapt up thanks to the Galileo probe's exploration of Jupiter and its moons, and the Cassini/Huygens mission to Saturn and its moons.

The Columbia disaster has set back both the space station program and human space exploration as a whole. The Russian space agency has managed to keep ISS supplied with materials and people and there is the hope that the remaining Shuttle fleet (Discovery, Atlantis and Endeavour) will return to service this year. There have been other surprises such as the Chinese becoming only the third nation to self-launch humans into space, and the move towards commercial space capabilities with the amazing X-Prize success.

I have written the above to show not only how much of a complete space nut I am but to give you a context about where we are in space exploration. I think we are entering what could be an amazing period. President Bush put forward his vision of where NASA should be heading, with a focus on getting humans back to the Moon and then on to Mars. The European Space Agency has a fantastic schedule of robotic planetary missions for the next decade, and NASA will continue its exploration of Mars and beyond. I believe that the public are genuinely entranced by some of the pictures being returned from craft like the Mars rovers and the Huygens probe, and do understand the importance of these programs.

Both these programs are examples of truly great space exploration craft. The Mars rovers are now celebrating their first complete year on Mars. These were craft that were intended to only last for 90 days. When close to the end of that original mission deadline Steve Squyres (principal investigator for the Mars Rovers) said that they would use any extra lifetime they could get out of the Spirit rover to head towards some hills in the distance, and that although they would never get there they might find something interesting along the way. The Spirit rover spent Christmas exploring midway up those hills and is now getting some amazing views from near the top.

The Huygens probe was designed to drop through the atmosphere of Titan (one of Saturn's moons) take photos of the descent and hopefully survive an impact on the surface and return some photos from there, with an expected, top end, life span on the surface of 30mins before battery failure. Reports from ground based radio telescopes have so far pegged the surface life span of the probe at over one hour, and we may still find out that it lasted longer.

These projects have not only more than fulfilled their science goals but they have told us much about the application of technology to space exploration. NASA managed to debug the Spirit rover from a million miles away and get it working again, and then use the knowledge gained to prevent something similar from happening to its twin on the other side of Mars.

ESA found out that their use of redundant communications channels prevented a complete loss of data to a human error, a missing command on the Cassini probe telling it to listen on one of those channels.

Due to the massive extension in the lifespan of the Mars Rovers JPL have had to distribute the management of the probe, with the scientists on the project all returning to their home institutions and working by remote.

During the rest of this decade NASA will be sending one more lander to Mars and a new generation of Rover (in 2009). This rover will be about the size of an SUV. The ESA will be sending a rover of its own. In the decade after this, NASA intends to launch (about 2014) the Jupiter Icy Moons Orbiter (JIMO) probe. This probe will use a new design of nuclear propulsion (from the Prometheus project) and will orbit four of Jupiter's moons in succession.

What all of this screams out to me is why is the space exploration community always risking expensive projects with untested hardware/software. I understand the need to do this a lot of the time, but not exclusively. The next Mars rover will not take off till 2009, there are two more launch windows to Mars before then. Okay so they would not be able to do anything now for the August 2005 window. But in 2007 why not send 2 (or possible more) MER class rovers to Mars.

Launching probes is an expensive business, but that would be the main cost in this mission. NASA would have no need to design any hardware, thanless they really wanted to send something new, but even then you are probably changing little of the overall design. There are other improvements to be made, perhaps remove the direct to earth communications gear, as most of the MER program communications now go through one of the Mars Orbiters. This would reduce the weight of the probe and therefore the launch costs.

By the 2007 launch window the burgeoning commercial launch industry may well have managed to bring down these costs to a more manageable level. NASA could also gain sponsorship for these missions, easier when they are based on proven hardware. The complete mission could be run out of Universities around the world, which would also aid in NASA's out reach program.

You must ask yourselves why ESA are considering sending their own probe design to Mars. People suggest that they wish to gain experience in managing such missions, which is a good thing if you plan to participate more fully in man's exploration of the solar system. But why take the risk on new hardware, why doesn't NASA give them the designs to the MER rovers, take the hardware risk out of the equation.

Looking at the success of the Huygens mission to Titan, and the massive JIMO probe that NASA plans to send on a jaunt around the Jupiter system, why not strap 4 or more Huygen's class probes to the bottom of it. Let us get a look at the surface of these moons as well.

The technology of space exploration must move forward, this is true, but when you have technology that you know works, and has worked beyond all expectations, I see no good reason not to send it out again. The Mars rover team did an amazing job of hitting the middle of the expected landing site, I think they would relish the challenge of attempting something a little harder, perhaps trying to land in the caldera of Olympus Mons! That would be a hard and risky mission, but the possible results if successful would be massive, so it would be a perfect mission for sending reused hardware. Minimise the outlay to match the risk.

My desires for all of this to happen are totally selfish, because when I look at the images returned from Mars, Jupiter, Saturn and Titan I become that 12 year old again, I am absorbed and transfixed. I want to see what the surface of Europa looks like, I want to know all about Olympus Mons and how the largest volcano in the solar system came to be that way. The technology exists and the engineer in me says reuse it.

More project management tales...

This time not from me. I found this link on JavaBlogs (so applogies to anyone who reads this from there).

http://jroller.com/page/rsaunders/20040119#the_insane_project_manager_from

More updates to come tonight, been very busy over this last week. Will fill you all in later...

New computer at last...

On the third attempt I finally got my new computer. Suse installed like a dream, although took a couple of goes to get the official NVidia driver to install. So much more nice to work on a system with a decent speed. Just about to add in the extra RAM to bring it up to a nice 1Gb.

Of course I have my priorities right, so far I've installed Eclipse, Tomcat, JDK and a couple of Firefox extensions that I wouldn't live without. All is good. At last I will be able to get back to coding, once I get the old machine running next door and can get my source code off of there.

Lots to do.

Day estimates and the length of strings...

If one more person comes to me and asks for an estimate on how long it will take to do some work, without being able to tel me anything about the work or allowing me to specify it in any way, then I might just have to kill them.

I wasn't going to post about this, but one of the people involved in this persuaded me that I should and the more I think about this, the more I feel the need to talk about it (or basically to vent).

This has been building for some time, more and more over the last few years I have been asked to provide day estimates for unspecified work. In a lot of cases this is somewhat understandable given the type of work that we do. We are often responding to open tenders for public organisations and therefore they expect a fixed price contract, with that price fixed well before any specification work has begun. But recently several people have started asking me for estimates on work that could be specified, if only the project/client was managed better. These are estimates for clients we have a very good relationship with, people with whom, I am almost certain, we could take the time to explain the benefits of proper specification.

I should give you some examples of just how bad things have gotten. A project I recently worked on was delayed, on the client's request, becauee they were having meetings and agreeing things with their clients and partners. We were working on a time and materials contract, which was still to be signed, therefore I wasn't allowed to do any work. I had placed a period of specification work, into the project plan, that had to happen before any practical development take place. As it was the client who was delaying things people at our end were not that concerned, but I was. We knew that our client had immovable delivery dates with their clients, which meant that in reality the length of the project was being squeezed from the front end.

I finally managed to persuade the project manager that, as we were 99% certain to have to do this work, and by the original deadline, that we should begin the specification work. We ended up with less than half the time for this work, which meant I had to deliver very generic wireframes of the GUI. I explained to the client that this was all I had time to do, yet they still requested that I redo them with some real data in them, which pushed us further back.

The lack of proper specification time has added about 20% to the project, the delivery date for our end of which we have had to put back once already. The client has asked for an extension project, and wanted a day estimate based on a few lines of description. The project manager asked me for this and I spent about half an hour explaining to him why there was no way I could give him anything close to accurate figures, and that if he wanted them then I would have to build in such a margin of error that we probably wouldn't get the work. After all of this, after he agreed to go back to the client and explain the situation he said the following;

Project Manager: "So how long would it take to train people to use this?"

...then you could see the bulb over his head lighting up...

Project Manager: "That's a stupid question isn't it, how long will it take to train people on an application for which we have no specification and not a clue what it will look like. I can't ask that can I?"

Me: "No..."

If you think that's bad...

Oh it gets worse, yesterday was the proverbial straw. One of our analysts came to me for a day estimate, with absolutly no warning, no meetings, no meetings booked, nothing. This is how the conversation went;

Analyst: "How long would it take to translate some data?"

Me: "What data?"

Analyst: "An Excel spreadsheet."

Me: "What kind of spreadsheet? What format? Plain sheets or Workbooks?"

Analyst: "I don't know."

Me: "Well how much data?"

Analyst: "I don't know."

Me: "So what format has this got be be translated into?"

Analyst: "XML."

Me: "What specific format? Standards based or do we need to make up a grammer for it?"

Analyst: "Needs to be compatible with [some CMS thing]."

Me: "What does such a format look like?"

Analyst: "I don't know."

Me: "Can you find out?"

Analyst: "Not in time, I need the estimate now."

Me: "Well I just cannot tell you, there is no way that I can estimate that."

Analyst: "I fully understand..." brief pause "...so 10 days then."

I swear I almost ripped his throat out. This isn't a blind tender we are responding to, this company, while technically a client, is actually a strategic partner who we have been working with on this project for months. How on earth can we be in this situation.

The worst part is that these clients are shooting themselves in the foot, and so are we. The lack of specification raises the risk on projects massively, but this is not something that others recognise. There are a lot of people at work who would never ask this, who do understand the risks, but this small pocket of people that consistently ask for these things are driving me up the wall.

I had to write this, to vent my frustrations, but I am wandering if this is a more common problem in the induistry as a whole. Why is it that people will not listen to us developers, we have been doing this stuff for a long time. There is a lot of collective memory and knowledge about how to run these projects, and yet people still refuse to listen.

Let me know what you think, comments and e-mails always welcome.