When the new computer arrived this week I immediately got it up and running, of course. I installed Suse Linux, as that it now my OS of choice, I got all the important applications working and that was about all I had time for on that first night. Since then I've been quite busy with my real life, which I am determined to have. That has meant that today was the first day I could really begin to actually use the machine. What I had forgotten was just how important it is to have a work environment that you are happy with.
In preparation for the arrival of the new machine I had moved things around in my living room, and the workstation is in a much better position. I can sit and work and not feel as crowded as before. However, in moving things around and not really trying it out till today I had not realised just how little light I get in this corner. Need to fix that soon.
Your work environment goes much further than that though. I spent a lot of this morning getting my old machine set up next door simply so that I could get a lot of files off of there. Simple things like my collection of desktop ladies (very important to being able to work). I've spent a lot of time today ripping my CD collection. I dug out some old cables to hook this computer up to the Hi-Fi. Things like that also help.
Finally I started to look at my development environment. Previously when moving from one machine to another it has simply been a case of moving some files across. This time though I have gone from Windows to Linux, which means that I have a lot more new stuff to get used to and also it is not simply a matter of a one to one mapping. Simple things like all the little batch scripts that I used to rely on to make my life that bit easier. All of these now need to be recreated as shell scripts.
I am fairly religious about backing up data, I still have experiment write ups from when I was in high school. But stupid me, I didn't back up the Tomcat configurations from my old Windows machine where I was previously developing. Usually not a problem, but I had a very custom configuration in Axis to support the Quintanona code that I was developing. It's been so long since I touched that project that I cannot remember these settings, so I am having to go through my code and reverse engineer them. It won't take me too long, but it is just one more little annoyance.
After a day of all of this I am close to being able to finally do some development, although I still have some work to do on a set of Ant scripts that I used to use. Next week I should be back to development, I cannot wait...
Sunday, January 23, 2005
Saturday, January 22, 2005
Data interoperability, if you think you have problems...
A large part our work at Simualcra involves trying to establish data interoperability standards across organisations, often involving both the public and private sectors. Most of our work over the last five years has been in the UK education environment, attempting to get the various parties involved in ICT in education working together better. We have helped to create various metadata standards in the UK (both completely new and also creating profiles of international standards), we have helped to develop standard vocabularies and taxonomies for describing education learning materials and we have built one of the core educational websites.
It is easy for us to look at our work and admire how much we have achieved, and it is a lot. But most of the problems we have faced have been of a political nature, the problems of persuading competing companies and different government organisations to talk to each other. The technical challenges have not been so great, not tiny but also not massive. At least they are not massive when you compare them against the work of others in different fields.
If you have read my last post you will know that I am very interested in all things space and astronomy related. Being such a space nut, and a developer involved a lot in data interoperability standards and the practical technology to make this happen, you can understand my interest in the Virtual Observatory concept, well you will once I explain it.
Astronomy used be just people around the world using telescopes and writing to each other about what they have seen, with perhaps a sketch or two. As time went on information was shared more freely in journals and at regular international meetings. Then the use of photography allowed the recording of these observations and these photos could be reprinted. Finally we move into the digital age and CCD images can be shared around the world via the Internet almost as soon as they have been taken.
As well as the more traditional ground based optical telescopes we now have space based observatories (such as Hubble) and people are observing in wavelengths across the complete spectrum. This creates a hell of a lot of data.
While all observations of the heavens have a specific purpose it is reasonable to expect that the images and data produced can be used in different future research. A clear example of this is verifying the orbit of a newly discovered asteroid such as the recent 2004-MN4 which was initially thought to have a chance of hitting earth in 2029. Its orbit was correct when an image of it was found on an old observation which was not intended for asteroid hunting purposes.
This reuse of observation data and also the fact that there is so much of it streaming in constantly from all over the world, and from orbit, led people to suggest the Virtual Observatory concept. This is quite simple really, the idea is that standards for search observatory archives and exchanging data are created. Along with the use of Grid Computing techniques, this could lead to much more automation in the reuse of this data. For example it would be relatively easy to produce image recognition programs to search the worlds observation archives for asteroids or comets (i.e. things in the image that shouldn't be there). This of course is only the tip of the iceberg of possible usages.
While I said the concept is quite simple, the implementation of it isn't. The effort is being led through the International Virtual Observatory Alliance (IVOA), who are bring people from around the world together to help create the required standards. These are complex metadata schemes and interoperability standards that are being produced, you only have to look at a small subset of these to see that. Try the Space-Time Coordinate Metadata standard for example.
Beyond this there are standards or proposals for;
It isn't healthy to revel in the problems of others, and I don't believe that that is what I am doing, as I really hope that they succeed. However it is really nice to know that there are others in your field with tasks of scales that scare the crap out of you.
Sometimes the grass is greener on your side of the fence...
It is easy for us to look at our work and admire how much we have achieved, and it is a lot. But most of the problems we have faced have been of a political nature, the problems of persuading competing companies and different government organisations to talk to each other. The technical challenges have not been so great, not tiny but also not massive. At least they are not massive when you compare them against the work of others in different fields.
If you have read my last post you will know that I am very interested in all things space and astronomy related. Being such a space nut, and a developer involved a lot in data interoperability standards and the practical technology to make this happen, you can understand my interest in the Virtual Observatory concept, well you will once I explain it.
Astronomy used be just people around the world using telescopes and writing to each other about what they have seen, with perhaps a sketch or two. As time went on information was shared more freely in journals and at regular international meetings. Then the use of photography allowed the recording of these observations and these photos could be reprinted. Finally we move into the digital age and CCD images can be shared around the world via the Internet almost as soon as they have been taken.
As well as the more traditional ground based optical telescopes we now have space based observatories (such as Hubble) and people are observing in wavelengths across the complete spectrum. This creates a hell of a lot of data.
While all observations of the heavens have a specific purpose it is reasonable to expect that the images and data produced can be used in different future research. A clear example of this is verifying the orbit of a newly discovered asteroid such as the recent 2004-MN4 which was initially thought to have a chance of hitting earth in 2029. Its orbit was correct when an image of it was found on an old observation which was not intended for asteroid hunting purposes.
This reuse of observation data and also the fact that there is so much of it streaming in constantly from all over the world, and from orbit, led people to suggest the Virtual Observatory concept. This is quite simple really, the idea is that standards for search observatory archives and exchanging data are created. Along with the use of Grid Computing techniques, this could lead to much more automation in the reuse of this data. For example it would be relatively easy to produce image recognition programs to search the worlds observation archives for asteroids or comets (i.e. things in the image that shouldn't be there). This of course is only the tip of the iceberg of possible usages.
While I said the concept is quite simple, the implementation of it isn't. The effort is being led through the International Virtual Observatory Alliance (IVOA), who are bring people from around the world together to help create the required standards. These are complex metadata schemes and interoperability standards that are being produced, you only have to look at a small subset of these to see that. Try the Space-Time Coordinate Metadata standard for example.
Beyond this there are standards or proposals for;
- Observation information (location, telescope information etc)
- Tabular Data (the VOTable spec allows interoperability of tabular data)
- Datatype (for columns in the above tabular spec)
- Resource descriptions
- Archive search queries
- Image retrieval from archives
- Use Cases for all of this
- Global VO architecture
It isn't healthy to revel in the problems of others, and I don't believe that that is what I am doing, as I really hope that they succeed. However it is really nice to know that there are others in your field with tasks of scales that scare the crap out of you.
Sometimes the grass is greener on your side of the fence...
Off Topic: A Rant on space exploration's lack of reuse...
Those of you that know me will already appreciate that I am somewhat of a space nut. I've been fascinated by it for as long as I can remember. One of my greatest Christmas presents was from my Uncle, it was a copy of "The Illustrated Encyclopedia Of Space Technology (2nd edition)" by Kenneth Gatland. This book amazed me, I memorised almost everything in it, from the complete set of manned launches through to all the details about each of the world's launch sites. I eagerly awaited the, then named, US Space Station Alpha. I looked forward to man's return to the moon. The pictures of other planets of our solar system provided by the Pioneer, Voyager and Viking craft transfixed me. I was 12.
As I've grown up I've had to live through an almost complete lack of solar system exploration, the fact that human exploration has been limited to within low earth orbit. The slow recovery of the Shuttle program after the Challenger disaster. Finally in the late nineties and early 21st century space exploration began to take off again.
Space Station Alpha finally began to take shape, renamed the International Space Station due to the co-operation between US, Russian, European, Japanese, Italian and Canadian space agencies. NASA's Mars program took off in ernest, and despite some spectacular failures has returned amazing results. Our outer solar system understanding has leapt up thanks to the Galileo probe's exploration of Jupiter and its moons, and the Cassini/Huygens mission to Saturn and its moons.
The Columbia disaster has set back both the space station program and human space exploration as a whole. The Russian space agency has managed to keep ISS supplied with materials and people and there is the hope that the remaining Shuttle fleet (Discovery, Atlantis and Endeavour) will return to service this year. There have been other surprises such as the Chinese becoming only the third nation to self-launch humans into space, and the move towards commercial space capabilities with the amazing X-Prize success.
I have written the above to show not only how much of a complete space nut I am but to give you a context about where we are in space exploration. I think we are entering what could be an amazing period. President Bush put forward his vision of where NASA should be heading, with a focus on getting humans back to the Moon and then on to Mars. The European Space Agency has a fantastic schedule of robotic planetary missions for the next decade, and NASA will continue its exploration of Mars and beyond. I believe that the public are genuinely entranced by some of the pictures being returned from craft like the Mars rovers and the Huygens probe, and do understand the importance of these programs.
Both these programs are examples of truly great space exploration craft. The Mars rovers are now celebrating their first complete year on Mars. These were craft that were intended to only last for 90 days. When close to the end of that original mission deadline Steve Squyres (principal investigator for the Mars Rovers) said that they would use any extra lifetime they could get out of the Spirit rover to head towards some hills in the distance, and that although they would never get there they might find something interesting along the way. The Spirit rover spent Christmas exploring midway up those hills and is now getting some amazing views from near the top.
The Huygens probe was designed to drop through the atmosphere of Titan (one of Saturn's moons) take photos of the descent and hopefully survive an impact on the surface and return some photos from there, with an expected, top end, life span on the surface of 30mins before battery failure. Reports from ground based radio telescopes have so far pegged the surface life span of the probe at over one hour, and we may still find out that it lasted longer.
These projects have not only more than fulfilled their science goals but they have told us much about the application of technology to space exploration. NASA managed to debug the Spirit rover from a million miles away and get it working again, and then use the knowledge gained to prevent something similar from happening to its twin on the other side of Mars.
ESA found out that their use of redundant communications channels prevented a complete loss of data to a human error, a missing command on the Cassini probe telling it to listen on one of those channels.
Due to the massive extension in the lifespan of the Mars Rovers JPL have had to distribute the management of the probe, with the scientists on the project all returning to their home institutions and working by remote.
During the rest of this decade NASA will be sending one more lander to Mars and a new generation of Rover (in 2009). This rover will be about the size of an SUV. The ESA will be sending a rover of its own. In the decade after this, NASA intends to launch (about 2014) the Jupiter Icy Moons Orbiter (JIMO) probe. This probe will use a new design of nuclear propulsion (from the Prometheus project) and will orbit four of Jupiter's moons in succession.
What all of this screams out to me is why is the space exploration community always risking expensive projects with untested hardware/software. I understand the need to do this a lot of the time, but not exclusively. The next Mars rover will not take off till 2009, there are two more launch windows to Mars before then. Okay so they would not be able to do anything now for the August 2005 window. But in 2007 why not send 2 (or possible more) MER class rovers to Mars.
Launching probes is an expensive business, but that would be the main cost in this mission. NASA would have no need to design any hardware, thanless they really wanted to send something new, but even then you are probably changing little of the overall design. There are other improvements to be made, perhaps remove the direct to earth communications gear, as most of the MER program communications now go through one of the Mars Orbiters. This would reduce the weight of the probe and therefore the launch costs.
By the 2007 launch window the burgeoning commercial launch industry may well have managed to bring down these costs to a more manageable level. NASA could also gain sponsorship for these missions, easier when they are based on proven hardware. The complete mission could be run out of Universities around the world, which would also aid in NASA's out reach program.
You must ask yourselves why ESA are considering sending their own probe design to Mars. People suggest that they wish to gain experience in managing such missions, which is a good thing if you plan to participate more fully in man's exploration of the solar system. But why take the risk on new hardware, why doesn't NASA give them the designs to the MER rovers, take the hardware risk out of the equation.
Looking at the success of the Huygens mission to Titan, and the massive JIMO probe that NASA plans to send on a jaunt around the Jupiter system, why not strap 4 or more Huygen's class probes to the bottom of it. Let us get a look at the surface of these moons as well.
The technology of space exploration must move forward, this is true, but when you have technology that you know works, and has worked beyond all expectations, I see no good reason not to send it out again. The Mars rover team did an amazing job of hitting the middle of the expected landing site, I think they would relish the challenge of attempting something a little harder, perhaps trying to land in the caldera of Olympus Mons! That would be a hard and risky mission, but the possible results if successful would be massive, so it would be a perfect mission for sending reused hardware. Minimise the outlay to match the risk.
My desires for all of this to happen are totally selfish, because when I look at the images returned from Mars, Jupiter, Saturn and Titan I become that 12 year old again, I am absorbed and transfixed. I want to see what the surface of Europa looks like, I want to know all about Olympus Mons and how the largest volcano in the solar system came to be that way. The technology exists and the engineer in me says reuse it.
As I've grown up I've had to live through an almost complete lack of solar system exploration, the fact that human exploration has been limited to within low earth orbit. The slow recovery of the Shuttle program after the Challenger disaster. Finally in the late nineties and early 21st century space exploration began to take off again.
Space Station Alpha finally began to take shape, renamed the International Space Station due to the co-operation between US, Russian, European, Japanese, Italian and Canadian space agencies. NASA's Mars program took off in ernest, and despite some spectacular failures has returned amazing results. Our outer solar system understanding has leapt up thanks to the Galileo probe's exploration of Jupiter and its moons, and the Cassini/Huygens mission to Saturn and its moons.
The Columbia disaster has set back both the space station program and human space exploration as a whole. The Russian space agency has managed to keep ISS supplied with materials and people and there is the hope that the remaining Shuttle fleet (Discovery, Atlantis and Endeavour) will return to service this year. There have been other surprises such as the Chinese becoming only the third nation to self-launch humans into space, and the move towards commercial space capabilities with the amazing X-Prize success.
I have written the above to show not only how much of a complete space nut I am but to give you a context about where we are in space exploration. I think we are entering what could be an amazing period. President Bush put forward his vision of where NASA should be heading, with a focus on getting humans back to the Moon and then on to Mars. The European Space Agency has a fantastic schedule of robotic planetary missions for the next decade, and NASA will continue its exploration of Mars and beyond. I believe that the public are genuinely entranced by some of the pictures being returned from craft like the Mars rovers and the Huygens probe, and do understand the importance of these programs.
Both these programs are examples of truly great space exploration craft. The Mars rovers are now celebrating their first complete year on Mars. These were craft that were intended to only last for 90 days. When close to the end of that original mission deadline Steve Squyres (principal investigator for the Mars Rovers) said that they would use any extra lifetime they could get out of the Spirit rover to head towards some hills in the distance, and that although they would never get there they might find something interesting along the way. The Spirit rover spent Christmas exploring midway up those hills and is now getting some amazing views from near the top.
The Huygens probe was designed to drop through the atmosphere of Titan (one of Saturn's moons) take photos of the descent and hopefully survive an impact on the surface and return some photos from there, with an expected, top end, life span on the surface of 30mins before battery failure. Reports from ground based radio telescopes have so far pegged the surface life span of the probe at over one hour, and we may still find out that it lasted longer.
These projects have not only more than fulfilled their science goals but they have told us much about the application of technology to space exploration. NASA managed to debug the Spirit rover from a million miles away and get it working again, and then use the knowledge gained to prevent something similar from happening to its twin on the other side of Mars.
ESA found out that their use of redundant communications channels prevented a complete loss of data to a human error, a missing command on the Cassini probe telling it to listen on one of those channels.
Due to the massive extension in the lifespan of the Mars Rovers JPL have had to distribute the management of the probe, with the scientists on the project all returning to their home institutions and working by remote.
During the rest of this decade NASA will be sending one more lander to Mars and a new generation of Rover (in 2009). This rover will be about the size of an SUV. The ESA will be sending a rover of its own. In the decade after this, NASA intends to launch (about 2014) the Jupiter Icy Moons Orbiter (JIMO) probe. This probe will use a new design of nuclear propulsion (from the Prometheus project) and will orbit four of Jupiter's moons in succession.
What all of this screams out to me is why is the space exploration community always risking expensive projects with untested hardware/software. I understand the need to do this a lot of the time, but not exclusively. The next Mars rover will not take off till 2009, there are two more launch windows to Mars before then. Okay so they would not be able to do anything now for the August 2005 window. But in 2007 why not send 2 (or possible more) MER class rovers to Mars.
Launching probes is an expensive business, but that would be the main cost in this mission. NASA would have no need to design any hardware, thanless they really wanted to send something new, but even then you are probably changing little of the overall design. There are other improvements to be made, perhaps remove the direct to earth communications gear, as most of the MER program communications now go through one of the Mars Orbiters. This would reduce the weight of the probe and therefore the launch costs.
By the 2007 launch window the burgeoning commercial launch industry may well have managed to bring down these costs to a more manageable level. NASA could also gain sponsorship for these missions, easier when they are based on proven hardware. The complete mission could be run out of Universities around the world, which would also aid in NASA's out reach program.
You must ask yourselves why ESA are considering sending their own probe design to Mars. People suggest that they wish to gain experience in managing such missions, which is a good thing if you plan to participate more fully in man's exploration of the solar system. But why take the risk on new hardware, why doesn't NASA give them the designs to the MER rovers, take the hardware risk out of the equation.
Looking at the success of the Huygens mission to Titan, and the massive JIMO probe that NASA plans to send on a jaunt around the Jupiter system, why not strap 4 or more Huygen's class probes to the bottom of it. Let us get a look at the surface of these moons as well.
The technology of space exploration must move forward, this is true, but when you have technology that you know works, and has worked beyond all expectations, I see no good reason not to send it out again. The Mars rover team did an amazing job of hitting the middle of the expected landing site, I think they would relish the challenge of attempting something a little harder, perhaps trying to land in the caldera of Olympus Mons! That would be a hard and risky mission, but the possible results if successful would be massive, so it would be a perfect mission for sending reused hardware. Minimise the outlay to match the risk.
My desires for all of this to happen are totally selfish, because when I look at the images returned from Mars, Jupiter, Saturn and Titan I become that 12 year old again, I am absorbed and transfixed. I want to see what the surface of Europa looks like, I want to know all about Olympus Mons and how the largest volcano in the solar system came to be that way. The technology exists and the engineer in me says reuse it.
Friday, January 21, 2005
More project management tales...
This time not from me. I found this link on JavaBlogs (so applogies to anyone who reads this from there).
http://jroller.com/page/rsaunders/20040119#the_insane_project_manager_from
More updates to come tonight, been very busy over this last week. Will fill you all in later...
http://jroller.com/page/rsaunders/20040119#the_insane_project_manager_from
More updates to come tonight, been very busy over this last week. Will fill you all in later...
Wednesday, January 19, 2005
New computer at last...
On the third attempt I finally got my new computer. Suse installed like a dream, although took a couple of goes to get the official NVidia driver to install. So much more nice to work on a system with a decent speed. Just about to add in the extra RAM to bring it up to a nice 1Gb.
Of course I have my priorities right, so far I've installed Eclipse, Tomcat, JDK and a couple of Firefox extensions that I wouldn't live without. All is good. At last I will be able to get back to coding, once I get the old machine running next door and can get my source code off of there.
Lots to do.
Of course I have my priorities right, so far I've installed Eclipse, Tomcat, JDK and a couple of Firefox extensions that I wouldn't live without. All is good. At last I will be able to get back to coding, once I get the old machine running next door and can get my source code off of there.
Lots to do.
Saturday, January 08, 2005
Day estimates and the length of strings...
If one more person comes to me and asks for an estimate on how long it will take to do some work, without being able to tel me anything about the work or allowing me to specify it in any way, then I might just have to kill them.
I wasn't going to post about this, but one of the people involved in this persuaded me that I should and the more I think about this, the more I feel the need to talk about it (or basically to vent).
This has been building for some time, more and more over the last few years I have been asked to provide day estimates for unspecified work. In a lot of cases this is somewhat understandable given the type of work that we do. We are often responding to open tenders for public organisations and therefore they expect a fixed price contract, with that price fixed well before any specification work has begun. But recently several people have started asking me for estimates on work that could be specified, if only the project/client was managed better. These are estimates for clients we have a very good relationship with, people with whom, I am almost certain, we could take the time to explain the benefits of proper specification.
I should give you some examples of just how bad things have gotten. A project I recently worked on was delayed, on the client's request, becauee they were having meetings and agreeing things with their clients and partners. We were working on a time and materials contract, which was still to be signed, therefore I wasn't allowed to do any work. I had placed a period of specification work, into the project plan, that had to happen before any practical development take place. As it was the client who was delaying things people at our end were not that concerned, but I was. We knew that our client had immovable delivery dates with their clients, which meant that in reality the length of the project was being squeezed from the front end.
I finally managed to persuade the project manager that, as we were 99% certain to have to do this work, and by the original deadline, that we should begin the specification work. We ended up with less than half the time for this work, which meant I had to deliver very generic wireframes of the GUI. I explained to the client that this was all I had time to do, yet they still requested that I redo them with some real data in them, which pushed us further back.
The lack of proper specification time has added about 20% to the project, the delivery date for our end of which we have had to put back once already. The client has asked for an extension project, and wanted a day estimate based on a few lines of description. The project manager asked me for this and I spent about half an hour explaining to him why there was no way I could give him anything close to accurate figures, and that if he wanted them then I would have to build in such a margin of error that we probably wouldn't get the work. After all of this, after he agreed to go back to the client and explain the situation he said the following;
Project Manager: "So how long would it take to train people to use this?"
...then you could see the bulb over his head lighting up...
Project Manager: "That's a stupid question isn't it, how long will it take to train people on an application for which we have no specification and not a clue what it will look like. I can't ask that can I?"
Me: "No..."
If you think that's bad...
Oh it gets worse, yesterday was the proverbial straw. One of our analysts came to me for a day estimate, with absolutly no warning, no meetings, no meetings booked, nothing. This is how the conversation went;
Analyst: "How long would it take to translate some data?"
Me: "What data?"
Analyst: "An Excel spreadsheet."
Me: "What kind of spreadsheet? What format? Plain sheets or Workbooks?"
Analyst: "I don't know."
Me: "Well how much data?"
Analyst: "I don't know."
Me: "So what format has this got be be translated into?"
Analyst: "XML."
Me: "What specific format? Standards based or do we need to make up a grammer for it?"
Analyst: "Needs to be compatible with [some CMS thing]."
Me: "What does such a format look like?"
Analyst: "I don't know."
Me: "Can you find out?"
Analyst: "Not in time, I need the estimate now."
Me: "Well I just cannot tell you, there is no way that I can estimate that."
Analyst: "I fully understand..." brief pause "...so 10 days then."
I swear I almost ripped his throat out. This isn't a blind tender we are responding to, this company, while technically a client, is actually a strategic partner who we have been working with on this project for months. How on earth can we be in this situation.
The worst part is that these clients are shooting themselves in the foot, and so are we. The lack of specification raises the risk on projects massively, but this is not something that others recognise. There are a lot of people at work who would never ask this, who do understand the risks, but this small pocket of people that consistently ask for these things are driving me up the wall.
I had to write this, to vent my frustrations, but I am wandering if this is a more common problem in the induistry as a whole. Why is it that people will not listen to us developers, we have been doing this stuff for a long time. There is a lot of collective memory and knowledge about how to run these projects, and yet people still refuse to listen.
Let me know what you think, comments and e-mails always welcome.
I wasn't going to post about this, but one of the people involved in this persuaded me that I should and the more I think about this, the more I feel the need to talk about it (or basically to vent).
This has been building for some time, more and more over the last few years I have been asked to provide day estimates for unspecified work. In a lot of cases this is somewhat understandable given the type of work that we do. We are often responding to open tenders for public organisations and therefore they expect a fixed price contract, with that price fixed well before any specification work has begun. But recently several people have started asking me for estimates on work that could be specified, if only the project/client was managed better. These are estimates for clients we have a very good relationship with, people with whom, I am almost certain, we could take the time to explain the benefits of proper specification.
I should give you some examples of just how bad things have gotten. A project I recently worked on was delayed, on the client's request, becauee they were having meetings and agreeing things with their clients and partners. We were working on a time and materials contract, which was still to be signed, therefore I wasn't allowed to do any work. I had placed a period of specification work, into the project plan, that had to happen before any practical development take place. As it was the client who was delaying things people at our end were not that concerned, but I was. We knew that our client had immovable delivery dates with their clients, which meant that in reality the length of the project was being squeezed from the front end.
I finally managed to persuade the project manager that, as we were 99% certain to have to do this work, and by the original deadline, that we should begin the specification work. We ended up with less than half the time for this work, which meant I had to deliver very generic wireframes of the GUI. I explained to the client that this was all I had time to do, yet they still requested that I redo them with some real data in them, which pushed us further back.
The lack of proper specification time has added about 20% to the project, the delivery date for our end of which we have had to put back once already. The client has asked for an extension project, and wanted a day estimate based on a few lines of description. The project manager asked me for this and I spent about half an hour explaining to him why there was no way I could give him anything close to accurate figures, and that if he wanted them then I would have to build in such a margin of error that we probably wouldn't get the work. After all of this, after he agreed to go back to the client and explain the situation he said the following;
Project Manager: "So how long would it take to train people to use this?"
...then you could see the bulb over his head lighting up...
Project Manager: "That's a stupid question isn't it, how long will it take to train people on an application for which we have no specification and not a clue what it will look like. I can't ask that can I?"
Me: "No..."
If you think that's bad...
Oh it gets worse, yesterday was the proverbial straw. One of our analysts came to me for a day estimate, with absolutly no warning, no meetings, no meetings booked, nothing. This is how the conversation went;
Analyst: "How long would it take to translate some data?"
Me: "What data?"
Analyst: "An Excel spreadsheet."
Me: "What kind of spreadsheet? What format? Plain sheets or Workbooks?"
Analyst: "I don't know."
Me: "Well how much data?"
Analyst: "I don't know."
Me: "So what format has this got be be translated into?"
Analyst: "XML."
Me: "What specific format? Standards based or do we need to make up a grammer for it?"
Analyst: "Needs to be compatible with [some CMS thing]."
Me: "What does such a format look like?"
Analyst: "I don't know."
Me: "Can you find out?"
Analyst: "Not in time, I need the estimate now."
Me: "Well I just cannot tell you, there is no way that I can estimate that."
Analyst: "I fully understand..." brief pause "...so 10 days then."
I swear I almost ripped his throat out. This isn't a blind tender we are responding to, this company, while technically a client, is actually a strategic partner who we have been working with on this project for months. How on earth can we be in this situation.
The worst part is that these clients are shooting themselves in the foot, and so are we. The lack of specification raises the risk on projects massively, but this is not something that others recognise. There are a lot of people at work who would never ask this, who do understand the risks, but this small pocket of people that consistently ask for these things are driving me up the wall.
I had to write this, to vent my frustrations, but I am wandering if this is a more common problem in the induistry as a whole. Why is it that people will not listen to us developers, we have been doing this stuff for a long time. There is a lot of collective memory and knowledge about how to run these projects, and yet people still refuse to listen.
Let me know what you think, comments and e-mails always welcome.
Friday, January 07, 2005
I am an idiot, so much I need reminding...
I have set a new task for my collegues at work, specifically it was a task for one of them hoever he has been somewhat remiss in his duties therefore I am widening the call to arms. Someone is to remind me, I originally said only in the morning but I feel there is a requirement for a lunchtime spot as well, that I am an idiot. With the help of those around me I belive that I can remove the fruitless time that I spend each day working on a problem under the misapprehension that I cannot be wrong so it must be something else. Going into these problems with the knowledge that I am an idiot will encourage me to first look for the stupid thing that I am bound to have done.
This is going to form one of the precepts of my new guide to being a programmer which I will probably never actually write down other then here. Another is my innovative new approach to project management, using the technique "Disaster Points". My theory is that for any project you should plan at least one, although there could be many, "Disaster Point". This is, as the name suggests, a point at which you plan to have all the things that will go wrong, go wrong. Once you have your "Disaster Point" planned there is no way that it can sneak up on you and scare you. [Note: This is very similar to the theory of buying a bullet and scratching you name into it so you know exactly where the "bullet with your name on it" is...]
Now if only people in the office would remember to remind me that I am an idiot things will go much better, unlike the last few days where I have been an idiot several times. I may have "the '/' patter in XSLT is treated differently by different processors" branded into the back of my hand so that I don't ever forget it again, that was todays idiot moment (well 15 minutes of idiot moment)...
This is going to form one of the precepts of my new guide to being a programmer which I will probably never actually write down other then here. Another is my innovative new approach to project management, using the technique "Disaster Points". My theory is that for any project you should plan at least one, although there could be many, "Disaster Point". This is, as the name suggests, a point at which you plan to have all the things that will go wrong, go wrong. Once you have your "Disaster Point" planned there is no way that it can sneak up on you and scare you. [Note: This is very similar to the theory of buying a bullet and scratching you name into it so you know exactly where the "bullet with your name on it" is...]
Now if only people in the office would remember to remind me that I am an idiot things will go much better, unlike the last few days where I have been an idiot several times. I may have "the '/' patter in XSLT is treated differently by different processors" branded into the back of my hand so that I don't ever forget it again, that was todays idiot moment (well 15 minutes of idiot moment)...
Monday, January 03, 2005
The Blogger spell checker needs updating...
I find it strange that the Blogger spell checker does not recognise "Blogger", "Google" and "GMail", love them or hate them you have to admit that Microsoft get some details right and their spell checkers always recognise their various products/companies (although there are often omissions of competitive products/companies).
Even more strange that these words though is that fact that it will not recognise "blog", "weblog" and "blogging". These are really annoying, as these are words that I use a lot when writing posts.
It would also be wonderful if they provided a British English dictionary, as I really would like to spell things with fewer 'Z's...
Even more strange that these words though is that fact that it will not recognise "blog", "weblog" and "blogging". These are really annoying, as these are words that I use a lot when writing posts.
It would also be wonderful if they provided a British English dictionary, as I really would like to spell things with fewer 'Z's...
The next generation of Web UI is upon us...
Back at the start of November I told you all how, for my final year project at University I "Apparently built REST Web Services in 1999...". This was triggered by reading the report, that I produced for that project, for the first time since I handed it in. I was, if you recall, rather surprised that, without knowing it, I was at the vanguard of something that wouldn't really become popular until very recently. As a quick reminder let me just recap what that project was all about. I was aiming to prove that you could use a handful of very new technologies to produce an interactive, web enabled Project Management Tool. It loaded a JavaScript engine, into the web browser, which made REST calls to grab project data in XML and then render it on the client side in VML (Vector Markup Language, Microsoft's forunner to SVG).
I concluded the article by saying ;
"This trip down memory lane has been a strange experience for me, I am left with some questions. Looking back on this should I be proud of what I achieved in the context of the time? The title of this post says that I created REST Web Services in 1999, but I have to admit that I didn't really know what I was doing so perhaps it was random chance that gave me this opportunity."
The realisation that I did something that was fairly worthwhile so long ago, even though I didn't really realise its' potential, has stayed with me over the last couple of months. It really did give me a well needed boost in confidence. However I have been even more shocked by the recent heavy chatter about the next generation of Web User Interfaces as showcased in Google's GMail service.
When people talk about this they are really extolling the virtues of using the XMLHttpRequest object in the browser to do data loading into a dynamic html interface. This leads to a very responsive experience for the user. Google has used this technique again to produce the Google Suggest service which will give you a list of word/phrase completions as you type and also tell you how many results they would produce.
Joel Spolsky talks about this technique on his blog, http://www.joelonsoftware.com/items/2004/12/15.html, where he says that its use is "raising the bar" for Web User Interfaces.
If you haven't guessed yet, the reason that I am talking about this is that it is exactly the technique that I used back in 1999 to deliver the interface for my final year project. I did this for reasons of providing a responsive user interface and removing the interface generation load from a server (by having it only deliver the data).
Again, I am not suggesting that I invented anything, or that I had the wherewithal to realise what I did. What I am saying is that ideas have their time, and I am glad that in some small way I managed to be part of two big ideas, even though no one will ever know (thanless they read this blog of course).
Note: I have 4 GMail invites that I have no use for, so if anyone wants one, e-mail me at the link on the home page and I'll hook you up. This is of course on a first come, first served basis.
I concluded the article by saying ;
"This trip down memory lane has been a strange experience for me, I am left with some questions. Looking back on this should I be proud of what I achieved in the context of the time? The title of this post says that I created REST Web Services in 1999, but I have to admit that I didn't really know what I was doing so perhaps it was random chance that gave me this opportunity."
The realisation that I did something that was fairly worthwhile so long ago, even though I didn't really realise its' potential, has stayed with me over the last couple of months. It really did give me a well needed boost in confidence. However I have been even more shocked by the recent heavy chatter about the next generation of Web User Interfaces as showcased in Google's GMail service.
When people talk about this they are really extolling the virtues of using the XMLHttpRequest object in the browser to do data loading into a dynamic html interface. This leads to a very responsive experience for the user. Google has used this technique again to produce the Google Suggest service which will give you a list of word/phrase completions as you type and also tell you how many results they would produce.
Joel Spolsky talks about this technique on his blog, http://www.joelonsoftware.com/items/2004/12/15.html, where he says that its use is "raising the bar" for Web User Interfaces.
If you haven't guessed yet, the reason that I am talking about this is that it is exactly the technique that I used back in 1999 to deliver the interface for my final year project. I did this for reasons of providing a responsive user interface and removing the interface generation load from a server (by having it only deliver the data).
Again, I am not suggesting that I invented anything, or that I had the wherewithal to realise what I did. What I am saying is that ideas have their time, and I am glad that in some small way I managed to be part of two big ideas, even though no one will ever know (thanless they read this blog of course).
Note: I have 4 GMail invites that I have no use for, so if anyone wants one, e-mail me at the link on the home page and I'll hook you up. This is of course on a first come, first served basis.
Subscribe to:
Posts (Atom)