Thursday, September 30, 2004

Developer Support #1: IT Conversations...

My God, how can I not have heard about this site before. IT Conversations houses a vast archive of quality interviews with IT industry profressionals and audio recordings of major events, such as the Open Source Convention.

In the last few days, while trudging through some really boring work, I've listened to 8 of these interviews, and while it's true that some are more interesting that others, I've found something interesting in each one.

I highly recommend the site. I will be making sure that all our developers know about it.

IT Conversations

Wednesday, September 29, 2004

Warning about Axis and Saxon before v7.2

This was a weird one that caught us earlier today. One of our developers had implemented a local preview of pages in our client application. In the end this ran an XSLT over some content XML on the local machine, spat the resulting HTML out to a file and opened it in the registered browser.

He was getting different results from those when the processing happened on the server side, and it was also different if he ran it in Eclipse or via Java Web Start. While initial reactions are to jump to some sinister conclusions it turned out that on the server, and in his Eclipse project, he was using Saxon (v7.0 which will become important as you probably guessed from the title!), yet in the JNLP file for the Java Web Start version there was no Saxon, so it was using the JRE default of Xerces.

Now Xerces and Saxon have been know to behave a little differently when it comes to XPath short hand (e.g. match="/Bob/Sarah" as opposed to match="/child::Bob/child::Sarah"), so this wasn't unexpected. So he put Saxon into the JNLP file and uploaded the appropriate Jar.

Mere moments later cries of anguish went up from another developer who had loaded this version of the client through JWS and something important hadn't worked.

As mentioned in previouse posts, the main protocol between our client and server is WebDAV, however for things that are in no way covered by this specification we implement Web Services. There is one which checks some user details and this was failing, at least as far as the application was concerned.

When doing development work we always run the connection through the Apache Axis projects TCPMonitor programme. So any time there is a problem such as this we can look at the HTTP messages flying back and forth and see what's going on....actually so we know who's to blame, me for the client side or another guy for the server side.

After looking at the TCPMonitor output the SOAP request was fine, it had been processed by the server fine and then it had returned a correct SOAP response.

Turns out that Saxon, before v7.2, was packaged with the Ælfred XML parser, which Axis, so it seems, cannot use correctly to parse incoming responses. An quick upgrade to Saxonv7.2 and no more problems.
Lessons Learned #1: JComboBox is actually quite complex...

I talked a little while ago about taking an opportunity to rebuild our ResourceTree component, which is a reusable tree view over resources on our server. I talked about extending this into a FormResoruceTree which added checkboxes or radio buttons to each node. One place that the plain ResourceTree is used is in a JComboTree component. This is a JComboBox like component that contains a ResourceTree in the drop down instead of a list of options.

I first built this component months ago. We had a requirement to be able to select a single resource in forms, but we had limited space to layout in. A temporarily display ResourceTree seemed to be the way to go and my project manager being the guy he is(*) he "suggested" it should work like the JComboBox.

It took me almost a week to get this working. I had thought it would be quite quick. Turns out JComboBox does some interesting things to be able to display the drop down over all the other components, even over the edge of the main window.

What I produced in the end was a JPanel containing a JLabel and a JButton. The JLabel holds the selected value and the JButton opens the drop down with the ResourceTree in it. The drop down of course is no such thing, it's a floating borderless JWindow, which is aligned with the bottom of JLabel and JButton to look like a drop down.

The JWindow is hidden when a value is selected from the tree, or if the JButton is pressed again. The problem with this has been that if you don't close the JWindow, by selecting a value or pressing the button, the it will remain floating in space.

Obviously this has been reported as a bug...many times.

After creating the FormResourceTree that I talked about before, my project manager suggested that I use this in the JComboTree, especially where we allow multiple selections within it. This didn't take too long to do, and removed another of the many JTree implementations used throughout the code. He also suggested that I fix that "annoying" floating window problem, so that it disappears when you click away from it, just like a proper JComboTree.

So how can you do this with any certainty? If you are anything like me the first two things you will think about are that the solution is either something to do with FocusListeners or MouseListeners.

Well I'll tell you now that it's nothing to do with MouseListeners. What would you put them on? Everything else in the programme? No, nothing to do with mouse listeners.

The FocusListener is a good start. Listen for FocusLost events on your JWindow. This works to a degree, as focus in Swing is defined as following keyboard entry focus. So this will work for the cases where the user moves onto another component with some keyboard entry features. However if your JComboTree is in a tabbed pane and the user switches to another tab, this apparently doesn't count.

So where do you look for a solution. Well my next step was to realise that JComboBox works, and therefore I should check out how this does it. A quick check in the JavaSDK source later and I had the solution. AncestorListener. I'll admit I'd never heard of this before. According to the JavaSDK JavaDocs it is this;

"AncestorListener Interface to support notification when changes occur to a JComponent or one of its ancestors. These include movement and when the component becomes visible or invisible, either by the setVisible() method or by being added or removed from the component hierarchy."

This is nearly perfect. After implementing this our pesky JWindow disappears in almost all cases, at least I can be sure that it will go eventually. People seem a lot happier now in any case, and that will do me.

So what are the "Man at arms" style morals from this tale. Firstly Swing components can often be far more complicated than you realise until you try to replicate their functionality. Secondly, keep the JavaSDK source close to hand, it can often provide inspiration, or at least explain what's going on under the hood.

* Just to explain, my project manager on this project has very strong views on how interfaces should work. Often this is far from the accepted norm, and therefore what is directly supported in frameworks like Swing, but almost always the right thing to do. Hence why I've spend most of this project extending or replacing Swing components.

Friday, September 24, 2004

Checkboxes in a JTree.

I've been having fun over the last couple of weeks getting to tidy up some of the code that I created a long time ago. For the administration client application I had gone through several designs for a standard tree view of resources, based on the Java JTree class.

By the end of the project I had about five different versions, all sharing different bits of code, for example some had their own specific TreeNode implementations, most shared a TreeCellRenderer etc.

I had to go in and fix a bug in these trees and thought I would take the opportunity to rationalise all this work into one standard tree component and an extention of that which turned the tree into a form, with checkboxes or radio buttons either at the leaf nodes or on all the group nodes.

So I created a new TreeCellRenderer which created a TreeCell (one of my classes that extends JPanel) for each cell. In the TreeCell were the usual things, but also either a checkbox or a radio button. This all worked fine in a tree where you can only select the leaf nodes, however in a tree where the leaf nodes are not display and group nodes can be selected I ran into trouble.

You see, I could not use a mouse listener on the checkbox, or infact listen for events from the checkbox. I am guessing that the JTree has some glass pane over it thus preventing this from working. So I extended MouseAdapter, put that on the tree and got the tree row from the mouse click co-ordinates, thus giving me the TreeNode which I could now inform to de/select its checkbox.

All seemed well, except in some strange cases where clicking on the plus symbol to open a part of the tree would pass the mouse event down to the TreeNode. This would only happen when the group node in question didn't actually have any sub-groups underneath it, thus the plus symbol dissappeared.

It took me a long time to find the solution to this, I tried screen co-ordinate translation (note don't ever try to get this working with tree cells!). I added a tree selection listener, but of course the selection this is talking about is nothing to do with my checkboxes. I could select a node, but then not unselected it, because the underlying node was still seen as selected and so the event would not get passed to the listener.

In the end I had to create a class which implemented TreeSelectionListener and extended TreeSelectionModel. This picks up the tree selection ("valueChanged") event, finds the node from the selected TreePath and tells it to toggle its checkbox/radio button. Then I call this.clearSelection() to the super TreeSelectionModel. Unfortunately this then triggers another "valueChanged" event, so I had to add a re-entry boolean to stop this from happening.


public class TreeMouseListener
extends DefaultTreeSelectionModel
implements TreeSelectionListener, TreeSelectionModel {

private boolean m_bReEntryStop = false;

public void valueChanged(TreeSelectionEvent tse) {
if(!m_bReEntryStop) {
TreeNode node =
(TreeNode)tse.getPath().getLastPathComponent();
TreeCell cell = this.m_tree.getCellForNode(node);
if(cell.isEnabled()) {
cell.mouseClicked(null);
m_bReEntryStop = true;
this.clearSelection();
m_bReEntryStop = false;
}
}
}
}

Thursday, September 23, 2004

Gartner update on major Web-Servives-Enabled software players

Spotted this analysis of the market, it has some nice thoughts on the different approach to WS that these people are taking and their involvement in standards work. It's a bit of a shame that the entry point is capped at >$500million in revenue, but there is an honorable mention for one company that didn't make this cut. Worth a read.

Magic Quadrant for Web-Services-Enabled Software, 3Q04

Tuesday, September 21, 2004

Load testing surprises...

We've been doing a considerable amount of load testing on the new server code recently and we've had some shocking results. The main one was a concurrency with our caching classes. We had implemented a removal queue in which objects would rise and fall based on usage. Turned out that there were some serious locking problems going on here. One of our developers reimplemented it using Doug Lea's packages util.concurrent Release 1.3.4. and the LRUMap and we've gained a considerable amount of performance.

Moral of the story, check this stuff earlier.

Monday, September 20, 2004

More debate about the WS-* stack.

The recent flood of blog postings about the viability of the WS-* stack specifications has mostly stemmed from Microsoft's recently published whitepaper An Introduction to the Web Services Architecture and Its Specifications. I've read the paper now, and generally think it's a very good overview of the MS/IBM Web Services Architecture, however there are a few points I feel I need to raise.

1) This is the MS/IBM camps Architecture, and therefore this whitepaper must be read within that context. There are open standards, from W3C, OASIS, IEFT etc, talked about in there, but also there are a lot of specifications that have not been/are no where near submission to a standards body.

I think it is a missed opportunity for Microsft not to have made this more clear. If they had they may have managed to control the resulting debate to being about why these specifications are important and how best we, as a community of developers, can bring them about.

Worst yet, they present specifications here that have direct competition from open standards, for example the MS/IBM WS-ReliableMessaging which is basically the same as the OASIS WS-Reliabillity specification.

That said, read in this context this whitepaper is a great introductory article to the state of one Web Services Architecture and will set you thinking about a more general one.

2) After reading some of the follow on posts, for example "More WS-* specs, more questions about architectural viability" I can understand some of the problems people are having with this. Of course people are not going to see reason for all these specifications if they are not now, or probably never will be, interested in using them. However I don't agree with the following.

"Tim Bray's point about Amazon, eBay, etc. not needing the WS-* stuff to get their job done is well taken, but it's also quite clear that these were built from the ground up to work with the Web, whereas the fertile ground for WS-* are the enterprise systems that were not designed with the web in mind."

Amazon, eBay and the rest are at the vangard of such large Web Services implementations. I am sure they will have long term strategies which they are not sharing. But the services they have exposed so far are not critical services. Amazon does not come off too bas if an associates site fails to load a page, and they are not yet allowing seemless purchasing from associates. For this to happen specifications such as WS-Secutiry are essential. One of the Reliability specifications/standards will also be required if they are to puch the purchasing services this far out.

This is only the start, large corporate associates would probably wich to implement something like WS-SecureConversation to save the repeated single shot security problems with just having WS-Security.

Of course Amazon, eBay and the like have been happy with only SOAP and WSDL, because that's all they had, but I wouldn't be surprise to see them picking up on other standards and doing some interesting things with them. To address the second point in the quote, yes they were designed for the web, and yes enterprise systems will need some of these specifications, however I would be willing to bet that Amazon see their systems as "enterprise".

Also are we going to throw out all the ideas and experience from these "enterprise" systems? Currently most of the Web Service implementations out there are simple point-to-point systems, there are few, that I know of, compelex networks of services with many intermediaries and compound services which go on to trigger many more. These types of system will required the more advanced WS-* stack specifications. So let's start getting them ready now.

Friday, September 17, 2004

The state of the WS-* stack

If you've been reading my updates you will know that I've been very concerned about the status of the WS-* stack (the set of standards that can be implemented on top of the basic Web Service standards of SOAP, WSDL and UDDI).

My major concern has been that you cannot currently describe a service that utilises any more than the basic standards, my main case in point was that there is no standard way in WSDL to describe the WS-Security requirements of a service.

Since those recent postings, and the release of the WS-Reliability first draft, I've been catching up on the state of the various WS-* stack standards. Here is what I've found, some of which is definately not good news.

Firstly, here is a pretty good overview of the various standards:

The Web Services Protocol Stack

Next, going back to the WS-Security example that I have talked about before. My problem with this was that there appeared to be no way to describe the WS-Security requirements of a service in the WSDL file, and therefore no way to programatically create clients for that service without some form of human intervention.

After doing some research, which I should have done before, I now understand what is going on. I think my problem was in the fact that WS-Security is now an OASIS standard, but it had originally come from the Microsoft/IBM camp. In the MS/IBM camp, the prefered way of dealing with programatic descriptions of WS-* stack requirements is place the information into WS-Policy xml.

WS-Policy is simply a framework to contain policy assertions, these are standard specific and are described in other documents (for example the WS-Security Policy standard).

WS-Policy in itself does not describe a way of attatching the policies to service descriptions, this is dealt with by the WS-PolicyAttachment standard. In this standard there are descriptions of how to attach WS-Policies to both WSDL and UDDI.

Now I like the WS-Policy framework, I think it is nicely lightweight in its core specification, nicely extended for the standard specific assertions and I also like the separation of the attachment details. However this does not solve the WS-Security issue.

You see WS-Security is now an OASIS standard, but WS-Policy has not yet been submitted. So while we are all agreed on how to implement Web Services security, we have no (independantly standardised) way to attach these requirements to our service descriptions.

I need to do more thinking about this, but I am wondering where we go from here. In my mind I am formulating what I see as the WS-* stack Base Implementation, the minimum set of standards that need to be implemented before Web Services become truely usable for industry, and the more I look into it, the further away this goal becomes.

Anyway, time for Friday drinks in the pub. I will post more on this later, including any findings on the MS/IBM camp WS-ReliableMessaging standard verses the OASIS WS-Reliabillity standard.....surely things can only get better!

Tuesday, September 14, 2004

WS-Reliability specification released

Wow, no sooner do I start to moan about how WS-* stack specification are not taking WSDL into account than this is released from OASIS.

Cover Pages: OASIS WSRM TC Releases Web Services Reliable Messaging (WS-Reliability) Version 1.1.

I haven't had time to go over the specification in detail yet, but it does appear to deal with the implications for WSDL. I will let you know.

Monday, September 13, 2004

Why you cannot describe a Web Service ... or how I bullied an Oracle product manager.

I am embarking on a new pet project at home, to do with Web Services. I won't go into too much detail in this posting as I am sure I'll be going on and on about it in the coming months. Basically though it is to do with generating dynamic proxies for SOAP Web Services and doing this from service descriptions such as WSDL and UDDI.

For this project I would like to be able to support a more fully WS-* stack, including WS-Security, WS-ReliableMessaging and WS-Addressing. The problem I am having can be seen with WS-Security.

There is no where in the WS-Security specification that advises on how you declare this requirement in WSDL documents. Therefore from a WSDL there is no way to know that you need to apply WS-Security.

WSDL is meant to desribe the interfact to the service, the clues in the name, however if the service is any more complex than basic SOAP this is not possible. This seems to me to be a glaring ommision.

The WS-Security Specification states that "Advertisement and exchange of security policy." is a "Non-goal", but surely is cannot be the responsibility of the WSDL group to deal with this.

You may have read my previous post Our WebDAV love affair? where I have a bit of a go at the WebDAV specifications, however there is one place where they have done amazing work which is understanding how to build a stack of specifications. In any of the extending specifications there are clear statements on how these effect the other specifications.

My major concern is that I cannot find any work going on in this, the the WSDL 2.0 Feature element might be something for this, but if the extension specifications continue to see this as something for other people to deal with then who will come up with the vocabulary for this to work.

The second part of the title for this posting is about me bullying an Oracle product manager. On Friday I went to the London Java Special Interest Group meeting where there was a presentation from RSA on Identity and Federation and then one from Oracle on the new WS-Security features in their App Server and JDeveloper 10g.

Part of this was a demo of building a WS-Security enabled SOAP Service in JDeveloper, which I have to admit looked to be an extremely easy process. The simple service was exposed, including building the WSDL, and then WS-Security was added to it.

I asked if the WSDL file had in anyway been changed when adding the WS-Security, and as I expected it hadn't. A couple of other people then commented on this, obviously not have come up against this problem before. Unfortunately someone then asked to see the building of a client stub for that service, which was also a completely painless process.

My bullying then came in the form of pointing out that the client blatently would not work as it had been built from the WSDL which did not advertise the fact that encryption and digital signatures where required. I feel I should publicly appologise to this guy and state that aside from this, which is not the fault of JDeveloper or anyone at Oracle, the demo was really good and their next generation product line does seem to be very good.

As for the problems with WSDL and the WS-* stack, well I guess we are all going to have to wait for this, and perhapse try to get the WS-I to expand their remit to look at this kind of thing.
Back after a long break.....

So this is my first post in a while, mostly because I've been away from the office for the best part of seven weeks. This has been due to take-it-or-loose it holiday and training that my company has sent me on.

The holiday time was great, with some of it spent back home with the folks attending a family wedding and some of it spent in Rome.

Last weekend also saw the third annual 24x24, 24 hours watching the third season of 24 in sync, so starting at 1:00pm. Always much fun, this year was a little different as I'd not seen the season before this.

The training I went on was, hmmmm shall we say "interesting". It was a week long Business Analysis course and three days of Introduction to Consultancy Skills. Most of the company went on both of these, as they are seen as key skills, even for developers. Certainly opened my eyes to a different way of working.

As part of the Consultancy Skills course I took the Belbin test to discover my preferred team working roles. A little worryingly I got zero for the co-ordinator role, however I scored highly on the specialist and completer-finisher roles which does seems to make sense.

Anyway, more technical postings to come, just thought I'd get a hello post in there first.