Category: Technology

The need (or otherwise) for a technology roadmap: Part 1

This is a post in two parts, inspired by a recent conversation with a colleague. In this part I’ll set out what a technology roadmap is and circumstances in which it would be useful. In the next part I’ll set out why it might be best to go without.

Roadmaps can be usefulIntroduction

Let’s first make sure we know what we’re talking about. A roadmap is a series of milestones to achieve some end; a technology roadmap is a roadmap for the technology base of an organisation.

Simple examples might be: we will replace the following legacy systems so that we can respond more quickly to our users’ demands; we will move our non-core systems into the cloud, in this sequence; we will implement the following technologies so that we can operate our distribution system globally.

In reality such a roadmap is likely to be more complex, with several different elements.

People get very excited about roadmaps. If you have two departments about to spend your time and money, and one has a roadmap and the other doesn’t, then the one without is going to look pretty shabby. And that’s even without looking at the content of the roadmap…

Any roadmap has some positives

Now, setting aside for one moment the benefit that any particular roadmap or plan might or might not give us, any technology roadmap clearly has some positives. It demonstrates purpose to those outside it (“They’re on a mission…”) even if those outside don’t fully understand it (“…and I guess I have to trust them on that”). It gives purpose to those implementing it (“I have to go in today despite being sick; there’s a key deadline for our roadmap”). It shuts down any number of unwanted conversations (“Can you do this for me?” / “Sorry, we’re maxed out delivering the roadmap, and you know the board’s expecting results”).

Obviously this is an overly cynical perspective, but like I said, that’s setting aside the benefits of the roadmap itself. So roadmaps are good, and have benefits even if they don’t actually deliver any real value themselves.

But really and truly a roadmap does need to have a purpose, and (because technology for its own sake has no benefit) that purpose must be driven by something outside technology. Here are four suggestions of where that purpose might come from, in order of increasing maturity:

4. The bureaucratic roadmap

Here, a roadmap is created purely because corporate bureaucracy requires it. Maybe it’s budget time and your budget won’t be approved unless it looks like there’s a grand plan. Maybe the CEO — or the VCs who are bankrolling your startup — want to see things mapped out before they’ll give you the time of day.

This scenario is not much better (possibly worse) than the cynical reasons given above. Except this time it’s not you who is made to feel better for applying some arbitrary order to daily pressures, but the corporate powers that be.

3. The defensive roadmap

This is where a tech team realises it must do something for itself to head off something less palatable. An example might be integrating a corporately-approved web service, such as Basecamp, before too many staff sign up to similar services themselves, directly, and scatter company data all over the Internet. Another is modernising the mail system ahead of a merger with a larger company to avoid being utterly taken over by the larger company’s technology base.

Clearly it’s not great to be on the defensive, but at least we’re responding.

2. The directive roadmap

This is where a tech team recognises the company doesn’t have a particular direction, so it implements something in order to create one. An example here might be enhancing all the systems’ robustness and security because there’s an opportunity to shift from a consumer to a corporate clientbase, and the tech team realises this would be the way to open up those kind of opportunities.

The directive approach can really only be instigated by the most senior one or two people in the tech team. Also, it’s a bit risky because it could be called into question if there’s a lack of trust or change of leadership. The IT director could find themselves in a difficult situation if their peers suddenly thought they were running an expensive project which wasn’t aligned with any current company purpose. But if it works then it’s a good result: a previously directionless company gets a new lease of life. Or at least new opportunities, whether or not it chooses to exploit them.

1. The aligned roadmap

Almost certainly the highest form of technology roadmap. In this situation the organisation has a clear strategy which becomes the basis of every other department’s own, localised strategy — the the technology department is one of those.

If a company decides it needs to increase its overseas sales then there are implications for distribution, legal, advertising, procurement, and of course technology. Even without a single co-ordinated programme to achieve a specific goal every department suddenly has a purpose, each one can work out for itself what it needs to achieve that, and all departments are working to achieve the same thing.

That this is highest form of technology roadmap should not be surprising. It’s born of a very high level of organisational maturity which may seem easy but isn’t in practice. Just recall the story from eBay where the company had a long-term strategy to emphasize fixed-priced goods, but they managed to forget to tell the technology team.

In the next part of this article I’ll consider reasons why it might actually be better not to have a roadmap…

When board leaders fail to grasp technology

I’m amazed and disappointed that it’s still acceptable for people who run major companies to show wilful ignorance of technology. And I don’t mean what firewalls do or how to get wifi working on your laptop. I mean what it means to the companies they run, how it impacts their people and their customers, and how to make sure it’s an asset rather than a liability. And there was an exemplary demonstration of this on last week’s edition of The Bottom Line on Radio 4.

Another CEO prepares to demonstrate IT leadershipThe programme is a straight discussion of business issues with Evan Davis and a number of business leaders. It’s not dumbed down (any more than it has to be edited down) and is a consistently excellent listen. Last week the guests were Luke Johnson, chairman of Risk Capital Partners, Vincent de Rivaz, chief executive of EDF Energy, and Jacqueline de Rojas, UK and Ireland vice president of McAfee. And they dealt with two major topics: organic growth versus acquisitions, and problems with IT. Luke Johnson in particular seemed to really have problems with technology, and to me there was overwhelming evidence that some of the causes might be rather closer to home than he might like to think.

Coming up is some criticism of what Luke Johnson and (to a much lesser extent) Vincent de Rivaz say, but don’t mistake it for a criticism of them as individuals. The point of what follows is that their attitude is representative of so many board leaders. Even though there are many board leaders who have a very confident grasp of technology issues it’s still remarkable there are so many others who handle technology as if they were wearing boxing gloves.

Problems with technology

As the resident IT expert it was Jacqueline de Rojas who was asked first about her own experience of IT problems, and in particular the bad software update that her company issued in April which disabled many of its customers’ computers. Notably, she responded to a technology-related question with a positive and human answer: “The key in a crisis like that is how you respond; that’s how you’re judged”. Then Vincent de Rivaz was asked if he had ever had any IT disasters, and before he could finish his response (along the lines of “no actual disasters, but we have had problems”) Luke Johnson cut in:

LJ: “Every business I’ve ever come across has always been having problems with [its IT].”

VdR: “We have to invest in our IT and we have to develop our IT systems which we are doing at the moment [with a new online system to help our customers with their bills.]

ED: “What sort of problems have you had with your IT?”

VdR: “The problem is to control the costs. That is the permanent challenge for me, to be sure that we are spending the right amount of money on the IT systems.”

LJ: “All my experience of IT systems is they are always overdue and over budget, and I think it’s a built-in part of the IT industry that they deliberately under-quote and then they may get up on the extras.”

Now there are many problems board leaders could have with IT: We aren’t getting the best out of our technology investments. Our technology portfolio isn’t broad enough to help us in a constantly-shifting market. We don’t have the people who really understand how to manage and change our technology infrastructure.

These are all problems that require board-level investment and oversight. And they are also all questions that, if you substitute “financial” for “technology” all boards do deal with. But sadly it’s okay to ignore those matters if they’re technological, and so the top-of-mind IT issues for our token CEO and chairman are (1) cost control, and (2) IT people are sharks.

Indeed, Luke Johnson seems to have something of a chip on his shoulder about technologists. Jacqueline de Rojas goes on to explain a bit about the difficulty of  turning “woolly and aspirational” IT demands into concrete deliverables, but that doesn’t stop him:

LJ: “I think the IT experts blind clients with science, and I think very often they themselves overestimate the benefits, and overcharge for what they do. And the fact is there isn’t another industry, except perhaps pharmaceuticals, that makes the returns on capital and margins that software does. Huge margins.”

JdR: “The level of investment we have to make to stay ahead technologically is also huge. So where do you think that money goes? It goes right back into making sure we stay one step ahead of — in our case — the hackers and cyberterrorists and the competition”

ED: “Let me challenge you, Luke. Is there something different about IT from other areas of company purchasing? Is IT unique in being difficult and awkward for a company?”

LJ: “No, I don’t think it’s unique, but I think it is mission critical, and unquestionably we are all more dependent on IT than ever. I accept all that, and it can deliver huge values, and the very nature of it is it’s constantly evolving. But the truth of the matter is, what other sort of project or product, rather, do you buy that requires these perpetual upgrades and offers such super returns? I was involved with a retailer once, Whittard of Chelsea. We spent, many years ago, the best part of half a year’s profits on building a fully-integrated website. This was in the early days of the web. Within two years we had to throw it away. Because it just didn’t work. We used the wrong suppliers and we made a lot of mistakes.”

To him IT is a homogenous  blob, and he transfers his anxieties freely between commodity software sales (“returns on capital and margins… perpetual upgrades”) and individual business change projects (“a fully-integrated website”). Maybe he’s just exploiting the opportunity to have a go directly at a software vendor, because when Jacqueline de Rojas highlights how technology change really does improve our lives he’s having none of it:

LJ: “What are the cost of goods in software? What are the cost of goods? How much does each new program cost Microsoft? Zero.”

Which is silly, because he knows perfectly well that software (like pharmaceuticals) costs a lot of money to create, and Jacqueline de Rojas has already explained the cycle of investment, return, and re-investment that her market demands.

IT - easy moneyWhere the problems lie

I’m not for a second denying that Luke Johnson’s frustration is unfounded. It’s clearly borne from many hard experiences: this is a man who’s job is to cast his net widely across many, many companies, to look into them seriously, and having done this he finds that “every business [he’s] ever come across has always been having problems with [its IT]” and he’s witnessed one of his companies building one costly system only to have to replace it unexpectedly early.

But from where I sit a lot of his problems are of his own making. If you treat IT primarily as a cost then the best you’ll achieve with it is cost control, whereas the best you should get out of it is business transformation. If you think IT contractors under-quote on the basics and expect to make up for it on the extras, then you probably want to structure the next deal differently and be prepared to make it more of a partnership — if you can be confident of being able to act as a partner. If you think IT experts blind you with science then you’re talking to the wrong people — and if you don’t know who the right people are then you need to start filling that gap in your personal network. If you think IT experts overcharge then you’ve created a relationship with conflict at its core and the best you’ll get out of them is the minimum they’re contractually obliged to deliver, whereas you should be getting them to apply their creativity and rigour to help propel your business forward.

The thing that’s really shocking to me is that there are many company leaders like this, who are unashamed of their blunt handling of technology — which is undeniably mission critical — and yet the same people wouldn’t stay a week in their jobs if they demonstrated the same disregard for, say, legal matters or finance.

Which is rather ironic, because earlier in the programme it was Luke Johnson who was called on to be the resident expert on organic growth versus acquisitions, and one insight he provided was this:

LJ: “Many studies suggest that even perhaps a majority of all acquisitions fail to deliver shareholder value. So it’s important to put it into perspective and say if we can achieve returns through organic growth, but perhaps a little more slowly, then maybe that is the more sensible way to proceed.”

Let’s take a second to understand what we’ve just heard: possibly more than half of acquisitions fail to deliver value to the owners of the business — which sounds very similar to his problems with IT. And acquisitions are a significant part of his company’s business — mission critical, even. Just like IT. But in that conversation he wasn’t remotely upset or bitter about this, he just treated it as something to deal with. He could do that because he was comfortable with working in that sphere and, I suspect, because while he might have had his share of acquisition failures they will have been outweighed by his successes.

It seems to me that if board leaders were obliged to grasp technology to the same degree as they are obliged to deal with financial or legal matters then their companies — and their companies’ technology — would be in a much better place.

Two final things

A couple of parting thoughts.

First, some not-at-all-serious observations about Risk Capital Partners. While confirming that indeed they don’t invest in any IT or software companies, I found they do have an interest  in InterQuest, “a fast growing IT recruitment business”. So if Luke Johnson is concerned about the high cost of IT professionals then he might want to have a word with them. Also, I couldn’t help but think if software really is the fantastic high-margin business he thinks it is then RCP really should start making some investments there. I don’t know if they really will make a lot of money, but I do know they’ll learn a huge amount.

Second, a much more serious note. Earlier I said that if these sorts of leaders had IT problems then some of the causes were close to home. But it’s also true that some of the causes lie with us technology professionals. If technology really is seen above all else as a cost to be controlled, and if we are finding ourselves in unbalanced, conflict-driven relationships, then we really do need to do a much, much better job at explaining what we do. And then we have to carry that through into our actions. Technology in business is about making the business more effective and people’s working lives better. We ought to be able to find routes to even the most sceptical business leaders to explain that, and get a positive reception.

The Eigenharp, openness, and launching something (really) new

Today I attended the launch of something weird and wonderful: a new musical instrument, the Eigenharp. And although this is a hardware device the event, and the run up to it, brought to mind the launch of our own Open Platform six months ago. By seeing some commonalities between the two it gave me a whole new respect for the people who do marketing and PR, because it reminded me how much stuff needs to be planned. Common phases I saw are: (1) the controlled buzz, (2) openness at launch, and (3) the follow-through. But first a few words about the two products…

The Eigenharp and the Open Platform

The Eigenharp is a new kind of device. It’s a musical instrument with several patents already filed, and incorporates 132 keys with three-colour LEDs and a breath pipe. The keys operate in three dimensions (so you can exploit, say, pressure, pitch bend and filter effects with one finger) and are sensitive to within one micron. So while it’s compatible with MIDI, that’s an incredibly out-of-date protocol compared to what you could do with it.

The Open Platform has been well-documented here and elsewhere: a full content API directly into the Guardian’s content database, plus a store of raw data on which our journalists base some of their stories.

Although they are clearly two different kinds of beasts a remarkable similarity struck me in that they are both not just new products, but really new products. There is no clearly defined place for them in either of their markets, and no predictable success path for either one. The success of each is dependent entirely on other people’s innovation. The Eigenharp’s success is dependent on musicians taking their creativity into new directions — no-one to date has written music for an Eigenharp. The Open Platform’s success is dependent on developers doing innovative things with the new data they have access to.

So how do you launch a product that’s really new? The similar steps I can see are…

1. The controlled buzz

Eigenlabs (the creators of the Eigenharp) and the Guardian (creators of the Open Platform) managed to seed a few select people with their product — musicians and developers respectively — and some small details leaked out. This clearly did a couple of things.

First, it helps you get the product right. I know from my conversations around the Open Platform that what might seem a good idea to one or two people close to the product can provoke strong negative reactions when you float it past an informed outsider. When you’re inside the company you can easily think too much and lose the big picture, so it’s good to get an informed but independent view before you go public.

Second,  creating a buzz protects you from a bit of unforeseen negative reaction — if people are excited about your product and want it to succeed they will forgive some minor mistakes. You can see Eigenlabs achieved this with this low-fi video of two guys playing the James Bond/Moby theme on YouTube. Eigenlabs didn’t create this themselves, but they let “close friends” do it, and you can see the reaction in the comments underneath: “What the hell!! This is awesome”, “where can I buy one of these?”, “dear god this is epic” and so on.

The benefits of this are well explained by Lance Ulanoff at PCMag.com. Here he compares the lacklustre launch of the Motorola Cliq with the hype from the January 2009 announcement (six months before the launch) of the Palm Pre:

Speaking of Palm, it has a good bit in common with Motorola right now. The Pre (and now Pixi) is its hail-Mary pass. If the new phone and WebOS platform fail, Palm will be done. Back in January, however, Palm ran the best product rollout event I have ever attended. It perfectly conveyed the company’s excitement and all that is good about the Pre. I remember tweeting the event with mounting excitement. Today, I tweeted the Motorola event with mounting confusion.

What Palm did in January was give the Pre a good hard shove off the shore. Those waves of excitement produced a good six months of positive press. Eventually, Palm and the Pre hit some rough waters—a shipping delay, the slow delivery of the SDK and tiny list of apps that has yet to grow. Still, that first day set the tone.

Then after the buzz is…

2. Openness at launch

There’s much that doesn’t need to be said about the launch event, but one thing that struck me about both the launch of the Eigenharp and the launch of the Open Platform was a common ethos: the honesty and openness of the presenters, and the openness of the products.

At both events the “real users” (musicians/developers) were on-stage demonstrating their early adoption of the product and open to questions from the audience. These are the kinds of people who most marketing and PR people would lock in cupboards at a press event for fear they arrive without the regulation press-on smile and go off-message. But when a product’s success relies on innovation and creativity from its real users then it’s important that intended audience hears an authentic voice.

Also, there is an openness about both products. Aside from the very concept of opening up content and data, the Open Platform gives a lot of latitude to developers, including (unusually) commercial use. Meanwhile Eigenlabs are open-sourcing their software. Again, because each product’s success relies on people taking it in unexpected directions it’s important for there to be as many opportunities as possible for that to happen. Otherwise its success is stifled.

And then finally there’s…

3. The follow-through

Well, it’s early days for both products. The Open Platform API is still in beta, but you can see some developments now, such as the launch last week of the Applications Gallery. It’s even earlier for the Eigenharp. Company founder John Lambert was asked what well-known musicians he’d like to see using his creation and he said that a number of high profile people do have the instrument but that he cannot name any names yet. Clearly some really interesting things are going to happen some time soon.

If a product really does depend on the creativity of others then you can’t — and mustn’t — be too controlling of what those people do with it. (When I spoke to him John seemed slightly apprehensive about the quality of some of the things people might produce around the Eigenharp; I bet Matt McAlister didn’t anticipate a swearword tracker coming out of the Open Platform.) But if you can’t control what others do, you can at least show some examples of what might be achieved, and that’s what the follow-through is doing with each of these products.

1-2-3, easy as A-B-C

Now if any marketing people have stumbled across this blog post I wouldn’t be surprised if they were horrified by the naivety of these observations — all of this might just be first grade A-B-C of marketing for them. But I find this fascinating. As a tech person it’s very easy for me to focus on my own role and not spend much time wondering what challenges are faced by my colleagues in other departments. Here are two products whose paths to success are unusually dependent on the unknown and whose stories will be well worth watching. By looking at the communication that’s gone on around them the significance of other people’s role becomes much more apparent. Technology success is about much more than successful technology.

Discovering JavaFX with the Guardian Open Platform

Guardian tag bubbles - click to see the applicationI’ve spent a while learning JavaFX and developing an application which is now live. I’ve written about the application itself over on the Guardian’s Open Platform blog, because it makes use of the Open Platform’s Content API. But in this blog post I want to write less about the application and more about my experiences with the JavaFX technology.

In this article:

What is JavaFX?

JavaFX is probably best viewed as Sun’s answer to Flash, just as Silverlight is Microsoft’s response. So it’s a way of developing rich internet applications, or embedding fancy graphical thingies into web pages.

In some sense Sun might not have had any need for JavaFX. For example, take a look at We Feel Fine. This is an astonishingly beautiful and imaginative application, and though it might look like it’s been written as Flash it’s actually a Java applet.

So why did Sun feel the need for adding to Java? According to JavaFX’s creator, Chris Oliver, “its purpose was to explore making GUI programming easier in general”. In other words, the Java language is not really optimised for GUI development. In addition, the language needs a number of libraries. But all of this compiles down to Java bytecode and runs on the JVM, so JavaFX is really just Java with special add-ons to make graphical-oriented applications easier.

The language (good)

The most obvious initial difference with JavaFX is the language that comes with it: JavaFX Script. As implausible as it sounds Sun has combined the best bits of Javascript and Java, and come up with something really rather wonderful. Consider the following code snippet from my application:

ring = Circle {
    centerX: bind bubble.x
    centerY: bind bubble.y
    radius: bubble.diameterInScene / 2
    opacity: 0.0
    fill: null
    stroke: BUBBLE_INDICATOR_COLOUR
};

This creates a thin ring, which is actually just a circle with an outline and no fill. It starts off being invisible (opacity 0). The variable ring was declared on a previous line and is initialised here. I think this simple piece of code demonstrates a number of really nice things about the language. First, the code Circle { ... };. This is a constructor, which I really like. It feels lightweight, like Javascript, and it’s got Javascript’s named properties. But like Java, JavaFX Script is a statically typed language, so ring is of type Circle and if I forget this later on in my development then the compiler will signal an error — that error won’t make it into production. Second, the most exciting addition: the bind keyword. This means that the formula that follows will change dynamically when any of its component parts changes. So the x co-ordinate of the ring’s centre is bound to bubble.x — the x property of the variable bubble, which is the x co-ordinate of its centre. Similarly for the y co-ordinates. Now I know that the ring will always be centred on the bubble: whenever I update the bubble’s position the ring’s position will change, too. As one of our team’s developers explained when I started telling him about JavaFX binding (in fact he didn’t even let me finish my sentence) it’s just perfect for a GUI-oriented language where you want the view to track the model. Another great feature of the language is its native handling of timelines. For example, here’s a piece of code I use to fade something in or out over a period of half a second:

Timeline {
    repeatCount: 1
    keyFrames: [
        at (0.5s) { this.opacity => targetOpacity }
    ]
}.play();

This constructs a Timeline with a single keyframe (at the half second point), and JavaFX is told that by that point the opacity needs to be at the target opacity. On calling the play() method it will interpolate the object’s opacity value so that it fades in or out accordingly. Needless to say, you can have several keyframes, and interpolate several values simultaneously or in sequence.

You can also ensure specific actions at certain points. Here is some code which fades a message out, changes it, and fades it back in again. This mixes native timeline syntax with more usual JavaFX syntax:

Timeline {
    repeatCount: 1
    keyFrames: [
        at (0.5s) { barText.opacity => 0.0 }
        KeyFrame {
            time: 0.5s
            action: function():Void { barText.content = message; }
        }
        at (1.0s) { barText.opacity => 1.0 }
    ]
}.play();

You might also notice from this that functions are treated as a first class object: another very handy feature of the language.

Eclipse development with JavaFXDevelopment support (ugly)

As you might expect from a Java extension JavaFX integrates well with Java: you can use any Java class or object in your JavaFX code. In my case I wrote a small physics engine in Java and then accessed it via my JavaFX code; it was trivial to do.

On the other hand, JavaFX classes and methods can’t be accessed from Java without a lot of jumping through hoops (you have to create an interface in Java, implement it in JavaFX, and then you can reference that interface in your Java code). Also, Java generics aren’t available in JavaFX, which is a bit of loss.

The JavaFX platform itself is evolving. Version 1.0 was released in December 2008. Version 1.1 came out in February 2009 and changed not only the underlying libraries, but also the language, so even your source code needed be revised. Most recently version 1.2 came out in May 2009, with similar changes and similar consequences.

Overall, these changes are a very good thing. Each new version contains not only extra front-end features (charting capabilities appeared in 1.2) but also performance enhancements and changes which ensure greater API consistency and extensibility. It’s interesting to see Sun take a different approach from Java: there the remit has been to preserve backwards compatibility at all costs, because there are so many applications deployed in it, but JavaFX is still in its infancy and breaking backwards compability is a significantly lesser evil than consistency and extensibility. Nevertheless, the feeling is of a platform that’s yet to mature, and my productivity was knocked back by one or two platform bugs whose equivalents just wouldn’t be be dreamt of in, say, Java. Still, JavaFX is clearly actively evolving, and most certainly for the better.

More disappointing is the poor IDE support. Inevitably, support for Sun’s NetBeans is being actively pursued alongside the evolving platform. But my first experience with JavaFX in NetBeans earlier in 2009 found it to be an imperfect environment, and I opted to switch to Eclipse — I decided I could live with more quirks in return for a more familiar working environment. And after some considerable hassle I got it working.

But Eclipse support really is poor: there are a few annoyances documented over on pliq.com, and I found a few more. Overall it meant that even typing code was a bit of a minefield, and that’s without refactoring support.

There is also the inevitable problem with unit testing graphically-oriented applications. I admit I resorted to the old fashioned develop-debug cycle, which was a little disappointing and inevitably frustrating. But I can’t quite conceive of a unit test framework for a platform in which there is so much visually-oriented code. I suspect it would mean a slightly different approach to coding. My brain can’t quite work out what, though.

Swirly Java preloaderThe user experience (bad)

So developers have problems. Let’s pretend not to worry about them; their entire lives revolve around solving problems so they’ll find ways to solve those. Much more important is the experience of end users, and with Sun boasting that Java runs on “800 million desktops” (isn’t that more than the number of atoms in the universe…?) those people’s experience is paramount.

Well, if you’re running the right environment then the JavaFX experience isn’t at all bad. Unfortunately you’re very unlikely to be running in the right environment…

Sun have done an excellent job in making it simple and reliable to embed your applet into a web page: once you’ve configured your code you only need some very simple and clear JavaScript in your page. This JavaScript takes care of platform versioning, prompts the user for an update if necessary, shows a standard “Loading…” animation, and downloads and runs your code.

In the circumstances this clever; but the circumstances are far from ideal. For example, when did you last update your JVM? Probably not recently enough to have the latest JavaFX release, so your first use of a 1.2 application is going to prompt you for a very large download of the JavaFX 1.2 runtime. You’re also going to need to run a launcher applet from Sun, which then prompts you with a popup requesting you trust software from Sun. Anyone who wants a seamless user experience is going to take one look at JavaFX and run a mile.

Even if you’ve already got the 1.2 runtime and launcher applet the JavaScript embed code itself references a script on Sun’s website. That means your web page is always relying on Sun’s website being up, running and performant in order to load. For code on my own little blog that’s fine — Sun’s uptime will always be more reliable than mine. But if this is to be adopted by bigger players, such as my own employer, the Guardian, then the sysadmins responsible for page delivery time and reliability won’t be happy with another third party dependency.

And if you’re not impressed with that, then I hope you’re not an Apple user. Java support for Macs has long lagged behind that of Windows machines (I recall the Java 5 SDK came out over a year after the Windows release) and this shows again with JavaFX. In theory JavaFX runs on a Mac. In practice it’s a world of pain: it’s slow and rendering features such as antialiasing are dire. Experiments show that if you’re running a very high end Mac then it’s half-way approaching decent, but not actually decent.

As someone who spent a long time developing this on a Windows machine it was very disspiriting to see the fruits of my labour reduced to a syrupy judder when I wanted to demo it to someone with a Mac.

Dstandley chasm nt by *waterspriteConclusions and (inevitable) comparison with Flash

If Flash didn’t exist then my view of JavaFX would be different. But Flash does exist, it’s the dominant RIA platform — certainly within web pages — and so any view of JavaFX has to be assessed in that context.

I find it very amusing that Adobe and Sun have eyed up each others’ territory and each wanted a piece of the other’s action. Adobe has wanted to move out of the web page and onto the desktop, so they’ve introduced AIR, while ActionScript has gone all serious, ditching the JavaScript-like version 2 for the Java-like version 3, and encouraging a cavalcade of grown-up design patterns. Meanwhile Sun has left behind that heavyweight baggage of the Java programming language and gone all JavaScripty, with easy constructor syntax, type inference and treating functions as first-class objects. They’ve each adopted the other’s clothes, but changing clothes is only part of what’s needed.

For my money JavaFX Script is the superior language. It’s designed for graphics creation and management, while ActionScript looks like Java before generics. The code I’ve found online to draw shapes in ActionScript brings back nightmares from the early days of writing Java applets: get a graphics object, set some properties, start the filling, do the drawing, stop the filling… it’s unintuitive, uphill work. (What the hell is a “graphics object”, anyway?) With JavaFX you use a constructor like the one shown above.

Mind you, there’s a reason for this difference. Flash development is predicated on the use of some really well-developed proprietary, commercial tools to do the job: Flash Professional for designers and Flex Builder for application developers. Looking for an ActionScript Hello World program produces some entertaining results. Here’s one conversation in which the request for a Hello World app is met with incomprehension: can’t you just drop the text on the stage? asks one responder, who misses the point of a programming tutorial; I suppose you could use the trace function to print the text out, suggests someone else. Here’s an ActionScript Hello World tutorial which includes the instruction “Select the Text Tool…”. There is a purely programmatic tutorial which Andrew Payne created because “The other examples I could find all used Adobe’s paid tools, Flash Designer, etc.” — but that was written only in January 2009.

So Flash is deeply embedded into design houses, which have come to rely on some very mature tools in which programming is secondary, particularly when it comes to asset creation. And while this might make more traditional developers smile it does demonstrate the cultural chasm that seperates Flash from JavaFX. Designers demand a first rate user experience, or else their work won’t sell itself. That means Flash execution doesn’t suffer the popup bombardment of JavaFX. And it means Flash runs just as well on a Mac as on a PC. And it means Flash developers have become obsessed with performance in a way that JavaFX developers can’t even imagine, so there’s an entire mindset which has yet to be addressed. And it means component support in JavaFX is lacking where in Flash it’s commonplace. One example of this is the lack of a JavaFX preloader: with Flash it’s standard to have a progress bar as the application loads; with JavaFX the standard preloader is an animated image that continues to animate even if there’s a background error, and I can’t find any code which shows a progress bar. Another is the ease in which one Flash swf can load another; again, not readily possible in JavaFX.

Designers would also bring a huge benefit to JavaFX beyond setting high bars for user experience: they are the people who will make JavaFX applications look beautiful, and so attract others’ interest, and so start a rolling snowball of adoption. I think my own application is quite pretty, but it was quite different in the early stages. It was only when I studied and tried to emulate the work of the Guardian’s professional designers that it started to come alive: circles were a friendlier shape than rounded rectangles, the palette of guardian.co.uk was infinitely prettier than my original palette of browns, and so on.

Sun is not unaware that this cultural gap needs to be bridged. That will be one reason why they’ve created JavaFX Production Suite. But years of cultural awareness and workflow embedded into design studios’ daily operations will be very, very hard to reverse.

So my own thoughts about JavaFX reflect my own experience with it. My development experience was very positive: it’s a great language and a mediocre development environment with many teething troubles which will no doubt be ironed out over time. I had terrific fun with it, and getting fast results was very rewarding. But beyond the development experience the future seems more difficult. The user experience is very poor, with JavaFX throwing up barriers in front of the non-technical end user where Flash is invisible. And the leap from niche developer interest to general acceptance by design studios is just enormous.

Making JavaFX a real success is huge job for Sun. They’ve got off to an excellent start, with a language and runtime environment that allow for very quick development. But that, alas, is the easy bit compared to what lies ahead. I do wish them the very best of luck.

Solved: Annoying JavaFX Eclipse plugin problem

[I don’t usually post very technical material here, and apologies if you’re after higher level discussion, but it seemed the most appropriate place for this.]

I was quite thrilled last night to have solved a really frustrating bug in a piece of open source software. First the background, then the problem and finally, for the record, the solution.

Background

I’ve been messing around programming with with JavaFX and would have liked to have used my favourite IDE, Eclipse.

Unfortunately JavaFX has a bit of a poor pedigree. It’s a me-too product being Sun’s response to Adobe Flash and AIR, just as Silverlight is Microsoft’s response. It also doesn’t seem to have been given the heavy backing and attention within Sun that anyone would need going up against the incumbent RIA platform. I see this because (a) the language changed slightly after the first official release (1.0) to the next “minor” version, which smacks of carelessness (or pressures on dates of public announcements); (b) JavaFX application loading seems to be a variable experience, and certainly not as slick as Flash’s; (c) patchy documentation — for your reading pleasure I quote here Chapter 10 of the JavaFX language reference in its entirety:

Chapter 10. Animation

Table of contents

[To do: write chapter]

Added to that, Oracle has just bought Sun. Who knows what the new owners will think of this project… it’s a long way from their core business of providing large scale middleware solutions. It’s a shame, because JavaFX has a lot going for it, and as far as I can tell stands up well technically to Flash; but it needs investment in giving it a lot of polish if it’s to demonstrate this widely. Certainly there doesn’t seem to be a fully-featured IDE readily available.

Problem

However, a small team of independent developers wrote an open source Eclipse plugin which was released in December 2008, but the last commit was on 13 January 2009. Since that time JavaFX 1.1 was released in February 2009 which doesn’t play nicely with the Eclipse plugin, but no word from the maintainers on any new releases. It seems to be dormant.

The problem I (and many before me) found is bug 206, documented in bugzilla: when writing seemingly innocuous code the incremental compiler reports

incompatible types found   : Integer required: com.sun.javafx.runtime.location.FloatVariable

Which doesn’t read very well but I think means “I found an integer but requred a float”. The error is reported on about one line in five, swamping genuine error messages, and no amount of bracketing, casting or whatever seems to help. It’s clearly frustrated a lot of people and it certainly made me rather disappointed with the whole JavaFX experience.

Anyway, inspired by insights from some of those similarly affected, and after a lot of semi-random hacking and poking I found a solution, which at least works for me.

Solution

The solution is to build the plugin yourself from the source, and use that. The original documentation for this is a little terse (well, I struggled a couple of times) so here’s an augmented version:

How to build the plugins from sources

1. Requirements

  • Install the Eclipse IDE for Java EE Developers from http://www.eclipse.org/downloads/.
  • Install the DeltaPack from http://update.eclipse.org/downloads/drops/R-3.4-200806172000/index.php#DeltaPack. (I have no idea what this DeltaPack is, but I think it’s used in this instance as libraries against which the plugin sources are compiled. I unzipped the download and dropped the files into my Eclipse application directory as appropriate: features in the  features directory, plugins in the plugins directory.)
  • Get the JavaFX SDK from Sun in the form of a zip file. Since it’s currently only available as an executable, you need to download and  install that in the usual way (if you haven’t done so already) and then zip it up into a file. I called my file javafx-sdk1.1.zip (though I’ve actually got version 1.1.1 — using only major and minor seems to be the convention).

2. Get the plugin source files

  • On the Unix commandline the incantation is svn co https://kenai.com/svn/eplugin~subversion/trunk eplugin (though I’ve not checked this).
  • Alternatively on Windows you can use TortoiseSVN.
  • However my Eclipse was set up to use subversion so I imported the source directly into Eclipse:
  • Import the files into Eclipse. In Eclipse I did File > New > Project… > SVN > Checkout Projects From SVN. Then use the repository location https://kenai.com/svn/eplugin~subversion/trunk and get the entire contents. I had to accept a few certificates (temporarily) along the way. One time I tried this the new Eclipse project showed lots of Java errors (libraries not found), but they didn’t seem to cause a problem.

3. Amend and then run the build

Running the build that came with the source files didn’t work, and I needed to amend the Ant build.xml file. I had to change the original line

<property name="javafx.sdk.root.dir" value="javafx-sdk1.0"/>

to

<property name="javafx.sdk.root.dir" value="javafx-sdk1.1"/>

That is,  I had to change the javafx.sdk.root.dir variable to match the name of my JavaFX SDK zip file, but without .zip bit.

Now you can run the build from within Eclipse. To do this I did: Right click the build.xml file in the Eclipse package navigator, choose Run As > Ant Build… Go to the Main tab, then the Arguments box and fill in these arguments

-DeclipseDir=<path> -Djavafx.sdk.build.url=<url_to_zip> clean build-windows [build-macosx] [build-linux]

Note that you need one file path and one URL, and you need to name the operating system you’re building for. This took a few minutes to run, but it completed without problems.

If you’re not doing this from inside Eclipse you can use Ant directly.

4. Install the results

The result of this is in a new dist directory. Here you will find a few files, but the one you’re interested in is the .zip file. In my case it was called com.sun.javafx.eclipse.feature-win32-win32-x86-1.0.1.zip. This is an Eclipse plugin. Unzip it and drop the contents into the relevant places in your Eclipse install directory: features in features, plugins in plugins.

I hope you find this goes relatively smoothly.

Why did this happen?

Honestly, you think I know? My guess is that the plugin was built with JavaFX 1.0.x embedded, and those of us with JavaFX 1.1.x get compatibility errors; recompiling from sources ensures your new plugin in built with your current JavaFX SDK. If so, that’s a pain, as it may mean you’ll need to do all this again when the next JavaFX SDK comes out. But, hey, you’ll get a few hours of happy coding done in the meantime.

An ABC of R2: V is for video content

…which changed in the way we thought about it (and implemented it) a long time after it was released.

Originally any piece of content was designated as being of exactly one type: it was either an article, a video, a competition, and so on. To take the video content as an example, putting a video onto the site involves opening up a video content editor in the CMS, referencing the appropriate asset in the Brightcove system, and adding some video-specific metadata such as its source and preferred display size. Clearly you aren’t going to confuse that with writing an article, which is why it made sense to say that each piece of content was of exactly one type, no more and no less.

But one day one of the editorial staff made an unusual request: she had an article and wanted us to add a checkbox in the article editor marked “Display on video page”. This puzzled the tech team initially — if it was an article then people wanting videos wouldn’t want it to appear on the video page. Also, we were wary about adding a feature which would complicate the article editor but be used by few people. We quickly found the good reason for her request: she was creating an article with an embedded YouTube video, and the purpose of the content really was to showcase the video. Anyone looking for videos on the site would want to find the content, so it should appear on the video page, but the video should remain outside the Guardian CMS, so the video content editor wasn’t appropriate.

We realised this was a general need: content could fall into several categories at once and editorial staff should be able to flag it as such. The solution, then, was not to add a checkbox marked “Display on video page” because that wouldn’t solve the general problem. Instead we took an existing “labelling” facility present in all the content editors, and we extended it allow content-type labels to be added. Of course, we also needed to ensure the system responded to these new labels, but since it was an extension of an existing feature it was a more natural and reliable solution than we might otherwise have implemented.

The end result therefore didn’t introduce any complexity for end-users — no extra buttons or checkboxes — but it did give them the power to do what they wanted.

This is a good example both of how our understanding of apparently-basic questions (“what is video?”) can change with experience, and of how understanding the source of a requirement can lead to better solutions.

An ABC of R2: U is for URLs

…whose structure we worked hard to get right as part of the project. This was an important part of weaving ourselves into the fabric of the web: to ensure our referencing system had a useful structure to those outside our organisation.

Previously our URLs looked like this:

http://politics.guardian.co.uk/tonyblair/story/0,,2075005,00.html

today the same piece of content is referenced like this:

http://www.guardian.co.uk/politics/2007/may/10/tonyblair.labour3

There are two obvious changes here. First, the obscure numbers have gone — a legacy of our old Vignette StoryServer software, and utterly useless to anyone who happens to be a human being. Second, we’ve included the date in a friendly format, to provide some sense of orientation, and particularly useful if you happen to be looking at several URLs at a time.

There are other changes, too. The “politics” reference has come out of the domain name and into the path, to reflect the fact that all our content is part of the guardian.co.uk site. A couple of keywords also appear in the URL, partly to convey what the content is about, and partly to distinguish it from any other content that appeared in the same section on the same day. As it happens, quite a lot was written about Tony Blair and Labour on that day, so the “3” adds a further unique identifier.

In some ways URL structure is an obscure topic, but getting it right opens up access to more of our content in more ways to more people. Further examples have been given by Matt McAlister in his discussion of the RSS feeds.

An ABC of R2: L is for legacy systems

…which needed to be removed if we were to be productive. Just adding shiny new things would only add to our workload if we didn’t also get rid of the old ones.

In fact the most significant legacy systems needed to be removed (or at least isolated) before we could even start R2. These included internal dependencies on an unsupported browser, a cumbersome (and imminently unsupported) application server, and a particularly insidious database table.

Eliminating these was not a glamorous job by external standards, but it was a necessary one. And significantly it allowed us to prove and improve our Agile skills. Each project was more complicated than the one before. By the time we got to the last of these we believed we really could undertake the largest software project since the website began.

An ABC of R2: A is for article editor

…which is one of the first tools we built as part of the project. And it’s a lot more complex that most people expect, especially if their main exposure to a content management system is through blogging.

The main reason there’s so much to it is that there’s so much to the organisation it operates in. On a blog, if you write an article then you don’t need to worry about people finding it, because it will appear at the top of your homepage, which is more or less the only page that people will be watching if not the RSS feeds. Furthermore, a blog is really just a single stream of articles. So when you write a blog post you really only need to worry about the headline, the body text, and maybe an extract to entice people in — and you certainly don’t need to worry about the environment that the article appears in.

On a large website such as guardian.co.uk, by contrast, an article generally won’t appear on the homepage, and it has a less than 1% chance of appearing at the top of the homepage. But it will be linked from many other pages. It will also most likely be only one part of a whole mesh of other pieces of content, all touching on different aspects of that and much bigger stories (plural), so it’s important to link all these together. (This is part of what Lloyd Shepherd calls “slow news”.) Therefore the article editor has lots of boxes to fill in which ensure the article fits in properly with the rest of the site’s content. Keywords are central to link to and list it on appropriate subject pages; an alternative headline is essential to allow the article to be linked to from those subject pages and elsewhere (many newspaper headlines only make sense on the printed page with a particular layout and don’t translate to the web); a one- or two-sentence explainer is similarly important to go underneath the alternative headline.

And side from the importance of maximising cross-linking, there are other factors that explain the differences with an article editor for a blog post, but they are still all consequences of working in serious organisation. Rights issues mean the writer has to be identified exactly; design requirements will mean that different parts of the article (most obviously the standfirst) need to be separated so that they can be displayed differently; print publication times mean you’ll need to identify the print publication date and the web publication date separately; marketing decisions may mean you need to give a particular article a specific URL.

A while back, Martin Stabe lamented “Why can’t a newspaper CMS be as user-friendly as a blog?” I’m sure our article editor could be more user-friendly, but I doubt we’ll get it to be quite as user-friendly as many blog editors simply because it does a different and more complex job — a job in an organisation with hundreds of people and very sophisticated media and presentational requirements.