In general, and in both cases above, I would say less technology is better.

Less software

More software features can be better, but it comes at a price. More software means more staff to maintain it, or slower progress as new features are added onto a more complex system. Or both. That in turn means greater costs (and almost certainly decreasing profitability) and a loss of momentum in the market. Good architecture can help manage this (for example, see the Guardian’s micro-app framework), but it won’t solve it entirely. More features can also mean more complexity for users, and hence less engagement. At some point new features have to stop going in.

But less technology doesn’t necessarily mean less capability. To give one example: Guardian.co.uk has its own video player, but journalists can embed YouTube videos, too. Had we not have chosen to develop our own player (built on top of Brightcove’s technology) then the capability to put videos onto our website would remain.

(But not “no software”)

The Guardian software team, by the way, needs to enable not just video on web pages. If it did, then YouTube alone would be a sufficient solution. However, we also need to enable video distribution and syndication across a variety of platforms, so an in-house player fronting a well-managed back-end store is important. That also explains why “no technology” is also inappropriate, as I’ve said before — even though it looks like the logical conclusion of “less technology is better technology”.

Less hardware

Turning to the case of smartphones, what is the internal technology team to do? I disagree with the viewpoint of R “Ray” Wang on the HBR Blog Network. He seems to be saying that for the corporate technology function to remain relevant they must be able to bless consumer technologies through standardisation and commoditisation:

If IT slows down the business capability to innovate, the company will suffer as new business models emerge and infrastructure will fail to keep up. If business moves ahead of IT in technology, then the company fails because IT will spend years cleaning up technology messes. As technologies mature, IT should take over the commoditized technologies and drive efficiencies.

I think that means the IT department should be buying smartphones for everyone, getting a bulk discount, and then giving them standard configs. Unfortunately it’s exactly that approach that led us here in the first place. Once upon a time, just before smartphones, the IT function bought lots of BlackBerrys and gave them out to the staff. This worked really well for a long time, but of course not everyone got to have a BlackBerry and then smartphones overtook them, and everyone had a different smartphone that they liked and which wasn’t locked down. BlackBerrys-for-many-if-not-all became a burden that was seen to be holding the staff back. No wonder Ray says “We’ve yet to see many organizations succeed” in this.

Buying smartphones for everyone is a “more technology” solution, and causes problems.

Instead, I think Dan Tynan at InfoWorld has the right attitude when he says “bring your own device” is a reality, and the business has to embrace it. As one of their interviewees explains:

“Unless IT supports the devices and technologies users demand, the users will simply go around IT and use personal tech for business purposes [...] That is a much more dangerous situation from a security standpoint than supporting the consumer devices in the first place.”

I’m pleased to be working for an organisation which succeeds at this. You can use your smartphone to access your corporate mail as long as you adhere to certain reasonable security requirements. The company’s primary concern is managing the data.

Towards a less burdened future

As technology (software services like YouTube, and hardware like smartphones) becomes consumerised we in-house technologists have to question what we’re really here for. As Ray says, “For IT leaders accustomed to having control over corporate technology, this represents a huge challenge”. But the answer is not to find better ways of controlling what we used to control — it’s to verify what we should be enabling…

The enterprise tech team needs to be enabling secure e-mail access, and if it can do that without having to manage hundreds of devices then great. Similarly, as we saw above, the Guardian’s software team needs to enable distribution and syndication beyond the desktop web browser, so an in-house video player results. But YouTube is still an option — if it’s more convenient for journalists whose video content doesn’t need those additional channels then it’s a fine, lightweight option; and if those extra channels were never needed then we’d have been better off without the player and accompanying infrastructure.

Overall, having less technology is better.

Social decisions in software architecture

The other day in conversation about team structures the Guardian’s lead software architect, Mat Wall, mentioned that architecture is social. This is a good, and often disregarded, observation. In that context he meant that architectural decisions influence who works with who, what issues they need to sort out together, how they regard their work, and so on. Three examples…

The Guardian has an API that allows access to its content. That means that when developing an iPhone application the iOS team can work much more closely together, requiring less information about the guts of the back-end, and become more focused on the user experience.

Consider also the team embracing Scala. Our platform lead Graham Tackley has gone on record to say that one consequence of this is that it has “reinvigorated the team”. That’s not the only reason to take up a technology, but it is a significant consideration.

Third, I’ll relate the story of one of our recent projects. The team devised four candidate architectures and needed to choose between them. The first decision was to make the call in conjunction with the business owner. The second decision was for them to ask not “which architecture do you prefer?” but rather “what do you want your business to look like?” Because each architecture had a different impact on the end users inside and outside the company: certain things would be difficult, others would be easy, etc.

Social decisions in algorithms

And it’s not just about architecture and software design. It’s true also of the more abstract matter of algorithms, as seen by a couple of examples from the masters (or should that be “slaves”?) of the algorithm: Google.

The most recent example concerns Google+. Rocky Agrawal wrote:

I finally blocked Robert Scoble in Google+. I have absolutely nothing against Scoble. I quite admire him, actually. He’s a great asset to the startup scene and he works damn hard. I’ve met him a few times and I’m sure we’ll meet again. But he was just getting to be way too much.

My Google+ feed was dominated by him. I tried to take a half-step and just remove Scoble from my circles. But then he became Google’s perpetual #1 suggestion for a new friend.

Google have created an algorithm which provides recommendations, and of course highly-referenced people will be recommended more than others. But Robert Scoble’s star power is clearly so significant it’s ended up disrupting Rocky’s experience, and getting in the way of him using the service effectively. Clearly the algorithm needs to be tweaked for the optimum user experience, and how that is done is entirely at the discretion of Google engineers. They could cap the influence of exceptional stars like Scoble if they wanted. They have choices, and ultimately those choices are based on social, human instincts.

The second Google example comes from the spat earlier this year between the Google search team and Bing. Google noticed that Bing was copying some of its search results; they demonstrated this by deliberately generating irrelevant Google search results for particular nonsense queries, and then observing that the same results appeared in Bing for the same queries; Microsoft said, sure, “We use multiple signals and approaches”, including observing what search results people click on, even if those clicks happen to be by Google engineeers conducting a sting operation; Google cried foul; the world moved on.

What was happening was that the Bing toolbar was tracking search results that the user clicked on. If a user had the Bing toolbar installed, did a Google search, and clicked on a link, then the toolbar would send a message back to Bing HQ saying “Hey, someone thought this link was good for this query” and Bing would consider that next time it needed to respond to the same query.

I found the whole episode quite amusing and slightly baffling, most notably these rather indignant words on the Official Google Blog:

At Google we strongly believe in innovation and are proud of our search quality. We’ve invested thousands of person-years into developing our search algorithms because we want our users to get the right answer every time they search, and that’s not easy. We look forward to competing with genuinely new search algorithms out there—algorithms built on core innovation, and not on recycled search results from a competitor.

To be fair, I think that post was written before Google knew exactly how Bing was doing its job, but even so, they jumped to conclusions. My personal reading of the above is that “pure” algorithms using only digital, abstract data are good and acceptable, while algorithms that use human data and feedback from a wide variety of sources are cheating and bad.

Unfortunately that’s just wrong. Google may like to say they don’t editorialise and their search results are purely machine-generated. But what they choose to feed into their algorithm, and how they weight those things, is an entirely human affair influenced by very human traits: culture, prejudice, politics and more. That’s not a critique of Google or its staff; it’s a critique of human beings.

Jonathan Stray has made this point, too. He said “It’s impossible to build a computer system that helps people find or filter information without at some point making editorial judgements.” And he quoted Matt Cutts of Google: “In some sense when people come to Google, that’s exactly what they’re asking for — our editorial judgment. They’re expressed via algorithms.”

So however “pure” we think a technical decision might be, ultimately it’s not. Whether it’s software architecture, algorithms, or something else, it’s all influenced by and influences our social world.

We think software is soft, but that isn’t always true.

I was speaking recently to a friend with a large system he was struggling to scale. I was saying while the Guardian does have to spend tangible time (and sometimes money) to ensure its systems scale, we do tend to deal with the issues successfully. But that wasn’t his experience, to the extent he was considering tearing his system down and starting again.

We often think we can reconfigure and rework software, to adapt it to new circumstances. And often we’re correct. This is even the essence of agile development: get something useful out there, learn, and make it better.

But we need to think slightly differently when we’re dealing with proprietary software, as was at the heart of my friend’s system. It may be configurable, but it’s never as malleable as something open source or self-made. We’ll be working with the soft squishy stuff, then suddenly come to a hard black box which won’t flex. It’s like finding a piece of Lego while rolling out a ball of plasticine.

So there are different risks around using proprietary systems compared to more open ones. Proprietary systems allow us to do some things faster, and other things not at all. This doesn’t mean we should avoid them but it does mean we need to manage our projects accordingly. Activities need to be prioritised accordingly, and that may involve addressing earlier potential requirements which would lead to impossible changes. Thus it can have implications for project planning and (if it may mean changing our business-level expectations) governance.

Maybe you don’t need a roadmap

So those are all respectable reasons to have a technology roadmap. But there are also respectable reasons to not have a roadmap.

To start with, not all companies have a clear strategy which determines where it aims to be in the medium or long term. If our company is like this then we can’t produce an aligned roadmap.

(By the way, that doesn’t mean our company doesn’t have a clear driver. Steve Blank says that “the purpose of a startup is the search for a business model (not execution.)” So for startups at least it’s possible to have a strategy without it leading to a technology roadmap.)

Next, we might think about a directive roadmap: “my company doesn’t have a strategy or direction, so I’ll create opportunities and drive out a strategy or direction from that”. But as we saw before, that has risks associated with it, and ultimately the risks might not be worth the reward.

Then, we might recognise that our department is in quite a healthy organisational position, so doesn’t need to create a roadmap for defensive reasons. Good for us. No point making grand plans for the sake of it.

And finally, if no-one outside our team is demand a roadmap for its own sake we aren’t looking to create a one to satisfy bureaucracy.

So what are we left with? Not much material from which to create a legitimate roadmap. Now there are two options.

Option one: forget it, and don’t worry. We don’t have a grand plan, but it’s not as if there’s no work to do. There are still projects, bugfixing and general consultancy that needs to happen. And as long we don’t feel the need for structure in every aspect of our lives then we will find nobility in honest labour. In fact, most people’s work is like this.

Option two is what I’m going to call “lean roadmapping”. Just because we can’t produce a roadmap with several milestones it doesn’t follow that we can’t decide what the single next milestone is going to be. A plan with only one milestone is hardly a roadmap, but it’s certainly more than operational or project delivery.

As an example, we might see that the company is generally increasing its global sales, or that it’s broadly shifting to more corporate clients, or that customer responsiveness is tending to become more central to the business. There’s no explicit strategy, but there is a general direction nevertheless. It’s this kind of thing that we can respond to. Increasing global sales may mean increasing support for global working. More corporate clients may require a shift to a more robust infrastructure. Customer responsiveness may mean better integration of CRM-type systems.

By spotting a general direction we can work out our next milestone. In fact, it would be wrong to work out any further milestones: there isn’t a clear strategy and it would be an expensive mistake to act as if there was. We might line up a number of candidates for what might follow, but we don’t need to make a commitment immediately.

Lean roadmapping moves us forward strategically, beyond what a project or daily operations might do, but it does so without requiring wasteful planning or doomed prophesising. Only when we reach the end of this milestone will we determine the next one, and only then by reassessing the business context as it is at the time.

So in summary: There are a few reasons we might usefully create a technology roadmap, ranging from the not-so-great to the excellent. But while having a roadmap for the right reasons is a good thing, the best business solution may be just to plan one milestone at a time to honestly reflect the realities of an uncertain future.

Introduction

Let’s first make sure we know what we’re talking about. A roadmap is a series of milestones to achieve some end; a technology roadmap is a roadmap for the technology base of an organisation.

Simple examples might be: we will replace the following legacy systems so that we can respond more quickly to our users’ demands; we will move our non-core systems into the cloud, in this sequence; we will implement the following technologies so that we can operate our distribution system globally.

In reality such a roadmap is likely to be more complex, with several different elements.

People get very excited about roadmaps. If you have two departments about to spend your time and money, and one has a roadmap and the other doesn’t, then the one without is going to look pretty shabby. And that’s even without looking at the content of the roadmap…

Any roadmap has some positives

Now, setting aside for one moment the benefit that any particular roadmap or plan might or might not give us, any technology roadmap clearly has some positives. It demonstrates purpose to those outside it (“They’re on a mission…”) even if those outside don’t fully understand it (“…and I guess I have to trust them on that”). It gives purpose to those implementing it (“I have to go in today despite being sick; there’s a key deadline for our roadmap”). It shuts down any number of unwanted conversations (“Can you do this for me?” / “Sorry, we’re maxed out delivering the roadmap, and you know the board’s expecting results”).

Obviously this is an overly cynical perspective, but like I said, that’s setting aside the benefits of the roadmap itself. So roadmaps are good, and have benefits even if they don’t actually deliver any real value themselves.

But really and truly a roadmap does need to have a purpose, and (because technology for its own sake has no benefit) that purpose must be driven by something outside technology. Here are four suggestions of where that purpose might come from, in order of increasing maturity:

4. The bureaucratic roadmap

Here, a roadmap is created purely because corporate bureaucracy requires it. Maybe it’s budget time and your budget won’t be approved unless it looks like there’s a grand plan. Maybe the CEO — or the VCs who are bankrolling your startup — want to see things mapped out before they’ll give you the time of day.

This scenario is not much better (possibly worse) than the cynical reasons given above. Except this time it’s not you who is made to feel better for applying some arbitrary order to daily pressures, but the corporate powers that be.

3. The defensive roadmap

This is where a tech team realises it must do something for itself to head off something less palatable. An example might be integrating a corporately-approved web service, such as Basecamp, before too many staff sign up to similar services themselves, directly, and scatter company data all over the Internet. Another is modernising the mail system ahead of a merger with a larger company to avoid being utterly taken over by the larger company’s technology base.

Clearly it’s not great to be on the defensive, but at least we’re responding.

2. The directive roadmap

This is where a tech team recognises the company doesn’t have a particular direction, so it implements something in order to create one. An example here might be enhancing all the systems’ robustness and security because there’s an opportunity to shift from a consumer to a corporate clientbase, and the tech team realises this would be the way to open up those kind of opportunities.

The directive approach can really only be instigated by the most senior one or two people in the tech team. Also, it’s a bit risky because it could be called into question if there’s a lack of trust or change of leadership. The IT director could find themselves in a difficult situation if their peers suddenly thought they were running an expensive project which wasn’t aligned with any current company purpose. But if it works then it’s a good result: a previously directionless company gets a new lease of life. Or at least new opportunities, whether or not it chooses to exploit them.

1. The aligned roadmap

Almost certainly the highest form of technology roadmap. In this situation the organisation has a clear strategy which becomes the basis of every other department’s own, localised strategy — the the technology department is one of those.

If a company decides it needs to increase its overseas sales then there are implications for distribution, legal, advertising, procurement, and of course technology. Even without a single co-ordinated programme to achieve a specific goal every department suddenly has a purpose, each one can work out for itself what it needs to achieve that, and all departments are working to achieve the same thing.

That this is highest form of technology roadmap should not be surprising. It’s born of a very high level of organisational maturity which may seem easy but isn’t in practice. Just recall the story from eBay where the company had a long-term strategy to emphasize fixed-priced goods, but they managed to forget to tell the technology team.

In the next part of this article I’ll consider reasons why it might actually be better not to have a roadmap…

The programme is a straight discussion of business issues with Evan Davis and a number of business leaders. It’s not dumbed down (any more than it has to be edited down) and is a consistently excellent listen. Last week the guests were Luke Johnson, chairman of Risk Capital Partners, Vincent de Rivaz, chief executive of EDF Energy, and Jacqueline de Rojas, UK and Ireland vice president of McAfee. And they dealt with two major topics: organic growth versus acquisitions, and problems with IT. Luke Johnson in particular seemed to really have problems with technology, and to me there was overwhelming evidence that some of the causes might be rather closer to home than he might like to think.

Coming up is some criticism of what Luke Johnson and (to a much lesser extent) Vincent de Rivaz say, but don’t mistake it for a criticism of them as individuals. The point of what follows is that their attitude is representative of so many board leaders. Even though there are many board leaders who have a very confident grasp of technology issues it’s still remarkable there are so many others who handle technology as if they were wearing boxing gloves.

Problems with technology

As the resident IT expert it was Jacqueline de Rojas who was asked first about her own experience of IT problems, and in particular the bad software update that her company issued in April which disabled many of its customers’ computers. Notably, she responded to a technology-related question with a positive and human answer: “The key in a crisis like that is how you respond; that’s how you’re judged”. Then Vincent de Rivaz was asked if he had ever had any IT disasters, and before he could finish his response (along the lines of “no actual disasters, but we have had problems”) Luke Johnson cut in:

LJ: “Every business I’ve ever come across has always been having problems with [its IT].”

VdR: “We have to invest in our IT and we have to develop our IT systems which we are doing at the moment [with a new online system to help our customers with their bills.]

ED: “What sort of problems have you had with your IT?”

VdR: “The problem is to control the costs. That is the permanent challenge for me, to be sure that we are spending the right amount of money on the IT systems.”

LJ: “All my experience of IT systems is they are always overdue and over budget, and I think it’s a built-in part of the IT industry that they deliberately under-quote and then they may get up on the extras.”

Now there are many problems board leaders could have with IT: We aren’t getting the best out of our technology investments. Our technology portfolio isn’t broad enough to help us in a constantly-shifting market. We don’t have the people who really understand how to manage and change our technology infrastructure.

These are all problems that require board-level investment and oversight. And they are also all questions that, if you substitute “financial” for “technology” all boards do deal with. But sadly it’s okay to ignore those matters if they’re technological, and so the top-of-mind IT issues for our token CEO and chairman are (1) cost control, and (2) IT people are sharks.

Indeed, Luke Johnson seems to have something of a chip on his shoulder about technologists. Jacqueline de Rojas goes on to explain a bit about the difficulty of  turning “woolly and aspirational” IT demands into concrete deliverables, but that doesn’t stop him:

LJ: “I think the IT experts blind clients with science, and I think very often they themselves overestimate the benefits, and overcharge for what they do. And the fact is there isn’t another industry, except perhaps pharmaceuticals, that makes the returns on capital and margins that software does. Huge margins.”

JdR: “The level of investment we have to make to stay ahead technologically is also huge. So where do you think that money goes? It goes right back into making sure we stay one step ahead of — in our case — the hackers and cyberterrorists and the competition”

ED: “Let me challenge you, Luke. Is there something different about IT from other areas of company purchasing? Is IT unique in being difficult and awkward for a company?”

LJ: “No, I don’t think it’s unique, but I think it is mission critical, and unquestionably we are all more dependent on IT than ever. I accept all that, and it can deliver huge values, and the very nature of it is it’s constantly evolving. But the truth of the matter is, what other sort of project or product, rather, do you buy that requires these perpetual upgrades and offers such super returns? I was involved with a retailer once, Whittard of Chelsea. We spent, many years ago, the best part of half a year’s profits on building a fully-integrated website. This was in the early days of the web. Within two years we had to throw it away. Because it just didn’t work. We used the wrong suppliers and we made a lot of mistakes.”

To him IT is a homogenous  blob, and he transfers his anxieties freely between commodity software sales (“returns on capital and margins… perpetual upgrades”) and individual business change projects (“a fully-integrated website”). Maybe he’s just exploiting the opportunity to have a go directly at a software vendor, because when Jacqueline de Rojas highlights how technology change really does improve our lives he’s having none of it:

LJ: “What are the cost of goods in software? What are the cost of goods? How much does each new program cost Microsoft? Zero.”

Which is silly, because he knows perfectly well that software (like pharmaceuticals) costs a lot of money to create, and Jacqueline de Rojas has already explained the cycle of investment, return, and re-investment that her market demands.

Where the problems lie

I’m not for a second denying that Luke Johnson’s frustration is unfounded. It’s clearly borne from many hard experiences: this is a man who’s job is to cast his net widely across many, many companies, to look into them seriously, and having done this he finds that “every business [he's] ever come across has always been having problems with [its IT]” and he’s witnessed one of his companies building one costly system only to have to replace it unexpectedly early.

But from where I sit a lot of his problems are of his own making. If you treat IT primarily as a cost then the best you’ll achieve with it is cost control, whereas the best you should get out of it is business transformation. If you think IT contractors under-quote on the basics and expect to make up for it on the extras, then you probably want to structure the next deal differently and be prepared to make it more of a partnership — if you can be confident of being able to act as a partner. If you think IT experts blind you with science then you’re talking to the wrong people — and if you don’t know who the right people are then you need to start filling that gap in your personal network. If you think IT experts overcharge then you’ve created a relationship with conflict at its core and the best you’ll get out of them is the minimum they’re contractually obliged to deliver, whereas you should be getting them to apply their creativity and rigour to help propel your business forward.

The thing that’s really shocking to me is that there are many company leaders like this, who are unashamed of their blunt handling of technology — which is undeniably mission critical — and yet the same people wouldn’t stay a week in their jobs if they demonstrated the same disregard for, say, legal matters or finance.

Which is rather ironic, because earlier in the programme it was Luke Johnson who was called on to be the resident expert on organic growth versus acquisitions, and one insight he provided was this:

LJ: “Many studies suggest that even perhaps a majority of all acquisitions fail to deliver shareholder value. So it’s important to put it into perspective and say if we can achieve returns through organic growth, but perhaps a little more slowly, then maybe that is the more sensible way to proceed.”

Let’s take a second to understand what we’ve just heard: possibly more than half of acquisitions fail to deliver value to the owners of the business — which sounds very similar to his problems with IT. And acquisitions are a significant part of his company’s business — mission critical, even. Just like IT. But in that conversation he wasn’t remotely upset or bitter about this, he just treated it as something to deal with. He could do that because he was comfortable with working in that sphere and, I suspect, because while he might have had his share of acquisition failures they will have been outweighed by his successes.

It seems to me that if board leaders were obliged to grasp technology to the same degree as they are obliged to deal with financial or legal matters then their companies — and their companies’ technology — would be in a much better place.

Two final things

A couple of parting thoughts.

First, some not-at-all-serious observations about Risk Capital Partners. While confirming that indeed they don’t invest in any IT or software companies, I found they do have an interest  in InterQuest, “a fast growing IT recruitment business”. So if Luke Johnson is concerned about the high cost of IT professionals then he might want to have a word with them. Also, I couldn’t help but think if software really is the fantastic high-margin business he thinks it is then RCP really should start making some investments there. I don’t know if they really will make a lot of money, but I do know they’ll learn a huge amount.

Second, a much more serious note. Earlier I said that if these sorts of leaders had IT problems then some of the causes were close to home. But it’s also true that some of the causes lie with us technology professionals. If technology really is seen above all else as a cost to be controlled, and if we are finding ourselves in unbalanced, conflict-driven relationships, then we really do need to do a much, much better job at explaining what we do. And then we have to carry that through into our actions. Technology in business is about making the business more effective and people’s working lives better. We ought to be able to find routes to even the most sceptical business leaders to explain that, and get a positive reception.

The Eigenharp and the Open Platform

The Eigenharp is a new kind of device. It’s a musical instrument with several patents already filed, and incorporates 132 keys with three-colour LEDs and a breath pipe. The keys operate in three dimensions (so you can exploit, say, pressure, pitch bend and filter effects with one finger) and are sensitive to within one micron. So while it’s compatible with MIDI, that’s an incredibly out-of-date protocol compared to what you could do with it.

The Open Platform has been well-documented here and elsewhere: a full content API directly into the Guardian’s content database, plus a store of raw data on which our journalists base some of their stories.

Although they are clearly two different kinds of beasts a remarkable similarity struck me in that they are both not just new products, but really new products. There is no clearly defined place for them in either of their markets, and no predictable success path for either one. The success of each is dependent entirely on other people’s innovation. The Eigenharp’s success is dependent on musicians taking their creativity into new directions — no-one to date has written music for an Eigenharp. The Open Platform’s success is dependent on developers doing innovative things with the new data they have access to.

So how do you launch a product that’s really new? The similar steps I can see are…

1. The controlled buzz

Eigenlabs (the creators of the Eigenharp) and the Guardian (creators of the Open Platform) managed to seed a few select people with their product — musicians and developers respectively — and some small details leaked out. This clearly did a couple of things.

First, it helps you get the product right. I know from my conversations around the Open Platform that what might seem a good idea to one or two people close to the product can provoke strong negative reactions when you float it past an informed outsider. When you’re inside the company you can easily think too much and lose the big picture, so it’s good to get an informed but independent view before you go public.

Second,  creating a buzz protects you from a bit of unforeseen negative reaction — if people are excited about your product and want it to succeed they will forgive some minor mistakes. You can see Eigenlabs achieved this with this low-fi video of two guys playing the James Bond/Moby theme on YouTube. Eigenlabs didn’t create this themselves, but they let “close friends” do it, and you can see the reaction in the comments underneath: “What the hell!! This is awesome”, “where can I buy one of these?”, “dear god this is epic” and so on.

The benefits of this are well explained by Lance Ulanoff at PCMag.com. Here he compares the lacklustre launch of the Motorola Cliq with the hype from the January 2009 announcement (six months before the launch) of the Palm Pre:

Speaking of Palm, it has a good bit in common with Motorola right now. The Pre (and now Pixi) is its hail-Mary pass. If the new phone and WebOS platform fail, Palm will be done. Back in January, however, Palm ran the best product rollout event I have ever attended. It perfectly conveyed the company’s excitement and all that is good about the Pre. I remember tweeting the event with mounting excitement. Today, I tweeted the Motorola event with mounting confusion.

What Palm did in January was give the Pre a good hard shove off the shore. Those waves of excitement produced a good six months of positive press. Eventually, Palm and the Pre hit some rough waters—a shipping delay, the slow delivery of the SDK and tiny list of apps that has yet to grow. Still, that first day set the tone.

Then after the buzz is…

2. Openness at launch

There’s much that doesn’t need to be said about the launch event, but one thing that struck me about both the launch of the Eigenharp and the launch of the Open Platform was a common ethos: the honesty and openness of the presenters, and the openness of the products.

At both events the “real users” (musicians/developers) were on-stage demonstrating their early adoption of the product and open to questions from the audience. These are the kinds of people who most marketing and PR people would lock in cupboards at a press event for fear they arrive without the regulation press-on smile and go off-message. But when a product’s success relies on innovation and creativity from its real users then it’s important that intended audience hears an authentic voice.

Also, there is an openness about both products. Aside from the very concept of opening up content and data, the Open Platform gives a lot of latitude to developers, including (unusually) commercial use. Meanwhile Eigenlabs are open-sourcing their software. Again, because each product’s success relies on people taking it in unexpected directions it’s important for there to be as many opportunities as possible for that to happen. Otherwise its success is stifled.

And then finally there’s…

3. The follow-through

Well, it’s early days for both products. The Open Platform API is still in beta, but you can see some developments now, such as the launch last week of the Applications Gallery. It’s even earlier for the Eigenharp. Company founder John Lambert was asked what well-known musicians he’d like to see using his creation and he said that a number of high profile people do have the instrument but that he cannot name any names yet. Clearly some really interesting things are going to happen some time soon.

If a product really does depend on the creativity of others then you can’t — and mustn’t — be too controlling of what those people do with it. (When I spoke to him John seemed slightly apprehensive about the quality of some of the things people might produce around the Eigenharp; I bet Matt McAlister didn’t anticipate a swearword tracker coming out of the Open Platform.) But if you can’t control what others do, you can at least show some examples of what might be achieved, and that’s what the follow-through is doing with each of these products.

1-2-3, easy as A-B-C

Now if any marketing people have stumbled across this blog post I wouldn’t be surprised if they were horrified by the naivety of these observations — all of this might just be first grade A-B-C of marketing for them. But I find this fascinating. As a tech person it’s very easy for me to focus on my own role and not spend much time wondering what challenges are faced by my colleagues in other departments. Here are two products whose paths to success are unusually dependent on the unknown and whose stories will be well worth watching. By looking at the communication that’s gone on around them the significance of other people’s role becomes much more apparent. Technology success is about much more than successful technology.

In this article:

• What is JavaFX?
• The language (good)
• Development support (ugly)
• The user experience (bad)
• Conclusions and (inevitable) comparison with Flash

What is JavaFX?

JavaFX is probably best viewed as Sun’s answer to Flash, just as Silverlight is Microsoft’s response. So it’s a way of developing rich internet applications, or embedding fancy graphical thingies into web pages.

In some sense Sun might not have had any need for JavaFX. For example, take a look at We Feel Fine. This is an astonishingly beautiful and imaginative application, and though it might look like it’s been written as Flash it’s actually a Java applet.

So why did Sun feel the need for adding to Java? According to JavaFX’s creator, Chris Oliver, “its purpose was to explore making GUI programming easier in general”. In other words, the Java language is not really optimised for GUI development. In addition, the language needs a number of libraries. But all of this compiles down to Java bytecode and runs on the JVM, so JavaFX is really just Java with special add-ons to make graphical-oriented applications easier.

The language (good)

The most obvious initial difference with JavaFX is the language that comes with it: JavaFX Script. As implausible as it sounds Sun has combined the best bits of Javascript and Java, and come up with something really rather wonderful. Consider the following code snippet from my application:

ring = Circle {
    centerX: bind bubble.x
    centerY: bind bubble.y
    radius: bubble.diameterInScene / 2
    opacity: 0.0
    fill: null
    stroke: BUBBLE_INDICATOR_COLOUR
};

This creates a thin ring, which is actually just a circle with an outline and no fill. It starts off being invisible (opacity 0). The variable ring was declared on a previous line and is initialised here. I think this simple piece of code demonstrates a number of really nice things about the language. First, the code Circle { ... };. This is a constructor, which I really like. It feels lightweight, like Javascript, and it’s got Javascript’s named properties. But like Java, JavaFX Script is a statically typed language, so ring is of type Circle and if I forget this later on in my development then the compiler will signal an error — that error won’t make it into production. Second, the most exciting addition: the bind keyword. This means that the formula that follows will change dynamically when any of its component parts changes. So the x co-ordinate of the ring’s centre is bound to bubble.x — the x property of the variable bubble, which is the x co-ordinate of its centre. Similarly for the y co-ordinates. Now I know that the ring will always be centred on the bubble: whenever I update the bubble’s position the ring’s position will change, too. As one of our team’s developers explained when I started telling him about JavaFX binding (in fact he didn’t even let me finish my sentence) it’s just perfect for a GUI-oriented language where you want the view to track the model. Another great feature of the language is its native handling of timelines. For example, here’s a piece of code I use to fade something in or out over a period of half a second:

Timeline {
    repeatCount: 1
    keyFrames: [
        at (0.5s) { this.opacity => targetOpacity }
    ]
}.play();

This constructs a Timeline with a single keyframe (at the half second point), and JavaFX is told that by that point the opacity needs to be at the target opacity. On calling the play() method it will interpolate the object’s opacity value so that it fades in or out accordingly. Needless to say, you can have several keyframes, and interpolate several values simultaneously or in sequence.

You can also ensure specific actions at certain points. Here is some code which fades a message out, changes it, and fades it back in again. This mixes native timeline syntax with more usual JavaFX syntax:

Timeline {
    repeatCount: 1
    keyFrames: [
        at (0.5s) { barText.opacity => 0.0 }
        KeyFrame {
            time: 0.5s
            action: function():Void { barText.content = message; }
        }
        at (1.0s) { barText.opacity => 1.0 }
    ]
}.play();

You might also notice from this that functions are treated as a first class object: another very handy feature of the language.

Development support (ugly)

As you might expect from a Java extension JavaFX integrates well with Java: you can use any Java class or object in your JavaFX code. In my case I wrote a small physics engine in Java and then accessed it via my JavaFX code; it was trivial to do.

On the other hand, JavaFX classes and methods can’t be accessed from Java without a lot of jumping through hoops (you have to create an interface in Java, implement it in JavaFX, and then you can reference that interface in your Java code). Also, Java generics aren’t available in JavaFX, which is a bit of loss.

The JavaFX platform itself is evolving. Version 1.0 was released in December 2008. Version 1.1 came out in February 2009 and changed not only the underlying libraries, but also the language, so even your source code needed be revised. Most recently version 1.2 came out in May 2009, with similar changes and similar consequences.

Overall, these changes are a very good thing. Each new version contains not only extra front-end features (charting capabilities appeared in 1.2) but also performance enhancements and changes which ensure greater API consistency and extensibility. It’s interesting to see Sun take a different approach from Java: there the remit has been to preserve backwards compatibility at all costs, because there are so many applications deployed in it, but JavaFX is still in its infancy and breaking backwards compability is a significantly lesser evil than consistency and extensibility. Nevertheless, the feeling is of a platform that’s yet to mature, and my productivity was knocked back by one or two platform bugs whose equivalents just wouldn’t be be dreamt of in, say, Java. Still, JavaFX is clearly actively evolving, and most certainly for the better.

More disappointing is the poor IDE support. Inevitably, support for Sun’s NetBeans is being actively pursued alongside the evolving platform. But my first experience with JavaFX in NetBeans earlier in 2009 found it to be an imperfect environment, and I opted to switch to Eclipse — I decided I could live with more quirks in return for a more familiar working environment. And after some considerable hassle I got it working.

But Eclipse support really is poor: there are a few annoyances documented over on pliq.com, and I found a few more. Overall it meant that even typing code was a bit of a minefield, and that’s without refactoring support.

There is also the inevitable problem with unit testing graphically-oriented applications. I admit I resorted to the old fashioned develop-debug cycle, which was a little disappointing and inevitably frustrating. But I can’t quite conceive of a unit test framework for a platform in which there is so much visually-oriented code. I suspect it would mean a slightly different approach to coding. My brain can’t quite work out what, though.

The user experience (bad)

So developers have problems. Let’s pretend not to worry about them; their entire lives revolve around solving problems so they’ll find ways to solve those. Much more important is the experience of end users, and with Sun boasting that Java runs on “800 million desktops” (isn’t that more than the number of atoms in the universe…?) those people’s experience is paramount.

Well, if you’re running the right environment then the JavaFX experience isn’t at all bad. Unfortunately you’re very unlikely to be running in the right environment…

Sun have done an excellent job in making it simple and reliable to embed your applet into a web page: once you’ve configured your code you only need some very simple and clear JavaScript in your page. This JavaScript takes care of platform versioning, prompts the user for an update if necessary, shows a standard “Loading…” animation, and downloads and runs your code.

In the circumstances this clever; but the circumstances are far from ideal. For example, when did you last update your JVM? Probably not recently enough to have the latest JavaFX release, so your first use of a 1.2 application is going to prompt you for a very large download of the JavaFX 1.2 runtime. You’re also going to need to run a launcher applet from Sun, which then prompts you with a popup requesting you trust software from Sun. Anyone who wants a seamless user experience is going to take one look at JavaFX and run a mile.

Even if you’ve already got the 1.2 runtime and launcher applet the JavaScript embed code itself references a script on Sun’s website. That means your web page is always relying on Sun’s website being up, running and performant in order to load. For code on my own little blog that’s fine — Sun’s uptime will always be more reliable than mine. But if this is to be adopted by bigger players, such as my own employer, the Guardian, then the sysadmins responsible for page delivery time and reliability won’t be happy with another third party dependency.

And if you’re not impressed with that, then I hope you’re not an Apple user. Java support for Macs has long lagged behind that of Windows machines (I recall the Java 5 SDK came out over a year after the Windows release) and this shows again with JavaFX. In theory JavaFX runs on a Mac. In practice it’s a world of pain: it’s slow and rendering features such as antialiasing are dire. Experiments show that if you’re running a very high end Mac then it’s half-way approaching decent, but not actually decent.

As someone who spent a long time developing this on a Windows machine it was very disspiriting to see the fruits of my labour reduced to a syrupy judder when I wanted to demo it to someone with a Mac.

Conclusions and (inevitable) comparison with Flash

If Flash didn’t exist then my view of JavaFX would be different. But Flash does exist, it’s the dominant RIA platform — certainly within web pages — and so any view of JavaFX has to be assessed in that context.

I find it very amusing that Adobe and Sun have eyed up each others’ territory and each wanted a piece of the other’s action. Adobe has wanted to move out of the web page and onto the desktop, so they’ve introduced AIR, while ActionScript has gone all serious, ditching the JavaScript-like version 2 for the Java-like version 3, and encouraging a cavalcade of grown-up design patterns. Meanwhile Sun has left behind that heavyweight baggage of the Java programming language and gone all JavaScripty, with easy constructor syntax, type inference and treating functions as first-class objects. They’ve each adopted the other’s clothes, but changing clothes is only part of what’s needed.

For my money JavaFX Script is the superior language. It’s designed for graphics creation and management, while ActionScript looks like Java before generics. The code I’ve found online to draw shapes in ActionScript brings back nightmares from the early days of writing Java applets: get a graphics object, set some properties, start the filling, do the drawing, stop the filling… it’s unintuitive, uphill work. (What the hell is a “graphics object”, anyway?) With JavaFX you use a constructor like the one shown above.

Mind you, there’s a reason for this difference. Flash development is predicated on the use of some really well-developed proprietary, commercial tools to do the job: Flash Professional for designers and Flex Builder for application developers. Looking for an ActionScript Hello World program produces some entertaining results. Here’s one conversation in which the request for a Hello World app is met with incomprehension: can’t you just drop the text on the stage? asks one responder, who misses the point of a programming tutorial; I suppose you could use the trace function to print the text out, suggests someone else. Here’s an ActionScript Hello World tutorial which includes the instruction “Select the Text Tool…”. There is a purely programmatic tutorial which Andrew Payne created because “The other examples I could find all used Adobe’s paid tools, Flash Designer, etc.” — but that was written only in January 2009.

So Flash is deeply embedded into design houses, which have come to rely on some very mature tools in which programming is secondary, particularly when it comes to asset creation. And while this might make more traditional developers smile it does demonstrate the cultural chasm that seperates Flash from JavaFX. Designers demand a first rate user experience, or else their work won’t sell itself. That means Flash execution doesn’t suffer the popup bombardment of JavaFX. And it means Flash runs just as well on a Mac as on a PC. And it means Flash developers have become obsessed with performance in a way that JavaFX developers can’t even imagine, so there’s an entire mindset which has yet to be addressed. And it means component support in JavaFX is lacking where in Flash it’s commonplace. One example of this is the lack of a JavaFX preloader: with Flash it’s standard to have a progress bar as the application loads; with JavaFX the standard preloader is an animated image that continues to animate even if there’s a background error, and I can’t find any code which shows a progress bar. Another is the ease in which one Flash swf can load another; again, not readily possible in JavaFX.

Designers would also bring a huge benefit to JavaFX beyond setting high bars for user experience: they are the people who will make JavaFX applications look beautiful, and so attract others’ interest, and so start a rolling snowball of adoption. I think my own application is quite pretty, but it was quite different in the early stages. It was only when I studied and tried to emulate the work of the Guardian’s professional designers that it started to come alive: circles were a friendlier shape than rounded rectangles, the palette of guardian.co.uk was infinitely prettier than my original palette of browns, and so on.

Sun is not unaware that this cultural gap needs to be bridged. That will be one reason why they’ve created JavaFX Production Suite. But years of cultural awareness and workflow embedded into design studios’ daily operations will be very, very hard to reverse.

So my own thoughts about JavaFX reflect my own experience with it. My development experience was very positive: it’s a great language and a mediocre development environment with many teething troubles which will no doubt be ironed out over time. I had terrific fun with it, and getting fast results was very rewarding. But beyond the development experience the future seems more difficult. The user experience is very poor, with JavaFX throwing up barriers in front of the non-technical end user where Flash is invisible. And the leap from niche developer interest to general acceptance by design studios is just enormous.

Making JavaFX a real success is huge job for Sun. They’ve got off to an excellent start, with a language and runtime environment that allow for very quick development. But that, alas, is the easy bit compared to what lies ahead. I do wish them the very best of luck.

I was quite thrilled last night to have solved a really frustrating bug in a piece of open source software. First the background, then the problem and finally, for the record, the solution.

Background

I’ve been messing around programming with with JavaFX and would have liked to have used my favourite IDE, Eclipse.

Unfortunately JavaFX has a bit of a poor pedigree. It’s a me-too product being Sun’s response to Adobe Flash and AIR, just as Silverlight is Microsoft’s response. It also doesn’t seem to have been given the heavy backing and attention within Sun that anyone would need going up against the incumbent RIA platform. I see this because (a) the language changed slightly after the first official release (1.0) to the next “minor” version, which smacks of carelessness (or pressures on dates of public announcements); (b) JavaFX application loading seems to be a variable experience, and certainly not as slick as Flash’s; (c) patchy documentation — for your reading pleasure I quote here Chapter 10 of the JavaFX language reference in its entirety:

Chapter 10. Animation

Table of contents

[To do: write chapter]

Added to that, Oracle has just bought Sun. Who knows what the new owners will think of this project… it’s a long way from their core business of providing large scale middleware solutions. It’s a shame, because JavaFX has a lot going for it, and as far as I can tell stands up well technically to Flash; but it needs investment in giving it a lot of polish if it’s to demonstrate this widely. Certainly there doesn’t seem to be a fully-featured IDE readily available.

Problem

However, a small team of independent developers wrote an open source Eclipse plugin which was released in December 2008, but the last commit was on 13 January 2009. Since that time JavaFX 1.1 was released in February 2009 which doesn’t play nicely with the Eclipse plugin, but no word from the maintainers on any new releases. It seems to be dormant.

The problem I (and many before me) found is bug 206, documented in bugzilla: when writing seemingly innocuous code the incremental compiler reports

incompatible types found   : Integer required: com.sun.javafx.runtime.location.FloatVariable

Which doesn’t read very well but I think means “I found an integer but requred a float”. The error is reported on about one line in five, swamping genuine error messages, and no amount of bracketing, casting or whatever seems to help. It’s clearly frustrated a lot of people and it certainly made me rather disappointed with the whole JavaFX experience.

Anyway, inspired by insights from some of those similarly affected, and after a lot of semi-random hacking and poking I found a solution, which at least works for me.

Solution

The solution is to build the plugin yourself from the source, and use that. The original documentation for this is a little terse (well, I struggled a couple of times) so here’s an augmented version:

How to build the plugins from sources

1. Requirements

• Install the Eclipse IDE for Java EE Developers from http://www.eclipse.org/downloads/.
• Install the DeltaPack from http://update.eclipse.org/downloads/drops/R-3.4-200806172000/index.php#DeltaPack. (I have no idea what this DeltaPack is, but I think it’s used in this instance as libraries against which the plugin sources are compiled. I unzipped the download and dropped the files into my Eclipse application directory as appropriate: features in the  features directory, plugins in the plugins directory.)
• Get the JavaFX SDK from Sun in the form of a zip file. Since it’s currently only available as an executable, you need to download and  install that in the usual way (if you haven’t done so already) and then zip it up into a file. I called my file javafx-sdk1.1.zip (though I’ve actually got version 1.1.1 — using only major and minor seems to be the convention).

2. Get the plugin source files

• On the Unix commandline the incantation is svn co https://kenai.com/svn/eplugin~subversion/trunk eplugin (though I’ve not checked this).
• Alternatively on Windows you can use TortoiseSVN.
• However my Eclipse was set up to use subversion so I imported the source directly into Eclipse:
• Import the files into Eclipse. In Eclipse I did File > New > Project… > SVN > Checkout Projects From SVN. Then use the repository location https://kenai.com/svn/eplugin~subversion/trunk and get the entire contents. I had to accept a few certificates (temporarily) along the way. One time I tried this the new Eclipse project showed lots of Java errors (libraries not found), but they didn’t seem to cause a problem.

3. Amend and then run the build

Running the build that came with the source files didn’t work, and I needed to amend the Ant build.xml file. I had to change the original line

<property name="javafx.sdk.root.dir" value="javafx-sdk1.0"/>

to

<property name="javafx.sdk.root.dir" value="javafx-sdk1.1"/>

That is,  I had to change the javafx.sdk.root.dir variable to match the name of my JavaFX SDK zip file, but without .zip bit.

Now you can run the build from within Eclipse. To do this I did: Right click the build.xml file in the Eclipse package navigator, choose Run As > Ant Build… Go to the Main tab, then the Arguments box and fill in these arguments

-DeclipseDir=<path> -Djavafx.sdk.build.url=<url_to_zip> clean build-windows [build-macosx] [build-linux]

Note that you need one file path and one URL, and you need to name the operating system you’re building for. This took a few minutes to run, but it completed without problems.

If you’re not doing this from inside Eclipse you can use Ant directly.

4. Install the results

The result of this is in a new dist directory. Here you will find a few files, but the one you’re interested in is the .zip file. In my case it was called com.sun.javafx.eclipse.feature-win32-win32-x86-1.0.1.zip. This is an Eclipse plugin. Unzip it and drop the contents into the relevant places in your Eclipse install directory: features in features, plugins in plugins.

I hope you find this goes relatively smoothly.

Why did this happen?

Honestly, you think I know? My guess is that the plugin was built with JavaFX 1.0.x embedded, and those of us with JavaFX 1.1.x get compatibility errors; recompiling from sources ensures your new plugin in built with your current JavaFX SDK. If so, that’s a pain, as it may mean you’ll need to do all this again when the next JavaFX SDK comes out. But, hey, you’ll get a few hours of happy coding done in the meantime.