Disrupting innovations: Linux and mobile phones

Thoughts based on Clayton M. Christensen, and his book “Innovator’s Solution”. The basic tenent is that product (or service) quality increases faster than the consumers’ ability to utilize it. This is true of most industries. This has two major consequences:

  1. It’s possible for a company to start with a poor product and develop it to be better than what’s necessary.
  2. When a product line overshoots (becomes better than needed), a flip in the value chain will happen.

The first consequence means that a startup firm with a new low-quality product (like Sony’s first transistor radios) should not attempt to immediately develop it to match existing alternate solutions, because the cost of development will be huge. Instead it should target the market that currently cannot use the existing solutions – even a low-quality product is better than none. As the product improves in time, it will start taking over the existing solutions’ market shares.

When a product is not good enough, it needs to be improved and optimized. That means that the product layers above and below must be modularized (or commoditized) for the optimization to be possible (to optimize a CPU, the motherboard needs to be modular, as well as the silicon chips and transistors that are used in CPUs). When the product overshoots, it will be commoditized and modularized, allowing for the layers above and below it to start optimization increasing quality (when mobile phone architectures become good enough (as they soon will), they will become modular, which allows the subsystems to be optimized, as well as the services built on top of the architecture).

The organizational changes in the value-chain change radically when a flip occurs as a product becomes modular. Profit is always best made in the layers that are being optimized – since they are not good enough yet, customers will pay for improved versions. The modular layers, however, do not gather great profits, since they are good enough and anybody can produce them.

What does all this have to do with Linux, then? Well, Linux is a modular and open OS, unlike Unix and Windows. Windows and Unix have provided good revenue to their makers, since they’ve been integrated products with good profit margins – the producers of the subsystems have had to build deoptimized products, to fit the proprietary OS. Also the producers of higher level applications have had to adapt their products to fit into the OS. Linux (and OpenSolaris as well) is now changing that playing field: a modular OS is allows its subcomponents to be optimized, and also allows the applications to be optimized to specific uses. Instead of the development happening in the OS, the OS is now the modular architecture, that allows improvement of the subcomponents (drivers and lower level services) as well as applications (building Linux-based customized solutions, such as the Nokia 770 Internet tablet).

Why can Linux do this? Because it is not targeting the users of Microsoft Windows (even if Microsoft likes to think Linux competes with it, it actually does not), but the users that Windows does not serve. On the one hand Linux is good for geeks that need extra control of the OSs, but more importantly, Linux is good for those who cannot afford Windows. There’s a couple of billion people in southern Asia that can’t afford Windows licenses. While a Linux desktop might have problems and not be as good as Windows, it’s better than nothing, which the users have had before Linux.

Another area where Linux is gaining is in the web server sector. There’s no way Linux can compete with mainframe OSs, but the web servers are a new area where the traditional OSs can’t cut it.

And because technology always evolves faster than the users’ ability to use it, Linux will catch up. In some years (maybe a decade), it will be good enough for the average consumer and for corporate mainframes. But right now the modular Linux platform allows the optimization of services (like Apache, MySQL, the GNU C compiler, and countless others) as well as applications (Google, Yahoo, eBay, Nokia 770, and many more).

Linux is a good example of a disrupting innovation – it will disrupt the existing products and services, forcing them to either flee away (to concentrate on their own high-fidelity customers that demand the best) or to disintegrate (become modular and allow competition to form in the adjacent layers). Disintegration is the inevitable outcome for older players, and since modularization means lesser profits, they also need to change tactics – move up the value chain, to be specific. Microsoft will need to start concentrating on services that can be used on top of any OS. There’s no money to be made in OSs in a few years, since Linux (which cannot generate revenue as it is free as in beer) will take over the OS market. Money will be made in producing and selling better subsystems and better applications (the adjacent layers). Whether or not Microsoft can make this transition pretty much decides its fate. And the companies that have been creating added value to a modular OS during all this time will have an advantage.

The same trend is also happening with mobile phones. Nokia is a strong player in mobile phones, which have been proprietary, optimized products. It has has large profit margins, while its subcontractors have had no profits to talk about. But the mobile phone architecture it getting good enough. This is when the flip will happen: the phones will become modular, and the profits will move to the subcontractors who will start optimizing their components that any phone assembler can then use, as well as those who will start creating added value based on the modular mobile phone platform. Looking at the higher level, this has already started to happen – there’s only a couple of standards that can be used to create applications. But the challenge for Nokia is to survive the transition at the lower level. When the profits move from the phone assembler’s level to subsystem developers, Nokia’s current business model will be gone.

Nokia’s strategy has been chosen already: they’ve given away the OS for free, which means that they will start buying their component suppliers soon – not to integrate them into themselves, but just to get a hand on the profits they will get when their products start having better margins. Another possibility is to just stay at the integrator role, buy components (and encourage very high competition in that market to drive the prices down) and provide added value in the way of services and software components built on the modular, open OS that they’ve released. Time will tell.

Managing an agile software team in 4 countries

We’re now two months into CALIBRATE, a huge software development project funded by the European Schoolnet. It has close to 20 partners in different EU countries, and our team is coordinating one of the work packages, where we have 30 months to build a jazzy web portal for creating, browsing and using learning material in the EU. We’re calling it the “Toolbox” for now.

From the software development point of view, the most interesting aspect of this project is that we’re attempting to do agile development (using a modification of Scrum and XP) with a team of 8 developers, which reside in four countries (Estonia, Finland, Hungary, and Norway). And one of the most important principles in agile is to keep the team together. Well, also in a waterfall splitting the team apart is always an issue that needs addressing.

So the problem is that there’s a few people in each country. No location has enough people to form a working development team. So somehow we need to create a team that is present in all four locations simultaneously. Having attended Agile Finland’s seminar in Helsinki I was convinced that one of the key make-or-break issues in this project will be communication and team dynamics.

A good book to read was Organizational Patterns of Agile Software Development by James O. Coplien and Neil B. Harrison, which is basically a collection of best practices (or patterns) for software development, from the point of view of the people doing the work.

A plan was then set: Geographically distributed work can work, as long as no-one is left out of the loop. This means that no important decisions should be done by people in one location alone, and no information should be held in a place where others cannot access it. So everything should be done as openly as possible. After a quick review of available software, we decided to go with Trac, which is an open-source development platform that integrates a wiki, a ticket system and version control into one browsable, interlinked web interface. You can take a look at our project’s Trac site at http://goedel.uiah.fi/projects/calibrate/. The burndown chart and time tracking are additions to the plain Trac, which allow some measure of time estimation (the burndown chart is a Scrum practice).

When browsing our Trac, you’ll notice that everything is done completely visibly. All meeting minutes are there, all discussions about concepts and features are there. Every patch commited into the version control system (subversion, btw) is connected to a defect or enhancement ticket, which in turn is connected to a user story ticket. So the reason for everything that we do can be tracked down. Our team also has a mailing list, where Trac automatically sends all ticket changes, and Subversion sends all commit diffs.

And yes, we did start with a “Face to Face Before Working Remotely” by inviting everybody to Helsinki for a week for an intensive workshop, sauna, and swimming in the sea (in November). I’ll post another entry discussing the organizational patterns that we use more closely.

Turning the Buffalo wireless bridge into a firewall

I had an extra wireless bridge (Buffalo WLA2-G54L) lying around and our current firewalling wireless AP (D-Link) was as crappy as can be. It’s probably ok for just surfing and stuff, but for keeping ssh connections open it just didn’t cut it, since it severed tcp connections every now and then. Also it froze every now and then and needed rebooting. So replacing that with a better solution was on the todo list.

The WLA2-G54L is just a wireless bridge – it has no conception of a firewall in it. So time to flash OpenWRT into it. Setting up OpenWRT is a bit of an adventure in itself, but quite a nice one. Internally the Buffalo units contain the actual computer, a wireless radio, and a normal 5-6 port switch. The computer is connected to one of the ports in the switch, and in firewalling models the switch can present all ports as separate interfaces so that one of them can be treated as the WAN port. However, in WLA2-G54L the switch is a bit different and it doesn’t show the ports separately (tried with robocfg, which did not detect the proper model and couldn’t do anything with it).

OK, so how to setup a firewall with just basically one interface, which is connected to a four port switch? Well, I came up with the solution to just use IP based filtering. Better suggestions are welcome. This basically means that I created an alias to the br0 interface (replace with your own public IP, netmask, and gateway):

ifconfig br0:1 123.123.123.123 netmask 255.255.255.000
route add default gw 123.123.123.1

Meanwhile the normal br0 interface is given an IP address in the LAN, say 10.0.0.1. Now you can quite easily separate the WAN from the LAN using IP addresses instead of interfaces. So instead of

WAN=$(nvram get wan_iface)
iptables -A INPUT -i $WAN -j DROP

you’d have

LANIP=$(nvram get lan_ipaddr)
LANNET=$(echo ${LANIP}/8)
NOTLAN=$(echo \! $LANNET)
iptables -A INPUT -s $NOTLAN -j DROP

And so on. The only problem comes when doing DHCP queries, as the client is probably not in the local network to start with. I solved this by using DHCP only in the wireless network, which is on its separate interface. So for wired access you’ll need to manually setup the network parameters. I can live with that. Another option is to carefully allow DHCP queries with a suitable filter command, and by adding extra checks into the DHCP server so it only server those you want it to serve.

So not a perfect solution, but works OK for me. I now have a low-cost wireless AP that I can ssh into, setup pretty much any kind of firewalling with iptables and otherwise configure to my heart’s desire.

Oh yeah, one more thing: when using WPA in the wireless network, you use the nas software as usual. But in the nas init script you need to remove the parameter “-l br0″. nas is proprietary, so there’s not much of an idea what it’s for, but at least in WLA2-G54L I had a problem with WPA connections not working until this parameter was removed. Then everything worked like a charm, as long as you remember that the WPA-PSK key (if you use that) needs to be given in passphrase format, not in hexadecimal (while WEP keys need to be given in hexadecimal). Weird, that.