Archive for category complex systems

Two Problems Caused by the Innovation Diffusion Curve

Posted by Tim in book riffs, complex systems on 6 February 2012

The economist Rudi Dornbusch succinctly describes the way that ideas spread:

Things take longer to happen than you think they will and then they happen faster than you thought they could.

It’s the innovation S-Curve in words, this is what that looks like graphically:

And the problem is that the value for X is larger than we expect it to be – that’s the essence of Dornbusch’s quote.

I ran across the quote in a post by Andrew Hargadon discussing how sustainability in business is taking longer than expected to arrive. Hargadon explains why X is big:

Forget all the names and dates you learned in elementary school, great social and technical revolutions begin with a whisper, not a bang. They take decades to develop and then, when they do, they change everything overnight.

Take the industrial revolution. It started with a whisper: three different technologies slowly emerging in the 1700s. Coal slowly replaced wood as the dominant source of fuel; the steam engine slowly replaced animal and wind power (to pump water from coal mines); and large ironworks slowly replaced local craftsmen and blacksmiths. For decades, these technologies and the businesses and lifstyles that surrounded them grew and evolved. Then, all of the sudden, the last few decades of that century and the first few of the next saw an explosion of innovation across all industries—from textiles to shipping to railroads to iron and metalworks.

The impact didn’t come from any one of these technologies, it came from the interaction between them.

This slow diffusion causes two problems for firms. The first is that if you are a powerful incumbent, you see the slow diffusion and you think that it will continue to expand along path C – slow and steady. The consequence of this is that when the change does happen, even though there have been warning signs for ages, it still takes you by surprise.

There is a quote from the CEO of a major book store in Game-Changing Strategiesby Constantinos Markides:

We were late in implementing [the web] but not in evaluating it. And our evaluation was that this thing did not make sense. yet every time I tried to explain our reasons why we wouldn’t do it to Wall Street, my share price went down! Even in 1997 when online distribution of books went from zero to 6 percent, superstores increased their share from 10 percent to 22 percent – yet our stock price dropped by 40 percent. So in the end, we decided we had to do something.

This is exactly what path C thinking sounds like. And the problem it leads to is this (via Boing Boing):

But there are also problems for innovators in the S-Curve. The long delay in diffusion causes a lot of firms to go out of business trying to catch the new wave.

You can see this in the Kodak case. Here’s the world’s first digital camera:

It was invented by Kodak in 1975. The problem was, the rest of the economy wasn’t ready for digital cameras yet. Digital memory was still so expensive that you couldn’t actually take usable photos then. Does anyone else remember how crappy the first digital photos were in the late 90s? They were just awful. The supporting technology didn’t catch up with the camera technology for about 25 years.

That is exactly what Hargadon is talking about – it takes multiple innovations to disrupt an industry.

Kodak took this to mean that digital cameras would evolve along path C. So they kept the technology on the shelf and waited. If an independent entrepreneur had invented the digital camera, he or she would have gone bankrupt waiting for the supporting technologies. Or, if they were lucky, they might have sold the rights to a big company like Kodak.

The point is, when you’re early in the S-Curve, it usually takes a lot longer to get to the tipping point than you’d like it to.

The first step in addressing these problems is being aware of them. However, there are also some positive steps that you can take as well. I talk more about these here.

3 Comments

How to Think About the Future

Posted by Tim in complex systems, experiments on 2 February 2012

Imagine that 100 of us have gathered together in a room somewhere. It’s a social event, but I want you to think about a couple of numbers.

If we took the average height of all of us, it would be somewhere around 1.76 meters. What happens to this average if we’re joined by Sultan Kösen, the tallest man (2.51 meters) in the world? Our average height goes up to 1.767 meters. In other words, the average increased by about 0.4%.

Now think about our average wealth. The stats vary, but average net worth in the US is around $120,000. What happens to this average if we’re joined by Carlos Slim, the richest man ($63.3 billion) in the world? Our average net worth goes up to $745,544. In other words, the average increased by 521%! And that’s after Slim lost $11b due to the GFC.

The difference between 0.4% and 521% is the difference between normal and complex.

Height is distributed normally, and in a normal system, the average dominates the extremes. The economy is a complex system, and in a complex system, outliers matter.

That picture is from The Behavior Gap: Simple Ways to Stop Doing Dumb Things with Moneyby Carl Richards.

Here is what he says about the importance of outliers:

[O]utliers matter. In fact, they matter so much that they almost make the average meaningless. Because most of our lifetime return is determined by how many of these outliers we experience, it is time we stop ignoring them.

If we’re trying to innovate, our job is to invent the future. So the fact that the economy is a complex system is important.

First off, it’s important because returns to innovation follow the kinds of returns that we see in the wealth distribution. The average return to executing a new idea is small, but a small number are gigantic. This is why it’s important to manage innovation as a portfolio.

Secondly, this has a big impact on how to think about the future. Complex systems are impossible to predict. This is a problem, since we don’t like uncertainty.

Here is how Martin King frames the problem:

The problem with long term developments are that they are subject to exponential and combinatorial factors – chaotic things that we are not good at understanding at the best of times. To compound things change cycles themselves are becoming faster.

Instead of thinking of the future as something to predict, we should think about it as part of a pattern. Greg Fisher wrote an outstanding post (read it!) discussing the importance of pattern recognition in complex systems. Here is part of his prescription:

There is a relationship between patterns and prediction. In fact, I would note that not only do patterns exist and persist, we must rely on them in every day life. We make decisions in the present assuming the persistence of some patterns e.g. I will withdraw £50 from a cash machine today for spending over the next few days. I do not expect everyone else in the UK to switch to the Thai Baht during that period. Furthermore, it is particular patterns – many of which we might call institutions – that are responsible for our civilised society and a relatively high standard of living.

But – and this is to assert the point further – it is important to emphasise that the world will change and so too will the patterns around us. By “expect to persist” in my definition of patterns I was referring to making reasonable judgments that some patterns will remain broadly the same over a particular period.

How should we respond to this? Geoffrey Morton-Haworth has written an excellent post on Learning From Patterns. First he talks about the most common example of a complex system: the weather. And he says:

We cannot control the weather but if we recognize its patterns we can manage around them. And we can do the same in complex relationships.

He then discusses the work that Edward Tufte has done on effectively assessing complex data. Morton-Haworth includes this worksheet from Tufte:

Here is how he concludes:

Tufte argues for good method. That is “a shrewd intelligence about evidence, a clear logic of display and analysis, placing data in the appropriate context for assessing cause and effect”. In short, he talks about the need for “a coherent architecture for organizing and learning from images”.

A complex relationship outlasts its components, just as the ant colony outlives the individual ant, and in so doing develops a purpose of its own greater than the free will of its parts. While individuals may only be involved for a matter of months or just a few years, a complex relationship can learn, change, grow and adapt over five, ten, fifteen or more years. Nevertheless, because our lives take place at lower levels, we frequently don’t know the contribution we make to complex relationships. But we can help its intelligence to emerge.

This is why it’s important to think about inventing the future. Another important feature of complex systems is that the systems co-evolve with their parts. In simple terms, small changes among the parts can cause large changes within the system.

These are the small number of innovations that end up having big impacts. And how can we find these? We can’t predict which innovations will hit big – knowing this is one of the important outcomes of thinking about the economy as a complex system.

Harold Jarche does a nice job of framing some of the issues here:

When we move away from a “design it first, then build it” mindset, we can then engage everyone in critical and systems thinking. Workers in agile workplaces must be passionate, adaptive, innovative, and collaborative. Autonomy is the beginning.

Instead of innovating based on prediction (design, then build), which leads to big bets, we need to innovate based on experiments. This leads to little bets.

In a complex economy, the way to think about the future is this:

We can’t predict the future.
But we can learn about the patterns from which the future will emerge.
In fact, while we can’t control the future, we can influence it.
The best way to influence the future is by innovating through experiments.

What experiment can you try today?

9 Comments

Innovation Problem: New Ideas Spread Slowly

Posted by Tim in book riffs, complex systems, evolving economic entities on 30 January 2012

There’s a big problem with innovation: ideas spread much more slowly than we expect them to.

Ideas follow an S-Curve as they spread that looks like this:

They pick up steam very slowly, until they either die off or hit a tipping point and take off. The slow build-up is the time I’ve indicated as X in the drawing.

The idea for the S-Curve is based on the great work by Everett Rogers on innovation diffusion.

Based on his research, the population of users is divided into groups he called innovators, early adopters, the early and late majorities, and the laggards. In the populations that he looked at, the percentages of people in each group look like this:

You can see these numbers in the survey that Sophos Security released last week on the reactions of Facebook users to the new timeline feature:

This is being presented as a big problem for Facebook, but if you look at the numbers, they’re actually better than the stats from Rogers would lead us to expect. The survey doesn’t include the laggards, who probably still aren’t on Facebook, but the rest of the numbers map onto Rogers’ pretty well.

All of the people that hate the new timeline want to go back to the News Feed, another feature that had even worse approval numbers when it was introduced. And now people love it and don’t want it to change.

That’s the way that ideas spread. People resist, a small number adopt, and eventually over time, the idea wins. If you’re lucky.

There was another story over the weekend about the diffusion of Edison’t incandescent lightbulbs that tells the same story.

Here is what they say about adoption of electrical lighting:

By 1910, more than 30 years after Thomas Edison invented the incandescent bulb in 1879, only about 10 percent of American homes had been wired. Even in the glittering Roaring Twenties, only about 20 percent of homes had electricity — not because of a lack of electrical contractors, but because of a lack of consumer enthusiasm.

Advertisers proclaimed that homes with electricity would be brighter, cozier and happier, but the public wasn’t buying.

And this is for a product that was demonstrably better, cheaper and safer.

Again, the value for X was much longer than expected.

This is an issue that is addressed extremely well by James Gardner in his excellent new book Sidestep & Twist: How to create hit products and services that people will queue up to buy.

The book is worth reading and Gardner does a great job of explaining the S-Curve and its implications. One of the key outcomes of this is one that makes a lot of the people that have encountered Gardner’s ideas uncomfortable: breakthroughs don’t pay.

The long X shows us why. It takes so long for new ideas to spread that whoever introduces them is not always set up to capture the value from them.

This is kind of scary, because those of us that generate ideas want to think that a great idea will win. But they don’t automatically. One point that he makes is that you work around this by building on existing ideas:

A lack of genuine originality is a feature of almost every category-defining product in the last decade. Was Facebook the first social network? Certainly not: MySpace, Friendster and a host of others preceded it. In fact, the first real social network was a site called SixDegrees.com, and it was founded a decade before Facebook’s meteoric rise began. Was it Google that created web search? Of course not: the company’s contribution was to improve what Alta Vista and the other web search engines that had pioneered the field were doing already.

I could spend pages and pages going through examples like these, and will do so later on in this book. But one thing unites all these products and services: they’re built on something that was working well somewhere else.

Gardner has more good suggestions about what to do about this, and I discuss these more here. But for today, I just wanted to take the Facebook and Edison examples to illustrate the problem that we are trying to address. If you are trying to get ideas to spread, you must develop a good understanding of the idea diffusion S-Curves and what they mean.

The fact that ideas spread slowly is crucially important to understand. It is part of what makes it difficult to win through innovation. This is why we must manage innovation as a process.

It’s dangerous to think of innovation only as generating new ideas. That’s not enough. You also have to get the great ideas to spread. They spread through S-Curves, and we have to include these when we develop our innovation strategies.

11 Comments

Lessons From Kodak’s S-Curve Problems

Posted by Tim in complex systems, time on 20 January 2012

With Kodak in big trouble this week, a lot of people have been reflecting on what went wrong. While many are using this an opportunity to talk about bad management, or missing the digital photography trend, I think there’s more to the Kodak story than this. Kodak’s problems illustrate two very important innovation problems.

The first is that new innovative new ideas diffuse much more slowly than we expect them too. This causes problems when you have to respond to a new competitive innovation.

Larry Keeley has written an excellent post about Kodak, and he says:

The demise of Kodak isn’t merely the classic disruption story that everyone loves to tut tut over. Nor is the company’s downfall merely a result of recent bad decisions or the mismanagement of senior executives. It is the more nuanced story of how easy it can be to get things wrong, even when trying with the best of intentions to do everything right. It’s a cautionary tale of the need for deeper understanding of what innovation really means, and how it is infinitely more vital than most people think it is, even as it isn’t about any single product or widget or technology.

Kodak knew all about the impending disruption of digital technology. As many have noted, they own the primary patents on digital photography and built one of the world’s first digital cameras in 1975. As The Economist reported recently, a report circulated among senior executives in 1979 detailed how the market would shift permanently from film to digital by 2010. This disruption was no surprise.

Here is an illustration of the problem:

Innovations diffuse following an S-Curve, and this causes problems. When a new innovation is really hyped, people expect it to follow diffusion Curve A – where at the end of time X it has taken over the market. The problem here is that the value for X is usually much higher than we expect, but more importantly, once you get through that time, you have only gotten to the point where the innovation is starting to take off.

If the expectation is that the innovation should have taken over the world at after time X, but it has actually grown slowly, then there are three possible future paths. The innovation could die, and people often assume that this it what will happen if it hasn’t taken over the world yet – that is Curve B. Many people are talking as though this is the mistake that Kodak made – that they discounted the potential of digital.

However, the real problem is trickier. As Keeley points out, they were fully aware of the potential of digital photography. It wasn’t that they ignored it. The problem was that they thought it would grow in a slow, straight line, like Curve C. This mistake is incredibly common.

The first lessons from Kodak’s demise are:

New innovations take longer than we expect to become dominant, but when they finally take off, they move fast. Never assume that an idea will diffuse along Curves A or C – that never happens. Curve B is possible, and so is the S-Curve. Figuring out which is most likely is hard, but that’s what you need to focus on.
There are no straight lines in business. If you ever find yourself making a projection like Curve C, you are almost certainly wrong. Be cautious of this, especially if this projection suggests that you have a long time to respond to changes.

Heres the second issue: performance also improves following an S-Curve. Disruptive products (P2) replace existing ones (p1) in a pattern like this:

This is the Innovator’s Dilemma. New innovations start out with lousy performance. They are crappy. If you’re a dominant firm on the P1 Curve, and you correctly predict the shape of the new curve, the question then is: when do you jump to P2?

Consider this from Adrian Wooldridge in The Economist:

Like Kodak, Fujifilm realised in the 1980s that photography would be going digital. Like Kodak, it continued to milk profits from film sales, invested in digital technologies, and tried to diversify into new areas. Like Kodak, the folks in the wildly profitable film division were in control and late to admit that the film business was a lost cause. As late as 2000 Fujifilm counted on a gentle 15 or 20-year decline of film—not the sudden free-fall that took place. Within a decade, film went from 60% of Fujifilm’s profits to basically nothing.

If the market forecast, strategy and internal politics were the same, why the divergent outcomes? The big difference was execution.

Fujifilm realised it needed to develop in-house expertise in the new businesses. In contrast, Kodak seemed to believe that its core strength lay in brand and marketing, and that it could simply partner or buy its way into new industries, such as drugs or chemicals.

In other words, Fuji jumped onto the P2 curve early, while Kodak figured that its core competencies would allow it to jump to the P2 curve right around time TC – right when digital started to outperform film. Mike Ryall taught a lot of Kodak execs in his MBA classes, and this is how he describes their thinking:

Why were they so optimistic? When challenged to discuss it in class, they proudly explained that Kodak’s “core competency” was “color”. The reasoning went something like, “We understand color and its application to photography better than any other firm. This knowledge will be as important for success in digital applications as it was in analog film. Therefore, we are wonderfully positioned for whatever challenges the market presents.”

Simon Waldman’s book Creative Disruption: What you need to do to shake up your business in a digital worldis one my favourites on the topic of Kodak. In a recent blog post, he says:

Newspapers, music retailers, book publishers etc are all used to operating in a market with a lot of businesses that are essentially clones of each other [in terms of overall cost and revenue structure]. This is what ‘competition’ means to them. It is a world away from the sort of asymmetric warfare involved in dealing with a new, disruptive force – which will initially seem too small to even bother with compared to your traditional rivals. [eg: Craigslist vs Big Newspaper Co/ Play.com vs HMV/ Netflix vs Blockbuster]

He contends that Kodak was so busy fighting Fuji in the 80s and 90s that they seriously underinvested in digital. They just figured that they could jump onto the P2 curve when the time was right, using that core competency in color.

Scott Anthony comes up with two more lessons in this:

Start before you need to. … The challenge — what I call “The Innovator’s Paradox” — is when you have the freedom to change, you don’t feel the urgency. For example, in the early days of Kodak’s disruption, its core film business actually was growing. A lack of urgency allows a company to treat new growth efforts as science experiments that are academically interesting but not vital activities. However, once the urgency grows, freedom narrows rapidly, as attention goes to staunching the bleeding in the core business.

Place multiple bets. It’s always hard to know which idea is going to be “The One,” especially in fast-changing industries. An ideal response involves a portfolio and pipeline of growth strategies — again, started early enough that they have time to iterate, incubate, and grow.

The bottom line in all of this is that responding to innovations that disrupt your core business model is incredibly hard. You need to invest early, you need to have a clear idea of how you will compete in the new environment, and you have to have a reasonably accurate map of how the new innovation will disrupt your current products and services.

Having a good understanding of the innovation diffusion S-Curve will help with all of these steps.

8 Comments

There’s No Such Thing as Information Overload

Posted by Tim in complex systems, connect, filter, innovation, time on 13 January 2012

The size of your inbox or your RSS feed or your twitter stream might all argue otherwise, but there’s no such thing as information overload.

Or, at least, if there is, it’s not new. Check this out:

As long as the centuries continue to unfold, the number of books will grow continually, and one can predict that a time will come when it will be almost as difficult to learn anything from books as from the direct study of the whole universe. It will be almost as convenient to search for some bit of truth concealed in nature as it will be to find it hidden away in an immense multitude of bound volumes.

That was Denis Diderot in “Encyclopedie”, back in 1755. 1755!

The problems that we have with information isn’t that there’s too much of it – there has always been too much. Rather, there are two related problems with information: how do we filter out information that doesn’t help us, and how do we find information that we need.

Jorge Luis Borges touches on this in his story The Library of Babel. You should go read it here since everyone should be reading more Borges. The story is short, but packed with ideas. The library has an infinite number of rooms, all filled with books. Each book is the same length, with randomly assembled letters. The Men of the Library spend their lives wandering the shelves, reading the books. Since the library is infinite, it must contain all books ever written (and all that will be written!), but since the library is infinite, the odds of coming across even one sentence that makes sense are exceedingly small.

It is useless to observe that the best volume of the many hexagons under my administration is entitled The Combed Thunderclap and another The Plaster Cramp and another Axaxaxas mlö. These phrases, at first glance incoherent, can no doubt be justified in a cryptographical or allegorical manner; such a justification is verbal and, ex hypothesi, already figures in the Library. I cannot combine some characters

dhcmrlchtdj

which the divine Library has not foreseen and which in one of its secret tongues do not contain a terrible meaning. No one can articulate a syllable which is not filled with tenderness and fear, which is not, in one of these languages, the powerful name of a god. To speak is to fall into tautology. This wordy and useless epistle already exists in one of the thirty volumes of the five shelves of one of the innumerable hexagons — and its refutation as well. (An n number of possible languages use the same vocabulary; in some of them, the symbol library allows the correct definition a ubiquitous and lasting system of hexagonal galleries, but library is bread or pyramid or anything else, and these seven words which define it have another value. You who read me, are You sure of understanding my language?)

What do you do when you are faced with all of the information in the world? To make any sense of it, you have to find the information that is useful to you. So we filter.

As Borges suggests, each piece of information means something to someone, even if it’s gibberish to us. We need to knock out the stuff that’s gibberish. So we find ways to ignore information, by saying things like “Twitter is just 100 million people talking about what they ate for lunch, so why would I waste my time with that?” I do this by ignoring TV (unless I can find a hockey game on). Everyone makes choices about what they should be paying attention to.

The key to dealing with information is to be conscious of the choices that you’re making, and to develop a strategy or a set of routines for handling it. Howard Rheingold has created an outstanding set of resources for his classes on Mind Amplifiers and Infotention. Start with those to develop a filtering strategy.

We’ve always had too much information to handle, and we’ve always dealt with it by developing routines. The real difference now is not that there’s so much more information, it’s that we don’t have good routines to go with the new channels that the information is taking to get to us.

The danger in thinking that we have too much information is that we’ll start missing out on innovation opportunities. After all, the creative part of innovation is about making novel connections between ideas. So we actually have to seek out information that is a bit out of the ordinary (see the end of this post for some techniques for doing this).

If you think that the problem is information overload, then this will seem completely counterintuitive. That’s why it’s a dangerous idea – if you take it seriously, it makes it much harder to innovate.

That’s why I say that there’s no such thing as information overload. Even if that’s not strictly true, we’re better off acting as though that’s the case.

5 Comments

How to Build Business Metrics – revised

Posted by Tim in complex systems, innovation strategy on 14 December 2011

We’ve written a few posts criticising some of the more common innovation metrics in use, so I thought it would be smart to outline some ways that we can actually develop more effective metrics. Here’s a story that might help:

A while ago I was in charge of managing student recruitment for a tertiary education institution. One of the first things I looked into when I started the job was metrics – how did we measure how well my section was doing? The answer was one number: total number of enrolled students each year. The job that I was given was to increase that number by as much as possible (which begs all kinds of questions about quality, teaching and so on, but let’s set those aside for now…).

The problem was that managing that number as a standalone was hard. Well, impossible, actually. So I looked into what other numbers we had, and I found a that we had measures for total applications received, and total enrolments. I worked with my teams to figure out the path that people took to become students, and we then also figured out a way to measure enquiries. Once we had these numbers, here’s what we did:

We made three metrics: total number of enquiries, the ratio of applications/enquiries, and the ratio of enrolments/applications. Then I made the marketing team responsible for enquiries, the information team responsible for applications/enquiries, and the enrolments team responsible for enrolments/applications.

When my boss told me to increase enrolments as much as possible, he was hoping for a 5% increase. By breaking down the process, developing new metrics, and making people accountable for the measures, we were able to increase enrolments by 12%.

There are several lessons from improving innovation metrics in this:

Innovation is a process not an event: many things that we often think of as an event are actually processes. Enrolments is a good example – previously my institution only considered the end point, enrolled students. By breaking down the process that we went through to actually get an enrolled student, we were able to improve our ability to get enrolled students.
I think of innovation as a process too – this is the diagram that I use to describe it:

To improve our innovation metrics, we need to first think of it as a process, then build metrics to measure the intermediate steps as well as the outcomes.
Use multiple metrics: in the enrolments story, we used three metrics that led to the one that we were most interested in (total enrolments). We can do the same for innovation. Once we think of it as a process, then we need to develop metrics for each of the steps that lead to the outcomes that we are looking for from innovation. Innovation is a complex process, and to manage it we need to use multiple metrics.
Link Your Innovation Metrics to Your Strategy: my tertiary education institution saw increasing enrolments as a central part of its strategy. At the time, the educational sector in New Zealand was fairly turbulent, and there was a strong message from government that it wanted to see the sector consolidated. Increasing enrolments was seen as a way to signal that we were a thriving institution, making it less likely that we’d get absorbed by a larger polytechnic.
We need to do the same thing with innovation – link it to our overall strategy so that it can help drive success. There are a number of broader strategic goals that can be supported by innovation – we just need to be clear about which ones we’re targeting.
Improve the part of the process that is weakest: when we started tracking the enrolments process, we discovered that we were pretty good at generating enquiries, and very good at converting applications into enrolments. The weak link was converting enquiries into applications.
The information team had been given some sales training before I arrived, which they strongly resisted. They saw their role as helping people, not selling them. We implemented a lot of ideas, but the one that had the greatest impact was getting them to ask at the end of each enquiry that they handled “if you’re interested in the course, would you like to put in an application?”

When they started doing that, the applications/enquiries ratio shot up from about 12% to 18% in a couple of weeks. And we weren’t forcing people to apply for courses they didn’t really want to take – the enrolments/application ratio held steady. If the quality of applications had decreased, this metric would have gone down. It turned out that a lot of people really did want to start studying, but they just needed a small nudge to get started.

In looking at our innovation processes, we need to do the same thing: find the weak link, and figure out how to best improve it. As we’ve said many times before, usually the problem in organisations is not that they don’t have enough ideas, but rather that they need to get better at selecting ideas, or at getting them to spread. In any case, once we have identified the part of the process that is most in need of improvement, then we can figure out to best go about making it better.

Getting innovation metrics right is a challenging task. There is no single number that will tell us everything we need to know to manage innovation. I hope these ideas help you figure out how to measure it better in your organisation.

Note: There was also a good series on innovation metrics by Boris Plukowski on Innovation Excellence a while back: Part 1, Part 2 & Part 3.

1 Comment

The Exigency of Extrapolation

Posted by Tim in complex systems, experiments, selection on 12 December 2011

Noun 1. exigency – an unstable situation of extreme danger or difficulty;

I’ve had some jobs in which I’ve performed pretty well, and some where I haven’t been quite so good. Probably the worst job I’ve ever done was part of my portfolio when I was managing sales & marketing for a polytechnic in New Zealand. The specific job was new course evaluation.

We put in a modified stage-gate type process to evaluate potential new courses. It was my job to fill in the numbers. The one number that drove everything else was the expected number of students. If the expected number of students was high, we’d try to run the course. If not, we’d kill the proposal.

I developed a very elaborate model, based on historical data. I knew how many enquiries we could generate from advertising, how many of those we could convert into applications, and how many enrolments we got per application. In fact, figuring out these numbers was one of the biggest innovations that I executed there, and this model was of tremendous use in trying to figure out around November how many total students we could expect at the start of the new school year the next February.

However, the model was terrible at predicting new course enrolments. Why? In large part, because we’re really lousy at figuring out how something new will perform. We rejigged our new course approval process after I pointed out that we hadn’t approved a single new offering in over 6 months – our process was killing everything.

I was originally going to call this post The Perils of Prediction, but Greg Satell beat me to that title. Also, the specific problem that I’m talking about is really extrapolation. You should read all of Greg’s post, but here’s part of what he says:

The problem starts when smart people in nice suits and lab jackets proclaim that “the data says…” In truth, the data never says anything. We interpret it in one way or another and there are lots of ways to interpret it incorrectly.

Data is, after all, messy. It doesn’t spring forth whole, but must be collected in some way. We count, measure, survey, aggregate, slice and dice, picking up errors all the time. We need to make choices about which data we want to focus on and which fades into the background.

How do we deal with this? Usually by finding some numbers from the past and extrapolating them. However, there are a few problems with this approach, including:

We tend to think in straight lines, but there aren’t any straight lines in business: that’s really the point being made by the xkcd cartoon. Taking a straight line and extrapolating it into the future almost never gives us the right answer.
It’s really hard to tell what kind of curved line we’re on: this complicates things too. Even when we have historical data, it is nearly impossible to figure out what kind of engine is generating the output. Take a look at this data from an interesting post on climate change:
Is it likely that the data will progress in a straight line? Or will it level out at some point? Or will it increase exponentially? We don’t know. But when we’re predicting, it pays to consider what circumstances might lead to each of these outcomes.
Even when things are accelerating quickly, they tend to level out: innovations spread through an s-curve, and this is a very common pattern in business.

This is one of the issues with everyone talking about the singularity – it assumes that exponential growth will continue forever. It might, but usually exponential growth levels out, and then it looks like an s-curve.
However, by the far the biggest problem with extrapolation is that if we depend on extrapolation for predicting, we will never anticipate something new happening: extrapolation can only predict that things in the future will be mostly like things in the past. Here’s how Greg puts it:

And that’s what most analysts miss. The future is hard to predict not just because of our cognitive biases or inexplicable natural events, but because we have the power to make our own future.

The first new course that we approved at my Polytechnic after we scrapped the stage-gate process was a program that offered free computer and internet lessons to people in the community, particularly targeted at older adults. And the number of enrolments that we got went so far beyond anything that we had ever seen before that it was almost impossible to believe.

None of my models could have predicted that. When we innovate, our job is to invent the future. The exigency of extrapolation is that if that is the tool we use to predict, we won’t be able to invent anything that doesn’t already exist. And what kind of innovation is that?

5 Comments

The Jenga Theory of Creativity

Posted by Tim in complex systems, experiments on 8 December 2011

I think I actually made yesterday’s post on simplicity too complex. Here’s another try.

Earlier this week I edited two different papers for journals. My main contribution was that I cut 2,000 words out of each. I also wrote about 400 words in each, but it was the cutting that helped the papers.

This reminds me of the drawings by Matisse I talked about yesterday – he was more interested in what he could take out than what he should put in. The key question that he asked was: what is the minimum number of lines that I need to capture the essence of what I’m drawing?

Creativity is often about subtraction as much as it’s about addition – it’s really like playing Jenga. You need to pull out as many pieces as possible while still retaining the shape of the idea that you’re working on.

Or, as Austin Kleon put it his great post How to Steal Like an Artist:

The key issue is how do you know what to take out? That is where experimentation, failure and learning come in. The only way that you identify the essential pieces to keep in is through testing (prototyping). Here is how Joe McCarthy (please go read his blog, it’s awesome) put it in a comment:

And just to bring it full circle, while I agree that getting it right requires learning and skill, I believe that learning and skill often arise primarily through making lots of mistakes (i.e., being wrong alot … but with an open mind).

And that’s what I’m trying here. I gave it a go yesterday, I’m not sure if it worked, so I’m trying a different way today.

If I keep working on it, eventually I’ll get it right.

The way to make something simple is to cut out all the extra bits. But you can only know what to cut when you have a deep understanding of the system in which you’re working. That’s the Jenga Theory of Creativity.

(Jenga picture from flickr/riNux under a Creative Commons License)

5 Comments

Life’s What You Make It

Posted by Tim in complex systems, time on 2 December 2011

Well, we’re all getting older. What do you make of it? I ran across an interesting post by Ben Casnocha, which referenced an article by Benjamin Schwarz which includes this comment on John Updike:

Above all, and most poignantly, this collection highlights Updike’s evaluation of the slackening of his own mental and athletic prowess… A generous and companionable critic and an avowed Christian, Updike met the decline of his powers with courage and good humor, but also with a clear-eyed recognition that the compensations of old age—a hard-won sagacity, a bemused detachment—don’t make up for the irretrievable losses.

Here’s the thing – you have a choice about whether or not the compensations of old age make up for the losses.

I was in the best physical condition of my life when I was 20. And I was a wreck. I was a befuddling mix of arrogant and insecure, I was struggling at university, I was depressed. I had gifts that were not yet developed, and potential that seemed to be fading rather than emerging.

In short, I was an idiot. But probably not that far off the norm for a 20 year old either.

By the time I hit 30, things were a bit better. Nancy and I had just gotten married, which was great. Work was still a bit of a struggle – I still hadn’t figured out how to best use my talent. Physically, I was in ok shape, but nowhere near as fit as I was at 20.

At 40, I was in the middle of making a career change that was the smartest career move I’ve managed to make. There were high levels of uncertainty over whether or not it would work, but life was a lot better than it had been at 30 – even though I was in the worst physical shape of my life.

Now I’m rocketing towards 50 – and things are even better than they had been. Work is good, and some of that potential from when I was 20 is finally turning into something meaningful. Physically, I’m fitter than I was at 40, but I’m starting to lose a few things too, as you do.

I know I’m lucky, but for me, life has just gotten better and better as I’ve aged. Now, Ben Casnocha has been precociously successful, so maybe things will be different for him. I don’t think so though.

Why I am happier now than I was at 20 – despite the irretrievably physical losses? Because of the things I’ve learned, and that I only could have learned through experience. I’m better at executing ideas now because I’ve learned how to do it. This has been the key to developing that long-dormant potential.

This isn’t to say that bad things haven’t happened over the years, or that more won’t happen in the future. Of course they will. But one thing that I’ve learned is that the best way to ensure that your life gets worse as you get older is to convince yourself that life must get worse as you age. It doesn’t have to. The things around you don’t determine how you must live your life – read Man’s Search For Meaning by Victor Frankl for insight into that. The knowledge that you gain as you experience life is unobtainable when you’re young. You’d be smart to place a pretty high value on that.

At all ages, life’s what you make it.

Here’s a song by Talk Talk from my fit dance-club days that I stole the post title from:

5 Comments

The Most Important Innovation of All Time

Posted by Tim in complex systems, experiments, innovation on 23 November 2011

What is the most important innovation ever?

There are plenty of candidates. Fire, the telegraph, electricity, and the internet would all have to be candidates.

There’s another one though, that has had an enormous impact on every single one of us. And surprisingly, it’s not a whiz-bang piece of technology. It’s a simple process innovation.

The most important innovation of all time is: medical practitioners washing their hands before they touch patients.

Hand washing has been an unbelievably important medical breakthrough. It is one of the main reasons that we actually live long enough to retire now.

As with many great innovations, hand washing started with a scientific discovery – the germ theory of disease. And as with some innovations, the theory was driven by beer. Louis Pasteur’s work was motivated by brewers who couldn’t figure out why some batches of beer fermented well, while others failed. So in trying to make better beer, Pasteur made us all healthier.

There are some critical innovation lessons here:

Ideas need to be executed to create value: the germ theory of disease is an important scientific breakthrough, but a theory isn’t an innovation. Theories are often great ideas, but to become an innovation they have to be turned into something that can be executed to create value. Germ theory led to many important innovations: pasteurization, antibiotics, and hand washing. These innovations have had impact on a wide range of industries and activities, and that is where the value has been created.
Innovation isn’t just about new technology: hand washing in hospitals isn’t a sexy new piece of technology (which is maybe part of why it’s still hard to get everyone to do it consistently). Hand washing is a process, and process innovation can be incredibly important. Just think about the assembly line, lean management, or agile software development. All are process innovations, all are important, like hand washing.
Small innovations can have enormous impacts: one feature of complex systems is that small changes can results in gigantic change. That’s what happened with hand washing. This new process has made childbirth much safer, it improved the success rate of all surgeries, and it greatly reduced the chance of secondary infection in medical procedures. And it’s about the simplest thing imaginable!

So the next time you wash your hands before eating, think about what a great breakthrough you’re participating in. And when you’re thinking about innovation, remember that it’s often the smallest ideas that can make the biggest difference.

2 Comments