Continuous Integration + Internet of Things = Continuous Deployment

I have a hunch – Continuous deployment will be the next important change for the embedded industry. Think about it, continuous integration work methods meet device Internet connectivity, what do you get? Continuous deployment of new services and capabilities directly into the live networks and devices connected to the network.

Continuous deployment is already in use in some industries, like web hosted services (Facebook, Netflix) and television broadcasting for example. But it has not made sense for embedded devices, as they are often just stand alone equipment. But not anymore…In a world of IoT and Internet connectivity, where devices get connected to the network, suddenly Continuous Deployment makes a lot of sense.

And since agile methodologies have entered the embedded space and are commonly applied, the technical foundation for continuous deployment is in place.  Agile methodologies are no longer viewed as just “hipster engineering”, it’s accepted and broadly introduced everywhere, even in the most careful engineering environments. Alongside with agile methodologies often comes the practice of continuous integration, and sometimes also “squad teams” – engineering teams that are organized into smaller units that take responsibility for everything from design, implementation and test, all the way to final integration. Top it off with test automation and a nightly test-and-build system. What you get now is new feature development, fully production tested and integrated, built into the final system with every sprint, which should be every 1 – 3 weeks.

Taking the step to continuous deployment is not big. Well, ok, it is a large step considering the effort involved, and the change and impact it will have. What I mean is: Continuous deployment is a natural extension of practices already in use if you do agile and continuous integration. A friend of mine working with set top boxes told the story “we had already implemented Continuous Integration, and everyone was in such a rush all the time, so the step to Continuous Deployment just happened”.

So now you may think “Continuous deployment…. in embedded development… I don’t think so!” I choose to disagree. How come continuous deployment is not out of reach in the embedded space?

As with any continuous practice, there are some special challenges with embedded software development that needs to be solved. As this blog from Mike Bria from 2009 states, and is still relevant today:

• It is more difficult to test evolving software since the corresponding hardware may not be readily available.
• There is less freedom to change your mind, because the corresponding hardware change may come with unacceptably high cost
• Less ability to leverage “learn as you go” techniques, considering the hardware construction may demand a more upfront style of planning and design.

Certainly, since software for embedded devices is so tightly connected to special built hardware, the situation is different compared to the general world of IT. David Rosen’s blog from last year about continuous delivery also describes it well:

• The enormous amount of legacy. In terms of codebases but also product architectures, team organizations, build systems and test environments.
• The insatiable need for compute infrastructure. Build times when using C/C++ is well known for being long and CPU intensive.
• The difficulty to efficiently integrate and manage test automation when you’re dependent on manual configuration and deployment of physical targets.
• The long lead times for a typical baseline build and analysis is far longer in the C/C++ programming paradigm vs managed environments such as Java and .NET.
• The costly and time-consuming need for verifying with regulatory requirements such as IEC 61508, DO 178-B and others.

All of the above complicating factors, make it feel like true agile and continuous practices, including continuous delivery and continuous deployment, is simply utopia for embedded developers. How can we ever manage to have a quick and easy release process? The common threads that stand out to me, and shine through all of this is a) lack of immediate and ubiquitous access to target hardware, and b) lack of flexibility to change and modify hardware.

But…. What if you had instant access to all the target hardware you needed (so much that you could parallelize any activity possible to parallelize), and what if you could change the target hardware in a blink of an eye, without cost, to try out your new software change – what would it mean? How would it affect your development process, your tools infrastructure and test automation?

A virtual target, a full system simulator that can run unchanged production binaries in a managed simulated environment solves exactly this.

One of our successful Simics customers who pilots a continuous deployment service with one of their end-customer now cannot enough stress the importance of short lead times. As the above blog post says “If your current build, test and analysis process are any closer to being longer than what it take your developers to refill their cup of coffee, my recommendation would be to prioritize this as a key improvement to address.” Making the end-to-end time of the build-test-release cycle short is perhaps the most important aspect to address. And when everyone in the organization has unlimited access to virtual target hardware, which is possible to freely change and scale, suddenly you have a technical infrastructure that makes it possible to dramatically shorten lead times.

So that’s why my conclusion is that when agile practices and continuous integration meet device internet connectivity, and when the embedded challenges can be overcome by using full system simulation as the tooling infrastructure – we are seeing the light of continuous deployment in the embedded industry.

PS.

And by the way, the difference between continuous delivery and continuous deployment is removing the manual step before deploying to production. Read more from Carl Caum here.