How DevOps with Simulation Can Solve Your Hardware Problem — Part 2

In the first installment of this series, I discussed some of the challenges to up-leveling your DevOps practices and offered some thoughts on how to approach ROI. I also introduced the idea that if you are developing for the intelligent edge, you have a problem with hardware. The good news is that simulation is the answer to that problem. In this installment, I explore four ways hardware may be causing a problem and how simulation can fix it. Let’s jump in.

1. Development Lifecycle Bottlenecks

Let’s say the design team has determined that the combination of processor/OS that’s optimal for the device or system you are designing includes components that aren’t widely available in production yet. Or perhaps they are in production, but they’re too costly to equip each team member with a lab setup to work on. Meanwhile, if your team doesn’t start soon, they risk not making their deadline. Add to this scenario the fabulous DevOps pipeline you’ve gone to great lengths to design and implement. It’s sitting there, starving for input. It’s a bit like owning a fancy sportscar that only burns a specific type of fuel that you can’t find anywhere.

Maybe you can get your hands on something similar and, depending on what you’re building, perhaps that’s good enough. However, many of our customers are responsible for building high-stakes devices, whether it’s an industrial robot that needs to perform a task more than a million times precisely and without fail, or an aircraft that routinely transports hundreds of passengers from point A to point B safely. These are the types of scenarios where “good enough” … isn’t.

2. Accuracy vs. Speed

Some companies have addressed the issue of not having hardware by using some form of model or simulation of the hardware or system. When selecting a model, some see a spectrum between accuracy and speed, where one trades off to the other. I don’t think it’s that simple, for two reasons. You won’t use the same model for every use case, and for some you won’t use a model at all. The use case needs to drive the level of clarity that you will need in your model. For example, if you are developing for a specific SoC based on an Intel^® chip that you are not able to obtain, you could develop and test on an emulated x86 system, or engage in more generic x86 development on a similar device. You might find some defects, or you could have a false sense of security that your design is solid — which could prove unwarranted once you can develop on the final boards.

If you want to fine-tune your design, or build complex logic around the actual devices and peripherals your solution will deploy on, andbegin testing earlier, then having a high-fidelity model that you can integrate into your DevOps pipeline early in development is a superior choice.

With better clarity, you get better accuracy from your tests and a higher degree of confidence in your results. If you had a low-res model with an 80% pass rate, how comfortable would that make you? If you had a highly accurate model, you’d feel a lot better about that 80% pass. The bottom line is that the more accurate the simulation in your DevOps pipeline, the higher your confidence will be in code and readiness to ship software.

This degree of accuracy also becomes important when you are preparing for a certification. While in most cases you cannot use simulation to actually certify, you can prepare yourself for certification with a much higher degree of confidence (and a whole lot faster) by using simulation in the testing leading up to the actual cert.

3. Nondestructive Testing

This is an area where the benefits of simulation are pretty obvious. As mentioned in the previous point, you may need to certify your device. Again, you probably can’t use simulation for actual certification, but as you prepare, you need to be looking for all the vulnerabilities that might destroy your hardware. You need to see what happens under multiple scenarios that might stress your device. The question is, how do you stress test without blowing up your lab, or having to destroy physical models of the device or even the actual device itself? If you have to repeatedly test the device and destroy it to find all the vulnerabilities, that would become pretty expensive, not to mention potentially hazardous.

By using simulation, you expand the value of your DevOps practice to speed the path to certification while also slashing the costs of your hardware labs. By using preflight simulation, you can test a virtually endless combination of scenarios, a virtually endless number of times. The instead of swapping out hardware or rewiring and reconfiguring, you literally hit reset, modify the test scenario, and go again. You can even set this up on automation to run overnight and show you the results when you log in the next morning. No server is harmed.

4. One Device or Many

It’s one thing to set up a test to run on one device. I won’t say it’s easy, but testing on one device is, well, just one device. What if you are building a system that includes a constellation of devices? What if each device is going to be in a different environment, or needs to do different things but still be connected or even dependent on the network? It’s possible to set up a lab with 10, 100, or 1,000+ connected devices on which you can run your tests, but it certainly isn’t easy. One customer did just that by connecting all the test devices to their network, across their company campus. It worked, but it was painful and costly. You have to buy all of the equipment then spend the time to get everything networked together. Even if you don’t have devices scattered around every corner of your offices, the best-case scenario is that you have a lab with wires connected every which way, which becomes a mess as well as a real challenge to manage.

Then, once you have your physical lab set up, what happens to your DevOps pipeline when you need to make a change? Testing a piece of code on a single box can be challenging enough, but it’s just not sufficient if you are working on a network. You need, to the best of your ability, to test every piece of hardware, in the environment in which it will be deployed.

Another point is frequency. When you have a physical test lab with multiple components set up, how often can you really test? If you are shooting for more frequent deployment, can you really test every month, week, day, or multiple times a day?

Simulation allows you to set up your complete environment with as many devices as you need, with no limit. Whether it’s one device or 1,000, once you have your environment set up it’s easy to add additional devices and modify configurations as needed. If you want to test a new configuration or variable, no problem. Want to go back to the original configuration? Easy. No need to go next door and figure out which wire goes to which device. Plus, by having all of your models in simulation, you can test much more frequently, which will result in increased confidence in your testing and increased quality in your product.

So, again: If you are developing for the intelligent edge, you have a problem with hardware. The good news is that simulation is the answer to that problem.

As you can see, it is possible to test on a physical device, but there are many reasons why that might not be optimal. When you rely on physical hardware, you run the risk of driving up your costs and time to deploy, while simultaneously driving your confidence and quality down. Bottlenecks caused by hardware availability will either delay your ship date or compress your ability to fully test. Time-to-market delays might allow competitors to catch up or can result in an unhappy customer, while meeting the deadline but not giving yourself enough time to test increases your risk of delivering a product that does not meet the customer’s needs.

Choosing speed over accuracy when deciding on a simulated model runs the risk of actually slowing down your ability to deliver. When you work with something that is low-resolution or inaccurate, you don’t get test results you can rely on and end up needing to do more.

Nondestructive testing speaks for itself. Every time you blow up your hardware, you drive up your costs.

Testing many devices drives your cost up every time you add a device. And it’s not just the device cost; there is also the cost of connecting and managing the physical network you are testing.

On the other hand, with Simics simulation you can start to work right away, using an accurate model from the beginning. You don’t need to wait for the hardware to become available from the supplier or try to schedule lab time for the engineers who need it. Working on a hi-def model from the beginning helps you meet your deadline with plenty of time to run all the testing scenarios you need. By automating those tests, each change feeds directly into your DevOps pipeline, allowing you to deploy as often as you need to.

If a simulated model gets destroyed, just press the button and it is recreated instantly. Better yet, automate it to set itself up and run again so you can just view the results.

Need to test a network? Add as many components as you want; it’s basically a copy-paste situation. Depending on your use case, you can even link to other networks or physical devices and include them in your testing.

We’ve created some short videos to illustrate the ideas in this and the first installment of this series. You can use them to share these ideas with your team and colleagues. To watch, visit https://www.windriver.com/resources/webinars/devops-with-simulation-can-solve-your-hardware-problem.

To learn more about Wind River^® Simics^®, visit www.windriver.com/products/simics or contact us at salesinquiry@windriver.com.