Best practices for embedded software development and CI

Transcription

1 Tip Guide Best practices for embedded software development and CI Industry experts have found that organizations are continually running into the same types of issues that inhibit embedded software development from performing well and producing profits. In this Tip Guide, readers will learn: Best practices for software development Overview of the value and tips for continuous integration Strategies for creating workable solutions and producing profits And more Sponsored By:

2 Tip Guide Best practices for embedded software development and CI Table of Contents The seven deadly sins of embedded software development testing The motivation behind continuous integration in embedded software development Resources from IBM Sponsored By: Page 2 of 9

3 The seven deadly sins of embedded software development and testing By Howard Deiner You probably didn't think that Dante was speaking to you when he wrote The Divine Comedy," did you? Well, there's a lot to be learned from these sins as they relate to embedded software development and testing. We've all been tempted to sin, and many of us are guilty of committing one or more of these sins at one stage or other when developing the code for embedded systems. Pride At first glance, pride seems desirable. But, the human condition turns good pride in craftsmanship and quality into something bad. Pride can keep us from producing the type of code that makes for better profits. When we let down our guard down on basics like good code readability because we are so proud of what we did that we figure "no one else needs to know about this than me," we quickly find that we create difficult-to-understand code that is also difficult to maintain. This is not a call to more comments in the code. It is a call to unit tests, which are the best way to convey the sort of reuse that "clipboard inheritance" gives us. Our unit tests become the home to short and well-targeted code snippets that show how to use the code, and under what circumstances. That sort of humility is what we are looking for in the quest for sustainable engineering. Greed Greed is something that sneaks up on you. When developing embedded systems, many times, we are forced to think laterally to come up with a solution to a coding or testing problem that we face. Yet, we are constantly being seduced by various vendors attempting to sell us solutions to solve problems for us that we may or may not be facing. We must balance our actual needs with the solutions available to uphold product quality. Sponsored By: Page 3 of 9

4 Envy This is something that happens each and every time that we see others succeed where we fail in our product. We may lose the "first-to-market battle," but that doesn't preclude eventually winning the war for the hearts, minds and pocketbooks of our customers and the overall market. When envy occurs, we start to think "what corners can we cut to get this out the door quickly?" Sadly, one of the first things that we are willing to lose is quality. And diminished quality is what will convert potential lifelong customers into customers for the competing product. Good software development practices will help us turn envy into cash. Wrath Wrath occurs when we forget to develop our code correctly. One can avoid the ugly face of wrath by using test-driven development (TDD). We can also test both white-box solidness as well as black-box functionality. It is also especially important to develop our tests as regression suites and to automate the build and test cycle in a continuous integration server environment. Lust We succumb to lust when we code for our benefit, rather than the benefit of the product that we are developing. We find that there are things that we want to use, such as a new and cool framework, which have one minor flaw. They don't fit into the product that we are developing. But rather than simply sigh and walk away, we scheme to find a way to make the framework needed for our solution, perhaps invoking the ever popular "future proofing" argument. We can harm ourselves by adding unnecessary weight to the code, and perhaps adding extra burden on limited hardware resources. Gluttony When we "gold plate" our code, adding extra and unnecessary code to handle "the next requirements" that we think will be coming, we bulk up unnecessarily. The problem is that we, as developers, really aren't all that good at predicting the future. But I can easily predict what your future will hold if you succumb to this temptation -- extra code to Sponsored By: Page 4 of 9

5 maintain which will drain testing resources. A simple application of Ward Cunningham's YAGNI (Ya Ain't Gonna Need It) is usually enough to remind ourselves to stay trim. Sloth Sloth is present when we think that our embedded system software is incapable of being tested in any way other than in a fully integrated manner. In extreme cases, this means actual integration on the actual hardware that the system is destined for. While it is true that eventually we will have to perform final integration testing on the things that are actually delivered to the field, there are many different ways to test our code beforehand, using emulators, using test doubles and the like, before we commit to actual full integration. We can only make educated guesses as to the actual performance characteristics, and, yes, we won't be able to test all race and other timing issues, but we can still find an enormous amount of bugs in functionality using the low cost development environment before we commit to fully-integrated solutions. Conclusion By focusing on Agile and XP practices, we ll be able to avoid committing these seven deadly sins. In our endeavor to avoid the nine circles of Dante s hell, we will end up elevating ourselves to at least a workable solution, and perhaps we can ascend to the heavenly place in which our organization is performing well and producing profits. Sponsored By: Page 5 of 9

6 The motivation behind continuous integration in embedded software development By Howard Deiner Motivation Why is continuous integration (CI) such a popular topic? There are a number of vendors that are happy to interest you in their wares in this area. But is this something that you should invest in, or is this something in the things you own end up owning you (Tyler Durden, Flight Club, 1999, film distributed by 20 th Century Fox) arena because the cost for caring and feeding the CI server outweighs the benefits? The answer is that owning and operating a CI server is a wonderfully useful tool, and one that has the best payback potential of any Agile practice around. Agile expert Scott Ambler runs surveys every year on Agile process and practice adoption rates (for example, see His surveys consistently show CI as the most effective Agile practice and the least difficult to learn. Why is CI such a good practice for software development in general, and embedded development in particular? It s easy to say that quality will be enhanced, but it s important to understand the three basic reasons that we get to that conclusion. CI provides: Quicker and better development feedback cycles Reduced risk of defects showing up in final deliverables Easier recognition of where the defects are Why a continuous integration server is better than any human doing the same tasks Humans are a necessary part of developing software. Software development is painfully hard. It takes concentration and creativity. That s why machines can t be used to write code. Embedded software can get really hard once we introduce constraints that the hardware most embedded systems run on into the mix. And creating tests for software is Sponsored By: Page 6 of 9

7 just as hard. Good testing practice requires the same sort of concentration and creativity, especially if we want to write tests that are repeatable and aid us to quickly qualify a set of work as done. But running tests isn t the same sort of thing. Running tests are repetitive and boring, because we setup the environment, run the test, observe if we reached the correct result, and repeat for each test in the suite. Using humans to do these tasks is not just wasteful, it s actually risky. Why? Because humans tend to make mistakes in observations. Or forget to follow each and every step. Or interrupt themselves while they help answer a question for someone and then forget exactly where they were. Oh, and humans get tired and cranky if you work them 24 hours a day. But computers are just the opposite. Once they are programmed to run a test suite, they can do that time after time, 24 hours a day and always doing it right, and without complaints that they need a bathroom break. So, let s offload our boring and repetitive work to those machines of burden that we call computers. Now all we have to decide is what they should do to help us stay in the zone where we can concentrate and be creative on the things that we can do well. Let s first agree on what a CI server really is. Many people have a rudimentary CI server, which gets triggered by a check-in to the code repository. The workflow is: Before check-in, merge any conflicts that the developer code has with the repository code. Commit the code for check-in. The commit then is seen by the CI server, which checks out the code from the repository, builds the code, and reports any build errors back to the development team, usually through . This use of CI, which I will characterize as a continuous build server, has a really great benefit: frequent check-in s will help keep the conflict merging process easier, as there are less radical changes that occur when the merge conflict resolution process is done after every committed change than if we bunch up the changes for a huge conflict merge later. Easier conflict merging takes less time and will result in less wasteful rework to get the system back to working. And we, as humans, don t have to remember to do each and every little step along the way. Sponsored By: Page 7 of 9

8 Nothing wrong with that! But we can do so much more with little additional effort. Let s start with unit tests. For embedded software, we should be writing our code in a simulation environment for the target device using test-driven development (TDD) practices. TDD requires us to write a white box unit test that fails before we write our code and passes after we write it correctly. By using a unit testing framework (such as Unity, CxxTest, CppUnit, and others), we can easily tie these tests together and run them against our simulated device. The results of the tests are then available for automated parsing to determine which tests are passing, and which are failing, as well as where the failures are occurring. By ensuring that the unit tests run quickly (by the use of test doubles to isolate the system under test from depended-upon components), we can create an environment that can be run after a successful compilation of the code. That way, when we check-in our code, not only do we know that the code compiles, but we also ensure that any changes made to the code do not adversely affect any previously implemented functionality. This freedom from fear that the CI server gives you raises quality far more than any shallow assurance from a build server that your code still compiles. And we can do even better than that. Assume for a moment that we adopt a rule that every defect we find in the field and then fix has an associated black box test written for it. We place this test into our regression suite of integration tests, which we automate using the CI server. Now, assume that we schedule the regression suite to be run in the middle of every night. Voilà! We guarantee that every bug we ever swatted down stays fixed, or at least we know that we somehow unfixed it yesterday. It really beats trying to have QA continuously trying to run their regression suite all the time, which they don t have time to do anyway. And manual running of any test suite is repetitive, error prone, and boring. Remember, those sorts of tasks are error prone for humans, but perfect for computers. Conclusion Continuous integration servers are cheaper, faster, and just plain better than humans for activities related to the repetitive tasks associated with software development. Sponsored By: Page 8 of 9

9 Resources from IBM Delivering a smarter planet: an exploration of systems of systems Driving innovations through software in smart products ebook Systems engineering for dummies About IBM At IBM, we strive to lead in the creation, development and manufacture of the industry's most advanced information technologies, including computer systems, software, networking systems, storage devices and microelectronics. We translate these advanced technologies into value for our customers through our professional solutions and services businesses worldwide. Sponsored By: Page 9 of 9