Print

Software Engineering for Everyone

by Dennis E. Hamilton
04/04/2001

Learning to program is easier than ever before with excellent, freely available tools like Python and a community of contributors willing to help newcomers. However, most employers want more than just programmers, they want software engineers. It takes consistent discipline to deliver complex systems into an imperfect world. Whether for commercial air travel, distributing electrical power, operating the postal service, visiting a comet, or giving depositors reliable access to their own accounts via automated teller machines anywhere in the world, we rely on the fruits of engineering discipline daily. Even when developing simple systems, you'll encounter some of the same difficulties an engineer confronts in developing large-scale software.

The elements that help software engineers win at their game can help you. You can develop engineering discipline as you go about learning the craft of programming by keeping in mind three elements of engineering discipline: predictability, collegiality, and accountability. Working closely with engineers in my early days as a programmer taught me the importance of these elements. Together, they form the basis of what I call Software Engineering for Everyone.

Predictability: make it so, keep it so

The key principle of predictability is divide-and-conquer. To divide, create interface agreements. With interface agreements in hand you can break down implementations into modules, making parallel and independent effort possible. This is the most difficult aspect of engineering discipline that you will ever encounter. To succeed you must first define and organize programs and the work to produce them without having already done it.

In 1975, I provided system architecture for a nationwide on-line processing system. The programming team was short-handed. They gave me the opportunity to develop real-time display interface software for minicomputers to be installed in 100 business offices across the country. It was a critical element. We were outgrowing an initial release and urgently needed a new system. We rewrote all of the software, increasing performance and function all at once.

Applying the key principle of predictability, I redefined the application interfaces for interacting with display terminals first. I turned one ill-defined interface with a complex set of parameters into a set of simple interfaces. I introduced one interface per operation. I gave each operation a single well-defined function clearly defining all behavior, operation-by-operation. It was like handcrafting an object-oriented implementation. Designing in an informal dialect of Algol, with the implementation in assembly language, our adherence to an object model came entirely out of disciplined use of the available lower-level tools.

I was nervous about introducing this change in place of something already familiar. However, the simplicity of debugging and verifying correct operation with cleaner, separate interfaces was too valuable to pass up. It also made the inspection and verification of my real-time implementation much easier. Because I could build the new interfaces atop the old implementation using a simple shim layer, I had a working prototype immediately. We began testing the prototype on a system that was already known to be reliable.

Since I had stable software to work with on the other side of my interfaces I could easily avoid finger pointing, and I could get at defects of my new code when it didn't behave the way the shim already did. This became critical as my software became later and later, running far behind the original plan. I had an incremental, unit-level test plan that I was determined to follow rather than do what my boss wanted which was to turn on the whole thing at once and see what happened. Although that was temporarily career-limiting, I have never regretted it.

With a stable interface and a working shim, we quickly caught, repaired, or worked around defects in the prototype. In the end, integration and confirmation testing went off without a hitch. Only one bug was uncovered and that was in a new administrative interface having no counterpart in the old system.

I would love to offer this as a complete divide-and-conquer success story, but there is more, an unexpected lesson: when hardware developers start programming, it is easy to stop thinking like an engineer. The new software was so fast we uncovered timing problems in the firmware of the terminal hardware. Because the new interfaces completely hid the hardware, I was able to add special communication delays in places where the firmware needed more time to complete its operations. Application modules weren't touched. Still, some problems never went away. The terminal hardware would lock up from time to time. When the firmware became autistic, up to 16 displays simply stopped talking, and my software quietly timed out, shutting down user sessions that appeared to have simply gone away.

Hardware technicians pleaded for a way to diagnose the terminals from the minicomputer. I gave them a way to analyze the data my software had about all of the displays, but it was useless. The hardware interface didn't provide the kind of information that the technicians needed. We were dealing with a computer program hidden in firmware where none of us could get to it, let alone fix it. Well-known industry-standard peripheral interfaces didn't have these limitations, but the engineers who designed this equipment thought it was more important to use an inexpensive, limited interface. No amount of add-on software could compensate for that unfortunate point of inscrutable failure. When I had been at a mainframe manufacturer a decade earlier, you couldn't build, let alone ship, a computer system if you couldn't show people how to troubleshoot and repair it. That principle didn't transfer over so well when software started becoming part of everything.

Predefining and sticking to your interfaces allows modules to be brought together -- and substituted -- without surprises. Preserving predictability across many cycles of alteration and refinement is important. When you study the interfaces used by different software packages, you'll notice that some interfaces work better than others, and you'll begin to see why. It is the result of a wonderful paradox: freedom for design and innovation is carved out of the space created by agreeing to constraints that will never be violated. You must first set up the rules of the game so there is room to play, and to play again, and then to win.

Pages: 1, 2

Next Pagearrow