Viewpoint News, October 2009

Product Complexity

by James Campbell - jac@viewpointusa.com

Last month, I introduced the ‘7 Challenges for Highly Effective Test Engineers’. These challenges affect test engineers whether they are embarking on the development of a new test system, or upgrading an old one.

Last month gave a highlight of each Challenge. This month starts a more detailed exploration of each topic. First up is Product Complexity. Next month will be Development Tools.

Enhanced product complexity impacts two core aspects of test systems. First, the test requirements demand additional and perhaps difficult measurements. Second, the test equipment may not be capable of achieving measurement accuracy or timing.

After years of delivering test systems, Viewpoint test engineers have learned a few ways to address these issues. I’d like to share them with you, so please read on.

Testing for Design and Reliability

When a product needs testing for design validation or reliability, the main goal is performance characterization. The measured data are usually analyzed statistically across prototypes (looking for performance variations) or for failure modes. The latter is sometimes accompanied by prognostic data analysis.

The test engineer should work with the design team to understand as completely as possible the reason the particular data are needed. Then, generate the required measurement methods from this list of data. Finally, create the specifications for the test equipment and software needs. This “backwards” approach to creating requirements works very well: the design team knows what they want to measure and are happy to let the test engineer make it happen.

And do not forget the data analysis and generation summary reports. The design team typically has strong preferences regarding the presentation of data for either their own analysis tools or as final reports developed by the test application.

In summary, don’t start without details for the following:

• Accuracy and precision of all measurements.
• Acquisition rates and durations of all measurements.
• Timing and synchronization of various measurements.
• Suspected failure modes and methods to induce them.
• Data analysis and reporting.
• A catalog of other possible and desirable types of measurements, since sometimes it is easy to measure a parameter which the design team thinks is difficult.

If you do nothing else, be sure the software is modular and flexible, since the design team will request frequent changes as data analysis suggest new measurements or test methods.

Testing for Manufacturing

For manufacturing test, product complexity impacts the usability of existing equipment. Perhaps the existing equipment can be used, but you need to do research to know for sure. Perhaps the existing equipment is incapable of new measurements and new equipment must be purchased. My recommendations for these situations are:

• Understand the capabilities of your instruments to avoid purchasing unnecessary equipment.
• Discuss marginally capable measurements the design team in case the test specs can be relaxed a bit. I’ve seen the test specs be more than needed for pass/fail assessment because the design team wants the product performance data for design analysis.
• Don’t use some arcane feature on an existing instrument. When you upgrade, you’ll forget that feature and the new device won’t have the capability (Murphy’s Law).
• If you do have to get new hardware, understand future needs as well as the present ones so the hardware will be useable for a long time.

Increasing complexity also impacts test time. Usually the additional complexity creates both more steps and more complicated steps than prior products. These new steps affect software development. My recommendations for this situation are:

• Good test architecture is crucial. Without a solid base, new steps may break existing ones or test execution may become unmanageable.
• Modularity helps. If you can reuse prior steps or rework them, development time is reduced.
• Work with the design team to rearrange steps if test execution timing is too long. Sometimes reordering steps can speed execution since device and equipment states don’t need to be changed in subsequent steps.

I want to be realistic and make clear that increasing product complexity is usually an incremental phenomenon and not an abrupt transition in product capability. While this is a good thing from the perspective of being able to modify a test system without a complete rework, it is also a bad thing in that incremental product complexities usually transform once-pristine test systems into band-aided and patched test systems. I understand the motivations for this transformation, and I’ve seen it many times, but it should be avoided because, ultimately,

Conclusion

Product complexity is increasing relentlessly. Usually, the change is incremental, but large steps happen too. Test engineers needs to be prepared for both, but the incremental situation is where insidious patching and band-aiding happen. Do what you can to prevent traveling that path. If I have one mantra it is that product test for both manufacturing and design\reliability benefit from modular test code.