Reducing Errors with Electronic Documents – using data captured during a test to improve the utility of the manufacturing test system
Let’s talk about ways that enhanced test systems can reduce errors in a production environment.
For products that need to be tested during assembly by manual or semi-automated operations, moving data between electronic documents and records brings improved consistency, faster production rates, lower assembly errors, and visibility into the production process.
The first area we recommend is putting the assembly and test instructions into electronic form. Instructions go by many names at different companies. We hear terms like work instructions, assembly manual, and so on. The idea is to replace any printed material that the operator might use with content displayed on a computer screen.
When used in a test system, the written instructions are the basis of the test steps in an electronic test sequence.
I should note that using an electronic test sequence does not imply automation of the test steps. A test step can be completely manual. It could display some text and an image to describe the way to assembly a part or to take a measurement. No automated actions are done. Only the automation of the instructions.
Once you move to electronic test sequences, you can use document management. For example, you can use:
- Verification and Validation management so that only released manuals are presented and executed at the test station.
- Version control so that the latest version can be located and earlier versions can be accessed for older model parts.
Note that taking full advantage of document management means that business processes need to be in place.
The second area we recommend for electronic documents is to capture the results of the assembly and testing into electronic data so these results can be analyzed within, and even across, product lines.
Some aspects of a test sequence execution that can be revealed:
- Since a test sequence is driving the test step execution, the test system can be programmed to capture all results, such as reasons for skipping a test, observations during assembly, the time a step takes to complete, and others.
- If measurements are manually typed into the step results, the data can be sanity checked. If automatically measured from the equipment, manual typos are eliminated.
- These data should be made searchable (database, XML, etc.) to allow production managers visibility into the status of tested parts. You can then create reports for frequency of various failure modes, duration of steps as a function of operator, a graph of the quantity of parts tested versus time of day, and so on.
Both the document management and data capture often require IT activities, so get them involved early. We also noticed that some of our clients have IT create production dashboards with these results.
Questions to Ask
Here are some questions to ask before deciding to implement electronic documents:
- How many instruction documents do you have and how often do they change?
- If you only have one test system with one instruction manual that never changes, the effort to modify the test application software to work with electronic documentation can be as simple as scanning the pages of the manual and presenting these on the tester screen.
- Changing over to a full-fledged electronic test sequence method makes sense when you have more than around 3 combinations of test systems and products, such as one tester for 3 products or 3 testers for one product each.
- Are you in a regulated environment and you need to manage the assembly and test instructions to show an auditor that the right instructions were used on any given part and that the results from each part tested are traceable?
- Do you have support from whomever maintains the test systems?
- We have seen situations where management did not want to spend the effort to automate reporting because they wanted control over the content of each and every report – manual report generation ensued.
- Do you have a Lean initiative?
- The usefulness of the electronic assembly and test results is enhanced when you have a reason and means to analyze the results.
Here are some recommendations based on our experiences working with clients to add electronic documentation to a test system.
For operator test instructions, the biggest benefit we’ve seen happens on test systems that are used to test multiple different models of parts. Switching testing from one part to another can cause confusion.
So, we recommend the following things.
- Convert your instructions into an executable electronic test sequence.
- Have the test application choose the test sequence automatically based on part ID, rather than searching for the appropriate paper document.
- Use test limits files based on model and retrieve automatically with the test sequence.
- Enhance the test application to notice versions, so outdated versions are not used by mistake.
- Enhance the test application to recognize the release status of the instructions, so only qualified people are allowed appropriate access to the instructions.
These enhancements are among the most complex of the items mentioned. Not because of the technical nature, which is not huge, but because they impact other parts and people within your company. It can be challenging to convince others to make the enhancements.
For the electronic records, our recommendations are based on including the operational data with the test results. Besides collecting test results on product performance, include information useful to the production group such as:
- Test duration for each step.
- Comments about the results of the test steps.
- Date/time stamp of the tests.
- ID of the operator performing the tests.
- All the relevant raw data used to access the pass/fail disposition of each test step.
For example, the test duration data can be used to understand the test stand usage for Overall Equipment Effectiveness, or OEE, computations. OEE involves Availability (the time the tester is available), Performance (how fast it takes to test a part), and Quality (first pass yield). This OEE metric can help the production group improve on efficiency by identifying the poor performers.
An actual project that Viewpoint delivered might be useful to discuss. The company makes aerospace actuators. They are a high-mix, low volume production facility. Some parts are made at less than 10 per week and there are 1000s of part models.
I’ll concentrate on some of the fundamentals since the scope of this project has grown to encompass worldwide production.
First, the assembly and test instructions have gradually been moved into electronic test sequences. Many sequences are still semi-automated, since automating every step would have been expensive. The combination of the programmable test sequencer and the electronic instruction documents was the first hurdle that allowed all the rest to happen. We know this because the client originally had the instructions in XLS and database files with no automated sequencing through the instructions. That effort was abandoned due to an ineffective outcome and replaced with the automated test sequencer.
Also, this customer wanted the test steps to be configurable during the running of a test, so that engineers could assist with troubleshooting parts that were behaving oddly. All the results of are saved, normal operation or not.
Second, the data produced by the test system, including both the test results and the operational information, are used to understand things like:
- First pass yield pareto
- Product touch time and test cycle time
- SPC charts and usual stats, such as Cpk
- WIP dashboard
- Per unit device history brings visibility on each part across multiple test systems, even for parts that are reworked.
The client has put a lot of effort into this electronic documentation effort. After some initial proof of concept efforts, a substantial development effort was undertaken with payback in less than 2 years.
How to get Started
Begin by understanding of the level of effort to convert the documents into test sequences. How many test sequences will there be and how many steps do you have to convert?
Then, assess how much test data will be produced for all these sequences. Data storage is cheap nowadays, but still needs to be considered.
Also, scope out the effort to create methods to store the electronic test sequences and don’t forget the benefits of putting these sequences under version control. Also, all those test results need to be stored somewhere, so you’ll be designing the layout of the network folders and possibly database structures to hold all these results. IT will need to get involved.
Once that work is done, you can enhance your test system application software to use the test sequences and save the results.
Then you’ll need to developed one or more applications to “slice and dice” the results so you can create reports for production management. You might also want to create a metrics dashboard.
The end goal of all this is being able to make intelligent decisions that improve operations, a perfect goal for Lean initiatives.
Waste Reduction using Manufacturing Test as a Lean Enabler
Let’s dive into some details to better understand how test data can reduce waste in a production environment.
Many companies have embraced Lean initiatives to reduce waste in manufacturing. Lean looks for waste in areas such as Inventory (including WIP), Waiting, and Defects. Identifying the contributors to waste requires data.
The test system can help produce data that will lead to decisions about:
- Decreased cycle time
- Less inventory
- Increased productivity
- Increased capital equipment utilization
The misconception that many companies have are that these metrics are usually associated with manufacturing operations, not test systems. But, done properly, the data pulled from test systems give key information about these factors.
A hurdle often appears when integrating the test system data into these business metrics since interaction with the IT group is necessary. You’ll want them to help you connect data between the test system and the business databases, such as MES and ERP.
But management will likely need to approve that assistance. Why would they do that? Let’s look at how waste can be removed at the test system.
With test systems, you can gather information on the status of a part, how it was tested in prior test systems during the assembly process, and how it should be treated by the test system at which it is being tested at the moment. For example, waste can be removed by knowing:
- Has the part and all its subassemblies already passed all the required tests?
- Are there any test steps that are creating a bottleneck in production? If so, how can they be sped up?
- Is any test system being kept idle for unreasonable amounts of time?
- Are some test operators having difficulty with a particular assembly step?
These data feed directly into Lean initiatives.
Questions to Ask
Waste reduction depends on collecting information on the status of a part and how it was tested. The data collection needs to be tied to efforts to analyze the data, since there is no point to measuring if no one is going to look at the results.
So, questions worth considering are:
- Is management behind this effort to use test systems to improve production?
- Is the company embracing Lean?
- Since the data will be shared with business systems, the test system will need to interface with production databases, such as in your MES or ERP systems. Is IT ready to support this effort?
- Which test system or systems seem to be associated with the most waste? How do I make these decisions?
Review the cost volume of products going through each test system, including any rework expenses. Identify the testers that see the largest costs and target these first.
You’ll want to access the business production databases to put context around the test data. These databases are contained in the company MES and/or EPR application. Examples are things related to the work order for the part being tested, such as listed in the leftmost box labeled “inputs”.
IT will have to be involved to help you get access these business databases. If you think you will encounter resistance, you may want to consider gathering data on just one test system. You could do so crudely, perhaps even manually at first.
When you have been able to make a business ROI case with one test system, you gain enough credibility that management encourages you to automate this crude and/or manual process and apply it to other test systems.
Let’s review a project with an Aerospace client. The outcomes of this project have been so successful that the test system enhancements we developed in an initial few test systems is now being pushed out to many test systems at facilities around the world.
This client is a high-mix, low-volume producer of parts for aircraft and space vehicles. As such their test systems are manual or semi-automated with complex assemblies. The enhancements made to their test systems are as described earlier with electronic docs and test data coupled to their MES/ERP business data.
The table below shows the areas of waste reduction and the associated average gains in production capabilities. The charts in the upper right show the Value Stream Map for various products.
|Test executive & data management||10%|
|Unified Report Generation||5-8%|
As you can see, the client saw an average of 30% reduction in waste. Plus, the enhanced automation has allowed this client to manage seven times the production volume as 10 years ago without significant increase in production personnel.
Perhaps the biggest improvement in identifying waste has been their ability to understand WIP status and problem areas. In the past, some production problems were never addressed due to the attention required by the immediate “fire fight” of the day. Now, with rapid access to the test and assembly data provided by these automated database transactions, the client has a dashboard which can identify issues in real-time.
Assembly errors have reduced and traceability has been improved to such an extent that they have received accolades from their customers.
The payback for the original enhancement effort has been estimated to be about 1.5 years. After the initial development push off and on over the past several years, enhancing other test systems now takes about between 1 to 4 weeks for each test system, depending on the complexity of the part being tested. So, the payback time is much less than 1.5 years for these incremental enhancements.
How to get Started
Start by enhancing your test system to gather data on the performance of the operators, test systems, test steps, and part failures. For the first test system, you don’t need to get IT involved to in order to save the results. You can if you want, but that’s not needed.
Start by enhancing the usual pass/fail data for each test step by saving all the measured data, limits used during the test, any processed results, and so on, in addition to the usual final pass/fail disposition. And then enhance your test application to gather information about the operational aspects of the test, such as test start/stop timestamp for each step, execution time for each step, operator ID, and part ID.
The collection of this information will allow you to do some initial analyses on the performance of the test system and the parts being tested. These initial results will start discussions on ways to improve the making of the parts being tested at your test systems.
If you’d like to hear some thoughts on how you might enhance your test system for your scenario, feel free to reach out to us here. If you’d like to see more about how we can help with test system development, go here.