Beyond Failure Analysis

In any product development or manufacturing environment, failures are bound to occur. However, failures would be less frequent if the engineers were to characterize the product, process, and system before problems occurred. Our product, Explicore, is a data analysis scorecard utility that provides an automated data capture, characterization, and analysis on various types of data. As you will see, the product was extremely useful in resolving engineering issues.

In many cases, engineers may not always know the problems they are facing in manufacturing when failures occur. The engineers must perform painstaking analysis to get to root cause. Effective analysis is tedious and difficulty arises when engineers have to resolve these problems quickly. Here are three examples of failures that went drastically wrong.

Example 1: a company fielded a product with anticipation that the product would be highly successful. Unfortunately, the product operated less than 10 hours before it failed and many customers quickly returned the product. The engineers had to scramble to resolve the technical issues. They were not successful identifying the problems and did not know where to begin. The engineers felt considerable pain from management when they were unable to resolve the issues and the engineering groups started blaming each other. Engineering deployed Explicore to characterize and analyze all the parameters from functional test and Explicore immediately highlighted the problems; engineering corrected the design and the improved product was once again fielded. The cost of the product re-design was significant in terms of money, engineering resources, emotions, and customer loyalty. Based on the outcome of the product characterization and analysis, the organization predicted the mean time between failures (MTBF) to be 10,000 hours and later discovered the re-designed product exceeded this prediction. Customers were highly satisfied with the product, but it took a long time to re-gain the customer’s loyalty and trust. This characterization and analysis resulted in an annual warranty savings of nearly $9,000,000 year-over-year.

Example 2: a company completed a new design that was put into the initial production phase. The yield was an abysmal 10%. All the engineering groups assembled on the production floor to determine the root cause of the issues and were soon blaming each other for the poor yield. The customer was furious and informed the company to supply a down revision product until the issues were resolved and then the company would replace the down revision product with the new one. The engineering group ran the test data through Explicore which immediately characterized and analyzed all the parameters to determine what exactly was happening with the product. The top issues were uncovered quickly. The RF Circuits were found to be the major problem and design engineering redesigned the circuits. The updated design was put into a preliminary production run and the test data went back through Explicore and although there were minor issues with a few parameters, the product yield increased to 86%. The company sent the newly designed product to the customer and had to replace more than 500 units at a cost to the company of approximately $32,000,000. The company lost credibility with the customer and almost lost the account. It has taken a long time to re-gain the customer’s loyalty and trust.

Example 3: a company was to provide a combined technology product to a major customer. During the initial production run, the yield was a mere 15% and management and engineering attributed it to new processes being deployed in manufacturing. All the engineering groups were assembled to determine the problem and it became the ‘blame game’. It wasn’t a pretty sight with all the emotional upheaval. Engineering groups struggled to locate the root cause and were perplexed as to why the product did not work. Quality engineering used Explicore to characterize and analyze all the test data parameters to determine the problematic parameters. They uncovered design related issues as well as some potential test issues. The test engineers ran a Measurement System Evaluation (Gage R&R) on the test system and discovered there were issues with the test process (later discovery found issues with both the software and the hardware). Design had to scramble to get to the root cause of their issues and test engineering had to redo many tests. These efforts resulted in a 3 month delay in product launch. Finally, the yield increased to 92% with the redesign of the product and updates to test system. The company had the customer shut down and this delay cost the company thousands if not millions of dollars due to the delay – the penalty for being late. The estimated loss for the company was about $18,000,000 (in penalties and lost revenue). This little faux pas took the company a long time to re-gain the customer’s loyalty and trust.

If the engineers were to perform this characterization up-front in both design and production, there is a high likelihood that the amount of time spent on failures would have been minimized. The ideal situation is to generate a characterization baseline in the design process and manufacturing for the product, process, and system, which consists of capturing data, characterizing that data, and performing a preliminary analysis on the data. A baseline may be created manually which is extremely time consuming or automatically which literally only takes minutes to perform. Explicore is an excellent product to provide the automated data capture, characterization, and analysis. The product simplifies the process and provides a baseline characterization which leads to better visibility when a failure occurs. In addition, the engineers should periodically capture, characterize, and analyze their test data to continually monitor the health of the product, process, and system.

Engineers cannot learn from failures if they do not discuss and analyze them. However, the analysis needs to be thoughtful, fully discussed, and insightful. We know and understand that failure analysis is extremely important, but very tedious to carry out. Ask any engineer, examiner of an airliner crash, or a medical practitioner about failed systems. Unless deeper analysis of the nature of the error(s) is conducted, it is difficult to ascertain what needs to be corrected. Effective analysis of a failure is found in the meticulous and painstaking analysis that goes into understanding the problem. For example, hundreds of hours may go into gathering and analyzing data to sort out exactly what happened and what is learned from the analysis. When failure analysis consumes engineering, it is like a plague. It paralyzes the company, its people, and may stifle growth even if it is short in duration. Analysis can only be effective if the engineer speaks openly about what they know that enables a new understanding of what happened, especially when the engineer has characterized the product, process, and system. Typically engineers address only significant failures, rather than identifying and learning from all failures. We understand that failure analysis takes precious time and resources to address failures. We also realize that individuals will typically experience negative emotions when examining failures, so analysis of a failure requires openness, patience, and acceptance of ambiguity.

However, failure analysis is not enough. We need to consider doing something before the failures create larger problems. There are alternatives to preclude the need for failure analysis. One alternative is to use the automated process Explicore, a data analysis scorecard utility. Explicore is a utility that reviews all data for a product, process, and system, and statistically determines which parameter(s) need improvement. Here are some questions to consider. What would life be like if engineers were able to understand their product more fully before it is launched? Would it help people to glean a comprehensive understanding of the product, process, and system before a failure occurred? Would it be helpful if the product, process, and system were characterized (baselined) before failures were to occur? Would the company then be able to concentrate more on growth and profitability?

Failures would be less frequent if the engineers were able to understand the nature of the problems before they occurred. The best alternative is to characterize the entire data set early in the life cycle. This becomes a 3-pronged approach: identify areas of product improvement during the design and development phase, baseline the manufacturing process; and baseline the system. Note that this process may be applied during any portion of the life cycle to baseline and periodically monitor them to determine the health and integrity of a product, process, and system. The data provides in-depth information about product design, manufacturing, test, and service processes. It simply leads to improving the product and process development cycle, and the manufacturing cycle faster through this continuous improvement effort. This applies to the service industry as well – they will be better prepared when things go wrong.

This idea of generating a baseline and periodically monitoring the product, process, and system ensures that continuous improvement will ultimately decrease the overall cost of product ownership (design, manufacturing, service, and warranty cost). Explicore provides the automated method to generate a solid baseline for data capture, characterization and analysis, and this baseline will provide more insight when discussing, analyzing, and applying failure analysis. Management and engineering would minimize potential negative emotions since they will have a higher level of confidence in the product, process, and system of the product. In addition, a periodic analysis, based on product volume, will continually monitor the product, process, and system to validate their health, especially when any changes occur (i.e. design changes, process changes, or test data changes).

This recurring data capture, characterization, and analysis helps to reduce the cost of failure analysis if continuous improvement actions are implemented during the entire life cycle. Note that a product, process, or system does not need to fail to be improved. A product, process, or system that is within the specification may be in need of improvement if the data is skewed (where the predicted yield is low).

The benefit of going beyond failure analysis is clearly a cost-benefit decision of the company, but it becomes a pay-me-now or pay-me-later decision. The pay-me-now decision may cost a little more up-front, but the benefits of performing a characterization baseline far outweigh the cost of warranty, lost customer loyalty and trust, or simply the loss of a customer. The pay-me-later decision can be very costly since the product is already fielded with significant warranty implications attached to it. These implications include embarrassment, loss of customer loyalty, potentially the loss of a customer, and the expense of redesign and fielding product upgrades, etc.

In addition to the technical aspects of periodic data capture, characterization, and analysis, improving the product, process, and system has important organizational benefits. First, the data provides an opportunity for others not involved in the data process to learn from it. Second, others may bring new perspectives and insights that further improve the product, process, and/or system (this applies to service as well) which help to counteract self-serving biases that may alter perceptions of those directly involved. With failures, the tendency is for people to blame others or forces beyond their control. If this tendency goes unchecked, it reduces the ability for the engineers to effectively learn. Explicore helps to glean an in-depth understanding of the product, process, and system which minimizes the emotional impact on a company.

Lastly, the value of learning from utilizing the data may prevent simple mistakes from being overlooked prior to launching a product or service. Many scientific discoveries have resulted from attentiveness to detail including the simple mistakes that are often overlooked. With attention to minor details that Explicore brings to the table, the fielded product, service, or process will have minimized the possibility for failure. Ultimately, if engineering were to use Explicore earlier in the life cycle and learn from it, then they may be able to preclude failures that could otherwise occur without the data characterization process being invoked.

Please visit TestSoft’s website www.testsoftinc.com to learn more about Explicore. We would like to show you first-hand why TestSoft stands above the competition. It would be our pleasure to demonstrate TestSoft’s capabilities and show you how the unique features of Explicore – a Data Analysis Scorecard Utility would benefit you.