The title is a bit misleading as I do not want to talk about our experiences about which factors will make an initiative a success. I would rather show a method, how we can measure what the employees of different organizations think, that makes the initiative successful. And when I say think, I really mean, what their belief is and not what they would say if asked this question.
The idea is to apply a statistical method employed in psychology (but other fields as well) to evaluate a carefully designed survey, to quantify the strength of the beliefs of the respondents concerning the link between different factors influencing our outcome variable. This sounds unnecessarily complicated, but let me give a concrete example.
Imagine we want to measure what people think of which factors determine the success of a lean initiative. Just asking this simple question is not very helpful because there is a near certainty that we will get all kinds of incommensurable answers – everybody understands “success” differently and the factors mentioned will also be wildly different, from the very concrete to the philosophical. So, obviously we will have to ask more specific questions.
Concerning success, for example, we might have our own definition of what a successful lean initiative looks like – many Kaizen events, Obeya rooms regularly visited, waste elimination, 5S in place etc. All these elements REFLECT the status of the initiative , meaning, that if we ask questions about these in a successful lean company we will generally get high scores and vice versa. We do not need to capture ALL that defines a successful implementation though, we just need to be reasonably sure that all successful companies will have reasonably high scores on most of our questions.
To put it in more special terms, we assume, that there is a hidden variable out there at the company that we can call Maturity of Lean. We can not measure this directly, indeed we do not even have an operational definition of it. However we can ask questions that we expect that will reflect on the state of our Maturity of Lean hidden variable, a bit like looking at mosaic stones and trying to figure out the whole picture. To get this information we shall define one question for each aspect of Maturity we came up with in our survey.
Using the same logic we can assume that there is another hidden variable at the company called Management Commitment and another one called Tool Proficiency and so on. To get an idea of the state of each of these we shall design several questions that we believe will reflect on the status of the hidden variable. E.g. for Tool Proficiency we might decide to ask the number of successful Kaizen events , the number of employees involved, the amount of money saved, the number of areas with visual management present and so on. In the same way we may define a number of questions around management commitment and so on.
As a side issue, wherever possible, we should use Likert scales for the answers to facilitate the analysis.
Now, once we collected the answers, we will want to analyse the relationship between these hidden variables and their effect on our similarly hidden outcome variable. In principle, it would be possible to analyse more detailed effects, like the impact of 5S on the number of Kaizen events, but this means building a large number of correlations (one between each success component and influencing factor) and this will be statistically absolutely unsound unless we apply some corrections (like the Bonferroni correction if we want to keep it simple) and really also way too detailed. Another problem will be that many of our independent variables (the answers to our questions) will be correlated , which will make a traditional regression analysis very difficult,
Anyway, he real questions are at the level of the hidden variables – e.g. does Tool Proficiency contribute to the success of the initiative and if yes how strong is it’s effect? Once we get answers at this level we can go one step deeper and analyse the contribution of each component to the outcome, like: does 5S have a strong contribution to the Tool Proficiency or is it the Visual Management? , and the like..
The statistical method to analyse our survey is called PLS SEM (Partial Least Squares – Structural Equation Modelling). Not delving into the mathematics , it has essentially 3 steps.
- We describe which survey questions relate to which hidden variable. As we designed the survey with hidden variables in mind, this will not be a difficult exercise. Based onthis info, the system will optimally construct the hidden synthetic variables, as linear combinations of the respective inputs. That will be roughly the PLS part of the method
- We start with some broad assumption on which hidden variable can impact which other hidden variable. E.g. we can assume that management commitment has an impact on tool proficiency or that leadership has an impact on management and also on tool proficiency etc. Based on these assumptions the system will calculate the strength of the influence of a hidden variable on another – that would be the SEM part.
With these two elements we can run the PLS SEM model and then we have the 3rd step which will be the interpretation of the model. Here we can quality check the structure of our hidden variables and see whether we picked the categories correctly . Then if all our hidden variables are correctly built we can check the model as if it were a normal multiple regression and reduce it based on the p values in the usual way. So we end up with is a statistically sound high level description or model ) of what the survey participants think of a such a complex issue as the success of a lean initiative.
The method is not limited to survey evaluation though. In any industry were we have some customer requirement that can be described by several measured values, we can apply the idea of hidden variables that are reflected by these measured values. Indeed my first exposure to the method was at a time we worked for a number of companies manufacturing paint. One characteristic of paint that is of interest to the customers is how “shiny” the paint is. This will be measured by a lab instrument at several angles, say, at 20 degrees, 40, 60 80 . We can model the customer requirement as the hidden variable “Shine” reflected by the measurement values at different angles. Then we can use the hidden variable as the Y in a regression model, that can have manufacturing parameters , components of recipes and the like giving us a lot of insight into the way we can improve our process. The same logic can be applied in the food industry in many places as well.
In summary, we know that the most important step in a process improvement is the accurate capture of the voice of the customer. As Six Sigma has a reputation of relying strongly on measurements and statistics, adopting a method that will link our strength to our aspiration is definitely something we should do, and do more often.