Six Sigma is now according to many business development and quality improvement experts, the most popular management methodology in history.
Table of contents
1.2. six sigma history
1.3.2. six sigma elevator speech
1.4.2. see also
Six sigma 
Six Sigma is now according to many business development and quality improvement experts, the most popular management methodology in history. Six Sigma is certainly a very big industry in its own right, and Six Sigma is now an enormous 'brand' in the world of corporate development. Six Sigma began in 1986 as a statistically-based method to reduce variation in electronic manufacturing processes in Motorola Inc in the USA. Today, twenty-something years on, Six Sigma is used as an all-encompassing business performance methodology, all over the world, in organizations as diverse as local government departments, prisons, hospitals, the armed forces, banks, and multi-nationals corporations. While Six Sigma implementation continues apace in many of the world's largest corporations, many organizations and suppliers in the consulting and training communities have also seized on the Six Sigma concept, to package and provide all sorts of Six Sigma 'branded' training products and consultancy and services. Six Sigma has also spawned manay and various business books on the subject. Six Sigma, it might seem, is taking over the world.
Interestingly while Six Sigma has become a very widely used 'generic' term, the name Six Sigma is actually a registered trademark of Motorola Inc., in the USA, who first pioneered Six Sigma methods in the 1980's. The original and technically correct spelling seems to be Six Sigma, rather than 6 Sigma, although in recent years Motorola and GE have each since developed their own sexy Six Sigma logos using the number six and the Greek sigma character s.
Six Sigma is now a global brand and something of a revolution. But what is Six Sigma?...
six sigma definitions
The answer is that Six Sigma is lots of things.
First, Six Sigma is arguably a very clever way of branding and packaging many aspects of Total Quality Management that exist in their own right, regardless of the development of Six Sigma. Read the section about Total Quality Management and 'Excellence' and you will understand this.
Six Sigma is lots of different things because it had different meanings over time, and also because it is now interpreted in increasingly different ways. And Six Sigma is still evolving.
The UK Department for Trade and Industry says Six Sigma is:
"A data-driven method for achieving near perfect quality. Six Sigma analysis can focus on any element of production or service, and has a strong emphasis on statistical analysis in design, manufacturing and customer-oriented activities." June 2005.
Here's the DTI fact-sheet on Six Sigma - please note this is Crown copyright.
Motorola Inc., who first developed the methodology in the mid-late1980's and who provide extensive Six Sigma training and consultancy services, provide the following definitions:
"...Six Sigma has evolved over the last two decades and so has its definition. Six Sigma has literal, conceptual, and practical definitions. At Motorola University (Motorola's Six Sigma training and consultancy division), we think about Six Sigma at three different levels:
- As a metric
- As a methodology
- As a management system
Essentially, Six Sigma is all three at the same time."
"...Six Sigma as a Metric: The term "Sigma" is often used as a scale for levels of 'goodness' or quality. Using this scale, 'Six Sigma' equates to 3.4 defects per one million opportunities (DPMO). Therefore, Six Sigma started as a defect reduction effort in manufacturing and was then applied to other business processes for the same purpose.."
"...Six Sigma as a Methodology: As Six Sigma has evolved, there has been less emphasis on the literal definition of 3.4 DPMO, or counting defects in products and processes. Six Sigma is a business improvement methodology that focuses an organization on:
- Understanding and managing customer requirements
- Aligning key business processes to achieve those requirements
- Utilizing rigorous data analysis to minimize variation in those processes
- Driving rapid and sustainable improvement to business processes.."
"..At the heart of the methodology is the DMAIC model for process improvement. DMAIC is commonly used by Six Sigma project teams and is an acronym for:
- Define opportunity
- Measure performance
- Analyze opportunity
- Improve performance
- Control performance.."
"...Six Sigma Management System: Through experience, Motorola has learned that disciplined use of metrics and application of the methodology is still not enough to drive desired breakthrough improvements and results that are sustainable over time. For greatest impact, Motorola ensures that process metrics and structured methodology are applied to improvement opportunities that are directly linked to the organizational strategy. When practiced as a management system, Six Sigma is a high performance system for executing business strategy. Six Sigma is a top-down solution to help organizations:
- Align their business strategy to critical improvement efforts
- Mobilize teams to attack high impact projects
- Accelerate improved business results
- Govern efforts to ensure improvements are sustained.."
"..The Six Sigma Management System drives clarity around the business strategy and the metrics that most reflect success with that strategy. It provides the framework to prioritize resources for projects that will improve the metrics, and it leverages leaders who will manage the efforts for rapid, sustainable, and improved business results.."
© Copyright 1994-2005 Motorola, Inc.
General Electric (GE), the first large-scale adopters and advocates of Six Sigma after Motorola, and considered by most experts to have been responsible for Six Sigma's rapidly achieved high profile, provide the following definitions of Six Sigma:
"...Six Sigma is a highly disciplined process that helps us focus on developing and delivering near-perfect products and services. Why 'Sigma'? The word is a statistical term that measures how far a given process deviates from perfection. The central idea behind Six Sigma is that if you can measure how many 'defects' you have in a process, you can systematically figure out how to eliminate them and get as close to 'zero defects' as possible. To achieve Six Sigma Quality, a process must produce no more than 3.4 defects per million opportunities. An 'opportunity' is defined as a chance for nonconformance, or not meeting the required specifications. This means we need to be nearly flawless in executing our key processes."
"...At its core, Six Sigma revolves around a few key concepts.
- Critical to Quality: Attributes most important to the customer
- Defect: Failing to deliver what the customer wants
- Process Capability: What your process can deliver
- Variation: What the customer sees and feels
- Stable Operations: Ensuring consistent, predictable processes to improve what the customer sees and feels
- Design for Six Sigma: Designing to meet customer needs and process capability..."
© Copyright General Electric Company 1997-2005
The Isixsigma organisation, which seems to be the biggest online 'community' of Six Sigma practitioners, was founded in 2000, and is owned and run by a number of 'quality professionals'. Isixsigma provides the following main definition of Six Sigma (which actually serves as an introduction to several other very detailed Six Sigma definitions contained in the Isixsigma resources):
"...Six Sigma is a rigorous and disciplined methodology that uses data and statistical analysis to measure and improve a company's operational performance by identifying and eliminating 'defects' in manufacturing and service-related processes. Commonly defined as 3.4 defects per million opportunities, Six Sigma can be defined and understood at three distinct levels: metric, methodology and philosophy..." July 2005.
Here's a brief history of Six Sigma, and the Six Sigma name. Additionally, comments I've received about Six Sigma contain aspects of Six Sigma history.
Since the 1920's the word 'sigma' has been used by mathematicians and engineers as a symbol for a unit of measurement in product quality variation. (Note it's sigma with a small 's' because in this context sigma is a generic unit of measurement.)
In the mid-1980's engineers in Motorola Inc in the USA used 'Six Sigma' an an informal name for an in-house initiative for reducing defects in production processes, because it represented a suitably high level of quality. (Note here it's Sigma with a big 'S' because in this context Six Sigma is a 'branded' name for Motorola's initiative.)
(Certain engineers - there are varying opinions as to whether the very first was Bill Smith or Mikal Harry - felt that measuring defects in terms of thousands was an insufficiently rigorous standard. Hence they increased the measurement scale to parts per million, described as 'defects per million', which prompted the use the the 'six sigma' terminology and adoption of the capitalised 'Six Sigma' branded name, given that six sigma was deemed to equate to 3.4 parts - or defects - per million.)
In the late-1980's following the success of the above initiative, Motorola extended the Six Sigma methods to its critical business processes, and significantly Six Sigma became a formalised in-house 'branded' name for a performance improvement methodology, ie., beyond purely 'defect reduction', in Motorola Inc.
In 1991 Motorola certified its first 'Black Belt' Six Sigma experts, which indicates the beginnings of the formalisation of the accredited training of Six Sigma methods.
In 1991 also, Allied Signal, (a large avionics company which merged with Honeywell in 1999), adopted the Six Sigma methods, and claimed significant improvements and cost savings within six months. It seems that Allied Signal's new CEO Lawrence Bossidy learned of Motorola's work with Six Sigma and so approached Motorola's CEO Bob Galvin to learn how it could be used in Allied Signal.
In 1995, General Electric's CEO Jack Welch (Welch knew Bossidy since Bossidy once worked for Welch at GE, and Welch was impressed by Bossidy's achievements using Six Sigma) decided to implement Six Sigma in GE, and by 1998 GE claimed that Six Sigma had generated over three-quarters of a billion dollars of cost savings. (Source: George Eckes' book, The Six Sigma Revolution.)
By the mid-1990's Six Sigma had developed into a transferable 'branded' corporate management initiative and methodology, notably in General Electric and other large manufacturing corporations, but also in organizations outside the manufacturing sector.
By the year 2000, Six Sigma was effectively established as an industry in its own right, involving the training, consultancy and implementation of Six Sigma methodology in all sorts of organisations around the world.
That is to say, in a little over ten years, Six Sigma quickly became not only a hugely popular methodology used by many corporations for quality and process improvement, Six Sigma also became the subject of many and various training and consultancy products and services around which developed very many Six Sigma support organizations.
You will gather from the definitions and history of Six Sigma that many people consider the model to be capable of leveraging huge performance improvements and cost savings.
None of this of course happens on its own. Teams and team leaders are an essential part of the Six Sigma methodology.
Six Sigma is therefore a methodology which requires and encourages team leaders and teams to take responsibility for implementing the Six Sigma processes. Significantly these people need to be trained in Six Sigma's methods - especially the use of the measurement and improvement tools, and in communications and relationship skills, necessary to involve and serve the needs of the internal and external customers and suppliers that form the critical processes of the organization'sdelivery chains.
Training is therefore also an essential element of the Six Sigma methodology, and lots of it.
Consistent with the sexy pseudo-Japanese 'Six Sigma' name (Sigma is in fact Greek, for the letter 's', and a long-standing symbol for a unit of statistical variation measurement), Six Sigma terminology employs sexy names for other elements within the model, for example 'Black Belts' and 'Green Belts', which denote people with different levels of expertise (and to an extent qualifications), and different responsibilities, for implementing Six Sigma methods.
Six Sigma teams and notably Six Sigma team leaders ('Black Belts') use a vast array of tools at each stage of Six Sigma implementation to define, measure, analyse and control variation in process quality, and to manage people, teams and communications.
When an organization decides to implement Six Sigma, first the executive team has to decide the strategy - which might typically be termed an improvement initiative, and this base strategy should focus on the essential processes necessary to meet customer expectations.
This could amount to twenty or thirty business process. At the top level these are the main processes that enable the organization to add value to goods and services and supply them to customers. Implicit within this is an understanding of what the customers - internal and external - actually want and need.
A team of managers ('Black Belts' normally) who 'own' these processes is responsible for:
- identifying and understanding these processes in detail, and also
- understanding the levels of quality (especially tolerance of variation) that customers (internal and external) expect, and then
- measuring the effectiveness and efficiency of each process performance - notably the 'sigma' performance - ie., is the number of defects per million operations (pro-rate if appropriate of course).
The theory is entirely logical: understanding and then improving the most important 'delivery-chain' processes will naturally increase efficiency, customer satisfaction, competitive advantage, and profitability.
Easily said - tricky to achieve - which is what the Six Sigma methodology is for.
Most practitioners and users of Six Sigma refer to Motorola's early DMAIC acronym (extended since to DMAICT) as a way of reinforcing and reminding participants what needs to be done:
- D - Define opportunity
- M - Measure performance
- A - Analyse opportunity
- I - Improve performance
- C - Control performance, and optionally:
- T - Transfer best practice (to spread the learning to other areas of the organization)
Motorola emphasises that in order for Six Sigma to achieve 'breakthrough improvements' that are sustainable over time, Six Sigma's 'process metrics' and 'structured methodology' must be extended and applied to 'improvement opportunities' that are directly linked to 'organizational strategy'. It is difficult to argue with the logic. There is little point in measuring and improving things that have no significant impact on the strategically important organizational processes.
Six Sigma team leaders (Black Belts) work with their teams (team members will normally be people trained up to 'Green Belt' accreditation) to analyse and measure the performance of the identified critical processes. Measurement is typically focused on highly technical interpretations of percentage defects (by a which a 'sigma' measurement is arrived at - see the one-to-six sigma conversion scale below), and a deep detailed analysis of processes, involving organizational structures and flow-charts. Many other tools for performance measurement and analysis are used, for example the 'balanced scorecard' method, and 'process mapping', etc., depending on the processes and systems favoured by the team leaders and project statisticians, and what needs to be measured and analysed. Six Sigma does not stipulate specifically what analytical methods must be used - the organization and particularly the team leaders decide these things, which is why implementation and usage of Six Sigma varies so widely, and why Six Sigma will continue to evolve. Any analytical tool can be included within Six Sigma implementation.
Six Sigma experts and commentators commonly refer to typical failure rates of organizations that have not put particular pressure on their quality performance levels. Aside from anything else this at least helps to put the 'Sigma' terminology into a simpler mathematical context:
It is said that many ordinary businesses actually operate at between three and two and sigma performance. This equates to between approximately 66,800 and 308,500 defects per million operations, (which incidentally is also generally considered to be an unsustainable level of customer satisfaction - ie., the business is likely to be in decline, or about to head that way). Bear in mind that an 'operation' is not limited to the manufacturing processes - an 'operation' can be any process critical to customer satisfaction, for example, the operation of correctly understanding a customer request, or the operation of handling a customer complaint. Six Sigma is not restricted to engineering and production - Six Sigma potentially covers all sorts of service-related activities. What matters is that the operation is identified as being strategically critical and relevant to strategy and customer satisfaction.
A measurement of four sigma equates to approximately 6,200 DPMO, or around 99.4% perfection. This would arguably be an acceptable level of quality in certain types of business, for instance a roadside cafe, but a 99.4% success rate is obviously an unacceptable level of quality in other types of business, for example, passenger aircraft maintenance.
A measurement of five sigma equates to just 233 defects per million opportunities, equivalent to a 99.98% perfection rate, and arguably acceptable to many businesses, although absolutely still not good enough for the aircraft industry.
Here's a simplified one-to-six sigma conversion scale:
one to six sigma conversion table
|'Long Term Yield' (basically the percentage of successful outputs or operations) |
|Defects Per Million Opportunities (DPMO)||'Processs Sigma'|
You can see from the conversions above that the sigma scale is exponential. The difference between the DPMO equating to each whole number more than doubles as you move up through the scale. By my rough calculation, 'seven sigma' would equate to about 2 defects per 100 million opportunities (correct me if I'm wrong), which is perhaps a little over-demanding even for the aircraft industry, and that's perhaps why nobody bothers much with anything over six sigma.
Motorola and many other devotees of Six Sigma are increasingly at pains to point out that Six Sigma is nowadays far less concerned with the mathematical theory of the Sigma calculations, and a lot more concerned with the model's broader performance improvement methods, nevertheless, Six Sigma's complexity and variable interpretations are not helped by the difficulty in penetrating the original mathematical reasoning behind the essential Six Sigma metric: just exactly why does Six Sigma equate to 3.4 defects per million? What are the calculations which take us from 3.4 PPM (parts per million), ie., 0.0000034%, to 'Six Sigma'? Mathematical interpretations vary apparently. (If you can explain this in simple language, and less than a couple of hundred words, please do, and I'll gladly add the explanation to this page).
There is also difficulty in phrasing a single simple definition of Six Sigma. For example, the task of creating a Six Sigma 'elevator speech' (in other words - explain Six Sigma inside 30 seconds) continues to challenge many of the Six Sigma enthusiasts who frequent the growing Six Sigma web forums. If you have a good Six Sigma 'elevator speech' please send it, and I'll gladly add it to this page.
Here is a suggested Six Sigma elevator speech (thanks Steven, Jun 2010): "Sigma is the symbol of standard deviation, a measurement of deviation of a sample from the population average. Each sigma you depart from the average, the event, in this case failure, becomes more an more improbable. At 6 sigma, the probability is about 3.5/million. But this is just the statistical side of Six Sigma. The bulk of the work in a Six Sigma project would be in defining failures, measuring deviations, and other activities which ultimately lead to product quality. In fact, Six Sigma is used as a term for a management style, with the ultimate goal of high levels of customer satisfaction."
Can you offer a better Six Sigma elevator speech than this? If so please send it.
See also the Six Sigma elevator speech funny story below.
Aside from its definitions, the Six Sigma concept now has a life of its own, open to a range of interpretations, beyond the control and reach of the early Six Sigma originators.
I heard someone say once that Six Sigma is a bit like Naomi Campbell - an attractive, seductive, yet highly complex model. (Also, sexy, expensive, and has been known to fall over...)
Advocates of Six Sigma, which include many highly respected people such as Jack Welch, are in no doubt that Six Sigma can produce immense results, and quickly too. You will see claims that Motorola saved in excess of $16bn resulting from implementing Six Sigma.
The Six Sigma model may or may not be the most popular ever, but ultimately - as with any business methodology - it relies not on how it is defined, it relies instead on how well people use it.
First and simply, Six Sigma is a quality improvement methodology.
Six Sigma has also become a generic 'brand' for a set of concepts that many organizations have used, and continue to use, to improve quality, and to provide quality and performance improvement services and training.
In this respect Six Sigma has captured corporate imagination. Six Sigma is an immensely popular vehicle for initiating and supporting the process of organizational change. Six Sigma has become an industry in its own right. See the names of some of the major US organizations that have adopted Six Sigma in recent times.
Six Sigma is a very flexible concept: to an statistical engineer Six Sigma might be a production quality metric; to a customer service employee, or a CEO, Six Sigma can represent a corporate culture.
The expression Six Sigma was first used in the context of quality improvement by American Motorola engineers in the mid 1980's.
Initially within Motorola Six Sigma was purely a quality metric that was used to reduce defects in the production of electronic components.
Six Sigma was then simply a statistical term that specifically referred to a performance target of 3.4 defects per million operations or 'opportunities' (DPMO).
The target of 3.4 defects per million operations which was set by Motorola engineers was to an extent arbitrary and subjective. Even the calculations which arrive at 3.4 defects per million and which correlate to precisely six sigma, are open to debate and different interpretation. At this level, Six Sigma is a highly complex science, so it is not surprising that the meaning of Six Sigma had to change in order for it to become something that managers and employees could relate to.
Sigma is Greek for the letter 'S', and the term 'sigma' has been used for many years by statisticians, mathematicians and engineers, as a measurement unit of statistical variation.
During the mid to late 1980's Motorola developed its Six Sigma ideas, which extended to and embraced many existing quality improvement methods and tools.
Motorola quickly realised that they could extend Six Sigma principles beyond manufacturing - to reduce variation and defects in all aspects of organizational performance.
Following Motorola's success in defining and applying the Six Sigma methodology, Six Sigma became a transferable model. The early adopters of Six Sigma aside from Motorola were Allied Signal (a large avionics company which merged with Honeywell in 1999), and then more significantly the massive GE (General Electric) corporation; (according to most commentators the Six Sigma model was transferred between the Chief Executives of the respective organizations).
GE particularly trumpeted its successes and multi-billion dollars of bottom-line improvements derived from Six Sigma, and by the end of the millennium Six Sigma was established as a mainstream management methodology, and had been adopted by very many of the world's largest corporations.
Strictly speaking the Six Sigma brand is trade-marked in the USA and belongs to Motorola Inc.. Motorola has since developed its own accredited, certified services and training for Six Sigma, within what is called the 'Motorola University'.
Many other organizations and consultancies of all sizes also develop and deliver Six Sigma training, and this activity seems not to be subject to particular mandatory control or accreditation (although Motorola certainly do have established structures and competencies). Seemingly anyone can start up as a Six Sigma consultant, just like anyone can start up as a quality management consultant, or a performance management consultant.
Six Sigma grew quickly from a statistical process for reducing defects in production, to become a 'branded' and yet generic management methodology, whose elements extend far beyond the meaning of the original Six Sigma expression.
So, Six Sigma is very flexible, and it continues to evolve, and it's difficult to describe.
Perhaps the most objective way of looking at Six Sigma is to recognise that the Six Sigma methodology essentially provides a framework, and importantly a strongly branded corporate initiative, for an organization to:
- train its people to focus on key performance areas
- understand where the organization wants to go (its strategy, related to its market-place)
- understand the services that the organization's customers need most
- understand and better organize main business processes that deliver these customer requirements
- measure (in considerable detail) and improve the effectiveness of these processes.
Motorola, and as a rule other advocates of Six Sigma, say that as a management system, Six Sigma is a top-down method (ie., instigated at CEO-level) for executing business strategy by using and optimising these process elements:
- Aligning critical improvement efforts to business strategy.
- Mobilizing teams to attack high-impact projects.
- Accelerating the improvement of business results.
- Governing efforts (of teams and people) to achieve and sustain improvements.
Central also to Six Sigma purpose and method is increasing the clarity of business strategy and the metrics that most reflect success within it. Other more recognizable terms for these might be KRA's (Key Results Areas) and KPI's (Key Performance Indicators).
While Six Sigma's attention to process quality variation is arguably greater than most other performance improvement methodologies, the basic principles of establishing and measuring critical processes are not earth-shatteringly new. What is new is arguably Six Sigma's focus (some would say obsessive focus) on detailed analysis.
In this respect Six Sigma's emphasis on detail will logically appeal to organizations with a 'detail culture' and, organizations that have a high proportion of managers who enjoy focusing on accuracy, for example corporations in industries such as engineering, technology, manufacturing, finance, etc.
(I'd be interested to know of any great successes of applying Six Sigma in fields where the organizational culture, service and managerial profiles lean more towards people, communications, relationships, creativity, etc., for example advertising and design, news and media, leisure and entertainment, sport and the arts, research and development, and teaching, training and coaching. Theoretically, Six Sigma is unlikely to prove hugely successful in environments where people are not good at or inclined to a lot of detailed measurement, processing and checking, but I'm open to evidence to the contrary...)
Incidentally, the above paragraph has featured on this webpage since 2005. The page is visited several thousand times each month. As at 2011, after more than 300,000 page views, I have yet to receive any evidence of Six Sigma being used successfully in work environments that are not strongly process/systems-driven.
I draw your attention to some of the significant aspects of Six Sigma, which have some implications for organizational culture, and for the decision whether to adopt Six Sigma in the first place:
Six Sigma, while involving and relying on teams is a top-down methodology. This implies quite strongly centralised operating structures and behaviours. Many organizations thrive and depend on such dynamics, but some don't.
Words like 'mobilize' and 'accelerate' and 'high-impact projects' imply that people need mobilizing, that improvement needs accelerating, and that people are not already engaged on high-impact projects. If your organization already has lots of highly mobilised people, is successfully achieving fast-moving improvements, and people engaged on high-impact projects, then probably Six Sigma is not for you.
Six Sigma is likely to produce far greater returns in organizations that need to achieve these things compared to organizations that are already doing them.
I would like to say at this point that there are thousands of people out there who know a great deal about Six Sigma. If you have comments that would help improve this overview of the Six Sigma methodology please send your suggestions.
six sigma and quality management glossary
Many of these terms are very specifically related to Six Sigma. Others are used in a general 'quality management' context and also in Six Sigma. As already explained, Six Sigma tends to embrace many other methodologies. A few of these terms are quite technical since they occur in the statistical, engineering and mathematical aspects of Six Sigma. The more complex mathematical terms and acronyms are included in this glossary not to provide detailed explanations, but instead to enable initial recognition and a basis for further investigation, if you are so inclined. This small glossary is not exhaustive because it would take about ten years to compile an exhaustive Six Sigma and Quality Management glossary. This is just a few highlights, some points of clarification, words of warning, items of mild amusement, and terms of special note. The really obvious STBO terms have not been included. If you need a more detailed listing try the one on the isixsigma website which could keep you occupied for days. If you wish to nominate an item of Six Sigma or Quality Management terminology for inclusion here - especially an amusing or intriguing example - please send to me. Despite being completely fascinating of course, Six Sigma is possibly is one of the driest subjects I've ever encountered and so will benefit from as much light relief as you can suggest.
acceptance, and acceptable quality level (ACL) - Acceptance has at least two different meanings in Six Sigma terminology, so be careful to understand which one is being referred to. Firstly, acceptance relating to quality is the quality expectation of the customer, internal or external. Acceptable Quality Level (ACL) means the same basically, in more formal Six Sigma-speak, and which will frequently be expressed in terms of percentage defects. Secondly acceptance refers to the buy-in or agreement of people affected by proposed actions and changes, notably stakeholders. While not strictly part of the Six Sigma battery of supporting tools, I can strongly recommend Sharon Drew Morgen's facilitative communications concepts for anyone struggling with stakeholder acceptance (and wholesale organisational change as well for that matter.)
activity report - A simple tool which enables teams and team leaders to manage project management tasks, responsibilities and timescales.
affinity diagram - A diagrammatic method of capturing, analysing and organising lots of ideas, elements, activities, etc., that together represent or influence an overall category, such as a process or issue. The brainstorming method is central to structuring an affinity diagram, and 'post-it' or sticky notes are commonly used as a way of generating and organising data. Commonly used in brainstorming solutions during the Improve stage of DMAIC.
analysis - Analysis of all sorts of data is a critical component within the Six Sigma model, which involves using various analytical methods to identify and quantify the causes of quality variation and failure in specific processes. Various analysis perspectives are adopted, for example:
- discrete - looking at a particular failure or problem - eg., using Pareto ('80:20') or pie-charts to show causes by percentage
- continuous - mapping performance variation and types, etc., over time, using distribution graphs
- process - creating detailed flow-diagrams to understand what's really going on in the process or sub-process
ANOVA, ANCOVA, MANOVA, MANCOVA - Despite first impressions these are nothing to do with Russion gymnastics or ice-skating moves. ANOVA is an acronym for analysis of variance, a specialised variation calculation method concerned with comparing means and testing hypotheses, best left to engineers and mathematicians. So are the related methods, ANCOVA (analysis of covariance), MANOVA (multiple analysis of variance), and MANCOVA (multiple analysis of covariance). Unless you are an engineer or a mathematician you will almost certainly have better things to do than get to grips with this level of statistical capability. Terms such as these illustrate why we need to work in multi-disciplined teams.
balanced scorecard - A sophisticated strategic analysis and improvement methodology developed by Kaplan and Norton which in its own right can sit outside Six Sigma, but which can be included within Six Sigma methods, and in any event might be used or referenced in the context of quality and performance improvement. The 'balanced scorecard' identifies, correlates, 'balances', measures and drives improvement across a wide variety of factors that are deemed responsible for overall organisational effectiveness, and for meeting customer expectations. The tool essentially translates strategy into operational metrics, and according to Motorola (ie., in a Six Sigma context) typically features the perspectives of, vision, current initiatives, business processes, and business results. 'Balanced Scorecard' became a generic 'brand' for business improvement in the 1990's, rather like Six Sigma, although arguably not on such a grand scale.
black belt - A specific Six Sigma term to describe a team leader and one who has achieved accredited 'Black Belt' qualification via an appropriate training course.
black noise/white noise - Technical terms relating to respectively non-random and random causes of variation.
business improvement campaign - A Motorola Six Sigma buzz-phrase, which represents a leadership initiative to improve the business's 'big Y's'.
business process management - A common generic expression in its own right, but also a Six Sigma term for the initial strategic element of Six Sigma. Six Sigma's strategic first phase is designed to develop management's commitment to Six Sigma, and also management's active participation in the Six Sigma process (which suggests why a powerful brand name for the initiative, ie., Six Sigma, is helpful..). This amounts to identifying the key processes within the organisation that determine effectively meeting customer expectations; then measuring the effectiveness and efficiency of the processes (notably measuring variation in quality and analysing the causes), and then initiating improvements in the weakest processes, which should logically yield the greatest results and return on effort.
cause-effect diagram - Also known as the fishbone diagram, this is a generally used tool for mapping and analysing causal factors towards an end output, so that contributing factors (and weaknesses can be more easily identified). Used especially in Six Sigma as a team brainstorming analysis tool. Called a fishbone diagram because the diagram plots contributing factors along parallel diagonal lines which each join a central horizontal time-line (like the back-bone) which culminates at one end with the main issue or question.
CTQ - Critical To Quality - An element within a process that has a major influence on the process quality, and typically the quality of a critical process, or it would be unlikely to be receiving Six Sigma attention.
defect - A vital and generic Six Sigma term for any failure in meeting customer expectation (internal and external customers) - any failure within the delivery process.
DFSS - Commonly used abbreviation in Six Sigma activities and communications, it means Design For Six Sigma, and describes the method of using tools, training, measurements, and verification so that products and processes are designed at the outset to meet Six Sigma requirements. A more specific version is DMADV: Define, Measure, Analyze, Design, and Verify. Both DFSS and DMAVD are concerned with, and emphasise the importance of, using Six Sigma principles in product/process design, not just for remedial improvements - rather advocating that prevention is better than cure. Thus, if Six Sigma capability is built into new organizational systems and products when they are designed, so performance will be better, and the need for Six Sigma remedial effort will be reduced.
DMAIC/DMAICT - Central Six Sigma process and acronym to ensure you remember it: Define, Measure, Analyse Improve, Control, more recently extended to DMAICT by others in the Six Sigma consulting and training communities, to Transfer(transfer best practice and thereby share learning).
DMADV - An alternative/substitute abbreviation to DFSS (Design For Six Sigma), and like DFSS DMADV is central to Six Sigma initiatives. DMADV more specifically describes a method comprising linked steps; Define, Measure, Analyze, Design, Verify,for ensuring that products and processes are designed at the outset to meet Six Sigma requirements.
frequency distribution/frequency distribution analysis or checksheet - Frequency distribution and the checksheets and other frequency distribution measurement tools form an essential aspect of Six Sigma data analysis. Identifying frequency of variation in processes is central to Six Sigma, since customers are particularly sensitive to variation, arguably even more than isolated failures. Therefore the sampling and collection of data over many operations and extended time periods, and the use of this data to indicate the frequency (number of times) that a variation occurs rather than the size of isolated failures, is an essential perspective for truly understanding what's happening, and the causes, within any critical delivery process. Frequency distribution analysis is an excellent antidote for any temptation to respond to an isolated failure with a knee-jerk quick fix, such as shooting the messenger or bollocking the workers when something deeper in the process is awry.
green belt - A Six Sigma team member who has received Green Belt training and who works part-time on Six Sigma projects under the guidance of a Black belt team leader.
just in time (JIT) - Just In Time, commonly abbreviated to JIT, describes operational or production methods based on minimising stock levels, the aim of which is to reduce capital employed in stock, which also has knock-on benefits to reducing storage space, decreasing dependence on logistics, easier supply chain management, and better overall quality. Just In Time is actually a capability arising from improvements within a business operation, rather than a cause of improvement itself. Introducing Just In Time methods without improving efficiency and reliability necessary to support it is not viable. Since Just In Time methods entail reducing stock levels to absolute minimum or even zero, JIT allows no room for error. Timing and predictibility are cruicial. JIT requires total commitment to quality and efficiency or the supply chain and related operations break down, the costs and implications of which can easily exceed any savings from JIT stock reductions. The term and methodology were developed by the Japanese during their post-war industrial revival (second half of the 1900s) as a logical progression from 'materials requirements planning' (MRP). The Japanese original terminology is 'kanban', and is important within 'lean production' methodology. The aim of kanban is actually zero inventory. JIT features in highly efficient manufacturing corporations, and has more recently been significantly enabled by computerization, especially to analyse and manage timings rather than stock levels. Noted authors to have covered the subject include Edwards Deming, Taiichi Ohno, and Yasuhiro Monden. The acronyms page contains a more amusing definition of JIT.
master black belt - A highly qualified Six Sigma practitioner, typically concerned with overseeing Six Sigma activities from an organizational perspective.
materials requirements planning (MRP) - production quality management methodology focusing on planning stock (materials and components of all sorts) levels and availability according to production schedules.
pareto principle, pareto diagram, pareto analysis - The Pareto Principle is otherwise and more commonly known as the 80:20 rule. The Pareto Principle was named after its originator Vilfredo Pareto, (1848-1923) an Italian economist and professor of political economics at Lausanne University, who first discovered the 80:20 'rule' of 'predictable imbalance', that (as far as Six Sigma is concerned) provides a basis for focusing on the 20% of activities that generate 80% of results, or the 20% of failures that are responsible for 80% of the waste, etc. Pareto first made his discovery while analysing wealth distribution among the British, in 1897. The Pareto Principle is also known as The Parato Law, The Principle Of Least Effort, and The Principle Of Imbalance, which in themselves provide an example of the Pareto Principle in action because despite all the options, hardly anyone ever uses any other name than 'The 80:20 Rule'. More Pareto explanation and examples in use.
process - The word process is worth mentioning because it is a fundamental cause of confusion (and not just in Six Sigma, but that's another story). The word process is used heavily in describing how Six Sigma works, and it's also used heavily in referring to the service or production activities (processes) on which the Six Sigma methods (or processes) are directed. You see what I mean... It is both the subject and the object. People easily get confused by terminology at the best of times, so it's worth taking extra care when using words like process which have at least two distinctly different meanings. For example avoid phrases such as "Six Sigma is a process that uses processes to improve processes." It's true, but its a load of bollocks. So, when using the word process, check that people know what process you are actually referring to, and then you will have a fighting chance of not disappearing up your own backside.
process mapping - diagrammatical representation of how processes work, as could be used and developed in team meetings on a flip-chart, or other media, to enable teams to understand processes, participants, and where and how improvements might be made.
production planning - generic term describing the over-arching methodology used in managing the supply process from receipt (or forecast) of customer requirements through to delivery notes and invoicing. Production planning therefore includes:
- interpretation of customer orders/requirements
- works orders
- schedules and computer programs/ implications
- parts, stocks and materials
- machinery, plant, equipment availability and allocation
- people and teams
- quality and other targets - setting and monitoring
- stock and purchasing monitoring and records
- order processing, administration and accounting
- necessary inter-departmental liaison (e.g., sales, export, etc)
Production planning is typically highly modularized and computerized since process reliability is crucial and is systematically repeated, although production planning must also allow for variation in response to sales or other changing demands and product specifications. Production planning is generally a weekly and monthly requirement, as well as incorporating longer-term commitments and considerations. The particular sales environment and predictability of the market and business have major impacts on production planning. Volatile markets and unpredictable sales obviously make production planning more difficult. Costs and budgets, health and safety, environmental, and other indirect considerations or compliances are of course relevant to production, but not directly, and so are not included as integral parts of the process.
Q x A = E - a natty little formula advanced by Six Sigma writer George Eckes for emphasising and assessing the need for Six Sigma projects to feature both strong technical quality (Q), and strong acceptance by the stakeholders of the project team's proposed solutions (A). E represents the excellence of the results, although why it should be E and not R rather defeats me. Whatever, the idea is a sound one, in that A is a multiplier and should along with Q should be assessed in simple terms at the early phase of a Six Sigma project. Eckes suggests scoring each of Q and A out of 10, and that if E equals anything less than 60 then the project is unlikely to succeed, with the implication to return to improving technical quality and stakeholder buy-in.
six sigma - how long have you got?.... at its most basic Six Sigma equates to 3.4 defects per million opportunities; at its most sophisticated (dare one suggest most hyped?..) Six Sigma is an organizational philosophy.
soft skills - skills required for managing people, relationships, acceptance and effective communications. A potential area of vulnerability in many Six Sigma implementations, because of the predominance of Six Sigma team leaders with strong process skills and attention to detail, which can sometimes be at odds with the abilities of intuition, empathy, rapport-building, relationship-building, and other 'soft' people-skills.
stakeholder - vitally important aspect, this one: stakeholders are not just customers, stakeholders are all the people who are affected by the solutions identified within a Six Sigma project, and all the people with some involvement in implementing the solutions.
tollgates - breaks for review between Six Sigma processes within any of the DMAIC stages.
tree diagram - pictorial representation of how a broad aim is broken down into detailed actions, and which belong to named individuals or departments. A mapping technique that promotes creative thinking towards detailed causes and effects and accountabilities. Helps to avoid tendencies for activities and accountabilities to be left too vague.
X's/big X's - Motorola Six Sigma-speak for factors or variables that have the greatest impact on the 'big Y's'.
Y's/big Y's - Motorola Six Sigma-speak for the most important business results and measures that are linked to critical customer requirements and expectations.
large organizations that have adopted six sigma
These are some the large US corporations referenced by Motorola Inc that have used Six Sigma. There are thousands more all over the world. Send me your own thoughts and comments about Six Sigma.
Advanced Micro Devices
Bank of USA
Bristol Myers Squibb
City of Dallas
Kraft General Foods
Rohm and Haas
US Air Force
feedback and observations about six sigma history and development, and quality management generally
Occasionally I receive interesting helpful comments and observations about Six Sigma, quality management, and performance management, etc.
The best examples are published here, with grateful acknowledgement to the contributors. Send your own comments.
From David Hutchins, 14 August 2008:
Whilst your article is extremely informative I am surprised that it makes no mention of Robert Galvin who was the CEO of Motorola at the time that Six Sigma was being developed in the company and who as a consequence was awarded the Juran Prize by the ASQ. Also that you make no mention of the fact that Six Sigma was spawned from the widespread use in Motorola of the Juran Managerial Breakthrough process using Dr Juran's widely acclaimed 16 video tape series of training materials for project by project improvement. Six Sigma came about as a consequence of an internal observation that those using these materials could be placed in two categories. Those who used a factual approach to make the journey from Symptom through Theories of Causes to true Cause or causes and those who made assumptions and implemented solutions based on these assumptions. Motorola observed that there was an order of magnitude difference between the results of the two categories of persons in favour of those who used the factual approach. In order to persuade everyone to do this, the concept of Six Sigma was born. How do I know all of this? The reason is that I was fortunate enough to have been present at the ASQ Conference in San Francisco where Bob Galvin announced Six Sigma to the world at large as a consequence of it having enabled the company to win the coveted Baldrige Award (American National Quality Award) in its first year. He also acknowledged the impact of Japanese competition by saying, "If the Japanese had not existed we would have to have invented them..." He was of course referring to the widespread use in Japan of the Taguchi Quadratic Loss Function which challenges all variation no matter how rare.
(Thanks David Hutchins)
From Tom Williams, 7 January 2008:
"Six Sigma holds no new place in our modern economy. It has been around in principle for several centuries, e.g., Abraham de Moivre (1773) when he identified the normal distribution of factors around the mean. This subsequently led to six sigma being +3 and -3 standard deviations from the mean (each deviation defined as being a 'sigma' for mathematical formulaic purposes) hence Six Sigma. If one is to use Six Sigma within a project environment then one has to assume that those environments fit within the 'normal distribution'. I don't think so. People are making today's money out of old statistical methods. Perhaps more recent methodologies like ANCOVA may be more relevant. But maybe the problem is that humans are not widgets? I add that +/- 3 standard deviations from the mean equates to approximately 99.6% of the population within the normal distribution."
(Thanks Tom Williams)
From John Johnson, 8 Oct 2007, in response to the John Mangold item below:
"I read with amusement about the '… no more than two bad parts per thousand etc…' and I have to tell you that I first heard this story in the early eighties, whilst undergoing indoctrination on my Total Quality Practitioner's course! I was told that the Japanese did the very same thing and queried why a company would actually order so many defective parts per batch..."
(Thanks John Johnson)
From John Mangold, 23 Jul 2007, in response to my request on this page for a Six Sigma 'elevator speech' to summarise what in essence Six Sigma means, here's a very effective and amusing example:
A company advertised its new Six Sigma approach. A customer placed an order for a thousand parts saying, "We don't want more than two bad parts per thousand." The Six Sigma company shipped a container with the thousand parts, on time, to the customer. Along with the container came a small parcel. The customer called and asked the supplier, "What is in the parcel?" The Six Sigma company answered, "The two bad parts." (Thanks John Mangold)
For the uninitiated, the term 'Six Sigma' derived originally from a reliability standard equating to no more than 3.4 defects per million opportunities (DPMO), which obviously is rather more challenging than two parts per thousand.
Further suggestions for conveying the Six Sigma concept in a nutshell are always welcome.
From Roy Bunyan, July 2007:
I am just starting in the Six Sigma training. In doing some additional research I ran across your page of the history of Six Sigma. I greatly appreciated your history and commentary on Six Sigma. It helped put several things in perspective. I thought you might find some anecdotal observations of interest. In the early 1980s, I was a product engineer of the 3880 (Z80) line of microprocessors at Mostek. We were having problems with yield and speed with the NMOS processor and some of its auxiliary devices. In an effort to determine what was happening, I discovered that standard statistical modeling did not work. Instead I started developing graphic methods based on historical layer process metrics and yield and speed results. We went from a 75% yield with 50% at top rated speed, to over a 90% yield with 80% at the top speed. When my management wanted to know how I accomplished the yield improvements, they found that I had abandoned their 'tried and true' statistical methodologies. My management was very upset when they found out that I was changing processing levels based on where the metrics of the previous layers fell within my graphics. But they could not argue with the results. Several others started to use the same methodology with comparable results. I have since learned that I was dealing with what has come to be known as fractal math. In a conversation with a design engineer at Motorola in the late 1990s, I found out that they were still using a variance of that process control methodology and it was major part of what allowed them to achieve the processing controls required for the newest generations of small dimension, high density devices. It was originally a very politically incorrect methodology that got utilized because of its results, but no one knew or was willing to admit where it came from. From what I understand, the very manager who refused to accept what I had done, or reward me for doing it, went to company in France and implemented the same methodology, and formally published it under his name. It was not until that engineer was allowed to publish the methodology by his management did it become politically correct and applied throughout the industry. The irony of this is that Mostek was attempting to implement the concepts of 'Total Quality' from Deming with presentations about the need to be able to 'think outside the box'. Yet management, in recognizing the need, could not do it themselves. Something else had to 'change the box' so that they could think within it. I use this story as lead in to what I have found to be somewhat true within any corporate culture. From Mostek, I went to Wang, and then to MCI. I spent 12 years at MCI doing things that were originally politically incorrect, but worked. When the management had to acknowledge the working model, a politically correct person was brought in and allowed to take credit for the 'thing'. In using that operational methodology I was able to do so many different things - more prototypes, proof of concepts - and more 'new' things than any one individual. Of course it was, for the most part, politically incorrect when initiated, and was only accomplished because the management was not aware of what was happening until the results were undeniable. And just like Mostek, upper management would find someone that was politically correct, and give them oversight. Even when the objective of the oversight was to dismantle what had been accomplished, because it was already a 'common knowledge', the management wound up embracing the new things within a new paradigm of technology. Even the ISO 2000 efforts at MCI in the early 1990s were allowed to operate only within the politically correct. From this perspective I see Six Sigma as a formal set of best practice methodologies that gives management the ability to change their own paradigm through politically correct management directives, allowing management to take ownership of changes that are needed without having the need to do it themselves. Even so, Six Sigma still functions within the politically correctness of management directive.
Six Sigma is a good set of best methods/practices. I have used several of the formal practices in an informal way to achieve the success in more than on project. But unlike the intentions of Six Sigma to be a 'top-down' management directed process, I used them 'bottom-up'. One of the first bits of misinformation about technology development is that new 'disruptive' technologies are welcomed by management. Nothing could be further from the truth. Truly 'disruptive' technologies are politically incorrect and challenge the status quo. Most 'disruptive' technologies take a long time for acceptance and implementation, and migration is slow and controlled. By using Six Sigma methods not only management directed projects can be a success, but employee directed projects can have the same success, in a very different way. Defining requirements, establishing working relationships with stakeholders, leveraging influence of other organizations and people, scoping resources and staying within budgets, and maintaining oversight of the project path are all parts of these methods, regardless of who uses them. When I was using these methods, I had task requirements that the politically correct technologies could not meet, which was one of the primary reasons why the task had not be achieved before. To use an example: At MCI, in 1990, we had what the LAN networking people referred to as the 'NOS wars'. There were several different vice presidents that had their own LAN and Network Operating System. The VP were manipulating for political gain their NOS, Novel, 3Com, Pathworks, Banyan, LAN Server, DecNet and what ever else was on the market at the time, all within the same city, in different buildings. None of the different NOSs could share data because each had its own protocol. It was very frustrating for all of the network analysts. A group of network analysts, including me, got together and decided that an independent protocol was the only answer to the problem. Because management each had their own politically correct vendor technology, with its own vendor proprietary protocol, it was decided to use TCP/IP, because it was not dependent on any one vendor. I also took on the role of finding the Host Master (the person who is responsible for the use of IP addresses) to get a range of addresses to use. I talked with Vent Cerf, only to discover that MCI had never deployed any officially registered IP addresses. With Vent's help, I became the first Host Master at MCI. I was working for the construction division of the company and had access to unused optical fibre between the different buildings. As a network analyst for construction I also had access to the technology development LAB and the people there. I developed a relationship with the marketing and development people and did a 'trade' of setting up a test bed of a long haul fibre transmission path in exchange for the use of a test router chassis that was part of a development project. I got each of the different stake holder analysts to purchase an interface module for the router out of their operations budget. Another set of smaller routers was obtained as part of the 'test' environment. We installed the routers in each of the buildings and I hooked up all of the fiber between them. I took on the role of getting the routing set up and the other analysts set up the TCP/IP on their on LAN NOS. Six months after we started the 'hidden' project I made a presentation before my management, and others. In January of 1991 I made a presentation on a 3Com network, with an application loaded from a Pathworks network, using a power point file loaded from a Novel network. The NOS wars were over. It did not matter anymore which NOS was better, we could all now share information data. It was a project that was politically incorrect and only succeeded because nothing out of a capital budget was used. Many of the best practice methods of what is now known as Six Sigma were used to accomplish what was impossible before. The barriers were management politics, the proprietary nature of the vendor technology, and the lack of capital budget. All of these were overcome. As a formal set of methods and practices, Six Sigma appears to be a good way to go. Management wants to control it. But it is the employees that have the power to actually accomplish something from it.
(With acknowledgements to Roy Bynum, Technical Solutions Advisor, Enterprise Operations Center, CompuCom Systems, Inc.)
From Gordon Stalker, July 2007:
An interesting corollary to Pareto's (80/20 Rule) observation is in engineering design, with specific reference to electronics and related areas. In any analysis of the performance of the most successful technology companies (e.g., Hewlett Packard) it is found that almost precisely 1 out of every 5 projects irretrievably fail final testing. That is to say that 20% of all projects are scrapped after completion, so that almost all of the income comes from the remaining 80%. This can be explained by the economics of product development. It is possible to spend a great amount of time and money carefully defining the product specification and testing the product at each stage of development. However, to half the notional 'error rate' requires twice the effort at each stage, and there comes a point where the cost of eliminating errors in development exceeds the benefit gained. For example if you are currently experiencing an 'error rate' of 1% is it really worth spending twice the time and effort to reduce it to 0.5%? Clearly not. It so happens that a 20% rate is not only acceptable but in fact optimal. To reduce the error rate to 10% means the product takes too long to develop, and competitors are able to provide a solution earlier and less expensively. Halving the development effort results in a 40% error rate so a reduction in successful products to 60% of all projects, which at first glance seems like a better economic outcome (ignoring the production costs!) But in reality you want to produce as many new products as you can if you wish to attract new customers and keep the ones you already have. So developing 25% fewer new products at 50% of the development cost is a false economy. This is also ignoring the fact that a well-designed product also takes production costs into consideration. It is all very well designing the perfect mouse trap, which guarantees 100% effective rodent control with zero maintenance over a 1000 year service life. But if it requires a team of highly trained engineers three months to assemble it, people ain't exactly going to beat a track to your door!
I'm not sure where I picked this up from, but it's something which I kept having to explain when I worked as a designer engineer. In my own experience as a design engineer it does seem about right, and highest 'productivity' does seem to result from an acceptance that one out of five designs will fall at the last hurdle (which is the most expensive time to find out!) I'm sure that it would not take too long to find other sources for that figure, although it is not something that companies like to boast about. The problem is that many customers don't seem to understand that while anything is possible, it depends on how much time and money they have to spend. Unfortunately many have already spent a large amount of their own time and effort on their idea and (I think) in many cases have lost sight of what they originally set out to do. I think success depends very much on the ability to admit that maybe it wasn't such a great idea in the first place, and get on with the next thing. There's a big difference between 'determined' and 'stubborn'. Determined is knowing that the goal is achievable, and accepts that there will be pitfalls along the way, and that they may have to return to the start again if the path is wrong. Stubborn just ignores the pitfalls, and refuses to change the path, and doing it their way becomes more important than the actual goal. As I was often fond of pointing out, it's part of an engineers job to look for the possible pitfalls, and plan a route to avoid them where possible, surmount them where necessary, and if an obstacle proves insurmountable, to try the next path until the specification is met. This often attracts the criticism that engineers are 'unenthusiastic', or worse 'obstructive', but I prefer to see this instead as being 'proactive', which happily is a response that most business people demand.
(With acknowledgements to Gordon Stalker.)
From James Diorio, Feb 2009:
Texas Instruments is on your list. In reality, only a part of TI adopted Six Sigma. The division of TI that adopted Six Sigma is the Sensors & Controls Group (motor controllers, automotive transducers, etc.). This group was spun off a few years ago, and became 'Sensata' (see sensata.com). However, several years prior to the spin-off, the Six Sigma movement lost momentum (1995 or so), and basically perished. The Semiconductor Division (essentially the core manufacturing business) of Texas Instruments apparently never embraced the movement.
(With acknowledgements to James Diorio.)
Feel free to send your own comments.
six sigma information and resources
- quality management
- balanced scorecard
- business process modelling
- decision-making and problem-solving
- project management
- running meetings
- swot analysis and templates
- tam - team, action, management model
- running team workshops
- modern facilitative change methodology