What is Benchmarking?
•Benchmarking is the process of comparing the business processes and performance metrics including cost, cycle time, productivity, or quality to another that is widely considered to be an industry standard benchmark or best practice. Essentially, benchmarking provides a snapshot of the performance of your business and helps you understand where you are in relation to a particular standard. The result is often a business case and "Burning Platform" for making changes in order to make improvements.
•Benchmarking can be simply defined as a continuous process to find and implement best practices that will lead to superior performance. As the definition implies, benchmarking is a process that will make a company s operations lean, and improve quality and productivity
In the quest for increased competitiveness, companies often ask themselves the question, "How are we doing?" Asking this question leads logically to the next question, "Compared to what?" To fully answer this second question involves an examination of a company's own operations, and subsequently comparing the operations with those of other organisations identified to be leaders in the field. Such comparisons are at the heart of benchmarking.
There are three major reasons for an organisation to embark upon benchmarking. These are:
•Benchmarking provides an objective evaluation of a company's business processes against similar processes in other organisations
•Benchmarking serves as a vehicle to source for improvement ideas from other organisations
•Benchmarking broadens an organisation's experience base by providing insights into systems and methods that work and those that don't. It therefore supports the notion of a learning organisation.
The benchmarking process can be applied to all facets of a company's business, be it in products, services or business processes. However, the focus of most benchmarking projects is on business processes because the effective management of these processes, including quality, speed, and service, is of vital importance to achieve superior performance and he more competitive.
There is no single benchmarking process that has been universally adopted. The wide appeal and acceptance of benchmarking has led to various benchmarking methodologies emerging. The first book on benchmarking, written by Kaiser Associates, offered a 7-step approach. Robert Camp (who wrote one of the earliest books on benchmarking in 1989) developed a 12-stage approach to benchmarking.
The 12 stage methodology consisted of 1. Select subject ahead 2. Define the process 3. Identify potential partners 4. Identify data sources 5. Collect data and select partners 6. Determine the gap 7. Establish process differences 8. Target future performance 9. Communicate 10. Adjust goal 11. Implement 12. Review/recalibrate
Types of Benchmarking
Depending on the objectives and scope of benchmarking, different types of benchmarking processes can be distinguished depending on what is compared and to whom it is being compared.
Let take a good look at this.
Benchmarking of What?
•Performance benchmarking is a brief evaluation process that compares company performance measures against a standard or target that has been established, or performance data of other organisations.
•Process benchmarking analyses and compares the methods and practices of a participating company's processes in order that another company can learn from the best and improve their own processes. In effect, it involves the identification of best practices that lie behind superior performance.
•Strategic benchmarking is an in-depth analysis aimed at identifying fundamental areas for improvement, i.e. a company's strengths and weak points. Information concerning other company's strategic choices is collected in order to improve a company's own strategic planning and positioning.
Benchmarking against Whom?
•Internal benchmarking is the comparison between a company's different departments, units or subsidiaries, including those based in different countries.
•Competitive benchmarking entails the direct comparison of a company's own performance against its competitors. This is easier in some respects because many external factors that affect performance are similar between the benchmarked companies, but it may be more difficult because of the competitive relationship between the companies, which can make data collection difficult.
•Functional benchmarking involves the comparison of processes against non-competitor companies within the same industry or service area that share common technological or market characteristics. Compared to competitive benchmarking, it is easier to find benchmarking partners, since the relationship between companies is not one of direct competition.
Generic benchmarking focuses on the comparison of a company's own processes against best processes, irrespective of industry or service sector. It studies innovative methods or technologies with the aim of identifying technologies that will lead to breakthroughs. This is particularly relevant for environmental benchmarking because best environmental practices are rarely industry-specific.
Benefits from Benchmarking
•Improving communication
•Professionalizing the organization / processes, or for
•Budgetary reasons
•In outsourcing projects
Cost of Benchmarking
There are costs to benchmarking, although many companies find that it pays for itself. The three main types of costs are:
•Visit costs - This includes hotel rooms, travel costs, meals, a token gift, and lost labour time.
•Time costs - Members of the benchmarking team will be investing time in researching problems, finding exceptional companies to study, visits, and implementation. This will take them away from their regular tasks for part of each day so additional staff might be required.
•Benchmarking database costs - Organizations that institutionalize benchmarking into their daily procedures find it is useful to create and maintain a database of best practices and the companies associated with each best practice
Limitation of Benchmarking
•Benchmarking is a tough process that needs a lot of commitment to succeed.
•Time-consuming and expensive.
•More than once benchmarking projects end with the 'they are different from us' syndrome or competitive sensitivity prevents the free flow of information that is necessary.
•Comparing performances and processes with 'best in class' is important and should ideally be done on a continuous basis (the competition is improving its processes also...).
•Is the success of the target company really attributable to the practice that is benchmarked? Are the companies comparable in strategy, size, model, culture?
•What are the downsides of adopting a practice?
Showing posts with label Marketing Research. Show all posts
Showing posts with label Marketing Research. Show all posts
Tuesday, November 10, 2009
Wednesday, November 4, 2009
Porter’s Five Forces Analysis
Porter's five forces analysis is a framework for the industry analysis and business strategy development developed by Michael E. Porter of Harvard Business School in 1979. It uses concepts developed in Industrial Organization (IO) economics to derive five forces which determine the competitive intensity and therefore attractiveness of a market. Attractiveness in this context refers to the overall industry profitability. An "unattractive" industry is one where the combination of forces acts to drive down overall profitability. A very unattractive industry would be one approaching "pure competition".
Porter referred to these forces as the micro environment, to contrast it with the more general term macro environment. They consist of those forces close to a company that affect its ability to serve its customers and make a profit. A change in any of the forces normally requires a company to re-assess the marketplace. The overall industry attractiveness does not imply that every firm in the industry will return the same profitability. Firms are able to apply their core competences, business model or network to achieve a profit above the industry average. A clear example of this is the airline industry. As an industry, profitability is low and yet individual companies, by applying unique business models have been able to make a return in excess of the industry average.
Main Aspects of Porter’s Five Forces Analysis
The original competitive forces model, as proposed by Porter, identified five forces which would impact on an organization’s behaviour in a competitive market. These include the following:
•The rivalry between existing sellers in the market.
•The power exerted by the customers in the market.
•The impact of the suppliers on the sellers.
•The potential threat of new sellers entering the market.
•The threat of substitute products becoming available in the market.
Understanding the nature of each of these forces gives organizations the necessary insights to enable them to formulate the appropriate strategies to be successful in their market.
Force 1: The Degree of Rivalry
The intensity of rivalry, which is the most obvious of the five forces in an industry, helps determine the extent to which the value created by an industry will be dissipated through head-to-head competition. The most valuable contribution of Porter's “five forces” framework in this issue may be its suggestion that rivalry, while important, is only one of several forces that determine industry attractiveness.
•This force is located at the centre of the diagram;
•Is most likely to be high in those industries where there is a threat of substitute products; and existing power of suppliers and buyers in the market.
Force 2: The Threat of Entry
Both potential and existing competitors influence average industry profitability. The threat of new entrants is usually based on the market entry barriers. They can take diverse forms and are used to prevent an influx of firms into an industry whenever profits, adjusted for the cost of capital, rise above zero. In contrast, entry barriers exist whenever it is difficult or not economically feasible for an outsider to replicate the incumbents’ position (Porter, 1980b; Sanderson, 1998) The most common forms of entry barriers, except intrinsic physical or legal obstacles, are as follows:
•Economies of scale: for example, benefits associated with bulk purchasing;
•Cost of entry: for example, investment into technology;
•Distribution channels: for example, ease of access for competitors;
•Cost advantages not related to the size of the company: for example, contacts and expertise;
•Government legislations: for example, introduction of new laws might weaken company’s competitive position;
•Differentiation: for example, certain brand that cannot be copied (The Champagne)
Force 3: The Threat of Substitutes
The threat that substitute products pose to an industry's profitability depends on the relative price-to-performance ratios of the different types of products or services to which customers can turn to satisfy the same basic need. The threat of substitution is also affected by switching costs – that is, the costs in areas such as retraining, retooling and redesigning that are incurred when a customer switches to a different type of product or service. It also involves:
•Product-for-product substitution (email for mail, fax); is based on the substitution of need;
•Generic substitution (Video suppliers compete with travel companies);
•Substitution that relates to something that people can do without (cigarettes, alcohol).
Force 4: Buyer Power
Buyer power is one of the two horizontal forces that influence the appropriation of the value created by an industry (refer to the diagram). The most important determinants of buyer power are the size and the concentration of customers. Other factors are the extent to which the buyers are informed and the concentration or differentiation of the competitors. Kippenberger (1998) states that it is often useful to distinguish potential buyer power from the buyer's willingness or incentive to use that power, willingness that derives mainly from the “risk of failure” associated with a product's use.
•This force is relatively high where there a few, large players in the market, as it is the case with retailers an grocery stores;
•Present where there is a large number of undifferentiated, small suppliers, such as small farming businesses supplying large grocery companies;
•Low cost of switching between suppliers, such as from one fleet supplier of trucks to another.
Force 5: Supplier Power
Supplier power is a mirror image of the buyer power. As a result, the analysis of supplier power typically focuses first on the relative size and concentration of suppliers relative to industry participants and second on the degree of differentiation in the inputs supplied. The ability to charge customers different prices in line with differences in the value created for each of those buyers usually indicates that the market is characterized by high supplier power and at the same time by low buyer power (Porter, 1998). Bargaining power of suppliers exists in the following situations:
•Where the switching costs are high (switching from one Internet provider to another);
•High power of brands (McDonalds, British Airways, Tesco);
•Possibility of forward integration of suppliers (Brewers buying bars);
•Fragmentation of customers (not in clusters) with a limited bargaining power (Gas/Petrol stations in remote places).
The nature of competition in an industry is strongly affected by suggested five forces. The stronger the power of buyers and suppliers, and the stronger the threats of entry and substitution, the more intense competition is likely to be within the industry. However, these five factors are not the only ones that determine how firms in an industry will compete – the structure of the industry itself may play an important role. Indeed, the whole five-forces framework is based on an economic theory know as the “Structure-Conduct-Performance” (SCP) model: the structure of an industry determines organizations’ competitive behaviour (conduct), which in turn determines their profitability (performance). In concentrated industries, according to this model, organizations would be expected to compete less fiercely, and make higher profits, than in fragmented ones. However, as Haberberg and Rieple (2001) state, the histories and cultures of the firms in the industry also play a very important role in shaping competitive behaviour, and the predictions of the SCP model need to be modified accordingly.
Strengths of the Five Competitive Forces Model Benefits
•The model is a strong tool for competitive analysis at industry level. Compare: PEST Analysis
•It provides useful input for performing a SWOT Analysis.
Limitation of Porter’s Five Forces Model
•Care should be taken when using this model for the following: do not underestimate or underemphasize the importance of the (existing) strengths of the organization (Inside-out strategy).
•The model was designed for analyzing individual business strategies. It does not cope with synergies and interdependencies within the portfolio of large corporations.
•From a more theoretical perspective, the model does not address the possibility that an industry could be attractive because certain companies are in it.
•Some people claim that environments which are characterized by rapid, systemic and radical change require more flexible, dynamic or emergent approaches to strategy formulation. Sometimes it may be possible to create completely new markets instead of selecting from existing ones.
Porter's Six Forces model and its relationship to the standard Five Forces model
Porter’s Five Forces model actually has an extension referred to as Porter’s Six Forces model. It is considerably less popular than the Five Forces model as its acceptance has been less positive than the Five Forces model. The Six Forces model though is very similar to the Five Forces model with the only difference being the addition of the sixth force in the framework. This sixth force in the model is termed as the relative power of other stakeholders, and can refer to a number of other groups or entities, depending on the factor which has the greatest influence including:
• Complementors – One school of thought looks at the sixth force to be complementors, which are businesses offering complementary products to the sector in focus and being analysed (Grove 1996). The author states that these complementary businesses, as a sixth factor, affect the industry as changes in these businesses (such as new techniques, approaches or technologies) can impact on the dynamics between the industry and the complementors.
• The government – The sixth force in the framework can also be considered to be the government, and is included in the framework if it has potential to impact on all the other five forces (Gordon, 1997). Thus, the government can have direct impact in the industry as the sixth force, but can also have indirect impact or influence by affecting the other five forces, whether favourably or unfavourably.
• The public – Yet other viewpoints look at the public as the sixth force in the model, particularly if the public has a strong influence in the dynamics of the sector resulting in changes to the other forces or in the sector as a whole.
• Shareholders – This group can also be considered potentially as the sixth force. This is more important in recent years where shareholder activity has increased significantly in the boardroom, and management of firms has been scrutinised much more and even given ‘threats’ if certain actions favoured by the shareholders were not pursued.
• Employees – Employees could also be considered as the sixth force if they wielded extraordinarily strong influence on the firm in a particular sector. The status of employees seems to follow similar rules in certain sectors, and thus could be considered a strong influence in these sectors. For example, in the automobile sector in the US, a large part of the work force are unionised, and thus could be considered the sixth force instead of the government or complementors.
While a sixth force has been added to Porter’s original Five Forces model, the acceptance of this framework has been somewhat limited. This could be for two reasons. First, is that there is no definite and specific sixth force in all sectors, as it is different for each sector. Second, while a sixth force could be defined for all sectors, the influence of this factor can also be captured in the other five forces and thus the necessity of having it in the framework is less compelling.
Porter referred to these forces as the micro environment, to contrast it with the more general term macro environment. They consist of those forces close to a company that affect its ability to serve its customers and make a profit. A change in any of the forces normally requires a company to re-assess the marketplace. The overall industry attractiveness does not imply that every firm in the industry will return the same profitability. Firms are able to apply their core competences, business model or network to achieve a profit above the industry average. A clear example of this is the airline industry. As an industry, profitability is low and yet individual companies, by applying unique business models have been able to make a return in excess of the industry average.
Main Aspects of Porter’s Five Forces Analysis
The original competitive forces model, as proposed by Porter, identified five forces which would impact on an organization’s behaviour in a competitive market. These include the following:
•The rivalry between existing sellers in the market.
•The power exerted by the customers in the market.
•The impact of the suppliers on the sellers.
•The potential threat of new sellers entering the market.
•The threat of substitute products becoming available in the market.
Understanding the nature of each of these forces gives organizations the necessary insights to enable them to formulate the appropriate strategies to be successful in their market.
Force 1: The Degree of Rivalry
The intensity of rivalry, which is the most obvious of the five forces in an industry, helps determine the extent to which the value created by an industry will be dissipated through head-to-head competition. The most valuable contribution of Porter's “five forces” framework in this issue may be its suggestion that rivalry, while important, is only one of several forces that determine industry attractiveness.
•This force is located at the centre of the diagram;
•Is most likely to be high in those industries where there is a threat of substitute products; and existing power of suppliers and buyers in the market.
Force 2: The Threat of Entry
Both potential and existing competitors influence average industry profitability. The threat of new entrants is usually based on the market entry barriers. They can take diverse forms and are used to prevent an influx of firms into an industry whenever profits, adjusted for the cost of capital, rise above zero. In contrast, entry barriers exist whenever it is difficult or not economically feasible for an outsider to replicate the incumbents’ position (Porter, 1980b; Sanderson, 1998) The most common forms of entry barriers, except intrinsic physical or legal obstacles, are as follows:
•Economies of scale: for example, benefits associated with bulk purchasing;
•Cost of entry: for example, investment into technology;
•Distribution channels: for example, ease of access for competitors;
•Cost advantages not related to the size of the company: for example, contacts and expertise;
•Government legislations: for example, introduction of new laws might weaken company’s competitive position;
•Differentiation: for example, certain brand that cannot be copied (The Champagne)
Force 3: The Threat of Substitutes
The threat that substitute products pose to an industry's profitability depends on the relative price-to-performance ratios of the different types of products or services to which customers can turn to satisfy the same basic need. The threat of substitution is also affected by switching costs – that is, the costs in areas such as retraining, retooling and redesigning that are incurred when a customer switches to a different type of product or service. It also involves:
•Product-for-product substitution (email for mail, fax); is based on the substitution of need;
•Generic substitution (Video suppliers compete with travel companies);
•Substitution that relates to something that people can do without (cigarettes, alcohol).
Force 4: Buyer Power
Buyer power is one of the two horizontal forces that influence the appropriation of the value created by an industry (refer to the diagram). The most important determinants of buyer power are the size and the concentration of customers. Other factors are the extent to which the buyers are informed and the concentration or differentiation of the competitors. Kippenberger (1998) states that it is often useful to distinguish potential buyer power from the buyer's willingness or incentive to use that power, willingness that derives mainly from the “risk of failure” associated with a product's use.
•This force is relatively high where there a few, large players in the market, as it is the case with retailers an grocery stores;
•Present where there is a large number of undifferentiated, small suppliers, such as small farming businesses supplying large grocery companies;
•Low cost of switching between suppliers, such as from one fleet supplier of trucks to another.
Force 5: Supplier Power
Supplier power is a mirror image of the buyer power. As a result, the analysis of supplier power typically focuses first on the relative size and concentration of suppliers relative to industry participants and second on the degree of differentiation in the inputs supplied. The ability to charge customers different prices in line with differences in the value created for each of those buyers usually indicates that the market is characterized by high supplier power and at the same time by low buyer power (Porter, 1998). Bargaining power of suppliers exists in the following situations:
•Where the switching costs are high (switching from one Internet provider to another);
•High power of brands (McDonalds, British Airways, Tesco);
•Possibility of forward integration of suppliers (Brewers buying bars);
•Fragmentation of customers (not in clusters) with a limited bargaining power (Gas/Petrol stations in remote places).
The nature of competition in an industry is strongly affected by suggested five forces. The stronger the power of buyers and suppliers, and the stronger the threats of entry and substitution, the more intense competition is likely to be within the industry. However, these five factors are not the only ones that determine how firms in an industry will compete – the structure of the industry itself may play an important role. Indeed, the whole five-forces framework is based on an economic theory know as the “Structure-Conduct-Performance” (SCP) model: the structure of an industry determines organizations’ competitive behaviour (conduct), which in turn determines their profitability (performance). In concentrated industries, according to this model, organizations would be expected to compete less fiercely, and make higher profits, than in fragmented ones. However, as Haberberg and Rieple (2001) state, the histories and cultures of the firms in the industry also play a very important role in shaping competitive behaviour, and the predictions of the SCP model need to be modified accordingly.
Strengths of the Five Competitive Forces Model Benefits
•The model is a strong tool for competitive analysis at industry level. Compare: PEST Analysis
•It provides useful input for performing a SWOT Analysis.
Limitation of Porter’s Five Forces Model
•Care should be taken when using this model for the following: do not underestimate or underemphasize the importance of the (existing) strengths of the organization (Inside-out strategy).
•The model was designed for analyzing individual business strategies. It does not cope with synergies and interdependencies within the portfolio of large corporations.
•From a more theoretical perspective, the model does not address the possibility that an industry could be attractive because certain companies are in it.
•Some people claim that environments which are characterized by rapid, systemic and radical change require more flexible, dynamic or emergent approaches to strategy formulation. Sometimes it may be possible to create completely new markets instead of selecting from existing ones.
Porter's Six Forces model and its relationship to the standard Five Forces model
Porter’s Five Forces model actually has an extension referred to as Porter’s Six Forces model. It is considerably less popular than the Five Forces model as its acceptance has been less positive than the Five Forces model. The Six Forces model though is very similar to the Five Forces model with the only difference being the addition of the sixth force in the framework. This sixth force in the model is termed as the relative power of other stakeholders, and can refer to a number of other groups or entities, depending on the factor which has the greatest influence including:
• Complementors – One school of thought looks at the sixth force to be complementors, which are businesses offering complementary products to the sector in focus and being analysed (Grove 1996). The author states that these complementary businesses, as a sixth factor, affect the industry as changes in these businesses (such as new techniques, approaches or technologies) can impact on the dynamics between the industry and the complementors.
• The government – The sixth force in the framework can also be considered to be the government, and is included in the framework if it has potential to impact on all the other five forces (Gordon, 1997). Thus, the government can have direct impact in the industry as the sixth force, but can also have indirect impact or influence by affecting the other five forces, whether favourably or unfavourably.
• The public – Yet other viewpoints look at the public as the sixth force in the model, particularly if the public has a strong influence in the dynamics of the sector resulting in changes to the other forces or in the sector as a whole.
• Shareholders – This group can also be considered potentially as the sixth force. This is more important in recent years where shareholder activity has increased significantly in the boardroom, and management of firms has been scrutinised much more and even given ‘threats’ if certain actions favoured by the shareholders were not pursued.
• Employees – Employees could also be considered as the sixth force if they wielded extraordinarily strong influence on the firm in a particular sector. The status of employees seems to follow similar rules in certain sectors, and thus could be considered a strong influence in these sectors. For example, in the automobile sector in the US, a large part of the work force are unionised, and thus could be considered the sixth force instead of the government or complementors.
While a sixth force has been added to Porter’s original Five Forces model, the acceptance of this framework has been somewhat limited. This could be for two reasons. First, is that there is no definite and specific sixth force in all sectors, as it is different for each sector. Second, while a sixth force could be defined for all sectors, the influence of this factor can also be captured in the other five forces and thus the necessity of having it in the framework is less compelling.
Ecological Model of Competition
The ecological model of competition is a reassessment of the nature of competition in the economy. Traditional economics models the economy on the principles of physics (force, equilibrium, inertia, momentum, and linear relationships). This can be seen in the economics lexicon: terms like labour force, market equilibrium, capital flows, and price elasticity. This is probably due to historical coincidence. Classical Newtonian physics was the state of the art in science when Adam Smith was formulating the first principles of economics in the 1700s.
According to the ecological model, it is more appropriate to model the economy on biology (growth, change, death, evolution, survival of the fittest, complex inter-relationships, and non-linear relationships). Businesses operate in a complex environment with interlinked sets of determinants. Companies co-evolve: they influence, and are influenced by, competitors, customers, governments, investors, suppliers, unions, distributors, banks, and others. We should look at this business environment as a business ecosystem that both sustains, and threatens the firm. A company that is not well matched to its environment might not survive. Companies that are able to develop a successful business model and turn a core competency into a sustainable competitive advantage will thrive and grow. Very successful firms may come to dominate their industry (referred to as category killers).
According to the ecological model, it is more appropriate to model the economy on biology (growth, change, death, evolution, survival of the fittest, complex inter-relationships, and non-linear relationships). Businesses operate in a complex environment with interlinked sets of determinants. Companies co-evolve: they influence, and are influenced by, competitors, customers, governments, investors, suppliers, unions, distributors, banks, and others. We should look at this business environment as a business ecosystem that both sustains, and threatens the firm. A company that is not well matched to its environment might not survive. Companies that are able to develop a successful business model and turn a core competency into a sustainable competitive advantage will thrive and grow. Very successful firms may come to dominate their industry (referred to as category killers).
Monday, October 26, 2009
SWOT Analysis
Definition
SWOT is an abbreviation for Strengths, Weaknesses, Opportunities and Threats
An assessment of Strengths, Weaknesses, Opportunities, and Threats. SWOT analysis is used within organizations in the early stages of strategic and marketing planning. It is also used in problem solving, decision making, or for making staff aware of the need for change. It can be used at a personal level when examining your career path or determining possible career development
SWOT analysis is an important tool for auditing the overall strategic position of a business and its environment.
Once key strategic issues have been identified, they feed into business objectives, particularly marketing objectives. SWOT analysis can be used in conjunction with other tools for audit and analysis, such as PEST analysis and Porter's Five-Forces analysis. It is also a very popular tool with business and marketing students because it is quick and easy to learn
The Key Distinction - Internal and External Issues
Strengths and weaknesses are Internal factors. For example, strength could be your specialist marketing expertise. A weakness could be the lack of a new product.
Opportunities and threats are External factors. For example, an opportunity could be a developing distribution channel such as the Internet, or changing consumer lifestyles that potentially increase demand for a company's products. A threat could be a new competitor in an important existing market or a technological change that makes existing products potentially obsolete.
It is worth pointing out that SWOT analysis can be very subjective - two people rarely come-up with the same version of a SWOT analysis even when given the same information about the same business and its environment. Accordingly, SWOT analysis is best used as a guide and not a prescription. Adding and weighting criteria to each factor increases the validity of the analysis.
Where is S.W.O.T. being applied?
S.W.O.T. Analysis as it may sometimes being called can be performed in a variety of application or situation. It can be used as a situation analysis as an input into a strategic planning process at corporate of company level. It can also apply to evaluate the situation in terms of its capabilities. We use S.W.O.T. as a situation analysis tool.
When do we Perform a S.W.O.T. Analysis?
In common practice, S.W.O.T. Analysis is performed during the Strategic Planning or Business budget session normally done at the end of a financial year. But to perform a S.W.O.T. should not be limited to a yearly affair. You may perform a S.W.O.T. Analysis whenever it is needed to help you to identify causes of a non-conformance and you needed a new solution or strategy.
Who would Perform a S.W.O.T. Analysis?
In most cases, leaders of an organization perform a S.W.O.T. Analysis. However, it should not be limited to this group of people. In fact, anyone who has an interest and trained can perform a S.W.O.T. Analysis for the situation they are in. I have many situations where heads of a department perform a S.W.O.T. Analysis for their own operation issues because they want to develop solutions based on facts.
Why do you need to Perform a S.W.O.T. Analysis?
As it can be seen by now, data gathering is an essential part of S.W.O.T. Analysis. Hence, the information collected is likely to be more factual. Any solution derived from S.W.O.T. will be more realistic and reliable.
How to Perform a S.W.O.T. Analysis?
As data collection is one of the key activities in S.W.O.T. analysis, it should allow enough time to bring back the data. 1-3 month before a S.W.O.T. Analysis session is conducted. Once the data is collected, it should be grouped into the four factors. This can be done individually or in a team.
In summary, with some basic understanding of S.W.O.T. Analysis, the solution derived from it can be value add to the organization.
How SWOT Analysis is used to formulate Strategies
This is perhaps the most powerful usage of SWOT Analysis in the Strategic Planning Process. I am going to show you how to used the four factors of SWOT to develop Strategies
By now, you would have collected several data pertaining to the Strengths, Weaknesses, Opportunities and Threats. Then you will use them to formulate strategy. Not sure how to do it? Don't worry, I take you through the steps.
Step 1 – Evaluate the Surrounding
Let's take a moment to think about both of us as the coach for two teams of football teams.
Before the game starts, you and I have certain strategies that we want the team to follow. As the game progresses, there is sign of difference between the two teams in terms of the game as well as the condition of the team members.
Step 2 – Identify the Strengths, Weaknesses, Opportunities and Threats
Now, it is time to evaluate the teams in the four factors of SWOT. Let’s take the following examples as the result of the evaluation:-
Strengths - Your team full of fighting spirit
Weaknesses - One of your team members is hurt
Opportunities - Your opposition team seems to loose stamina
Threats - Your opposition team is full of energy
Note: Some of these factors seem to be conflicting each other. For the purpose of this step, this conflict is ignored.
Step 3 - Pair the SWOT factors to formulate Strategies
Now, you would start to formulate strategies in the four categories. Namely:-
• SO Strategies (Strengths and Opportunities Strategy)
• ST Strategies (Strengths and Threats Strategy)
• WO Strategies (Weaknesses and Opportunities Strategy)
• WT Strategies (Weaknesses and Threats Strategy)
In this case, your strength is "your team is full of fighting spirit " and paired with your opportunities is " Opposite team is losing stamina" . With this scenario, what would you do? Perhaps you formulate a strategy to " ATTACK ". There it goes, you just formulate a attacking strategy.
Then you do the same procedure for SW Strategies, WO strategies and WT strategies.
Step 4 – Evaluate the Strategic Options
At the end of this paring of SWOT factors, you would have end up several strategic options. Do a quick evaluation of each of these strategies to the extent of meeting the company objectives.
Step 5 – Selecting Strategic Options
At this step, you would have a long list of strategic options. Too many strategies to implement may not be practical. Therefore, you need to shorten the list to perhaps maximum three strategies.
After you have completed all the 5 steps to use SWOT Analysis to Formulate Strategies, you have a list of strategies for you to implement to your business.
Pros and Cons of using SWOT in Strategic Planning
You may have gained some basic understanding of SWOT Analysis. You like to start using it for your work or your personal objectives. Whichever way you do it, it will bring about a your desired outcome because the data you collected for the Four factors of S.W.O.T. is objective and relevant.
If you have put the SWOT Analysis into real life practice, you could have faced with some difficulties in using it. But don't worry too much, as more practice would gain better experience with SWOT Analysis.
In this chapter, I will point out some of the Pros and Cons of using SWOT Analysis in Strategic Planning so you are aware of it. The sample list below should help you to reinforce your understanding of the SWOT Analysis.
PROS
1) Factual data are available to understand external factors as well as internal capabilities
2) Get a chance to evaluate the external opportunities and threats
3) A factual evaluation of own strengths and Weaknesses as compared with competitors
4) Open up a new dimension of competitive position
CONS
1) Time consuming
2) Data collected may not be current (member may take past single even to make conclusion)
3) Differences in opinion due to difference understanding of the SWOT process
4) Form own opinion of an event instead of base on factual information
SWOT is an abbreviation for Strengths, Weaknesses, Opportunities and Threats
An assessment of Strengths, Weaknesses, Opportunities, and Threats. SWOT analysis is used within organizations in the early stages of strategic and marketing planning. It is also used in problem solving, decision making, or for making staff aware of the need for change. It can be used at a personal level when examining your career path or determining possible career development
SWOT analysis is an important tool for auditing the overall strategic position of a business and its environment.
Once key strategic issues have been identified, they feed into business objectives, particularly marketing objectives. SWOT analysis can be used in conjunction with other tools for audit and analysis, such as PEST analysis and Porter's Five-Forces analysis. It is also a very popular tool with business and marketing students because it is quick and easy to learn
The Key Distinction - Internal and External Issues
Strengths and weaknesses are Internal factors. For example, strength could be your specialist marketing expertise. A weakness could be the lack of a new product.
Opportunities and threats are External factors. For example, an opportunity could be a developing distribution channel such as the Internet, or changing consumer lifestyles that potentially increase demand for a company's products. A threat could be a new competitor in an important existing market or a technological change that makes existing products potentially obsolete.
It is worth pointing out that SWOT analysis can be very subjective - two people rarely come-up with the same version of a SWOT analysis even when given the same information about the same business and its environment. Accordingly, SWOT analysis is best used as a guide and not a prescription. Adding and weighting criteria to each factor increases the validity of the analysis.
Where is S.W.O.T. being applied?
S.W.O.T. Analysis as it may sometimes being called can be performed in a variety of application or situation. It can be used as a situation analysis as an input into a strategic planning process at corporate of company level. It can also apply to evaluate the situation in terms of its capabilities. We use S.W.O.T. as a situation analysis tool.
When do we Perform a S.W.O.T. Analysis?
In common practice, S.W.O.T. Analysis is performed during the Strategic Planning or Business budget session normally done at the end of a financial year. But to perform a S.W.O.T. should not be limited to a yearly affair. You may perform a S.W.O.T. Analysis whenever it is needed to help you to identify causes of a non-conformance and you needed a new solution or strategy.
Who would Perform a S.W.O.T. Analysis?
In most cases, leaders of an organization perform a S.W.O.T. Analysis. However, it should not be limited to this group of people. In fact, anyone who has an interest and trained can perform a S.W.O.T. Analysis for the situation they are in. I have many situations where heads of a department perform a S.W.O.T. Analysis for their own operation issues because they want to develop solutions based on facts.
Why do you need to Perform a S.W.O.T. Analysis?
As it can be seen by now, data gathering is an essential part of S.W.O.T. Analysis. Hence, the information collected is likely to be more factual. Any solution derived from S.W.O.T. will be more realistic and reliable.
How to Perform a S.W.O.T. Analysis?
As data collection is one of the key activities in S.W.O.T. analysis, it should allow enough time to bring back the data. 1-3 month before a S.W.O.T. Analysis session is conducted. Once the data is collected, it should be grouped into the four factors. This can be done individually or in a team.
In summary, with some basic understanding of S.W.O.T. Analysis, the solution derived from it can be value add to the organization.
How SWOT Analysis is used to formulate Strategies
This is perhaps the most powerful usage of SWOT Analysis in the Strategic Planning Process. I am going to show you how to used the four factors of SWOT to develop Strategies
By now, you would have collected several data pertaining to the Strengths, Weaknesses, Opportunities and Threats. Then you will use them to formulate strategy. Not sure how to do it? Don't worry, I take you through the steps.
Step 1 – Evaluate the Surrounding
Let's take a moment to think about both of us as the coach for two teams of football teams.
Before the game starts, you and I have certain strategies that we want the team to follow. As the game progresses, there is sign of difference between the two teams in terms of the game as well as the condition of the team members.
Step 2 – Identify the Strengths, Weaknesses, Opportunities and Threats
Now, it is time to evaluate the teams in the four factors of SWOT. Let’s take the following examples as the result of the evaluation:-
Strengths - Your team full of fighting spirit
Weaknesses - One of your team members is hurt
Opportunities - Your opposition team seems to loose stamina
Threats - Your opposition team is full of energy
Note: Some of these factors seem to be conflicting each other. For the purpose of this step, this conflict is ignored.
Step 3 - Pair the SWOT factors to formulate Strategies
Now, you would start to formulate strategies in the four categories. Namely:-
• SO Strategies (Strengths and Opportunities Strategy)
• ST Strategies (Strengths and Threats Strategy)
• WO Strategies (Weaknesses and Opportunities Strategy)
• WT Strategies (Weaknesses and Threats Strategy)
In this case, your strength is "your team is full of fighting spirit " and paired with your opportunities is " Opposite team is losing stamina" . With this scenario, what would you do? Perhaps you formulate a strategy to " ATTACK ". There it goes, you just formulate a attacking strategy.
Then you do the same procedure for SW Strategies, WO strategies and WT strategies.
Step 4 – Evaluate the Strategic Options
At the end of this paring of SWOT factors, you would have end up several strategic options. Do a quick evaluation of each of these strategies to the extent of meeting the company objectives.
Step 5 – Selecting Strategic Options
At this step, you would have a long list of strategic options. Too many strategies to implement may not be practical. Therefore, you need to shorten the list to perhaps maximum three strategies.
After you have completed all the 5 steps to use SWOT Analysis to Formulate Strategies, you have a list of strategies for you to implement to your business.
Pros and Cons of using SWOT in Strategic Planning
You may have gained some basic understanding of SWOT Analysis. You like to start using it for your work or your personal objectives. Whichever way you do it, it will bring about a your desired outcome because the data you collected for the Four factors of S.W.O.T. is objective and relevant.
If you have put the SWOT Analysis into real life practice, you could have faced with some difficulties in using it. But don't worry too much, as more practice would gain better experience with SWOT Analysis.
In this chapter, I will point out some of the Pros and Cons of using SWOT Analysis in Strategic Planning so you are aware of it. The sample list below should help you to reinforce your understanding of the SWOT Analysis.
PROS
1) Factual data are available to understand external factors as well as internal capabilities
2) Get a chance to evaluate the external opportunities and threats
3) A factual evaluation of own strengths and Weaknesses as compared with competitors
4) Open up a new dimension of competitive position
CONS
1) Time consuming
2) Data collected may not be current (member may take past single even to make conclusion)
3) Differences in opinion due to difference understanding of the SWOT process
4) Form own opinion of an event instead of base on factual information
Thursday, October 22, 2009
Environmental Scanning
Environmental Scanning
Definition
Careful monitoring of a firm's internal and external environments for detecting early signs of opportunities and threats that may influence its current and future plans.
Objectives of an Environmental Scanning System
•Detecting scientific, technical, economic, social, and political trends and events important to the institution,
•Defining the potential threats, opportunities, or changes for the institution implied by those trends and events,
•Promoting a future orientation in the thinking of management and staff, and
•Alerting management and staff to trends that are converging, diverging, speeding up, slowing down, or interacting.
Fahey and Naravanan (1986) suggest that an effective environmental scanning program should enable decision makers to understand current and potential changes taking place in their institutions' external environments. Scanning provides strategic intelligence useful in determining organizational strategies. The consequences of this activity include fostering an understanding of the effects of change on organizations, aiding in forecasting, and bringing expectations of change to bear on decision making.
Experimental Research Designs
In an attempt to control for extraneous factors, several experimental research designs have been developed, including:
•Classical pretest-post test - The total population of participants is randomly divided into two samples; the control sample, and the experimental sample. Only the experimental sample is exposed to the manipulated variable. The researcher compares the pretest results with the post test results for both samples. Any divergence between the two samples is assumed to be a result of the experiment.
•Solomon four group design - The population is randomly divided into four samples. Two of the groups are experimental samples. Two groups experience no experimental manipulation of variables. Two groups receive a pretest and a post test. Two groups receive only a post test. This is an improvement over the classical design because it controls for the effect of the pretest.
•Factorial design - this is similar to a classical design except additional samples are used. Each group is exposed to a different experimental manipulation
Advantages and Disadvantages of Experimental Research
Advantages
*Gain insight into methods of instruction
*Intuitive practice shaped by research
*Teachers have bias but can be reflective
*Researcher can have control over variables
*Humans perform experiments anyway
*Can be combined with other research methods for rigor
*Use to determine what is best for population
*Provides for greater transferability than anecdotal research
Disadvantages
*Subject to human error
*Personal bias of researcher may intrude
*Sample may not be representative
*Can produce artificial results
*Results may only apply to one situation and may be difficult to replicate
*Groups may not be comparable
*Human response can be difficult to measure
*Political pressure may skew results
Definition
Careful monitoring of a firm's internal and external environments for detecting early signs of opportunities and threats that may influence its current and future plans.
Objectives of an Environmental Scanning System
•Detecting scientific, technical, economic, social, and political trends and events important to the institution,
•Defining the potential threats, opportunities, or changes for the institution implied by those trends and events,
•Promoting a future orientation in the thinking of management and staff, and
•Alerting management and staff to trends that are converging, diverging, speeding up, slowing down, or interacting.
Fahey and Naravanan (1986) suggest that an effective environmental scanning program should enable decision makers to understand current and potential changes taking place in their institutions' external environments. Scanning provides strategic intelligence useful in determining organizational strategies. The consequences of this activity include fostering an understanding of the effects of change on organizations, aiding in forecasting, and bringing expectations of change to bear on decision making.
Experimental Research Designs
In an attempt to control for extraneous factors, several experimental research designs have been developed, including:
•Classical pretest-post test - The total population of participants is randomly divided into two samples; the control sample, and the experimental sample. Only the experimental sample is exposed to the manipulated variable. The researcher compares the pretest results with the post test results for both samples. Any divergence between the two samples is assumed to be a result of the experiment.
•Solomon four group design - The population is randomly divided into four samples. Two of the groups are experimental samples. Two groups experience no experimental manipulation of variables. Two groups receive a pretest and a post test. Two groups receive only a post test. This is an improvement over the classical design because it controls for the effect of the pretest.
•Factorial design - this is similar to a classical design except additional samples are used. Each group is exposed to a different experimental manipulation
Advantages and Disadvantages of Experimental Research
Advantages
*Gain insight into methods of instruction
*Intuitive practice shaped by research
*Teachers have bias but can be reflective
*Researcher can have control over variables
*Humans perform experiments anyway
*Can be combined with other research methods for rigor
*Use to determine what is best for population
*Provides for greater transferability than anecdotal research
Disadvantages
*Subject to human error
*Personal bias of researcher may intrude
*Sample may not be representative
*Can produce artificial results
*Results may only apply to one situation and may be difficult to replicate
*Groups may not be comparable
*Human response can be difficult to measure
*Political pressure may skew results
Observational Techniques
What is Observational Techniques?
•Observational Techniques (or field research) is a social research technique that involves the direct observation of phenomena in their natural setting.
•Observational Techniques, a form of naturalistic inquiry, allow investigation of phenomena in their naturally occurring settings.
Participant observation is where the researcher joins the population or its organisation or community setting to record behaviours, interactions or events that occur. He or she engages in the activities that s/he is studying, but the first priority is the observation. Participation is a way to get close to the action and to get a feel for what things mean to the actors. As a participant, the evaluator is in a position to gain additional insights through experiencing the phenomena for themselves. Participant observation can be used as a long or short term technique. The evaluator/researcher has to stay long enough however to immerse him /herself in the local environment and culture and to earn acceptance and trust from the regular actors.
Observation consists of observing behaviour and interactions as they occur, but seen through the eyes of the researcher. There is no attempt to participate as a member of the group or setting, although usually the evaluator has to negotiate access to the setting and the terms of research activity. The intention is to ‘melt into the background’ so that an outsider presence has no direct effect on the phenomena under study. He or she tries to observe and understand the situation ‘from the inside’.
Observational techniques share similarities with the ethnographic approach that anthropologists use in studying a culture although typically they spend a long time in the field. Aspects of the ethnographic approach are sometimes incorporated into observational methods, as for example where interest is not just in behaviours and interactions but also in features and artefacts of the physical, social and cultural setting. These are taken to embed the norms, values, procedures and rituals of the organisation and reflect the ‘taken for granted’ background of the setting which influences behaviours understandings, beliefs and attitudes of the different actors.
Another form of naturalistic inquiry that complements observational methods is conversation and discourse analysis. This qualitative method studies naturally occurring talk and conversation in institutional and non-institutional settings, and offers insights into systems of social meaning and the methods used for producing orderly social interaction. It can be a useful technique for evaluating the conversational interaction between public service agents and clients in service delivery settings.
Main Steps in Observational Techniques
Observational methods generally involve the following steps.
Step 1. Choice of situations for observation: The settings for observation are defined in advance in relation to the interests of the evaluation commissioners and other key stakeholders. They consist of settings of interaction or of negotiation between public actors and the beneficiaries of the evaluated policy. The researcher negotiates access to the sites of observation with the relevant parties (informally, in the case of participant observation).
Step 2. Observation: The observer observes the course of interaction, taking care to disturb the behaviour of the actors as little as possible. This work consists of note-taking and audio-visual recordings (as discretely as possible). The observer can take notes away from research subjects or immediately after the visit.
This step cannot be limited to simple observation but must be complemented by organisational or institutional analysis so as to identify the ways in which social, cultural and physical features of the setting impinge on relations between the actors. The observer must record as much information as possible and capture an insider view of the setting.
Step 3. Analysing the material: One approach to processing the material gathered is to analyse the events observed in terms of characteristic sequences. Each recording is ‘cut up’ just as one would edit a film into sequences.
The observer identifies the ‘evaluative assertions’, that is to say, the sentences which convey an explicit or implicit value judgement. Typical sequences and their analysis are concentrated on these assertions, and reveal the way in which the policy is judged in the field. Used in this way, the tool can shed important new light on the validity and effectiveness of the policy.
Step 4. Analysis of typical sequences with the actors. The typical sequences and assertions are rewritten or modified to make them anonymous. They are then given to representatives of the people observed, for the purpose of collecting their comments and reactions. This step serves to verify that no bias has been created by taking the sequences out of their context. It gives, for each sequence, keys for interpretation which are recognised and validated by the ‘community’ under study.
Comments about the above section – only one method is described, analysis of sequences (or conversations?). More common, general observation technique is to write notes and code them afterwards, an ethnographic method and with this it is not usually returned to subjects to verify.
Types of Observation Technique
The most frequently used types of observational techniques are:
•Personal observation
1.Observing products in use to detect usage patterns and problems
2.Observing license plates in store parking lots
3.Determining the socio-economic status of shoppers
4.Determining the level of package scrutiny
5.Determining the time it takes to make a purchase decision
•Mechanical observation
1.Eye-tracking analysis while subjects watch advertisements
(a)Oculometers - what the subject is looking at
(b)Pupilometers - how interested is the viewer
(2)Electronic checkout scanners - records purchase behavior
(3)On-site cameras in stores
(4)Nielsen box for tracking television station watching
(5)Voice pitch meters - measures emotional reactions
(6)Psychogalvanometer - measures galvanic skin response
•Audits
i)Retail audits to determine the quality of service in stores
ii)Inventory audits to determine product acceptance
iii)Shelf space audits
•Trace Analysis
i)Credit card records
ii)Computer cookie records
iii)Garbology - looking for traces of purchase patterns in garbage
iv)Detecting store traffic patterns by observing the wear in the floor (long term) or the dirt on the floor (short term)
v)Exposure to advertisements
•Content analysis
i)Observe the content of magazines, television broadcasts, radio broadcasts, or newspapers, either articles, programs, or advertisements
Strengths and Limitations of Observational Techniques
Observation is a generic method that involves the collection, interpretation and comparison of data. It shares these characteristics with the case study method. It is therefore particularly well suited to the analysis of the effects of an intervention that is innovative or unfamiliar, and especially the clarification of confounding factors that influence the apparent success or failure of the interventions evaluated.
Observational techniques serve to reveal the discrepancy between the way in which public interventions are understood high up at decision-making level, and the way in which it is understood in the field; it highlights the interpretation made of it by individuals in an operational situation.
The observation is generally limited to a small number of settings. Generalisation is therefore possible only if the intervention is sufficiently homogeneous across sites.
It is based on spontaneous or naturalistic data, gathered by an independent and experienced observer. The reliability of the observation depends to a large extent on the professional know-how of the observer-analyst. It is however possible to introduce a structured observational template that can be used by less experienced researchers, when gathering data across a large number of settings.
Despite its advantages, observation requires meticulous preparation to enable the observer to fit into the observed context without disturbing anyone [what sort of preparation?], as well as considerable time for data collection. making it an expensive method.
The technique allows data to be gathered in difficult situations where other survey techniques cannot be used.
A major strength of using observational techniques, especially those based on Grounded Theory, is that they can capture unexpected data which other methods can miss. The researcher does not define categories of data before going out into the field but is open to “what’s there” – the theory emerges from the data on the ground rather than pre-defined theory influencing what data is collected.
The extent to which the observer can be present without disturbing or influencing research subjects is never nil; it is usually recommended that observers maintain self-awareness about how they impact the environment they are researching and to take account of it in their data collection. In participant observation the researcher aims to become part of a community or environment rather than maintaining a detached status.
•Observational Techniques (or field research) is a social research technique that involves the direct observation of phenomena in their natural setting.
•Observational Techniques, a form of naturalistic inquiry, allow investigation of phenomena in their naturally occurring settings.
Participant observation is where the researcher joins the population or its organisation or community setting to record behaviours, interactions or events that occur. He or she engages in the activities that s/he is studying, but the first priority is the observation. Participation is a way to get close to the action and to get a feel for what things mean to the actors. As a participant, the evaluator is in a position to gain additional insights through experiencing the phenomena for themselves. Participant observation can be used as a long or short term technique. The evaluator/researcher has to stay long enough however to immerse him /herself in the local environment and culture and to earn acceptance and trust from the regular actors.
Observation consists of observing behaviour and interactions as they occur, but seen through the eyes of the researcher. There is no attempt to participate as a member of the group or setting, although usually the evaluator has to negotiate access to the setting and the terms of research activity. The intention is to ‘melt into the background’ so that an outsider presence has no direct effect on the phenomena under study. He or she tries to observe and understand the situation ‘from the inside’.
Observational techniques share similarities with the ethnographic approach that anthropologists use in studying a culture although typically they spend a long time in the field. Aspects of the ethnographic approach are sometimes incorporated into observational methods, as for example where interest is not just in behaviours and interactions but also in features and artefacts of the physical, social and cultural setting. These are taken to embed the norms, values, procedures and rituals of the organisation and reflect the ‘taken for granted’ background of the setting which influences behaviours understandings, beliefs and attitudes of the different actors.
Another form of naturalistic inquiry that complements observational methods is conversation and discourse analysis. This qualitative method studies naturally occurring talk and conversation in institutional and non-institutional settings, and offers insights into systems of social meaning and the methods used for producing orderly social interaction. It can be a useful technique for evaluating the conversational interaction between public service agents and clients in service delivery settings.
Main Steps in Observational Techniques
Observational methods generally involve the following steps.
Step 1. Choice of situations for observation: The settings for observation are defined in advance in relation to the interests of the evaluation commissioners and other key stakeholders. They consist of settings of interaction or of negotiation between public actors and the beneficiaries of the evaluated policy. The researcher negotiates access to the sites of observation with the relevant parties (informally, in the case of participant observation).
Step 2. Observation: The observer observes the course of interaction, taking care to disturb the behaviour of the actors as little as possible. This work consists of note-taking and audio-visual recordings (as discretely as possible). The observer can take notes away from research subjects or immediately after the visit.
This step cannot be limited to simple observation but must be complemented by organisational or institutional analysis so as to identify the ways in which social, cultural and physical features of the setting impinge on relations between the actors. The observer must record as much information as possible and capture an insider view of the setting.
Step 3. Analysing the material: One approach to processing the material gathered is to analyse the events observed in terms of characteristic sequences. Each recording is ‘cut up’ just as one would edit a film into sequences.
The observer identifies the ‘evaluative assertions’, that is to say, the sentences which convey an explicit or implicit value judgement. Typical sequences and their analysis are concentrated on these assertions, and reveal the way in which the policy is judged in the field. Used in this way, the tool can shed important new light on the validity and effectiveness of the policy.
Step 4. Analysis of typical sequences with the actors. The typical sequences and assertions are rewritten or modified to make them anonymous. They are then given to representatives of the people observed, for the purpose of collecting their comments and reactions. This step serves to verify that no bias has been created by taking the sequences out of their context. It gives, for each sequence, keys for interpretation which are recognised and validated by the ‘community’ under study.
Comments about the above section – only one method is described, analysis of sequences (or conversations?). More common, general observation technique is to write notes and code them afterwards, an ethnographic method and with this it is not usually returned to subjects to verify.
Types of Observation Technique
The most frequently used types of observational techniques are:
•Personal observation
1.Observing products in use to detect usage patterns and problems
2.Observing license plates in store parking lots
3.Determining the socio-economic status of shoppers
4.Determining the level of package scrutiny
5.Determining the time it takes to make a purchase decision
•Mechanical observation
1.Eye-tracking analysis while subjects watch advertisements
(a)Oculometers - what the subject is looking at
(b)Pupilometers - how interested is the viewer
(2)Electronic checkout scanners - records purchase behavior
(3)On-site cameras in stores
(4)Nielsen box for tracking television station watching
(5)Voice pitch meters - measures emotional reactions
(6)Psychogalvanometer - measures galvanic skin response
•Audits
i)Retail audits to determine the quality of service in stores
ii)Inventory audits to determine product acceptance
iii)Shelf space audits
•Trace Analysis
i)Credit card records
ii)Computer cookie records
iii)Garbology - looking for traces of purchase patterns in garbage
iv)Detecting store traffic patterns by observing the wear in the floor (long term) or the dirt on the floor (short term)
v)Exposure to advertisements
•Content analysis
i)Observe the content of magazines, television broadcasts, radio broadcasts, or newspapers, either articles, programs, or advertisements
Strengths and Limitations of Observational Techniques
Observation is a generic method that involves the collection, interpretation and comparison of data. It shares these characteristics with the case study method. It is therefore particularly well suited to the analysis of the effects of an intervention that is innovative or unfamiliar, and especially the clarification of confounding factors that influence the apparent success or failure of the interventions evaluated.
Observational techniques serve to reveal the discrepancy between the way in which public interventions are understood high up at decision-making level, and the way in which it is understood in the field; it highlights the interpretation made of it by individuals in an operational situation.
The observation is generally limited to a small number of settings. Generalisation is therefore possible only if the intervention is sufficiently homogeneous across sites.
It is based on spontaneous or naturalistic data, gathered by an independent and experienced observer. The reliability of the observation depends to a large extent on the professional know-how of the observer-analyst. It is however possible to introduce a structured observational template that can be used by less experienced researchers, when gathering data across a large number of settings.
Despite its advantages, observation requires meticulous preparation to enable the observer to fit into the observed context without disturbing anyone [what sort of preparation?], as well as considerable time for data collection. making it an expensive method.
The technique allows data to be gathered in difficult situations where other survey techniques cannot be used.
A major strength of using observational techniques, especially those based on Grounded Theory, is that they can capture unexpected data which other methods can miss. The researcher does not define categories of data before going out into the field but is open to “what’s there” – the theory emerges from the data on the ground rather than pre-defined theory influencing what data is collected.
The extent to which the observer can be present without disturbing or influencing research subjects is never nil; it is usually recommended that observers maintain self-awareness about how they impact the environment they are researching and to take account of it in their data collection. In participant observation the researcher aims to become part of a community or environment rather than maintaining a detached status.
Tuesday, October 20, 2009
Sampling
Sampling Ratio
This is the proportion of elements in the population that are selected (one name for every two respondent in the class).
Sampling ratio= Sample size/pop size.
Sampling Interval
This is the standard distance between elements selected in the sample population size/sample size
Sampling Methods
Sampling is a very important part of the Market Research process. If you have surveyed using an appropriate sampling technique, you can be confident that your results will be generalised to the population in question. If the sample were biased in any way, for example, if the selection technique gave older people more of a chance of selection than younger people, it would be inadvisable to make generalisations from the findings.
There are essentiality two types of sampling: probability and non-probability sampling.
• Probability Sampling Methods
Probability or random sampling gives all members of the population a known chance of being selected for inclusion in the sample and this does not depend upon previous events in the selection process. In other words, the selection of individuals does not affect the chance of anyone else in the population being selected.
Many statistical techniques assume that a sample was selected on a random basis. There are four basic types of random sampling techniques:
1) Simple Random Sampling
This is the ideal choice as it is a ‘perfect’ random method. Using this method, individuals are randomly selected from a list of the population and every single individual has an equal chance of selection.
This method is ideal, but if it cannot be adopted, one of the following alternatives may be chosen if any shortfall in accuracy.
2) Systematic Sampling
Systematic sampling is a frequently used variant of simple random sampling. When performing systematic sampling, every element from the list is selected (this is referred to as the sample interval) from a randomly selected starting point. For example, if we have a listed population of 6000 members and wish to draw a sample of 2000, we would select every 30th (6000 divided by 200) person from the list. In practice, we would randomly select a number between 1 and 30 to act as our starting point.
The one potential problem with this method of sampling concerns the arrangement of elements in the list.? If the list is arranged in any kind of order e.g. if every 30th house is smaller than the others from which the sample is being recruited, there is a possibility that the sample produced could be seriously biased.
3) Stratified Sampling
Stratified sampling is a variant on simple random and systematic methods and is used when there are a number of distinct subgroups, within each of which it is required that there is full representation. A stratified sample is constructed by classifying the population in sub-populations (or strata), base on some well-known characteristics of the population, such as age, gender or socio-economic status. The selection of elements is then made separately from within each strata, usually by random or systematic sampling methods.
Stratified sampling methods also come in two types – proportionate and disproportionate.
In proportionate sampling, the strata sample sizes are made proportional to the strata population sizes. For example if the first strata is made up of males, then as there are around 50% of males in the UK population, the male strata will need to represent around 50% of the total sample.
In disproportionate methods, the strata are not sampled according to the population sizes, but higher proportions are selected from some groups and not others. This technique is typically used in a number of distinct situations:
The costs of collecting data may differ from subgroup to subgroup.
We might require more cases in some groups if estimations of populations values are likely to be harder to make i.e. the larger the sample size (up to certain limits), the more accurate any estimations are likely to be.
We expect different response rates from different groups of people. Therefore, the less co-operative groups might be ‘over-sampled’ to compensate.
4) Cluster or Multi-stage Sampling
Cluster sampling is a frequently-used, and usually more practical, random sampling method. It is particularly useful in situations for which no list of the elements within a population is available and therefore cannot be selected directly. As this form of sampling is conducted by randomly selecting subgroups of the population, possibly in several stages, it should produce results equivalent to a simple random sample.
The sample is generally done by first sampling at the higher level(s) e.g. randomly sampled countries, then sampling from subsequent levels in turn e.g. within the selected countries sample counties, then within these postcodes, the within these households, until the final stage is reached, at which point the sampling is done in a simple random manner e.g. sampling people within the selected households. The ‘levels’ in question are defined by subgroups into which it is appropriate to subdivide your population.
Cluster samples are generally used if:
- No list of the population exists.
- Well-defined clusters, which will often be geographic areas exist.
- A reasonable estimate of the number of elements in each level of clustering can be made.
- Often the total sample size must be fairly large to enable cluster sampling to be used effectively.
•Non-probability Sampling Methods
Non-probability sampling procedures are much less desirable, as they will almost certainly contain sampling biases. Unfortunately, in some circumstances such methods are unavoidable.
In a Market Research context, the most frequently-adopted form of non-probability sampling is known as quota sampling.? In some ways this is similar to cluster sampling in that it requires the definition of key subgroups. The main difference lies in the fact that quotas (i.e. the amount of people to be surveyed) within subgroups are set beforehand (e.g. 25% 16-24 yr olds, 30% 25-34 yr olds, 20% 35-55 yr olds, and 25% 56+ yr olds) usually proportions are set to match known population distributions. Interviewers then select respondents according to these criteria rather than at random. The subjective nature of this selection means that only about a proportion of the population has a chance of being selected in a typical quota sampling strategy.
If you are forced into using a non-random method, you must be extremely careful when drawing conclusions. You should always be honest about the sampling technique used and that a non-random approach will probably mean that biases are present within the data. In order to convert the sample to be representative of the true population, you may want to use weighting techniques.
The importance of sampling should not be underestimated, as it determines to whom the results of your research will be applicable. It is important, therefore to give full consideration to the sampling strategy to be used and to select the most appropriate. Your most important consideration should be whether you could adopt a simple random sample.? If not, could one of the other random methods be used? Only when you have no choice should a non-random method be used.
All too often, researchers succumb to the temptation of generalising their results to a much broader range of people than those from whom the data was originally gathered. This is poor practice and you should always aim to adopt an appropriate sampling technique. The key is not to guess, but take some advice?
General Advantages
•Typicality of subjects is aimed for
•Permits exploration
- General Disadvantage
•Unrepresentative
Calculating a Sample Size
A frequently asked question is “How many people should I sample?” It is an extremely good question, although unfortunately there is no single answer! In general, the larger the sample size, the more closely your sample data will match that from the population. However in practice, you need to work out how many responses will give you sufficient precision at an affordable cost.
Calculation of an appropriate sample size depends upon a number of factors unique to each survey and it is down to you to make the decision regarding these factors. The three most important are:
- How accurate you wish to be
- How confident you are in the results
- What budget you have available
The temptation is to say all should be as high as possible. The problem is that an increase in either accuracy or confidence (or both) will always require a larger sample and higher budget. Therefore a compromise must be reached and you must work out the degree of inaccuracy and confidence you are prepared to accept.
There are two types of figures that you may wish to estimate in your Market Research project: values such as mean income, mean height etc. and proportions (the percentage of people who intend to vote for party X). There are slightly different sample size calculations for each:
For a mean
The required formula is: s = (z / e)2
Where:
s = the sample size
z = a number relating to the degree of confidence you wish to have in the result. 95% confidence* is most frequently used and accepted. The value of ‘z’ should be 2.58 for 99% confidence, 1.96 for 95% confidence, 1.64 for 90% confidence and 1.28 for 80% confidence.
e = the error you are prepared to accept, measured as a proportion of the standard deviation (accuracy)
For example, imagine we are estimating mean income, and wish to know what sample size to aim for in order that we can be 95% confident in the result. Assuming that we are prepared to accept an error of 10% of the population standard deviation (previous research might have shown the standard deviation of income to be 8000 and we might be prepared to accept an error of 800 (10%)), we would do the following calculation:
s = (1.96 / 0.1)2
Therefore s = 384.16
In other words, 385 people would need to be sampled to meet our criterion.
*Because we interviewed a sample and not the whole population (if we had done this we could be 100% confident in our results), we have to be prepared to be less confident and because we based our sample size calculation on the 95% confidence level, we can be confident that amongst the whole population there is a 95% chance that the mean is inside our acceptable error limit. There is of course a 5% chance that the measure is outside this limit. If we wanted to be more confident, we would base our sample size calculation on a 99% confidence level and if we were prepared to accept a lower level of confidence, we would base our calculation on the 90% confidence level.`
For a Proportion
Although we are doing the same thing here, the formula is different:
s = z2(p(1-p))
???????? e2
Where:
s = the sample size
z = the number relating to the degree of confidence you wish to have in the result
p = an estimate of the proportion of people falling into the group in which you are interested in the population
e = the proportion of error we are prepared to accept
As an example, imagine we are attempting to assess the percentage of voters who will vote for candidate X. If we assume that we wish to be 99% confident of the result i.e. z = 2.85 and that we will allow for errors in the region of +/-3% i.e. e = 0.03. But in terms of an estimate of the proportion of the population who would vote for the candidate (p), if a previous survey had been carried out, we could use the percentage from that survey as an estimate. However, if this were the first survey, we would assume that 50% (i.e. p = 0.05) of people would vote for candidate X and 50% would not. Choosing 50% will provide the most conservative estimate of sample size. If the true percentage were 10%, we will still have an accurate estimate; we will simply have sampled more people than was absolutely necessary. The reverse situation, not having enough data to make reliable estimates, is much less desirable.
In the example:
s = 2.582(0.5*0.5)
???????? 0.032
Therefore s = 1,849
This rather large sample was necessary because we wanted to be 99% sure of the result and desired and desired a very narrow (+/-3%) margin of error. It does, however reveal why many political polls tend to interview between 1,000 and 2,000 people.
This is the proportion of elements in the population that are selected (one name for every two respondent in the class).
Sampling ratio= Sample size/pop size.
Sampling Interval
This is the standard distance between elements selected in the sample population size/sample size
Sampling Methods
Sampling is a very important part of the Market Research process. If you have surveyed using an appropriate sampling technique, you can be confident that your results will be generalised to the population in question. If the sample were biased in any way, for example, if the selection technique gave older people more of a chance of selection than younger people, it would be inadvisable to make generalisations from the findings.
There are essentiality two types of sampling: probability and non-probability sampling.
• Probability Sampling Methods
Probability or random sampling gives all members of the population a known chance of being selected for inclusion in the sample and this does not depend upon previous events in the selection process. In other words, the selection of individuals does not affect the chance of anyone else in the population being selected.
Many statistical techniques assume that a sample was selected on a random basis. There are four basic types of random sampling techniques:
1) Simple Random Sampling
This is the ideal choice as it is a ‘perfect’ random method. Using this method, individuals are randomly selected from a list of the population and every single individual has an equal chance of selection.
This method is ideal, but if it cannot be adopted, one of the following alternatives may be chosen if any shortfall in accuracy.
2) Systematic Sampling
Systematic sampling is a frequently used variant of simple random sampling. When performing systematic sampling, every element from the list is selected (this is referred to as the sample interval) from a randomly selected starting point. For example, if we have a listed population of 6000 members and wish to draw a sample of 2000, we would select every 30th (6000 divided by 200) person from the list. In practice, we would randomly select a number between 1 and 30 to act as our starting point.
The one potential problem with this method of sampling concerns the arrangement of elements in the list.? If the list is arranged in any kind of order e.g. if every 30th house is smaller than the others from which the sample is being recruited, there is a possibility that the sample produced could be seriously biased.
3) Stratified Sampling
Stratified sampling is a variant on simple random and systematic methods and is used when there are a number of distinct subgroups, within each of which it is required that there is full representation. A stratified sample is constructed by classifying the population in sub-populations (or strata), base on some well-known characteristics of the population, such as age, gender or socio-economic status. The selection of elements is then made separately from within each strata, usually by random or systematic sampling methods.
Stratified sampling methods also come in two types – proportionate and disproportionate.
In proportionate sampling, the strata sample sizes are made proportional to the strata population sizes. For example if the first strata is made up of males, then as there are around 50% of males in the UK population, the male strata will need to represent around 50% of the total sample.
In disproportionate methods, the strata are not sampled according to the population sizes, but higher proportions are selected from some groups and not others. This technique is typically used in a number of distinct situations:
The costs of collecting data may differ from subgroup to subgroup.
We might require more cases in some groups if estimations of populations values are likely to be harder to make i.e. the larger the sample size (up to certain limits), the more accurate any estimations are likely to be.
We expect different response rates from different groups of people. Therefore, the less co-operative groups might be ‘over-sampled’ to compensate.
4) Cluster or Multi-stage Sampling
Cluster sampling is a frequently-used, and usually more practical, random sampling method. It is particularly useful in situations for which no list of the elements within a population is available and therefore cannot be selected directly. As this form of sampling is conducted by randomly selecting subgroups of the population, possibly in several stages, it should produce results equivalent to a simple random sample.
The sample is generally done by first sampling at the higher level(s) e.g. randomly sampled countries, then sampling from subsequent levels in turn e.g. within the selected countries sample counties, then within these postcodes, the within these households, until the final stage is reached, at which point the sampling is done in a simple random manner e.g. sampling people within the selected households. The ‘levels’ in question are defined by subgroups into which it is appropriate to subdivide your population.
Cluster samples are generally used if:
- No list of the population exists.
- Well-defined clusters, which will often be geographic areas exist.
- A reasonable estimate of the number of elements in each level of clustering can be made.
- Often the total sample size must be fairly large to enable cluster sampling to be used effectively.
•Non-probability Sampling Methods
Non-probability sampling procedures are much less desirable, as they will almost certainly contain sampling biases. Unfortunately, in some circumstances such methods are unavoidable.
In a Market Research context, the most frequently-adopted form of non-probability sampling is known as quota sampling.? In some ways this is similar to cluster sampling in that it requires the definition of key subgroups. The main difference lies in the fact that quotas (i.e. the amount of people to be surveyed) within subgroups are set beforehand (e.g. 25% 16-24 yr olds, 30% 25-34 yr olds, 20% 35-55 yr olds, and 25% 56+ yr olds) usually proportions are set to match known population distributions. Interviewers then select respondents according to these criteria rather than at random. The subjective nature of this selection means that only about a proportion of the population has a chance of being selected in a typical quota sampling strategy.
If you are forced into using a non-random method, you must be extremely careful when drawing conclusions. You should always be honest about the sampling technique used and that a non-random approach will probably mean that biases are present within the data. In order to convert the sample to be representative of the true population, you may want to use weighting techniques.
The importance of sampling should not be underestimated, as it determines to whom the results of your research will be applicable. It is important, therefore to give full consideration to the sampling strategy to be used and to select the most appropriate. Your most important consideration should be whether you could adopt a simple random sample.? If not, could one of the other random methods be used? Only when you have no choice should a non-random method be used.
All too often, researchers succumb to the temptation of generalising their results to a much broader range of people than those from whom the data was originally gathered. This is poor practice and you should always aim to adopt an appropriate sampling technique. The key is not to guess, but take some advice?
General Advantages
•Typicality of subjects is aimed for
•Permits exploration
- General Disadvantage
•Unrepresentative
Calculating a Sample Size
A frequently asked question is “How many people should I sample?” It is an extremely good question, although unfortunately there is no single answer! In general, the larger the sample size, the more closely your sample data will match that from the population. However in practice, you need to work out how many responses will give you sufficient precision at an affordable cost.
Calculation of an appropriate sample size depends upon a number of factors unique to each survey and it is down to you to make the decision regarding these factors. The three most important are:
- How accurate you wish to be
- How confident you are in the results
- What budget you have available
The temptation is to say all should be as high as possible. The problem is that an increase in either accuracy or confidence (or both) will always require a larger sample and higher budget. Therefore a compromise must be reached and you must work out the degree of inaccuracy and confidence you are prepared to accept.
There are two types of figures that you may wish to estimate in your Market Research project: values such as mean income, mean height etc. and proportions (the percentage of people who intend to vote for party X). There are slightly different sample size calculations for each:
For a mean
The required formula is: s = (z / e)2
Where:
s = the sample size
z = a number relating to the degree of confidence you wish to have in the result. 95% confidence* is most frequently used and accepted. The value of ‘z’ should be 2.58 for 99% confidence, 1.96 for 95% confidence, 1.64 for 90% confidence and 1.28 for 80% confidence.
e = the error you are prepared to accept, measured as a proportion of the standard deviation (accuracy)
For example, imagine we are estimating mean income, and wish to know what sample size to aim for in order that we can be 95% confident in the result. Assuming that we are prepared to accept an error of 10% of the population standard deviation (previous research might have shown the standard deviation of income to be 8000 and we might be prepared to accept an error of 800 (10%)), we would do the following calculation:
s = (1.96 / 0.1)2
Therefore s = 384.16
In other words, 385 people would need to be sampled to meet our criterion.
*Because we interviewed a sample and not the whole population (if we had done this we could be 100% confident in our results), we have to be prepared to be less confident and because we based our sample size calculation on the 95% confidence level, we can be confident that amongst the whole population there is a 95% chance that the mean is inside our acceptable error limit. There is of course a 5% chance that the measure is outside this limit. If we wanted to be more confident, we would base our sample size calculation on a 99% confidence level and if we were prepared to accept a lower level of confidence, we would base our calculation on the 90% confidence level.`
For a Proportion
Although we are doing the same thing here, the formula is different:
s = z2(p(1-p))
???????? e2
Where:
s = the sample size
z = the number relating to the degree of confidence you wish to have in the result
p = an estimate of the proportion of people falling into the group in which you are interested in the population
e = the proportion of error we are prepared to accept
As an example, imagine we are attempting to assess the percentage of voters who will vote for candidate X. If we assume that we wish to be 99% confident of the result i.e. z = 2.85 and that we will allow for errors in the region of +/-3% i.e. e = 0.03. But in terms of an estimate of the proportion of the population who would vote for the candidate (p), if a previous survey had been carried out, we could use the percentage from that survey as an estimate. However, if this were the first survey, we would assume that 50% (i.e. p = 0.05) of people would vote for candidate X and 50% would not. Choosing 50% will provide the most conservative estimate of sample size. If the true percentage were 10%, we will still have an accurate estimate; we will simply have sampled more people than was absolutely necessary. The reverse situation, not having enough data to make reliable estimates, is much less desirable.
In the example:
s = 2.582(0.5*0.5)
???????? 0.032
Therefore s = 1,849
This rather large sample was necessary because we wanted to be 99% sure of the result and desired and desired a very narrow (+/-3%) margin of error. It does, however reveal why many political polls tend to interview between 1,000 and 2,000 people.
Non- Sampling Error
What is Non- Sampling Error?
Definition
Any error affecting a survey or census estimate apart from sampling error
Occurs in complete censuses as well as in sample surveys
Types of Non- Sampling Error
•Non-Response Error
•Response Error
•Processing Error
•Coverage Error
Definition
Any error affecting a survey or census estimate apart from sampling error
Occurs in complete censuses as well as in sample surveys
Types of Non- Sampling Error
•Non-Response Error
•Response Error
•Processing Error
•Coverage Error
Standard Error (SE)
Definition
A measure of the variability of an estimate due to sampling
Depends on variability in the population and sample size
Foundational measure
Margin of Error (MOE)
Definition
A measure of the precision of an estimate at a given level of confidence (90%, 95%, 99%)
Confidence level of a MOE
MOEs at the 90% confidence level for all published ACS estimates
Margin of Error (MOE)
Confidence Interval
Definition
A range that is expected to contain the population value of the characteristic with a known probability.
Formula
Where
LCL is the lower bound at the desired confidence level
UCL is the upper bound at the desired confidence level
is the ACS estimate and
is the margin of error at the desired confidence level
Confidence Interval computation
Coefficient of Variation (CV)
Definition
The relative amount of sampling error associated with a sample estimate
Sampling Error is related to Sample Size.
.The larger the sample size, the smaller the uncertainty or sampling error
•Combining ACS data from multiple years increases sample size and reduces sampling error
•All sample surveys have sampling error – including decennial census long-form data
How to Use Measures Associated With Sampling Error
How are Measures of Sampling Error Used?
•To indicate the statistical reliability and usability of estimates
•To make comparisons between estimates
•To conduct tests of statistical significance
•To help users draw appropriate conclusions about data
Test of Statistical Significance
Definition
A test to determine if it is unlikely that something has occurred by chance
A “statistically significant difference” means there is statistical evidence that there is a difference
A measure of the variability of an estimate due to sampling
Depends on variability in the population and sample size
Foundational measure
Margin of Error (MOE)
Definition
A measure of the precision of an estimate at a given level of confidence (90%, 95%, 99%)
Confidence level of a MOE
MOEs at the 90% confidence level for all published ACS estimates
Margin of Error (MOE)
Confidence Interval
Definition
A range that is expected to contain the population value of the characteristic with a known probability.
Formula
Where
LCL is the lower bound at the desired confidence level
UCL is the upper bound at the desired confidence level
is the ACS estimate and
is the margin of error at the desired confidence level
Confidence Interval computation
Coefficient of Variation (CV)
Definition
The relative amount of sampling error associated with a sample estimate
Sampling Error is related to Sample Size.
.The larger the sample size, the smaller the uncertainty or sampling error
•Combining ACS data from multiple years increases sample size and reduces sampling error
•All sample surveys have sampling error – including decennial census long-form data
How to Use Measures Associated With Sampling Error
How are Measures of Sampling Error Used?
•To indicate the statistical reliability and usability of estimates
•To make comparisons between estimates
•To conduct tests of statistical significance
•To help users draw appropriate conclusions about data
Test of Statistical Significance
Definition
A test to determine if it is unlikely that something has occurred by chance
A “statistically significant difference” means there is statistical evidence that there is a difference
Definition of Sampling Error
What is Sampling Error?
Definition
The uncertainty associated with an estimate that is based on data gathered from a sample of the population rather than the full population
Calculating a Sampling Error
In estimating the accuracy of a sample (sampling error), or selecting a sample to meet a required level of accuracy, there are two critical variables; the size of the sample and the measure being taken which for simplicity we shall take as a single percentage e.g. the percentage aware of a brand. A common mistake about sample size is to assume that accuracy is determined by the proportion of a population included in a sample (e.g. 10% of a population). Assuming a large population, this is not the case and what matters is the absolute size of the sample regardless of the size of the population – a sample of 500 drawn from a population of one million will be as accurate as a sample of 500 from a population of five million (assuming both are truly random samples of the respective populations).
Where:
e = sampling error (the proportion of error we are prepared to accept)
s = the sample size
z = the number relating to the degree of confidence you wish to have in the result
p = an estimate of the proportion of people falling into the group in which you are interested in the population
By applying the formula it can be calculated, for example, that from a sample of 500 respondents (s), a measure of 20% aware of a brand (p), will have a sample error of +/-3.5% at the 95% confidence level.
e = 1.96√(20(80))
??????????? √ 500
This means, therefore, that based on a sample of 500 we can be 95% sure that the true measure (e.g. of brand awareness) among the whole population from which the sample was drawn will be within +/-3.5% of 20% i.e. between 16.5% and 23.5%
The relationship between sampling error, a percentage measure and a sample size can be expressed as a formula.
e = z√(p%(100-p%))
?????????????? √ s
Measures Associated with Sampling Error
Measures Associated with Sampling Error includes:
•Standard Error (SE)
•Margin of Error (MOE)
•Confidence Interval (CI)
•Coefficient of Variation (CV)
Definition
The uncertainty associated with an estimate that is based on data gathered from a sample of the population rather than the full population
Calculating a Sampling Error
In estimating the accuracy of a sample (sampling error), or selecting a sample to meet a required level of accuracy, there are two critical variables; the size of the sample and the measure being taken which for simplicity we shall take as a single percentage e.g. the percentage aware of a brand. A common mistake about sample size is to assume that accuracy is determined by the proportion of a population included in a sample (e.g. 10% of a population). Assuming a large population, this is not the case and what matters is the absolute size of the sample regardless of the size of the population – a sample of 500 drawn from a population of one million will be as accurate as a sample of 500 from a population of five million (assuming both are truly random samples of the respective populations).
Where:
e = sampling error (the proportion of error we are prepared to accept)
s = the sample size
z = the number relating to the degree of confidence you wish to have in the result
p = an estimate of the proportion of people falling into the group in which you are interested in the population
By applying the formula it can be calculated, for example, that from a sample of 500 respondents (s), a measure of 20% aware of a brand (p), will have a sample error of +/-3.5% at the 95% confidence level.
e = 1.96√(20(80))
??????????? √ 500
This means, therefore, that based on a sample of 500 we can be 95% sure that the true measure (e.g. of brand awareness) among the whole population from which the sample was drawn will be within +/-3.5% of 20% i.e. between 16.5% and 23.5%
The relationship between sampling error, a percentage measure and a sample size can be expressed as a formula.
e = z√(p%(100-p%))
?????????????? √ s
Measures Associated with Sampling Error
Measures Associated with Sampling Error includes:
•Standard Error (SE)
•Margin of Error (MOE)
•Confidence Interval (CI)
•Coefficient of Variation (CV)
Sampling
What is sampling?
•Sampling is the process of selecting units (e.g., people, organizations) from a population of interest so that by studying the sample we may fairly generalize our results back to the population from which they were chosen.
•Sampling is the act, process, or technique of selecting a suitable sample, or a representative part of a population for the purpose of determining parameters or characteristics of the whole population
Process of Sampling
The sampling process comprises several stages:
•Defining the population of concern
•Specifying a sampling frame, a set of items or events possible to measure
•Specifying a sampling method for selecting items or events from the frame
•Determining the sample size
•Implementing the sampling plan
•Sampling and data collecting
•Reviewing the sampling process
•Sampling is the process of selecting units (e.g., people, organizations) from a population of interest so that by studying the sample we may fairly generalize our results back to the population from which they were chosen.
•Sampling is the act, process, or technique of selecting a suitable sample, or a representative part of a population for the purpose of determining parameters or characteristics of the whole population
Process of Sampling
The sampling process comprises several stages:
•Defining the population of concern
•Specifying a sampling frame, a set of items or events possible to measure
•Specifying a sampling method for selecting items or events from the frame
•Determining the sample size
•Implementing the sampling plan
•Sampling and data collecting
•Reviewing the sampling process
Sunday, October 18, 2009
Question Wording
The wording of a question is extremely important. Researchers strive for objectivity in surveys and, therefore, must be careful not to lead the respondent into giving a desired answer. Unfortunately, the effects of question wording are one of the least understood areas of questionnaire research.
Many investigators have confirmed that slight changes in the way questions are worded can have a significant impact on how people respond. Several authors have reported that minor changes in question wording can produce more than a 25 percent difference in people's opinions.
Several investigators have looked at the effects of modifying adjectives and adverbs. Words like usually, often, sometimes, occasionally, seldom, and rarely are "commonly" used in questionnaires, although it is clear that they do not mean the same thing to all people. Some adjectives have high variability and others have low variability. The following adjectives have highly variable meanings and should be avoided in surveys: a clear mandate, most, numerous, a substantial majority, a minority of, a large proportion of, a significant number of, many, a considerable number of, and several. Other adjectives produce less variability and generally have more shared meaning. These are: lots, almost all, virtually all, nearly all, a majority of, a consensus of, a small number of, not very many of, almost none, hardly any, a couple, and a few.
The Length of a Questionnaire
As a general rule, long questionnaires get less response than short questionnaires. However, some studies have shown that the length of a questionnaire does not necessarily affect response. More important than length is question content. A subject is more likely to respond if they are involved and interested in the research topic. Questions should be meaningful and interesting to the respondent.
Anonymity and Confidentiality
An anonymous study is one in which nobody (not even the researcher) can identify who provided data. It is difficult to conduct an anonymous questionnaire through the mail because of the need to follow-up on non responders. The only way to do a follow-up is to mail another survey or reminder postcard to the entire sample. However, it is possible to guarantee confidentiality, where those conducting the study promise not to reveal the information to anyone. For the purpose of follow-up, identifying numbers on questionnaires are generally preferred to using respondents' names. It is important, however, to explain why the number is there and what it will be used for.
Some studies have shown that response rate is affected by the anonymity/confidentiality policy of a study. Others have reported that responses became more distorted when subjects felt threatened that their identities would become known. Others have found that anonymity and confidentiality issues do not affect response rates or responses.
Qualities of a Good Question
There are good and bad questions. The qualities of a good question are as follows:
•Evokes the truth. Questions must be non-threatening. When a respondent is concerned about the consequences of answering a question in a particular manner, there is a good possibility that the answer will not be truthful. Anonymous questionnaires that contain no identifying information are more likely to produce honest responses than those identifying the respondent. If your questionnaire does contain sensitive items, be sure to clearly state your policy on confidentiality
•Asks for an answer on only one dimension. The purpose of a survey is to find out information. A question that asks for a response on more than one dimension will not provide the information you are seeking. For example, a researcher investigating a new food snack asks "Do you like the texture and flavor of the snack?" If a respondent answers "no", then the researcher will not know if the respondent dislikes the texture or the flavor, or both. Another questionnaire asks, "Were you satisfied with the quality of our food and service?" Again, if the respondent answers "no", there is no way to know whether the quality of the food, service, or both were unsatisfactory. A good question asks for only one "bit" of information.
•Can accommodate all possible answers. Multiple choice items are the most popular type of survey questions because they are generally the easiest for a respondent to answer and the easiest to analyze. Asking a question that does not accommodate all possible responses can confuse and frustrate the respondent. For example, consider the question:
What brand of computer do you own? __
A. IBM PC
B. Apple
Clearly, there are many problems with this question. What if the respondent doesn't own a microcomputer? What if he owns a different brand of computer? What if he owns both an IBM PC and an Apple? There are two ways to correct this kind of problem.
The first way is to make each response a separate dichotomous item on the questionnaire. For example:
Do you own an IBM PC? (circle: Yes or No)
Do you own an Apple computer? (circle: Yes or No)
Another way to correct the problem is to add the necessary response categories and allow multiple responses. This is the preferable method because it provides more information than the previous method.
What brand of computer do you own?
(Check all that apply)
Do not own a computer
IBM PC
Apple
Other
•Has mutually exclusive options. A good question leaves no ambiguity in the mind of the respondent. There should be only one correct or appropriate choice for the respondent to make. An obvious example is:
Where did you grow up? __
A. country
B. farm
C. city
A person who grew up on a farm in the country would not know whether to select choice A or B. This question would not provide meaningful information. Worse than that, it could frustrate the respondent and the questionnaire might find its way to the trash
•Produces variability of responses. When a question produces no variability in responses, we are left with considerable uncertainty about why we asked the question and what we learned from the information. If a question does not produce variability in responses, it will not be possible to perform any statistical analyses on the item. For example
What do you think about this report? __
A. It's the worst report I've read
B. It's somewhere between the worst and best
C. It's the best report I've read
Since almost all responses would be choice B, very little information is learned. Design your questions so they are sensitive to differences between respondents. As another example:
Are you against drug abuse? (circle: Yes or No)
Again, there would be very little variability in responses and we'd be left wondering why we asked the question in the first place.
•Follows comfortably from the previous question. Writing a questionnaire is similar to writing anything else. Transitions between questions should be smooth. Grouping questions that are similar will make the questionnaire easier to complete, and the respondent will feel more comfortable. Questionnaires that jump from one unrelated topic to another feel disjointed and are not likely to produce high response rates.
•Does not presuppose a certain state of affairs. Among the most subtle mistakes in questionnaire design are questions that make an unwarranted assumption. An example of this type of mistake is:
Are you satisfied with your current auto insurance? (Yes or No)
This question will present a problem for someone who does not currently have auto insurance. Write your questions so they apply to everyone. This often means simply adding an additional response category.
Are you satisfied with your current auto insurance?
___ Yes
___No
__ Don't have auto insurance
One of the most common mistaken assumptions is that the respondent knows the correct answer to the question. Industry surveys often contain very specific questions that the respondent may not know the answer to. For example:
What percent of your budget do you spend on direct mail advertising?
Very few people would know the answer to this question without looking it up, and very few respondents will take the time and effort to look it up. If you ask a question similar to this, it is important to understand that the responses are rough estimates and there is a strong likelihood of error.
It is important to look at each question and decide if all respondents will be able to answer it. Be careful not to assume anything. For example, the following question assumes the respondent knows what Proposition 13 is about.
Are you in favor of Proposition 13?
___ Yes
___ No
___ Undecided
If there is any possibility that the respondent may not know the answer to your question, include a "don't know" response category
•Does not imply a desired answer. The wording of a question is extremely important. We are striving for objectivity in our surveys and, therefore, must be careful not to lead the respondent into giving the answer we would like to receive. Leading questions are usually easily spotted because they use negative phraseology. As examples:
Wouldn't you like to receive our free brochure?
Don't you think the Congress is spending too much money?
•Does not use emotionally loaded or vaguely defined words. This is one of the areas overlooked by both beginners and experienced researchers. Quantifying adjectives (e.g., most, least, majority) are frequently used in questions. It is important to understand that these adjectives mean different things to different people.
•Does not use unfamiliar words or abbreviations. Remember who your audience is and write your questionnaire for them. Do not use uncommon words or compound sentences. Write short sentences. Abbreviations are okay if you are absolutely certain that every single respondent will understand their meanings. If there is any doubt at all, do not use the abbreviation. The following question might be okay if all the respondents are accountants, but it would not be a good question for the general public.
What was your AGI last year? ______
.Is not dependent on responses to previous questions. Branching in written questionnaires should be avoided. While branching can be used as an effective probing technique in telephone and face-to-face interviews, it should not be used in written questionnaires because it sometimes confuses respondents. An example of branching is:
1. Do you currently have a life insurance policy? (Yes or No) If no, go to question 3
2. How much is your annual life insurance premium? _________
These questions could easily be rewritten as one question that applies to everyone:
1. How much did you spend last year for life insurance? ______
•Does not ask the respondent to order or rank a series of more than five items. Questions asking respondents to rank items by importance should be avoided. This becomes increasingly difficult as the number of items increases, and the answers become less reliable. This becomes especially problematic when asking respondents to assign a percentage to a series of items. In order to successfully complete this task, the respondent must mentally continue to re-adjust his answers until they total one hundred percent. Limiting the number of items to five will make it easier for the respondent to answer.
Many investigators have confirmed that slight changes in the way questions are worded can have a significant impact on how people respond. Several authors have reported that minor changes in question wording can produce more than a 25 percent difference in people's opinions.
Several investigators have looked at the effects of modifying adjectives and adverbs. Words like usually, often, sometimes, occasionally, seldom, and rarely are "commonly" used in questionnaires, although it is clear that they do not mean the same thing to all people. Some adjectives have high variability and others have low variability. The following adjectives have highly variable meanings and should be avoided in surveys: a clear mandate, most, numerous, a substantial majority, a minority of, a large proportion of, a significant number of, many, a considerable number of, and several. Other adjectives produce less variability and generally have more shared meaning. These are: lots, almost all, virtually all, nearly all, a majority of, a consensus of, a small number of, not very many of, almost none, hardly any, a couple, and a few.
The Length of a Questionnaire
As a general rule, long questionnaires get less response than short questionnaires. However, some studies have shown that the length of a questionnaire does not necessarily affect response. More important than length is question content. A subject is more likely to respond if they are involved and interested in the research topic. Questions should be meaningful and interesting to the respondent.
Anonymity and Confidentiality
An anonymous study is one in which nobody (not even the researcher) can identify who provided data. It is difficult to conduct an anonymous questionnaire through the mail because of the need to follow-up on non responders. The only way to do a follow-up is to mail another survey or reminder postcard to the entire sample. However, it is possible to guarantee confidentiality, where those conducting the study promise not to reveal the information to anyone. For the purpose of follow-up, identifying numbers on questionnaires are generally preferred to using respondents' names. It is important, however, to explain why the number is there and what it will be used for.
Some studies have shown that response rate is affected by the anonymity/confidentiality policy of a study. Others have reported that responses became more distorted when subjects felt threatened that their identities would become known. Others have found that anonymity and confidentiality issues do not affect response rates or responses.
Qualities of a Good Question
There are good and bad questions. The qualities of a good question are as follows:
•Evokes the truth. Questions must be non-threatening. When a respondent is concerned about the consequences of answering a question in a particular manner, there is a good possibility that the answer will not be truthful. Anonymous questionnaires that contain no identifying information are more likely to produce honest responses than those identifying the respondent. If your questionnaire does contain sensitive items, be sure to clearly state your policy on confidentiality
•Asks for an answer on only one dimension. The purpose of a survey is to find out information. A question that asks for a response on more than one dimension will not provide the information you are seeking. For example, a researcher investigating a new food snack asks "Do you like the texture and flavor of the snack?" If a respondent answers "no", then the researcher will not know if the respondent dislikes the texture or the flavor, or both. Another questionnaire asks, "Were you satisfied with the quality of our food and service?" Again, if the respondent answers "no", there is no way to know whether the quality of the food, service, or both were unsatisfactory. A good question asks for only one "bit" of information.
•Can accommodate all possible answers. Multiple choice items are the most popular type of survey questions because they are generally the easiest for a respondent to answer and the easiest to analyze. Asking a question that does not accommodate all possible responses can confuse and frustrate the respondent. For example, consider the question:
What brand of computer do you own? __
A. IBM PC
B. Apple
Clearly, there are many problems with this question. What if the respondent doesn't own a microcomputer? What if he owns a different brand of computer? What if he owns both an IBM PC and an Apple? There are two ways to correct this kind of problem.
The first way is to make each response a separate dichotomous item on the questionnaire. For example:
Do you own an IBM PC? (circle: Yes or No)
Do you own an Apple computer? (circle: Yes or No)
Another way to correct the problem is to add the necessary response categories and allow multiple responses. This is the preferable method because it provides more information than the previous method.
What brand of computer do you own?
(Check all that apply)
Do not own a computer
IBM PC
Apple
Other
•Has mutually exclusive options. A good question leaves no ambiguity in the mind of the respondent. There should be only one correct or appropriate choice for the respondent to make. An obvious example is:
Where did you grow up? __
A. country
B. farm
C. city
A person who grew up on a farm in the country would not know whether to select choice A or B. This question would not provide meaningful information. Worse than that, it could frustrate the respondent and the questionnaire might find its way to the trash
•Produces variability of responses. When a question produces no variability in responses, we are left with considerable uncertainty about why we asked the question and what we learned from the information. If a question does not produce variability in responses, it will not be possible to perform any statistical analyses on the item. For example
What do you think about this report? __
A. It's the worst report I've read
B. It's somewhere between the worst and best
C. It's the best report I've read
Since almost all responses would be choice B, very little information is learned. Design your questions so they are sensitive to differences between respondents. As another example:
Are you against drug abuse? (circle: Yes or No)
Again, there would be very little variability in responses and we'd be left wondering why we asked the question in the first place.
•Follows comfortably from the previous question. Writing a questionnaire is similar to writing anything else. Transitions between questions should be smooth. Grouping questions that are similar will make the questionnaire easier to complete, and the respondent will feel more comfortable. Questionnaires that jump from one unrelated topic to another feel disjointed and are not likely to produce high response rates.
•Does not presuppose a certain state of affairs. Among the most subtle mistakes in questionnaire design are questions that make an unwarranted assumption. An example of this type of mistake is:
Are you satisfied with your current auto insurance? (Yes or No)
This question will present a problem for someone who does not currently have auto insurance. Write your questions so they apply to everyone. This often means simply adding an additional response category.
Are you satisfied with your current auto insurance?
___ Yes
___No
__ Don't have auto insurance
One of the most common mistaken assumptions is that the respondent knows the correct answer to the question. Industry surveys often contain very specific questions that the respondent may not know the answer to. For example:
What percent of your budget do you spend on direct mail advertising?
Very few people would know the answer to this question without looking it up, and very few respondents will take the time and effort to look it up. If you ask a question similar to this, it is important to understand that the responses are rough estimates and there is a strong likelihood of error.
It is important to look at each question and decide if all respondents will be able to answer it. Be careful not to assume anything. For example, the following question assumes the respondent knows what Proposition 13 is about.
Are you in favor of Proposition 13?
___ Yes
___ No
___ Undecided
If there is any possibility that the respondent may not know the answer to your question, include a "don't know" response category
•Does not imply a desired answer. The wording of a question is extremely important. We are striving for objectivity in our surveys and, therefore, must be careful not to lead the respondent into giving the answer we would like to receive. Leading questions are usually easily spotted because they use negative phraseology. As examples:
Wouldn't you like to receive our free brochure?
Don't you think the Congress is spending too much money?
•Does not use emotionally loaded or vaguely defined words. This is one of the areas overlooked by both beginners and experienced researchers. Quantifying adjectives (e.g., most, least, majority) are frequently used in questions. It is important to understand that these adjectives mean different things to different people.
•Does not use unfamiliar words or abbreviations. Remember who your audience is and write your questionnaire for them. Do not use uncommon words or compound sentences. Write short sentences. Abbreviations are okay if you are absolutely certain that every single respondent will understand their meanings. If there is any doubt at all, do not use the abbreviation. The following question might be okay if all the respondents are accountants, but it would not be a good question for the general public.
What was your AGI last year? ______
.Is not dependent on responses to previous questions. Branching in written questionnaires should be avoided. While branching can be used as an effective probing technique in telephone and face-to-face interviews, it should not be used in written questionnaires because it sometimes confuses respondents. An example of branching is:
1. Do you currently have a life insurance policy? (Yes or No) If no, go to question 3
2. How much is your annual life insurance premium? _________
These questions could easily be rewritten as one question that applies to everyone:
1. How much did you spend last year for life insurance? ______
•Does not ask the respondent to order or rank a series of more than five items. Questions asking respondents to rank items by importance should be avoided. This becomes increasingly difficult as the number of items increases, and the answers become less reliable. This becomes especially problematic when asking respondents to assign a percentage to a series of items. In order to successfully complete this task, the respondent must mentally continue to re-adjust his answers until they total one hundred percent. Limiting the number of items to five will make it easier for the respondent to answer.
Statistical Surveys
What is Statistical Surveys?
Statistical surveys are used to collect quantitative information in the fields of marketing, political polling, and social science research. A survey may focus on opinions or factual information depending on its purpose, but all surveys involve administering questions to individuals. When the questions are administered by a researcher, the survey is called a structured interview or a researcher administered survey. When the questions are administered by the respondent, the survey is referred to as a questionnaire or a self-administered survey
Tactics used to Increase Response Rates
•Brevity - single page if possible
•Financial incentives
•Prepaid in advance
•Paid at completion
•Non-monetary incentives
•Commodity giveaways (pens, notepads)
•Entry into a lottery, draw or contest
•Discount coupons
•Promise of contribution to charity
•Preliminary notification
•Foot-in-the-door techniques - start with a small inconsequential request
•Personalization of the request - address specific individuals
•Follow-up requests - multiple requests
•Claimed affiliation with universities, research institutions, or charities
•Emotional appeals
•Bids for sympathy
Advantages and Disadvantages of Surveys
Advantages
•It is an efficient way of collecting information from a large number of respondents. Very large samples are possible. Statistical techniques can be used to determine validity, reliability, and statistical significance.
•Surveys are flexible in the sense that a wide range of information can be collected. They can be used to study attitudes, values, beliefs, and past behaviors.
•Because they are standardized, they are relatively free from several types of errors.
•They are relatively easy to administer.
•There is an economy in data collection due to the focus provided by standardized questions. Only questions of interest to the researcher are asked, recorded, codified, and analyzed. Time and money is not spent on tangential questions.
•Cheaper to run.
Disadvantages
•They depend on subjects’ motivation, honesty, memory, and ability to respond. Subjects may not be aware of their reasons for any given action. They may have forgotten their reasons. They may not be motivated to give accurate answers, in fact, they may be motivated to give answers that present themselves in a favorable light.
•Structured surveys, particularly those with closed ended questions, may have low validity when researching affective variables.
•Although the chosen survey individuals are often a random sample, errors due to non response may exist. That is, people who choose to respond on the survey may be different from those who do not respond, thus biasing the estimates.
•Survey question answer-choices could lead to vague data sets because at times they are relative only to a personal abstract notion concerning "strength of choice". For instance the choice "moderately agree" may mean different things to different subjects, and to anyone interpreting the data for correlation. Even yes or no answers are problematic because subjects may for instance put "no" if the choice "only once" is not available.
Characteristics of Researcher-administered Surveys
•Fewer misunderstood questions and inappropriate responses.
•Fewer incomplete responses.
•Generally higher response rates and better information on no response, but...
•Respondents may be unwilling to discuss sensitive topics with a stranger.
•Greater control over the environment that the survey is administered in.
•Additional information can be collected from respondent.
•Subject to interviewer bias (e.g. answers influenced by desire to impress interviewer).
•Generally expensive/time-consuming to run.
Characteristics of Self-administered Surveys
•Respondents are more likely to stop participating mid-way through the survey (drop-offs).
•Respondents cannot ask for clarification.
•Low response rate in some modes.
•Often respondents returning survey represent extremes of the population - skewed responses (consequence of low response rates).
•Allows shy respondents to answer sensitive questions in private.
•No interviewer intervention available for probing or explanation.
•Respondents can read the whole questionnaire before answering any questions.
•Free of interviewer bias.
Modes of Data Collection in Survey
There are six common ways to get information. These are: literature searches, talking with people, focus groups, personal interviews, telephone surveys, and mail surveys.
•A literature search involves reviewing all readily available materials. These materials can include internal company information, relevant trade publications, newspapers, magazines, annual reports, company literature, on-line data bases, and any other published materials. It is a very inexpensive method of gathering information, although it generally does not yield timely information. Literature searches take between one and eight weeks.
•Talking with people is a good way to get information during the initial stages of a research project. It can be used to gather information that is not publicly available, or that is too new to be found in the literature. Examples might include meetings with prospects, customers, suppliers, and other types of business conversations at trade shows, seminars, and association meetings. Although often valuable, the information has questionable validity because it is highly subjective and might not be representative of the population.
•A focus group is used as a preliminary research technique to explore people’s ideas and attitudes. It is often used to test new approaches (such as products or advertising), and to discover customer concerns. A group of 6 to 20 people meet in a conference-room-like setting with a trained moderator. The room usually contains a one-way mirror for viewing, including audio and video capabilities. The moderator leads the group's discussion and keeps the focus on the areas you want to explore. Focus groups can be conducted within a couple of weeks and cost between two and three thousand dollars. Their disadvantage is that the sample is small and may not be representative of the population in general.
•Personal interviews are a way to get in-depth and comprehensive information. They involve one person interviewing another person for personal or detailed information. Personal interviews are very expensive because of the one-to-one nature of the interview ($50+ per interview). Typically, an interviewer will ask questions from a written questionnaire and record the answers verbatim.
Sometimes, the questionnaire is simply a list of topics that the research wants to discuss with an industry expert. Personal interviews (because of their expense) are generally used only when subjects are not likely to respond to other survey methods.
•Telephone surveys are the fastest method of gathering information from a relatively large sample (100-400 respondents). The interviewer follows a prepared script that is essentially the same as a written questionnaire. However, unlike a mail survey, the telephone survey allows the opportunity for some opinion probing. Telephone surveys generally last less than ten minutes. Typical costs are between four and six thousand dollars and they can be completed in two to four weeks.
•Mail surveys are a cost effective method of gathering information. They are ideal for large sample sizes, or when the sample comes from a wide geographic area. They cost a little less than telephone interviews, however, they take over twice as long to complete (eight to twelve weeks). Because there is no interviewer, there is no possibility of interviewer bias. The main disadvantage is the inability to probe respondents for more detailed information.
•E-mail and internet surveys are relatively new and little is known about the effect of sampling bias in internet surveys. While it is clearly the most cost effective and fastest method of distributing a survey, the demographic profile of the internet user does not represent the general population, although this is changing. Before doing an e-mail or internet survey, carefully consider the effect that this bias might have on the results.
Questionnaire Design - General Considerations
Most problems with questionnaire analysis can be traced back to the design phase of the project. Well-defined goals are the best way to assure a good questionnaire design. When the goals of a study can be expressed in a few clear and concise sentences, the design of the questionnaire becomes considerably easier. The questionnaire is developed to directly address the goals of the study.
One of the best ways to clarify your study goals is to decide how you intend to use the information. Do this before you begin designing the study. This sounds obvious, but many researchers neglect this task. Why do research if the results will not be used?
Be sure to commit the study goals to writing. Whenever you are unsure of a question, refer to the study goals and a solution will become clear. Ask only questions that directly address the study goals. Avoid the temptation to ask questions because it would be "interesting to know".
As a general rule, with only a few exceptions, long questionnaires get less response than short questionnaires. Keep your questionnaire short. In fact, the shorter the better. Response rate is the single most important indicator of how much confidence you can place in the results. A low response rate can be devastating to a study. Therefore, you must do everything possible to maximize the response rate. One of the most effective methods of maximizing response is to shorten the questionnaire.
If your survey is over a few pages, try to eliminate questions. Many people have difficulty knowing which questions could be eliminated. For the elimination round, read each question and ask, "How am I going to use this information?" If the information will be used in a decision-making process, then keep the question... it's important. If not, throw it out.
One important way to assure a successful survey is to include other experts and relevant decision-makers in the questionnaire design process. Their suggestions will improve the questionnaire and they will subsequently have more confidence in the results.
Formulate a plan for doing the statistical analysis during the design stage of the project. Know how every question will be analyzed and be prepared to handle missing data. If you cannot specify how you intend to analyze a question or use the information, do not use it in the survey.
Make the envelope unique. We all know how important first impressions are. The same holds true for questionnaires. The respondent's first impression of the study usually comes from the envelope containing the survey. The best envelopes (i.e., the ones that make you want to see what's inside) are colored, hand-addressed and use a commemorative postage stamp. Envelopes with bulk mail permits or gummed labels are perceived as unimportant. This will generally be reflected in a lower response rate.
Provide a well-written cover letter. The respondent's next impression comes from the cover letter. The importance of the cover letter should not be underestimated. It provides your best chance to persuade the respondent to complete the survey.
Give your questionnaire a title that is short and meaningful to the respondent. A questionnaire with a title is generally perceived to be more credible than one without.
Include clear and concise instructions on how to complete the questionnaire. These must be very easy to understand, so use short sentences and basic vocabulary. Be sure to print the return address on the questionnaire itself (since questionnaires often get separated from the reply envelopes).
Begin with a few non-threatening and interesting items. If the first items are too threatening or "boring", there is little chance that the person will complete the questionnaire. People generally look at the first few questions before deciding whether or not to complete the questionnaire. Make them want to continue by putting interesting questions first.
Use simple and direct language. The questions must be clearly understood by the respondent. The wording of a question should be simple and to the point. Do not use uncommon words or long sentences. Make items as brief as possible. This will reduce misunderstandings and make the questionnaire appear easier to complete. One way to eliminate misunderstandings is to emphasize crucial words in each item by using bold, italics or underlining.
Leave adequate space for respondents to make comments. One criticism of questionnaires is their inability to retain the "flavor" of a response. Leaving space for comments will provide valuable information not captured by the response categories. Leaving white space also makes the questionnaire look easier and this increases response.
Place the most important items in the first half of the questionnaire. Respondents often send back partially completed questionnaires. By putting the most important items near the beginning, the partially completed questionnaires will still contain important information.
Hold the respondent's interest. We want the respondent to complete our questionnaire. One way to keep a questionnaire interesting is to provide variety in the type of items used. Varying the questioning format will also prevent respondents from falling into "response sets". At the same time, it is important to group items into coherent categories. All items should flow smoothly from one to the next.
If a questionnaire is more than a few pages and is held together by a staple, include some identifying data on each page (such as a respondent ID number). Pages often accidentally separate.
Provide incentives as a motivation for a properly completed questionnaire. What does the respondent get for completing your questionnaire? Altruism is rarely an effective motivator. Attaching a dollar bill to the questionnaire works well. If the information you are collecting is of interest to the respondent, offering a free summary report is also an excellent motivator. Whatever you choose, it must make the respondent want to complete the questionnaire.
Use professional production methods for the questionnaire--either desktop publishing or typesetting and key lining. Be creative. Try different colored inks and paper. The object is to make your questionnaire stand out from all the others the respondent receives.
Make it convenient. The easier it is for the respondent to complete the questionnaire the better. Always include a self-addressed postage-paid envelope. Envelopes with postage stamps get better response than business reply envelopes (although they are more expensive since you also pay for the non-respondents).
The final test of a questionnaire is to try it on representatives of the target audience. If there are problems with the questionnaire, they almost always show up here. If possible, be present while a respondent is completing the questionnaire and tell her that it is okay to ask you for clarification of any item. The questions she asks are indicative of problems in the questionnaire (i.e., the questions on the questionnaire must be without any ambiguity because there will be no chance to clarify a question when the survey is mailed).
Statistical surveys are used to collect quantitative information in the fields of marketing, political polling, and social science research. A survey may focus on opinions or factual information depending on its purpose, but all surveys involve administering questions to individuals. When the questions are administered by a researcher, the survey is called a structured interview or a researcher administered survey. When the questions are administered by the respondent, the survey is referred to as a questionnaire or a self-administered survey
Tactics used to Increase Response Rates
•Brevity - single page if possible
•Financial incentives
•Prepaid in advance
•Paid at completion
•Non-monetary incentives
•Commodity giveaways (pens, notepads)
•Entry into a lottery, draw or contest
•Discount coupons
•Promise of contribution to charity
•Preliminary notification
•Foot-in-the-door techniques - start with a small inconsequential request
•Personalization of the request - address specific individuals
•Follow-up requests - multiple requests
•Claimed affiliation with universities, research institutions, or charities
•Emotional appeals
•Bids for sympathy
Advantages and Disadvantages of Surveys
Advantages
•It is an efficient way of collecting information from a large number of respondents. Very large samples are possible. Statistical techniques can be used to determine validity, reliability, and statistical significance.
•Surveys are flexible in the sense that a wide range of information can be collected. They can be used to study attitudes, values, beliefs, and past behaviors.
•Because they are standardized, they are relatively free from several types of errors.
•They are relatively easy to administer.
•There is an economy in data collection due to the focus provided by standardized questions. Only questions of interest to the researcher are asked, recorded, codified, and analyzed. Time and money is not spent on tangential questions.
•Cheaper to run.
Disadvantages
•They depend on subjects’ motivation, honesty, memory, and ability to respond. Subjects may not be aware of their reasons for any given action. They may have forgotten their reasons. They may not be motivated to give accurate answers, in fact, they may be motivated to give answers that present themselves in a favorable light.
•Structured surveys, particularly those with closed ended questions, may have low validity when researching affective variables.
•Although the chosen survey individuals are often a random sample, errors due to non response may exist. That is, people who choose to respond on the survey may be different from those who do not respond, thus biasing the estimates.
•Survey question answer-choices could lead to vague data sets because at times they are relative only to a personal abstract notion concerning "strength of choice". For instance the choice "moderately agree" may mean different things to different subjects, and to anyone interpreting the data for correlation. Even yes or no answers are problematic because subjects may for instance put "no" if the choice "only once" is not available.
Characteristics of Researcher-administered Surveys
•Fewer misunderstood questions and inappropriate responses.
•Fewer incomplete responses.
•Generally higher response rates and better information on no response, but...
•Respondents may be unwilling to discuss sensitive topics with a stranger.
•Greater control over the environment that the survey is administered in.
•Additional information can be collected from respondent.
•Subject to interviewer bias (e.g. answers influenced by desire to impress interviewer).
•Generally expensive/time-consuming to run.
Characteristics of Self-administered Surveys
•Respondents are more likely to stop participating mid-way through the survey (drop-offs).
•Respondents cannot ask for clarification.
•Low response rate in some modes.
•Often respondents returning survey represent extremes of the population - skewed responses (consequence of low response rates).
•Allows shy respondents to answer sensitive questions in private.
•No interviewer intervention available for probing or explanation.
•Respondents can read the whole questionnaire before answering any questions.
•Free of interviewer bias.
Modes of Data Collection in Survey
There are six common ways to get information. These are: literature searches, talking with people, focus groups, personal interviews, telephone surveys, and mail surveys.
•A literature search involves reviewing all readily available materials. These materials can include internal company information, relevant trade publications, newspapers, magazines, annual reports, company literature, on-line data bases, and any other published materials. It is a very inexpensive method of gathering information, although it generally does not yield timely information. Literature searches take between one and eight weeks.
•Talking with people is a good way to get information during the initial stages of a research project. It can be used to gather information that is not publicly available, or that is too new to be found in the literature. Examples might include meetings with prospects, customers, suppliers, and other types of business conversations at trade shows, seminars, and association meetings. Although often valuable, the information has questionable validity because it is highly subjective and might not be representative of the population.
•A focus group is used as a preliminary research technique to explore people’s ideas and attitudes. It is often used to test new approaches (such as products or advertising), and to discover customer concerns. A group of 6 to 20 people meet in a conference-room-like setting with a trained moderator. The room usually contains a one-way mirror for viewing, including audio and video capabilities. The moderator leads the group's discussion and keeps the focus on the areas you want to explore. Focus groups can be conducted within a couple of weeks and cost between two and three thousand dollars. Their disadvantage is that the sample is small and may not be representative of the population in general.
•Personal interviews are a way to get in-depth and comprehensive information. They involve one person interviewing another person for personal or detailed information. Personal interviews are very expensive because of the one-to-one nature of the interview ($50+ per interview). Typically, an interviewer will ask questions from a written questionnaire and record the answers verbatim.
Sometimes, the questionnaire is simply a list of topics that the research wants to discuss with an industry expert. Personal interviews (because of their expense) are generally used only when subjects are not likely to respond to other survey methods.
•Telephone surveys are the fastest method of gathering information from a relatively large sample (100-400 respondents). The interviewer follows a prepared script that is essentially the same as a written questionnaire. However, unlike a mail survey, the telephone survey allows the opportunity for some opinion probing. Telephone surveys generally last less than ten minutes. Typical costs are between four and six thousand dollars and they can be completed in two to four weeks.
•Mail surveys are a cost effective method of gathering information. They are ideal for large sample sizes, or when the sample comes from a wide geographic area. They cost a little less than telephone interviews, however, they take over twice as long to complete (eight to twelve weeks). Because there is no interviewer, there is no possibility of interviewer bias. The main disadvantage is the inability to probe respondents for more detailed information.
•E-mail and internet surveys are relatively new and little is known about the effect of sampling bias in internet surveys. While it is clearly the most cost effective and fastest method of distributing a survey, the demographic profile of the internet user does not represent the general population, although this is changing. Before doing an e-mail or internet survey, carefully consider the effect that this bias might have on the results.
Questionnaire Design - General Considerations
Most problems with questionnaire analysis can be traced back to the design phase of the project. Well-defined goals are the best way to assure a good questionnaire design. When the goals of a study can be expressed in a few clear and concise sentences, the design of the questionnaire becomes considerably easier. The questionnaire is developed to directly address the goals of the study.
One of the best ways to clarify your study goals is to decide how you intend to use the information. Do this before you begin designing the study. This sounds obvious, but many researchers neglect this task. Why do research if the results will not be used?
Be sure to commit the study goals to writing. Whenever you are unsure of a question, refer to the study goals and a solution will become clear. Ask only questions that directly address the study goals. Avoid the temptation to ask questions because it would be "interesting to know".
As a general rule, with only a few exceptions, long questionnaires get less response than short questionnaires. Keep your questionnaire short. In fact, the shorter the better. Response rate is the single most important indicator of how much confidence you can place in the results. A low response rate can be devastating to a study. Therefore, you must do everything possible to maximize the response rate. One of the most effective methods of maximizing response is to shorten the questionnaire.
If your survey is over a few pages, try to eliminate questions. Many people have difficulty knowing which questions could be eliminated. For the elimination round, read each question and ask, "How am I going to use this information?" If the information will be used in a decision-making process, then keep the question... it's important. If not, throw it out.
One important way to assure a successful survey is to include other experts and relevant decision-makers in the questionnaire design process. Their suggestions will improve the questionnaire and they will subsequently have more confidence in the results.
Formulate a plan for doing the statistical analysis during the design stage of the project. Know how every question will be analyzed and be prepared to handle missing data. If you cannot specify how you intend to analyze a question or use the information, do not use it in the survey.
Make the envelope unique. We all know how important first impressions are. The same holds true for questionnaires. The respondent's first impression of the study usually comes from the envelope containing the survey. The best envelopes (i.e., the ones that make you want to see what's inside) are colored, hand-addressed and use a commemorative postage stamp. Envelopes with bulk mail permits or gummed labels are perceived as unimportant. This will generally be reflected in a lower response rate.
Provide a well-written cover letter. The respondent's next impression comes from the cover letter. The importance of the cover letter should not be underestimated. It provides your best chance to persuade the respondent to complete the survey.
Give your questionnaire a title that is short and meaningful to the respondent. A questionnaire with a title is generally perceived to be more credible than one without.
Include clear and concise instructions on how to complete the questionnaire. These must be very easy to understand, so use short sentences and basic vocabulary. Be sure to print the return address on the questionnaire itself (since questionnaires often get separated from the reply envelopes).
Begin with a few non-threatening and interesting items. If the first items are too threatening or "boring", there is little chance that the person will complete the questionnaire. People generally look at the first few questions before deciding whether or not to complete the questionnaire. Make them want to continue by putting interesting questions first.
Use simple and direct language. The questions must be clearly understood by the respondent. The wording of a question should be simple and to the point. Do not use uncommon words or long sentences. Make items as brief as possible. This will reduce misunderstandings and make the questionnaire appear easier to complete. One way to eliminate misunderstandings is to emphasize crucial words in each item by using bold, italics or underlining.
Leave adequate space for respondents to make comments. One criticism of questionnaires is their inability to retain the "flavor" of a response. Leaving space for comments will provide valuable information not captured by the response categories. Leaving white space also makes the questionnaire look easier and this increases response.
Place the most important items in the first half of the questionnaire. Respondents often send back partially completed questionnaires. By putting the most important items near the beginning, the partially completed questionnaires will still contain important information.
Hold the respondent's interest. We want the respondent to complete our questionnaire. One way to keep a questionnaire interesting is to provide variety in the type of items used. Varying the questioning format will also prevent respondents from falling into "response sets". At the same time, it is important to group items into coherent categories. All items should flow smoothly from one to the next.
If a questionnaire is more than a few pages and is held together by a staple, include some identifying data on each page (such as a respondent ID number). Pages often accidentally separate.
Provide incentives as a motivation for a properly completed questionnaire. What does the respondent get for completing your questionnaire? Altruism is rarely an effective motivator. Attaching a dollar bill to the questionnaire works well. If the information you are collecting is of interest to the respondent, offering a free summary report is also an excellent motivator. Whatever you choose, it must make the respondent want to complete the questionnaire.
Use professional production methods for the questionnaire--either desktop publishing or typesetting and key lining. Be creative. Try different colored inks and paper. The object is to make your questionnaire stand out from all the others the respondent receives.
Make it convenient. The easier it is for the respondent to complete the questionnaire the better. Always include a self-addressed postage-paid envelope. Envelopes with postage stamps get better response than business reply envelopes (although they are more expensive since you also pay for the non-respondents).
The final test of a questionnaire is to try it on representatives of the target audience. If there are problems with the questionnaire, they almost always show up here. If possible, be present while a respondent is completing the questionnaire and tell her that it is okay to ask you for clarification of any item. The questions she asks are indicative of problems in the questionnaire (i.e., the questions on the questionnaire must be without any ambiguity because there will be no chance to clarify a question when the survey is mailed).
Friday, October 16, 2009
Quantitative Research
What is Quantitative Research?
Quantitative market research studies are designed to assess, predict, and estimate buyer attitudes and behaviors, used for market sizing, market segmentation, and uncovering "drivers" for brand and product preference
Quantitative research is about measuring a market and quantifying that measurement with data. Most often the data required relates to market size, market share, penetration, installed base and market growth rates.
However, quantitative research can also be used to measure customer attitudes, satisfaction, commitment and a range of other useful market data that can tracked over time.
Quantitative research can also be used to measure customer awareness and attitudes to different manufacturers and to understand overall customer behaviour in a market by taking a statistical sample of customers to understand the market as a whole. Such techniques are extremely powerful when combined with techniques such segmentation analysis and mean that key audiences can be targeted and monitored over time to ensure the optimal use of the marketing budget.
At the heart of all quantitative research is the statistical sample. Great care has to be taken in selecting the sample and also in the design of the sample questionnaire and the quality of the analysis of data collected.
Market research involves the collection of data to obtain insight and knowledge into the needs and wants of customers and the structure and dynamics of a market. In nearly all cases, it would be very costly and time-consuming to collect data from the entire population of a market. Accordingly, in market research, extensive use is made of sampling from which, through careful design and analysis, Marketers can draw information about the market.
Quantitative market research is numerically oriented, requires significant attention to the measurement of market phenomena and often involves statistical analysis. For example, a bank might ask its customers to rate its overall service as excellent, good, poor or very poor.
This will provide quantitative information that can be analysed statistically. The main rule with quantitative market research is that every respondent is asked the same series of questions. The approach is very structured and normally involves large numbers of interviews/questionnaires.
Perhaps the most common quantitative technique is the ‘market research survey’. These are basically projects that involve the collection of data from multiple cases – such as consumers or a set of products. Quantitative market research surveys can be conducted by using post (self-completion), face-to-face (in-street or in-home), telephone, email or web techniques. The questionnaire is one of the more common tools for collecting data from a survey, but it is only one of a wide ranging set of data collection aids.
Types of Quantitative Researches
•Descriptive: Descriptive research involves collecting data in order to test hypotheses or answer questions concerning the current status of the subjects of the study. It determines and reports the way things are.
•Correlational: Correlational research attempts to determine whether and to what degree a relationship exists between two or more quantifiable variables. However, it never establishes a cause-effect relationship. The relationship is expressed by correlation coefficient, which is a number between .00 and 1.00.
•Cause-comparative: Causal-comparative research: establishes the cause-effect relationship, compares the relationship, but the cause is not manipulated, such as "gender."
•Experimental: Experimental research establishes the cause-effect relationship and does the comparison, but the cause is manipulated. The cause, independent variable makes the difference. The effect, dependent variable is dependent on the independent variable.
Before Conducting a Quantitative Research
•Research Plan: Research plan must be completed before a study is begun. Why?
1.The plan makes a research to think;
2.A written plan facilitates evaluation of the proposed study;
3.The plan provides a guide for conducting the study.
Components of a Research Plan :
1.Introduction: It includes a statement of the problem, a review of related literature, and a statement of the hypothesis.
2.Method: This part includes subjects, instruments-- materials if appropriate, design procedure.
3.Data analysis: A description of the statistical technique or techniques that will be sued to analyze study data.
4.Time schedule: The time schedule is equally important for both beginning researchers working on the thesis or dissertation and for experienced researchers working under the deadlines of a research grant or contract. It basically includes a listing of major activities or phases of the proposed study and a corresponding expected completion time for each activity.
5.Budget: It should list all tentative expenses specifically and submitted to funding agency. It includes such items as personnel, clerical assistance, travel and postage and other expenses, equipment, and fringe benefits etc.
•Ethical Consideration:
THREE ethical considerations are:
1)The subjects should not be harmed in any way (physically or mentally) in the name of science. If an experiment involves any risk to subjects, they should be completely informed concerning the nature of the risk and the permission for participation in the experiment should be acquired in writing from the subjects themselves, or from persons legally responsible for the subjects if they are not of age. If school children are involved, it is a good idea to inform parents before the study is conducted if possible.
2)Subject’s privacy should be strictly confidential. Individual scores should never be reported, or made public.
3)Ethical principle in the conduct of research with human participants is the most definitive source of ethical guidelines for researcher. It is prepared and published by the American Psychological Association (APA). “.... with respect and concern for the dignity and welfare of the people who participate and with cognizance of federal and state regulations and professional standards governing the conduct of research with human participants.” That is “to respect and concern for the dignity and welfare of the people who participate.”
Basic Concepts of Quantitative Research
•Introduction
a.Defining a problem
b.Literature review
c.Hypotheses
•Method
a.Population and subjects
b.Instruments
c.Design and procedures
•Results
Data and statistics
1.Types of measurement scales
2.Descriptive statistics
1.Types of descriptive statistics
2.Calculation for interval data
Inferential statistics
1.Level of significance
2.Tests of significance
(a)Z test for independent variables
(b)Z test for dependent variables
(c)ANOVA
•Discussion
i)Interpretation of results
ii)Generalization
iii)Discussion of implications
•Conclusion and recommendation
i)Based on practical significance to draw conclusion and make suggestions.
Types of Hypothesis Tests use in Quantitative Research
This includes:
•Parametric tests of a single sample:
1.T test
2.Z test
•Parametric tests of two independent samples:
1.Two-group T test
2.Z test
•Parametric tests of paired samples:
1.Paired T test
•Nominal/ordinal level test of a single sample:
2.Chi-square
3.Kolmogorov-Smirnov one sample test
4.Runs test
5.Binomial test
•Nominal/ordinal level test of two independent samples:
1.Chi-square
2.Mann-Whitney U
3.Median
4.Kolmogorov-Smirnov two sample test
•Nominal/ordinal level test for paired samples:
1.Wilcoxon test
2.McNemar test
Point to remember:
•If a Variable (e.g. preference of the respondences on color of a product) is interval/ ratio scaled and meet some statistical assumption (e.g. Normality), then it is eligible for Parametric test.
•If a Variable (e.g. gender or rank order of few products on their certain attributes) is Nominal/ Ordinal scaled and/ or does not meet some statistical assumption (e.g. Normality), then it is not eligible for Parametric test. In this situation we have to use Non-parametric test.
We should use non-parametric test only if sample/ variable is not eligible for parametric test. Remember that, the non-parametric test is mostly used and misused technique in the world.
Inferential Techniques
Inferential techniques involve generalizing from a sample to the whole population. It also involves testing a hypothesis. A hypothesis must be stated in mathematical/statistical terms that make it possible to calculate the probability of possible samples assuming the hypothesis is correct. Then a test statistic must be chosen that will summarize the information in the sample that is relevant to the hypothesis. A null hypothesis is a hypothesis that is presumed true until a hypothesis test indicates otherwise. Typically it is a statement about parameter that is a property of a population. The parameter is often a mean or a standard deviation.
Not unusually, such a hypothesis states that the parameters, or mathematical characteristics, of two or more populations are identical. For example, if we want to compare the test scores of two random samples of men and women, the null hypothesis would be that the mean score in the male population from which the first sample was drawn, was the same as the mean score in the female population from which the second sample was drawn:
H0:μ1 = μ2
Where:
H0 = the null hypothesis
μ1 = the mean of population 1, and
μ2 = the mean of population 2.
The equality operator makes this a two-tailed test. The alternative hypothesis can be either greater than or less than the null hypothesis. In a one-tailed test, the operator is an inequality, and the alternative hypothesis has directionality:
H0:μ1 = or < μ2
These are sometimes called a hypothesis of significant difference because you are testing the difference between two groups with respect to one variable.
Alternatively, the null hypothesis can postulate that the two samples are drawn from the same population:
H0:μ1 − μ2 = 0
A hypothesis of association is where there is one population, but two traits being measured. It is a test of association of two traits within one group.
The distribution of the test statistic is used to calculate the probability sets of possible values (usually an interval or union of intervals). Among all the sets of possible values, we must choose one that we think represents the most extreme evidence against the hypothesis. That is called the critical region of the test statistic. The probability of the test statistic falling in the critical region when the hypothesis is correct is called the alpha value of the test. After the data is available, the test statistic is calculated and we determine whether it is inside the critical region. If the test statistic is inside the critical region, then our conclusion is either the hypothesis is incorrect, or an event of probability less than or equal to alpha has occurred. If the test statistic is outside the critical region, the conclusion is that there is not enough evidence to reject the hypothesis.
The significance level of a test is the maximum probability of accidentally rejecting a true null hypothesis (a decision known as a Type I error).For example, one may choose a significance level of, say, 5%, and calculate a critical value of a statistic (such as the mean) so that the probability of it exceeding that value, given the truth of the null hypothesis, would be 5%. If the actual, calculated statistic value exceeds the critical value, then it is significant "at the 5% level".
Types of Errors
Random Sampling Errors:
•Sample too small
•Sample not representative
•Inappropriate sampling method used
•Random errors
Research design Errors:
•Bias introduced
•Measurement error
•Data analysis error
•Sampling frame error
•Population definition error
•Scaling error
•Question construction error
Interviewer Errors:
•Recording errors
•Cheating errors
•Questioning errors
•Respondent selection error
Respondent Errors:
•Non-response error
•Inability error
•Falsification error
Hypothesis Errors:
•Type I error (also called alpha error)
i)The study results lead to the rejection of the null hypothesis even though it is actually true
•Type II error (also called beta error)
1.The study results lead to the acceptance (non-rejection) of the null hypothesis even though it is actually false
Quantitative market research studies are designed to assess, predict, and estimate buyer attitudes and behaviors, used for market sizing, market segmentation, and uncovering "drivers" for brand and product preference
Quantitative research is about measuring a market and quantifying that measurement with data. Most often the data required relates to market size, market share, penetration, installed base and market growth rates.
However, quantitative research can also be used to measure customer attitudes, satisfaction, commitment and a range of other useful market data that can tracked over time.
Quantitative research can also be used to measure customer awareness and attitudes to different manufacturers and to understand overall customer behaviour in a market by taking a statistical sample of customers to understand the market as a whole. Such techniques are extremely powerful when combined with techniques such segmentation analysis and mean that key audiences can be targeted and monitored over time to ensure the optimal use of the marketing budget.
At the heart of all quantitative research is the statistical sample. Great care has to be taken in selecting the sample and also in the design of the sample questionnaire and the quality of the analysis of data collected.
Market research involves the collection of data to obtain insight and knowledge into the needs and wants of customers and the structure and dynamics of a market. In nearly all cases, it would be very costly and time-consuming to collect data from the entire population of a market. Accordingly, in market research, extensive use is made of sampling from which, through careful design and analysis, Marketers can draw information about the market.
Quantitative market research is numerically oriented, requires significant attention to the measurement of market phenomena and often involves statistical analysis. For example, a bank might ask its customers to rate its overall service as excellent, good, poor or very poor.
This will provide quantitative information that can be analysed statistically. The main rule with quantitative market research is that every respondent is asked the same series of questions. The approach is very structured and normally involves large numbers of interviews/questionnaires.
Perhaps the most common quantitative technique is the ‘market research survey’. These are basically projects that involve the collection of data from multiple cases – such as consumers or a set of products. Quantitative market research surveys can be conducted by using post (self-completion), face-to-face (in-street or in-home), telephone, email or web techniques. The questionnaire is one of the more common tools for collecting data from a survey, but it is only one of a wide ranging set of data collection aids.
Types of Quantitative Researches
•Descriptive: Descriptive research involves collecting data in order to test hypotheses or answer questions concerning the current status of the subjects of the study. It determines and reports the way things are.
•Correlational: Correlational research attempts to determine whether and to what degree a relationship exists between two or more quantifiable variables. However, it never establishes a cause-effect relationship. The relationship is expressed by correlation coefficient, which is a number between .00 and 1.00.
•Cause-comparative: Causal-comparative research: establishes the cause-effect relationship, compares the relationship, but the cause is not manipulated, such as "gender."
•Experimental: Experimental research establishes the cause-effect relationship and does the comparison, but the cause is manipulated. The cause, independent variable makes the difference. The effect, dependent variable is dependent on the independent variable.
Before Conducting a Quantitative Research
•Research Plan: Research plan must be completed before a study is begun. Why?
1.The plan makes a research to think;
2.A written plan facilitates evaluation of the proposed study;
3.The plan provides a guide for conducting the study.
Components of a Research Plan :
1.Introduction: It includes a statement of the problem, a review of related literature, and a statement of the hypothesis.
2.Method: This part includes subjects, instruments-- materials if appropriate, design procedure.
3.Data analysis: A description of the statistical technique or techniques that will be sued to analyze study data.
4.Time schedule: The time schedule is equally important for both beginning researchers working on the thesis or dissertation and for experienced researchers working under the deadlines of a research grant or contract. It basically includes a listing of major activities or phases of the proposed study and a corresponding expected completion time for each activity.
5.Budget: It should list all tentative expenses specifically and submitted to funding agency. It includes such items as personnel, clerical assistance, travel and postage and other expenses, equipment, and fringe benefits etc.
•Ethical Consideration:
THREE ethical considerations are:
1)The subjects should not be harmed in any way (physically or mentally) in the name of science. If an experiment involves any risk to subjects, they should be completely informed concerning the nature of the risk and the permission for participation in the experiment should be acquired in writing from the subjects themselves, or from persons legally responsible for the subjects if they are not of age. If school children are involved, it is a good idea to inform parents before the study is conducted if possible.
2)Subject’s privacy should be strictly confidential. Individual scores should never be reported, or made public.
3)Ethical principle in the conduct of research with human participants is the most definitive source of ethical guidelines for researcher. It is prepared and published by the American Psychological Association (APA). “.... with respect and concern for the dignity and welfare of the people who participate and with cognizance of federal and state regulations and professional standards governing the conduct of research with human participants.” That is “to respect and concern for the dignity and welfare of the people who participate.”
Basic Concepts of Quantitative Research
•Introduction
a.Defining a problem
b.Literature review
c.Hypotheses
•Method
a.Population and subjects
b.Instruments
c.Design and procedures
•Results
Data and statistics
1.Types of measurement scales
2.Descriptive statistics
1.Types of descriptive statistics
2.Calculation for interval data
Inferential statistics
1.Level of significance
2.Tests of significance
(a)Z test for independent variables
(b)Z test for dependent variables
(c)ANOVA
•Discussion
i)Interpretation of results
ii)Generalization
iii)Discussion of implications
•Conclusion and recommendation
i)Based on practical significance to draw conclusion and make suggestions.
Types of Hypothesis Tests use in Quantitative Research
This includes:
•Parametric tests of a single sample:
1.T test
2.Z test
•Parametric tests of two independent samples:
1.Two-group T test
2.Z test
•Parametric tests of paired samples:
1.Paired T test
•Nominal/ordinal level test of a single sample:
2.Chi-square
3.Kolmogorov-Smirnov one sample test
4.Runs test
5.Binomial test
•Nominal/ordinal level test of two independent samples:
1.Chi-square
2.Mann-Whitney U
3.Median
4.Kolmogorov-Smirnov two sample test
•Nominal/ordinal level test for paired samples:
1.Wilcoxon test
2.McNemar test
Point to remember:
•If a Variable (e.g. preference of the respondences on color of a product) is interval/ ratio scaled and meet some statistical assumption (e.g. Normality), then it is eligible for Parametric test.
•If a Variable (e.g. gender or rank order of few products on their certain attributes) is Nominal/ Ordinal scaled and/ or does not meet some statistical assumption (e.g. Normality), then it is not eligible for Parametric test. In this situation we have to use Non-parametric test.
We should use non-parametric test only if sample/ variable is not eligible for parametric test. Remember that, the non-parametric test is mostly used and misused technique in the world.
Inferential Techniques
Inferential techniques involve generalizing from a sample to the whole population. It also involves testing a hypothesis. A hypothesis must be stated in mathematical/statistical terms that make it possible to calculate the probability of possible samples assuming the hypothesis is correct. Then a test statistic must be chosen that will summarize the information in the sample that is relevant to the hypothesis. A null hypothesis is a hypothesis that is presumed true until a hypothesis test indicates otherwise. Typically it is a statement about parameter that is a property of a population. The parameter is often a mean or a standard deviation.
Not unusually, such a hypothesis states that the parameters, or mathematical characteristics, of two or more populations are identical. For example, if we want to compare the test scores of two random samples of men and women, the null hypothesis would be that the mean score in the male population from which the first sample was drawn, was the same as the mean score in the female population from which the second sample was drawn:
H0:μ1 = μ2
Where:
H0 = the null hypothesis
μ1 = the mean of population 1, and
μ2 = the mean of population 2.
The equality operator makes this a two-tailed test. The alternative hypothesis can be either greater than or less than the null hypothesis. In a one-tailed test, the operator is an inequality, and the alternative hypothesis has directionality:
H0:μ1 = or < μ2
These are sometimes called a hypothesis of significant difference because you are testing the difference between two groups with respect to one variable.
Alternatively, the null hypothesis can postulate that the two samples are drawn from the same population:
H0:μ1 − μ2 = 0
A hypothesis of association is where there is one population, but two traits being measured. It is a test of association of two traits within one group.
The distribution of the test statistic is used to calculate the probability sets of possible values (usually an interval or union of intervals). Among all the sets of possible values, we must choose one that we think represents the most extreme evidence against the hypothesis. That is called the critical region of the test statistic. The probability of the test statistic falling in the critical region when the hypothesis is correct is called the alpha value of the test. After the data is available, the test statistic is calculated and we determine whether it is inside the critical region. If the test statistic is inside the critical region, then our conclusion is either the hypothesis is incorrect, or an event of probability less than or equal to alpha has occurred. If the test statistic is outside the critical region, the conclusion is that there is not enough evidence to reject the hypothesis.
The significance level of a test is the maximum probability of accidentally rejecting a true null hypothesis (a decision known as a Type I error).For example, one may choose a significance level of, say, 5%, and calculate a critical value of a statistic (such as the mean) so that the probability of it exceeding that value, given the truth of the null hypothesis, would be 5%. If the actual, calculated statistic value exceeds the critical value, then it is significant "at the 5% level".
Types of Errors
Random Sampling Errors:
•Sample too small
•Sample not representative
•Inappropriate sampling method used
•Random errors
Research design Errors:
•Bias introduced
•Measurement error
•Data analysis error
•Sampling frame error
•Population definition error
•Scaling error
•Question construction error
Interviewer Errors:
•Recording errors
•Cheating errors
•Questioning errors
•Respondent selection error
Respondent Errors:
•Non-response error
•Inability error
•Falsification error
Hypothesis Errors:
•Type I error (also called alpha error)
i)The study results lead to the rejection of the null hypothesis even though it is actually true
•Type II error (also called beta error)
1.The study results lead to the acceptance (non-rejection) of the null hypothesis even though it is actually false
Subscribe to:
Posts (Atom)