Have you opened a survey and noticed that some questions seem irrelevant or confusing? The survey description and questions don’t match.

Face validity is the amount of time a survey or question seems to measure what it’s supposed to measure. It’s not a scientific or objective way of measuring validity; it’s subjective and based on intuition.

Surveys with low face validity can negatively impact the number of people who respond, the accuracy of the data, and the reliability of the results. If your survey has low face validity, respondents may be confused, annoyed, or find your questions suspicious, and they may abandon the survey.

Let’s explore the different types of face validity in surveys, their importance, and how you can improve them in your surveys.

Definition and Explanation of Face Validity

Face validity is a subjective judgment about whether the questions in a survey seem to make sense and are relevant to the topic at hand.

For example, if you want to measure intelligence, you might use a test that asks questions about logic, reasoning, and problem-solving. This test would have high face validity because it looks like it measures intelligence.

However, face validity is not an accurate or reliable measure of validity. It relies on subjective perceptions and beliefs rather than empirical data and does not ensure that a test or measure accurately measures the construct.

Face validity is often used in surveys and questionnaires to ensure that respondents understand the questions and their context.

For example, if you want to measure customer satisfaction with a product, you could use a survey that asks questions like “How satisfied are you with the product?” This question has a high face validity because it appears to measure customer satisfaction.

Types of Face Validity in Surveys

• Logical Face Validity

This shows if the questions in a survey make logical sense to the topic being studied.

For example, in customer satisfaction, you’d likely ask, “How likely are they to recommend your product to others?” This question has a logical basis because it is directly related to the notion of satisfaction.

• Psychological Face Validity

It evaluates how well a survey question appears to measure what it is supposed to measure from the respondent’s psychological perspective. It looks at how well the questions seem relevant and relevant to the respondent.

For example, if you want to assess your student’s level of anxiety before an exam, you could ask them questions like “How confident are you in your preparation?” These questions have psychological face validity because they reflect how respondents feel about the concept of anxiety.

Related – Reliability vs Validity in Research: Types & Examples 

Factors that Affect Face Validity

• Types of Questions in Surveys

Different types of questions can elicit different types of responses and evaluate various components of a concept. For example, if you want to gauge customer satisfaction, you can ask questions such as:

Likert scale questions: These are questions that ask respondents to rate their agreement or disagreement with a statement on a scale of 1 to 5 or higher. For example, “I am satisfied with the quality of the product.”

Semantic differential questions: These are questions that ask people to rate how they feel about a product, service, or concept based on two opposite adjectives. For example, the product could be described as “Good” or “Bad”. It could also be “Cheap” or “Expensive.”

Open-ended questions: These are questions that allow respondents to write their answers without any predefined options. “For example, why did you prefer product M over product N?”

The type of question you use should be dependent on the type of construct being measured and the level of detail you need. For example, if you want to know how satisfied customers are with their service, you could use Likert scale questions about quality, pricing, and more.

However, if you want to look at customer satisfaction as a concept, such as satisfaction with product features, you can ask semantic differential questions about the product’s characteristics. However, if you want to learn more about why customers are happy or unhappy, you can ask open-ended questions that allow people to explicitly share their opinion.

• Language and Terminology Used in Questions

The language and terminology should be clear, and easy to understand for the respondents.  Avoid using technical jargon or language that may be confusing or frustrating to respondents, reducing the face validity of the survey.

The language and terminology should also be consistent and unbiased throughout the survey. For example, if you want to measure customer satisfaction, you should avoid using:

• Ambiguous or Vague statements– they can cause confusion or misinterpretation. Instead of asking, “How frequently do you use this product?” you might want to ask, “How many times per week do you use the product”
• Leading or loaded terms that may influence or distort respondents’ answers. For example, asking “How happy are you with the product?”, instead of “How satisfied are you with the product?”
• Technical terms or jargon that are difficult to understand or make people feel anxious. Instead of asking, “What is the ergonomic design of this product?”  ask, “How easy is it to use?”

• Respondent Characteristics and Demographics

This is the respondent background information and personality traits such as age and gender, education and income, culture, and more. These factors affect how people feel and how they respond to the test or questionnaire. For example, if you want to know how satisfied your customers are, you can design your surveys based on:

• Respondent Age: Different age groups may have different expectations and preferences for a product or service. For example, younger customers may value uniqueness and innovation more than older customers, who may value reliability and durability more.
• Education Level– Different education levels may influence how respondents understand and respond to your questions. Higher-educated customers, for example, may be more critical and analytical than lower-educated customers, who may be more emotional and impulsive.

Methods to Improve Face Validity

• Use of Clear and Concise Language

Avoid using jargon or complicated language that may cause people to become confused or misunderstand the question Stick to clear, simple language that your audience will understand.

For example, if you are measuring self-esteem, you should use simple statements like “I feel good about myself” rather than “I have a positive self-concept”.

• Pretesting and Pilot Testing

This is the process of testing your survey questions on a small group of people before distributing them to a larger target audience. This allows you to identify any gaps or issues with the questions and modify them.

Pilot testing is when you test the questions on a small group of participants to see if they are relevant and effective. This allows you to adjust the questions and improve their face validity.

• Reviewing Questions for Cultural Biases and Sensitivity

The test or measure questions and options should be culturally appropriate and respectful of the participants’ cultural backgrounds and values. The questions shouldn’t make assumptions, stereotype, or use language that is offensive to the participants.

Applications of Face Validity in Survey Research

• Evaluation of Questionnaire Design

You can use face validity to ensure that your questions are specific, relevant, appropriate, and comprehensive enough to help you measure your research objectives. This way, you can identify any potential problems with the questionnaire and ensure that it is measuring what it should.

For example, if you want to assess customer satisfaction with a new product, you can use face validity to ensure that your questions cover all aspects of the product, such as quality, price, features, and usability.

• Ensuring Relevance and Clarity of Questions

Another way to use face validity is to ensure your questions are specific and easy to understand for your audience. It can help you avoid asking questions that are too ambiguous, vague, complex, or biased.

For example, when measuring employee engagement face validity helps you avoid generic questions such as “How do you like your job?” or leading questions such as “Do you think our company is a great place to work?”

• Assessing the Acceptability of Survey Measures

You can use face validity to determine whether your respondents can answer your questions honestly and accurately. For example, if you want to assess political opinions, you can use face validity to avoid asking questions that are too sensitive, personal, or controversial for your respondents.

Limitations of Face Validity in Surveys

• Limitations in the Assessment of Actual Behavior

Face validity does not necessarily measure the respondent’s actual behavior. For example, a survey asking respondents how often participants exercise may have high face validity but may not accurately reflect their actual exercise patterns.

People may exaggerate or understate how often or how hard they exercise, or they could overlook or leave out details. So, face validity does not guarantee that the survey will reflect how people actually behave.

• Social Desirability Bias

This is a tendency for respondents to answer questions in a way that makes them look good or conform to social norms. So, face validity does not ensure that the survey reflects the honest and authentic opinions of the respondents.

For example, a survey asking people what they think about controversial issues may have a low face value because people may be afraid to say what they really think for fear of being criticized or judged.

• The Potential for Respondent Acquiescence or Response Bias

This is when respondents agree with a statement or question regardless of what it says or what it means. Likert scale surveys, for example, have a high face value, but they can also lead to people having a bias toward agreeing or disagreeing with everything.

Conclusion

Face validity is a simple and quick way to evaluate your survey measure at a glance. It boosts your response rate and enhances the quality of your data.

However, without employing best practices for face validity, there is no guarantee of the overall validity and reliability of your survey results. Incorporating face validity best practices help ensure that survey questions are appropriate, relevant, and reliable, resulting in more accurate and reliable data.