Influencer Marketing
Expand your reach and engage with your target audience using this trending technique that blends celebrity endorsements with social media marketing.
Leading High Performing Remote Teams
How can leaders ensure that performance remains high in remote or hybrid-work environments?
Design Thinking
Learn the 5 phases of this problem-solving methodology and switch from technology-centered to user-centered thinking.
Gantt Chart
Invented in the early 20th century, the Gantt Chart is one of the building blocks of modern project management. In this online course, you'll learn how this tool can be used effectively to monitor progress and achieve your team's goals.
Navigating Change Successfully
The working landscape is continually shifting and being disrupted, so how to employees maintain a sense of stability? Listen to CEO and president of Carl ZEISS Japan Stefan Sacre share his expertise on dealing with change in organizations and entire industries.
Halo Effect
The halo effect is often leveraged for marketing and promotion. But as a type of cognitive bias, it can also have a subconscious impact on decision-making in the workplace. Learn why and (how to overcome it) in this online course.
Anchoring and Framing
Want to increase your confidence during negotiations? Master the principles of anchoring and framing to take your negotiation skills to the next level.
ZOPA and BATNA
Understanding ZOPA and BATNA will help you become a better negotiator, create more value, and feel more confident at the table.
Content Marketing
In this course, you’ll learn how compelling blogs, videos, podcasts, and other media can reach customers and drive sales. You’ll also learn steps for creating an effective content marketing plan, and some important ways to measure its impact and success.
Content marketing is a essential digital marketing strategy for companies looking to provide relevant and useful information to support your community and attract new customers.
Get started on your content marketing journey today.
Sustainable Innovation in Times of Disruption: Choices for a Better Society
There are opportunities for progress all around us. The key is to innovate on these opportunities sustainably.
To help identify most effective path forward, you'll need to gain a global perspective to these challenges in an open discussion. How can Japan and the world take action to create a more sustainable, innovative world? Where do you fit in?
It's time to find out.
Social Media & Digital Communications: Impact on Global Public Opinion
Social and digital media have dominated the communications industry for decades. But it's no secret that social media has the power to sway public opinion, and the way in which many companies use these platforms could be seen as manipulative.
What do companies need to be aware of when utilizing social and digital media? How can these mediums be used to better communicate strategically with the world?
Discover what top media and communications experts have to say.
Blockchain
Blockchain is one of the most captivating technologies out there. Learn what it is and how to make use of its opportunities in this short online course.
Mehrabian’s Rule
The 7-38-55 Rule, developed by Albert Mehrabian, suggests that effective communication relies less on the words we choose than on our tone of our voice, appearance, and body language. Learn how to put this theory to use for better communication in business.
Pareto Principle
Your time and resources are limited. Efficiency means learning to prioritize. The Pareto principle (also called the 80-20 rule) can help you identify the best way to use your time for maximum results.
Country Analysis Framework
Overseas expansion requires careful planning. The Country Analysis Framework can help you look beyond an industry-level analysis and reframe your view based on performance, strategy, and context. Try this short course to learn how it works.
SECI Model
The SECI model illustrates how knowledge is created and shared. Learn how to put it to use for best practices, and how the Japanese concept of “ba” fits in to broaden your perspective.
Johari Window Model
The Johari Window Model is a self-awareness framework that helps you better understand . . . you. Learn how its four quadrants can help you identify gaps between how you see yourself, and how others see you.
Sunk Costs
Wondering if you should continue an investment or look for something new? Sunk costs can have a powerful psychological impact on decision-making. Learn how to recognize them to ensure rational decisions.
CAGE Distance Framework
Want to expand overseas? The CAGE distance framework can help ensure you're constructing a solid global strategy in four areas: cultural, administrative, economic, and geographic. Learn how to leverage useful differences between countries, identify potential obstacles, and achieve global business success.
Groupthink
Groupthink refers to group pressure and the perception of consensus which together lead to ill-formed decisions—or even unnecessary risks. Learn to identify the warning signs of groupthink and apply countermeasures in this online course.
Deductive and Inductive Reasoning
Solving problems with the best results means using two types of thinking: deductive and inductive reasoning. In this online course, learn to form a broad premise, make observations, and form conclusions from different perspectives.
Critical Thinking: Hypothesis-Driven Thinking
Anyone can come up with a good idea. The real challenge is putting that idea into action. In this online course, explore how to form compelling, testable hypotheses and bring ideas to life in your own organization.
Critical Thinking: Structured Reasoning
Even a few simple techniques for logical decision making and persuasion can vastly improve your skills as a leader. Explore how critical thinking can help you evaluate complex business problems, reduce bias, and devise effective solutions.
Critical Thinking: Problem-Solving
Problem-solving is a central business skill, and yet it's the one many people struggle with most. This course will show you how to apply critical thinking techniques to common business examples, avoid misunderstandings, and get at the root of any problem.
How to Dream
Join globally renowned author and Columbia Business School professor Dr. Sheena Iyengar as she explains how to approach your dreams with a new perspective. Learn to reflect on what you long to accomplish and what stands in your way.
Logical Thinking
Logical thinking is at the heart of confident, persuasive decisions. This course will equip you with a five-point approach to more becoming a more logical thinker. Learn to classify ideas and distinguish fact from opinion.
Investing & Diversity: The Changing Faces of Venture Capitalists
Is the venture capital industry embracing diversity in investors? Watch global venture capitalists from around the world discuss the state of things and what needs to be done for a more inclusive future.
Servant Leadership
There's more to leadership than driving a team to profit. In fact, there's a word for looking beyond self-interest to prioritize individual growth: servant leadership. Try this course for a quick breakdown of what that is, how it works, and how it can lead to organizational success.
Organizational Behavior and Leadership
Ever wonder what makes a great leader? Whether your role requires leadership or not, understanding organizational behavior is useful for your career. This course from GLOBIS Unlimited can set you on your way.
Leadership vs. Management
Leadership and management are different skills, but today’s leaders must have both. Try out this course from GLOBIS Unlimited to understand the difference, as well as when and why each skill is necessary for motivation, communication, and value.
Strategy: Creating Value Inside Your Company
Have you ever wondered why certain companies are more successful than others? The answer is strategy: internal processes that control costs, allocate resources, and create value. This course from GLOBIS Unlimited can give you the tools you need for that strategic edge.
Strategy: Understanding the External Environment
To plan strategy on any level, you need to understand your company's external environment. In fact, your level of understanding can impact hiring, budgeting, marketing, or nearly any other part of the business world. Want to learn how to do all that? This course from GLOBIS Unlimited is the perfect first step!
Using Japanese Values to Thrive in Global Business
Japanese companies have unique cultural, communication, and operational challenges. But they also have values that have led to remarkable longevity. Check out this seminar to hear how these values help earn trust from overseas head offices and develop employees.
Turnaround Leadership: The Differences Between Japan and the West
What's the best way for leaders to communicate a shift in corporate strategy? How do you even know when it's time for such a change? This course explains how Japan might have one answer, Western companies another.
Conflict Management
Conflicts in the workplace are inevitable. But they can lead to positive outcomes if they’re managed well. Check out this online course for a two-step process that can help you manage conflict successfully.
Evernote Founder: How Tech Startups Can Break through in Japan
Can startup models from Hollywood and Silicon Valley succeed anywhere? Phil Libin, cofounder and CEO of startup incubator All Turtles, explains how AI can solve everyday problems to bring products to market.
Women Empowerment: Lessons from Cartier
How can women overcome gender inequality and reach their leadership goals? Cartier Japan CEO June Miyachi shares her secret in this special course from GLOBIS Unlimited.
Marketing: Reaching Your Target
Every company works hard to get its products into the hands of customers. Are you doing everything you can to compete? In this course, you’ll find a winning formula to turn a product idea into real sales. Follow along through the fundamentals of the marketing mix and see how companies successfully bring products to market.
Marketing Mix
Seeing good products into the hands of customers is no easy task. The marketing mix can help. It's a collection of strategies and tactics companies utilize to get customers to purchase their products or services, and is an essential part of the overall marketing process.
The Principles of Negotiation
With the proper skills and attitude, anyone can become a successful negotiator. But first, you'll need to learn the basics to prepare for, assess, and respond to offers for the best results. GLOBIS Unlimited can help.
Negotiation: Creating Value
Want to create more shared value between yourself and your negotiation opponent? Discover how cognitive bias affects the judgment of others. Try this course from GLOBIS Unlimited to master the value of negotiation.
Finding Your Life Purpose with Ikigai
Ikigai can guide you in your quest for self-discovery. Listen to Japanese brain scientist Ken Mogi explain why and how.
Maslow’s Hierarchy of Needs
Want to leverage Maslow's Hierarchy of Needs as a leader? Try this short course to see how the theory can be applied in practical work scenarios.
Confirmation Bias
We all subconsciously collect information that reinforces our preconceptions. It's natural . . . but it does lead to a kind of flawed decision-making called confirmation bias. To become more objective and impartial, check out this course from GLOBIS Unlimited!
An Investor's Lesson to Entrepreneurs
Entrepreneurs have the power to transform societies for the better. But how do you attract investors to start or grow a business? Or to sell one? Check out this seminar for the answers to these and more, straight from a master venture capitalist!
Managerial Accounting
Managerial accounting is a powerful way to measure progress, identify problems, and meet your goals. Check out this course to learn how data-backed decisions can help you run your business.
Finance Basics: 1
For a healthy mix of quantitative planning, evaluation, and management, you need solid decision-making. And finance is the secret sauce! Get the essentials of finance in this two-part course from GLOBIS Unlimited.
Basic Accounting: Financial Analysis
Want to compare your performance vs. a competitor? Or evaluate a potential vendor? Then you'll need to conduct a financial analysis. This course will teach you how to use three financial statements and evaluate financial performance in terms of profitability, efficiency, soundness, growth, and overall strength.
Career Anchors
What drives you to be good at your job?
Career anchors are based on your values, desires, motivations, and abilities. They are the immovable parts of your professional self-image that guide you throughout your career journey.
Try this short GLOBIS Unlimited course to identify which of the eight career anchors is yours!
Digital Marketing Psychology to Transform Your Business
How does digital marketing really differ from traditional marketing? How is social media changing things really? And what's going on in Asia?
Pyramid Structure
Having the pyramid structure in your communication toolkit can not only help you approach a problem, but convince others that your solution is valid. Break away from linear thinking and test your logical thinking with this course from GLOBIS Unlimited!
Leadership with Passion through Kokorozashi
The key ingredient to success? Passion.
Finding your kokorozashi will unify your passions and skills to create positive change in society. This GLOBIS Unlimited course will help you develop the values and lifelong goals you need to become a strong, passion-driven leader.
AI First Companies – Implementation and Impact
AI is changing the way companies operate. How do you structure teams to increase efficiency?
Technovate in the Era of Industry 4.0
Is Industry 4.0 is the next step of human evolution human civilization? Dr. Jorge Calvo seems to think so. Join him to learn how the past can help you set goals for an exciting future of digital innovation.
Technovate Thinking
Business leaders of tomorrow need to harness the power of technology and innovation. That means understanding algorithms and how they drive business results. Discover opportunities to make technology work for your competitive edge.
Product Life Cycle
Every product takes a natural course through the market—there's a how, when, and why customers adopt products at different stages. Check out this course from GLOBIS Unlimited to find out how a product you use every day is part of this cycle.
Logic Tree
Logical thinking is the most valuable asset any business professional can have. That's why logic trees are such a valuable tool—they can help you identify a problem, break it down, and build it back up to a solution.
MECE Principle
Using the MECE principle can help ensure you categorize without gaps or overlaps. Check out this course from GLOBIS Unlimited for a practical demonstration of how it works!
In the first part of this series, we introduced the core psychometric concepts of validity, reliability, and error. Now, it’s time to get practical. The smallest details in an assessment’s design—the response options and the wording of the questions—have the largest impact on data quality.
This “quality assurance checklist” provides HR teams with an expert’s view on what to look for, helping you spot the design choices that introduce errors and compromise your results. The quality of your data is decided before the first employee ever clicks a response.
The guidance below will help HR teams spot design choices that lead to the kinds of error we discussed previously (method effects, inconsistent interpretation, and random responding).
A Quality Assurance Checklist: Assessing Content Design
1. What Response Options Are Used?
Research shows that the response options have the single largest impact on an assessment’s quality. Below are some of the most crucial response option characteristics that affect assessment quality.
Response Format
The format of the response options dictates the type of information that the assessment can provide and the way results can be used. Let me give an example with a commonly used response option format: forced-choice.
- Forced-choice format describes a format in which respondents reply by indicating which of two options they rate higher (See Figure A: ).
- Forced-choice format is valid for situations in which you want to understand what is most important to a single employee—do they value complex work more than work that impacts society? But they cannot be used to compare one employee to another, to a team, or to benchmark data. A #1 ranking by Employee A and #4 by Employee B does not necessarily mean A values complex work more than B—B could in fact value it more.
- Additionally, the data produced by forced-choice assessments cannot be used with analytic methods commonly used by HR teams (e.g., correlation, regression), or the analytic methods commonly used to create assessments (e.g., factor analysis). If factor analysis was inadvertently used to create a forced-choice assessment, the results cannot be trusted.
Probably the most widely used response option format is the Likert scale (pronounced “lick-urt,” see Figure B: ). While not free from issues, Likert scales can assess a wide variety of variables, have been extensively studied, produce intuitive information, and are suitable for most use cases and analytics common in HR contexts.
Unipolar vs. Bipolar Rating Scales
“Strongly Disagree – Strongly Agree” is a bipolar scale, whereas “Do Not Agree at All – Strongly Agree” is a unipolar scale. The choice of unipolar vs. bipolar is hotly debated in the literature, but for most HR use cases, a unipolar rating scale will produce the most accurate results.
Number of Options
It is recommended to use Likert rating scales with 5 or 7 response options. More options overwhelm respondents, leading to random responding, while fewer options may limit nuance.
Labels for Response Options
Response option labels should be chosen so that they produce the following two qualities:
- Equal Spacing Labels should create equal spacing between options—for example, the difference from ‘slightly’ $\rightarrow$ ‘moderately’ should match ‘very’ $\rightarrow$ ‘extremely.’ If options are not equally spaced (e.g., because they are too similar: “a little bit” $\rightarrow$ “slightly”), respondents may choose at random. And unequally spaced data causes most statistical analyses to produce distorted or misleading results.
- Nicely Spread Response Distribution Labels should result in a nice spread of answers across the rating scale (see Figure B). The converse situations are when everyone picks responses only at the extremes (skew), or when nearly everyone selects the same response (kurtosis). A nice spread makes results more useful–for example, no meaningful differences are revealed if every team scores a “3.” It is also required for most common statistical analyses to return accurate results.
Assessments are more likely to meet the above two criteria when they have the following characteristics:
- Equally Spaced Adjectives: Research shows the following labels are equally spaced:
- Intensity: (1) not at all, (2) slightly (or a little), (3) somewhat (or moderately), (4) very, (5) extremely (see Figures A-D: )
- Frequencies: (1) never, (2) rarely, (3) sometimes, (4) very often, (5) extremely often (or always) (when 100% of the time is relevant, “always” should be used)
- Quantity: (1) none, (2) a little (or a little bit), (3) some (or somewhat), (4) quite a bit, (5) a great deal.
- Encourage Variation: Labels encourage variation in responses. For example, to determine how employees feel when reporting a serious mistake, a “not at all – extremely comfortable” scale will produce skewed results because everyone feels at least a little uncomfortable. Whereas “not at all – extremely afraid” should produce more variation.
- Consider Culture: Labels should take cultural considerations into account. For example, research suggests that Japanese respondents are less likely to choose extreme-sounding options. Thus, in the Japanese organization, it is better to use a rating scale whose highest rating is “very” (とても) instead of “extremely” (非常に).
Additionally, rating scales with the following characteristics are less tiring for respondents to use, and reduce chances of inconsistent interpretation:
- Label all the response options, rather than just the highest and lowest (see Figure B).
- Reference the target construct in the labels (see Figure C: ).
- A “neither agree nor disagree” or “don’t know” label should NOT be used for the middle option on an odd-numbered bipolar rating scale.
Finally, HR teams should hesitate to use an assessment that allows those without an opinion to skip questions, such as with an “I don’t know” button next to the rating scale. Research shows that it has no benefits for data quality and results in less data being collected. (This is different from filter questions–e.g., letting men skip questions aimed at women–which are strongly encouraged).
2. How Are the Instructions and Questions Written?
Questions should possess the following characteristics to ensure consistent interpretation. This is especially important in global organizations where many respondents will be taking assessments in their second language:
- Easy-to-Understand Language: Simple (short, simple syntax, using concrete, specific and unambiguous language), free of idioms and double negatives (“I do not dislike the new initiative”), and written at around a sixth-grade reading level.
- Not Double-barreled: “Double-barreled” refers to two questions in one. For example, “I am motivated when my manager is friendly and caring”—if an employee feels motivated by a caring manager, but not a friendly manager, how should they answer?
- Not Negated: Negatively worded questions—such as those including the word “not” (e.g., “I do not feel stressed at work”) or using the negative form of a word (“I am dissatisfied by my job”)—are more likely to be misread or misinterpreted. Research shows respondents are also more hesitant to endorse negative statements even if they are true (details below).
The instructions should also be sufficiently clear to ensure consistent interpretation. As an example of poorly written instructions, consider Figure E: . Is it asking about emotional frequency, intensity, or something else? Is it asking only about emotions caused by the job, or also about emotions caused by personal issues that continue at work?
3. How Do All the Questions Relate to Each Other?
An individual question is not an island—each question’s quality is affected by the presence of other questions.
- Ordering of Questions: The ordering of questions can affect how respondents answer. For example, when a question about job quality is followed by a question about life quality, research shows respondents are likely to assume they should exclude the influence of their job on their rating of life quality. Thus, HR teams should hesitate to change question ordering, and before purchasing an assessment try to read through all questions in order.
- Positively and Negatively Worded Questions: Quality is degraded in a variety of ways when assessments include both positively (“My manager cares about me”) and negatively (“My manager dislikes me”) valenced questions. Instead, assessments should use all positively or negatively worded questions, or at least maintain an equal balance of both.
4. Are the Questions Capable of Producing the Information HR Needs?
Once questions and rating scales are determined to be of sufficient quality, the next step is to consider if they are going to be effective for the HR team’s use case:
- Capable of Capturing Change: HR often wants to use assessments to determine if a specific construct changes in response to interventions. In this case, questions should:
- Ask about constructs that can fluctuate and respond to HR intervention (e.g., “I feel satisfied working in the car industry” can’t capture HR-driven changes in job satisfaction).
- Reference a clear time frame (e.g., start every question with, “Over the past two weeks…”).
- Employees Willing to Answer Honestly: One goal is to write questions that employees will be willing to answer honestly. Often this means writing indirect and/or positively valenced questions. For example:
- “I work hard: True or False” $\rightarrow$ “I often feel highly motivated while I do my work.”
- “I hate my manager” $\rightarrow$ “My relationships with direct management are high quality”
- Provide Solution Guidance: HR typically wants to use assessments to identify and then solve problems. An assessment is better equipped to identify solutions when it has the following characteristics:
- Tied into theory. HR teams are encouraged to check if vendors can provide a clear and rational explanation of how their assessment is tied into theory, and how it can be used to identify solutions.
- Taps into Causal Factors. For example, Self-determination Theory shows autonomy increases employee motivation. Instead of “I am motivated: True/False”, a good assessment might ask, “I feel pressured to do things against my will at work.” Saying “true” to the latter question not only signals low motivation but suggests the solution is to increase autonomy.
Wrapping Up
The checklist above provides a clear framework for evaluating the visible parts of any employee assessment tool—the questions, instructions, and the response scales. Attention to detail here is critical, ensuring your data is accurate, sensitive to change, and robust enough to guide genuine solutions.
However, even perfectly written items are meaningless without rigorous scientific evidence to back them up and demonstrate they are in fact reliable and valid. In the final article of this series, we’ll unpack the standards psychometricians use to evaluate the statistical and research evidence of an assessment’s quality.
Related Articles
Japan’s Digital Nomad Visa offers a way to explore the startup ecosystem
The Basics of Employee Assessment Tools: What Every HR Team Needs to Know
