Conducting Empirical Validation and Adverse Impact Analysis

Yellow Blaze Candle Shops provides a full line of various types of candles and accessories such as candleholders. Yellow Blaze has 150 shops in shopping malls and strip malls throughout the country. Over 600 salespeople staff these stores, each of which has a full-time manager. Staffing the manager's position, by policy, must occur by promotion from within the sales ranks. The organization is interested in improving its identification of salespeople most likely to be successful store managers. It has developed a special technique for assessing and rating the suitability of salespeople for the manager's job.

To experiment with this technique, the regional HR department representative reviewed and rated the promotion suitability of each store's salespeople. They reviewed sales results, customer service orientation, and knowledge of store operations for each salesperson and then assigned a 1–3 promotion suitability rating (1 = not suitable, 2 = may be suitable, 3 = definitely suitable) on each of these three factors. Customer service orientation was rated based on supervisor and coworker observations of work behavior. These ratings incorporated evaluations of how often salespeople asked customers how they could help, how effectively salespeople were able to suggest products that matched customer requests, and how well salespeople ensured that customers were happy with their intended purchases at the end of the encounter. In most cases, ratings of customer service orientation were similar across managers and assistant managers, but there were some discrepancies. Knowledge of store operations was evaluated based on a standardized exam, consisting of a variety of questions related to facts ranging from managerial practices and procedures to refund and exchange policies to record-keeping requirements. A total promotion suitability (PS) score, ranging from 3 to 9, was then computed for each salesperson.

The PS scores were gathered for all salespeople but were not formally used in promotion decisions. Over the past year, 30 salespeople were promoted to store manager. Now it is time for the organization to preliminarily investigate the validity of the PS scores and see whether their use results in adverse impact against women or minorities. Each store manager's annual overall performance appraisal rating, ranging from 1 (low performance) to 5 (high performance), was used as the criterion measure in the validation study. The following data were available for analysis: