MANUAL FOR THE EMPLOYER OPENNESS SURVEY Dennis Gilbride, Kristie Golden ,David Vandergoot & Robert Stensrud Syracuse University Center for Essential Management Services South Oaks Hospital Drake University Introduction Despite continuing low unemployment rates, participation in the labor market by people with disabilities continues to be limited. The 1998 National Organization on Disability survey of people with disabilities found that, for those between the ages of 18 and 64, 29% were employed full or part time (National Organization on Disability, 1998). By the 2000 survey, this had not changed. Only 29% of persons with mild/moderate disabilities were working fulltime, but only 6% of those with severe disabilities (National Organization on Disability, 2000). This continuing problem became the focus of our research. We intended to create an innovative approach to establishing meaningful relationships with employers for the purposes of job placement. The overall project focused on the development of a tool that would help to assess an employer’s openness to hiring people with disabilities. Our goals included: • Development of an easy-to-use method for identifying new employers likely to be open to hiring and accommodating persons with disabilities; • A method not dependent on special skills or personality characteristics; • A method that is time sensitive and simple; and, • An approach that can be used by professionals and consumers alike. Successful employment of people with disabilities is complex and challenging, but of vital significance. We need to increase our understanding of employers and work environments, so that we can reduce the barriers to employment that many people with disabilities confront. At this stage of the research a final version of the Employer Openness Survey is available and this manual describes how it can be used. The manual will offer insight into understanding the use of the instrument, scoring it and using it in diverse settings. We are certain you will find it very useful in helping to increase job opportunities for individuals with disabilities. Acknowledgements The Employer Openness Survey was developed through a grant, #H133G000028, from the National Institute on Disability and Rehabilitation Research. The project was fully supported by staff from Syracuse University, Drake University and the Center for Essential Management Services, with the assistance of vocational rehabilitation professionals from South Oaks Hospital in New York. The Employer Openness Survey The Employer Openness Survey (EOS) is designed to be used with business persons to assess their openness to hiring persons with disabilities. Our research has found that there are a variety of characteristics that apply to companies that tend to hire people with disabilities and others representing a range of diversity. The responses obtained from these representatives of companies reveal a mindset on a continuum from closed to open. There are 16 items representing the characteristics associated with the openness/closedness continuum. Two additional items, questions 5 and 13, provide information specific to placement and are helpful to the rehabilitation professional, but, are not included in the scoring of the survey. The ordering of the questions is deliberate, with disability related questions near the close of the interview. The first and last questions are intended to influence the positive outlook of the employer during the completion of the survey. It is anticipated that with a positive question at the very beginning and at the very end, the rehabilitation professional will create an early and lasting good feeling with the employer about the development of a useful and fruitful relationship with the rehabilitation agency. The goals of this project reflected the potential impact it will have on the larger community. The history of the job placement process includes various attempts to improve the placement success of rehabilitation specialists when working with individuals with disabilities. In previous years, techniques used in marketing and placement have been studied, and plans to improve service delivery have been implemented. However, placement practices and employer development have not been done with a focus on strategy, nor has a systematic tool, such as the EOS, been developed for use until this time. The EOS, a two page interview, offers a simple, cost-effective way to look at an employer’s level of openness to hiring people with disabilities. Costs associated with duplicating the tool, and staff time to complete the questions with an employer are minimal. The tool, in its current form as a Microsoft Word document, can be emailed, faxed or photocopied by anyone who wishes to use it. At present, job developers and placement specialists spend time speaking to and visiting with employers with whom they wish to work in terms of placing individuals with disabilities. The EOS brings a strategy to this, thus streamlining the process of gathering employer data. In addition, it allows a rehabilitation provider to score the employer on openness. Understanding where an employer lies on a continuum of openness allows the rehabilitation provider to strategize about not only which employers to target immediately for placement, but also which ones need additional support, education and guidance. This saves time and effort on the part of the professional and leads to more satisfactory and expeditious placements. Hypothesis & Goals The purpose of the original study was to develop and test an instrument, the Employer Openness Survey, that was designed to operationalize the 3 categories (employer culture, work design, employer experience and support) and13 specific characteristics identified in the Gilbride et al. (2003) study, which identified organizational characteristics that suggested employer openness to hiring people with disabilities. It was initially hypothesized that a simple, non-threatening employer openness instrument could be used by rehabilitation professionals and consumers to identify the most promising employers to contact for placement and employer development. Further, such an instrument could be used as a consulting tool to help employers identify areas in which they could increase their openness and hiring of persons with disabilities. Finally, it was intended that the instrument be simple enough to not require any special skills or personality characteristics and that would be sensitive to time limits of both employers and rehabilitation professionals. Construct Validity and Reliability The next step was to test the convergent validity of the instrument by evaluating whether the concept of employer openness could be recognized and distinguished from the concept of closedness. Three composite sample instruments were created from the set of employer responses elicited during the pilot study. One sample EOS consisted of responses judged to be illustrative of closed employers, the second to illustrate neutral employers, and a third for open employers. Thirty-four rehabilitation professionals attending a disability awareness conference then reviewed these three sample instruments. These placement professionals were provided information on the Gilbride et al. (2003) model of employer openness, along with a description of the three categories and 13 specific characteristics identified in that study. Conference participants then rated the three sample EOS forms on the closedness-openness continuum, using the three point scale described above with 1= closed, 2= neutral, and 3= open. The average ratings of responses to all items of the closed EOS interview were 1.46, the neutral EOS interview was 2.01, and the open EOS interview was 2.82. These averages reflect well the expected values for closed, neutral, and open responses and were strong evidence that rehabilitation professionals could identify the extent of openness of an employer based on the results of the EOS. An analysis of variance was also conducted on these results confirming that the difference in the ratings on the 3 samples interviews were statistically significant (F= 144.89, p<.001) Field Test of EOS and Reliability We asked placement providers to nominate employers for inclusion into this phase of the project. Placement providers in two states (one in the Northeast, and the other in the Midwest) provided the names of employers they had worked with. Employer contact information was obtained, and research associates approached employers in an attempt to complete the EOS. Eighty-three employer interviews were conducted and scored. Scoring was based upon criteria developed during the earlier phases of the research, and upon the consensus of the researchers. A specific scoring protocol was developed that operationally defined each possible employer response and its corresponding score. For example, item number 12, “How do you communicate with, and get feedback from, your employees?” A proactive approach with evidence of reaching out to get input is scored as a 5; input possible but with no real active outreach, a passive approach is scored 3; no formal or systematic approach is scored as a 1. Three rehabilitation professionals were trained in scoring the surveys Several approaches were used to determine the reliability of the EOS using this scoring protocol. First, the percent of inter-rater agreement was computed. To do this, the total score for each company and rater was divided by the number of items to derive an overall “openness” average for each company as scored by each rater. This produced a potential range of scores from one to five, with one being closed and five open. The midpoint, 3, was neutral. With 83 companies and three raters, 249 scores were obtained. The inter-rater agreement was 85%. Second, the inter-rater correlations (Pearson r) were calculated by comparing the scores of each pair of raters. The correlations were as follows: Rater 1 with Rater 2 = .90 Rater 1 with Rater 3 = .85 Rater 2 with Rater 3 = .89. The average for these correlations was an acceptable .88. Next, a second inter-rater reliability was computed using the intra-class correlation. This approach is used to determine the source of variation in scores, whether it can be attributed to raters or to what is being rated, in this case, the openness of companies. As the correlation approaches one, the variation is due to what is being rated and not the raters. The intra-class correlation was .88, clearly showing that the agreement among raters is high. Evaluation of Usefulness of Instrument To determine if the EOS would be of value to rehabilitation professionals, presentations were made before two groups of placement providers attending professional conferences. The first was a group of rehabilitation professionals attending a regional National Rehabilitation Association conference (n = 39), the second was a group of practitioners attending a conference sponsored by the Association of Vocational Rehabilitation in Alcohol and Substance Abuse (n = 28 for a total of 67 providers). The Gilbride et al. (2003) model of employer openness, along with the instrument, scoring criteria and validation procedures were presented to group participants. At the conclusion of the presentation participants were asked to complete a one page feedback form that asked seven questions utilizing a Likert-style scale from 1 to 5 ( 1=strongly disagree, and 5= strongly agree) concerning the instrument. Results can be found in Table 1. Participants agreed that the instrument measured employer openness as defined by Gilbride et al. (M = 4.4, SD = .85), that it would be useful in developing relationships with employers (M = 4.4, SD = .80), and that they would consider using it in their placement work (M = 4.6, SD = .98). The only item rated lower than 4, was "Consumers could use this instrument" with a mean score of 3.9, (SD = .96). Following the conclusion of data collection a focus group was convened with the research associates who conducted the EOS interviews. Four associates were involved, all of whom were certified rehabilitation counselors (CRC’s), with Master’s degrees and employed as rehabilitation counselors with case management and placement responsibilities. Research associates were asked (1) How well did the process work in connecting with employers to complete the EOS; and (2) How useful is the EOS process and resulting information for placement? The research associates reported that on average it took about three calls to connect with the designated contact person. A number of contacts reported that they were initially reticent about discussing hiring procedures and policies with an unknown person. The research associates agreed that once a contact completed the EOS the information was useful for employer development and placement purposes. Several key topics were explored through the EOS, including revealing the needs of employers that rehabilitation professionals could address, identifying the attributes of applicants sought by employers, employers preferred recruiting resources, and learning about the employer’s prior experience with rehabilitation service providers. Table 1: Rehabilitation Providers Feedback on the Employer Openness Survey (N= 67) * Five level Likert scale: 1= Strongly disagree, 3= Moderately agree, 5= Strongly agree Summary The purpose of this study was to develop a reliable and valid instrument to measure employer openness as defined by Gilbride et al. (2003). The results provide preliminary support for the content, construct, and internal consistency of the EOS. The strong positive results found regarding its usefulness (provider review) indicates that the instrument is perceived as meaningful and useful to rehabilitation professionals engaged in placement activities. It must be noted that the EOS is a new instrument and further analysis and research is warranted. The EOS is based on a specific model of employer openness, which, while promising and grounded in past business and rehabilitation literature, is clearly not definitive. Further validation of the instrument is needed that would demonstrate increased hiring, accommodation, and advancement of workers with disabilities in company’s rated as more open. While further testing and analysis of the instrument is necessary, the strong consistent results found in the current research suggest that the EOS has the potential to help placement professionals identify employers most open to hiring and accommodating employees with disabilities. The results also suggest that employers don't find the EOS questions threatening or inappropriate. In fact, an unintended outcome of this study was the extent to which employers enjoyed and appreciated the discussion that resulted from answering the EOS questions. A very positive unexpected outcome of this study was that a number of placements were made during the pilot phase, and employer/provider relationships were strengthened by the process of engaging employers in discussing the EOS questions. There are a number of ways in which rehabilitation counselors and other placement providers can use the EOS to increase the employment of people with disabilities. The first use of the instrument is as an employer screening device. The EOS can be used to quickly identify the level of openness of a potential employer thus increasing the effectiveness of placement and employer development activities. Given the large caseloads of many rehabilitation professionals, using the EOS to help focus services on the most promising (open) employers may save valuable time and resources. The EOS may also have the potential to be used by consumers to help them understand the types of questions they can ask of employers that are useful, but will not result in employer defensiveness. A second use of the EOS is as a consulting and educational tool. To the extent that a provider or agency has the resources, the EOS can be used to help employers evaluate their actual (rather than publicized) level of openness. Consulting and educational services can then be directed to assist employers in honestly evaluating and changing organizational practices and procedures that may be limiting employment opportunities for people with disabilities. A third use of the EOS is as an employer development tool. Rehabilitation counselors and other professionals can use the questions on the EOS as a way to begin or deepen a relationship with an employer. The very strong positive response of employers to the questions on the EOS suggests that employers are willing to engage in conversations with providers if they feel that the rehabilitation professional has an authentic and thoughtful interest in them. Placement providers can thus use the EOS as a method to approach an employer and establish a meaningful relationship (Fry, 1997). These promising initial results, along with the concerns noted above, suggest that further research on the psychometric properties and uses of the EOS is warranted. A number of significant questions remain concerning the EOS, and employer openness in general. For example, are there a few particular questions (or employer characteristics) that are most important? Can the score on the EOS predict the likelihood of a consumer receiving a job offer? Is the score on the EOS an accurate measure of the employer’s level of openness, or is it more a reflection of the specific individual answering the questions? Are there meaningful cut-off scores that rehabilitation counselors can rely on to focus their placement efforts? Can the EOS be used with relatively closed employers to help them change their culture or procedures? Can consumers use the EOS to facilitate their self-directed job search? Is the model of employer openness that the EOS measures a valuable and useful concept? In summary, this initial research suggests that the EOS is a valid and reliable instrument that may have a number of practice applications for rehabilitation counselors. While further research is necessary, rehabilitation counselors and other placement professionals may find this instrument of immediate benefit in their work. Employer Interview Questions 1. What are your company’s key strengths? 2. What are the three top concerns of your company’s Chief Executive Officer? 3. What recruiting resources do you use to find good candidates? 4. What are your top two resources? 5. What experiences have you had with government recruiting resources such as the Department of Labor and the state vocational rehabilitation agency? 6. How has using automated electronic applications, such as resume scanning, using fax machines, email and the Internet, changed the make up of your applicant pool, if at all? 7. What jobs do you currently have openings for or that you will have in the near future? 8. What personal attributes or characteristics does your company look for in new hires? 9. What criteria or measures, such as interviews and reference checks, do you use to determine if an applicant is a good fit for the job? 10. If you had to choose, would you select someone who you knew to be a quick learner or someone who you knew to be very dependable? Why? 11. Do you provide internships and/or apprenticeships? 12. Can supervisors flexibly assign job tasks based on an employee’s strengths and weaknesses? 13. Does your company have special recognition programs or activities for individual employees, teams, departments or other groups? 14. How do you communicate with, and get feedback from, your employees? 15. What benefits do you offer? 16. What kind of social activities does your company have and who participates? 17. If a new hire or current employee asks for an accommodation who would you go to for technical assistance? 18. How do you handle it when an employee returns to work after an illness or injury? 19. What was the single most effective action your company took to promote diversity in the workplace? How did that come about? 20. What are you most proud of at your place of employment? Script Hello, my name is ___________________. I am calling from____________________. We are a (non-profit )organization that provides training and/or job placement services to individuals with disabilities. We are very interested in getting to know your company and how we can work together to create job opportunities for people with disabilities while meeting your recruiting needs. There is no cost to use our services and we are very interested in finding out what we can do to support your hiring process. We have a standard set of questions that we ask employers which can help us to better understand the types of skills and abilities you seek in new hires. We have completed the survey with many employers in the past and it has been received very positively by those with whom we have interacted. I would appreciate the opportunity to meet with you in person and see your company, however, if time does not permit, we can provide you with an electronic or fax copy which you can complete and return to us. We are not interested in collecting information about who you have hired and related statistics but we are more interested in how you go about the hiring process. Our goal is to work with you to successfully place more individuals with disabilities into employment while simultaneously providing you with highly qualified candidates. [At the conclusion] Thank you so much for working together with us toward achieving greater opportunities for the people in our community. Scoring Criteria It is not expected that employers will be open or closed consistently across all items. Therefore, it is not likely that the scoring for the interview will yield a result of either the company being open or closed. Rather, the scoring should reveal areas in which a company is open, other areas in which the company is neither, and some areas where the company appears closed. Where companies are closed, or at least not clearly open, rehabilitation professionals have opportunities to consult with employers and assist them in providing employment opportunities to persons with disabilities. This is an assessment tool indicating an employer’s readiness to hire and suggesting areas of need which can become opportunities to serve them for rehabilitation professionals. Actual scoring of items requires applying a scale from 1 to 5 where: Open responses score as 5 Neutral responses score as 3 Highest possible score: 80 Closed responses score as 1 Examples of each type of response are given for each item. Raters need to judge how closely the actual response matches one of the examples provided and score the response accordingly. If the scoring guidelines provided clearly do not fit the answer given by the employer, you may choose to use a rating of 2 for those answers that fall between 1 and 3, or a rating of 4 for those answers that fall between a 3 and a 5. Another words, use the mid-points between the three categories if a direct match is not apparent. If it is still not clear what rating to give, a general rule to follow is to give the response the highest rating possible. This approach will tend to score employers as open as possible and encourage further development and relationship building. Subsequent follow up with the employer will clarify the extent of the employer’s openness and readiness to hire. Do not let any prior response color any subsequent judgment you make. Each item should be rated independently of all other items. If a response does not seem to address the question asked, do not score the item. After learning the criteria by scoring two or three surveys, the average time to score each interview is approximately 5 to 6 minutes. * Items 5 and 13 are not scored as they are relevant only for collecting placement information. 1. What are your company’s key strengths? Open: focus is on employees, their positive attributes, or their opportunities as employees Neutral: focus is on customers Closed: focus is on production, services, technology, processes, etc., whatever the organization produces 2. What are the three top concerns of your company’s Chief Executive Officer? Open: focus is on employees, their well-being (note: concern for lack of staff is not to be interpreted as a concern for employees. This reflects more of a concern for production) Neutral: focus is on customers Closed: focus on production, provision of services, pricing, revenue 3. What recruiting resources do you use to find good candidates? Open: use of three of more types of resources that include some that are external to company, e.g., government agencies including at least one that focuses on persons with disabilities, web sites and recruiting at schools. Mention of external resources that do not include persons with disabilities, but other organizations that advocate for protected groups score as 4. Neutral: few types, most of which are primarily local resources, such as newspapers and job fairs Closed: use of internal recruiting (employee referrals), word-of-mouth, personal networks, or only one way of recruiting 4. What are your two top resources? Open: Both sources are non profit, education or social service oriented Neutral: One source is non profit, education or social service oriented Closed: Neither is non profit, education or social service oriented 5. What experiences have you had with government recruiting resources such as the Department of Labor and the state vocational rehabilitation agency? (Note: Raters may use information from Item #3 to help score this sub-item) Open: works with these agencies to hire or has tried Neutral: passive, mainly aware of agency services but hasn't been proactive Closed: Not aware, no experience, no interest 6. How has using automated electronic applications, such as resume scanning, using fax machines, email and the Internet, changed the make up of your applicant pool, if at all? Open: recognition that more persons with disabilities or from diverse backgrounds and culture are being attracted, and aware of implications of new technologies, or at least thinking about how new technologies have impacted on the make up of the applicant pool Neutral: no impact although used, not really thought about impact on the applicant pool Closed: not used, sticks with tried and true Not scored – a placement information item 7. What personal attributes or characteristics does your company look for in new hires? Open: An open employer will indicate a value for people for who they are rather than for what they can do and make reference to attributes or characteristics (rather than job-related skills) such as need for team player, readiness to learn (implies willingness to teach), employees described as valued resource, refers to having employees with a range/mix of attributes, not one type Neutral: A Neutral employer will indicate desire for a combination of skills as well as reference to desirable personal attributes. Closed: A closed employer references only required skills, looking for one specific type of applicant, or concerned with the personal appearance of employees. Even the reference for having employees with good interpersonal skills is closed when it is viewed as a skill required for certain jobs, such as sales clerks or receptionists. 8. What criteria or measures, such as interviews and reference checks, do you use to determine if an applicant is a good fit for the job? Open: An open employer will use a variety of approaches to get to know an applicant and use multiple contacts, both subjective and objective to accomplish this. There will be a balance of strategies, for example at least two objective and two subjective approaches. Interviews and reference checks are subjective, while work histories and tests are objective. Novel and creative/thoughtful approaches, such as taking a candidate on a tour, are signs of willingness to provide the person with an opportunity to make him/herself known. Neutral: Although both objective and subjective approaches are used, there is only one example of one of the types. There is not a balance or enough variety of objective or subjective approaches. Closed: Rigid and complete reliance one or the other kind of evidence such as only using drug tests, computer tests, screening tests or only using interviews and references, without the mix of objective and subjective input. 9. If you had to choose, would you select someone who you knew to be a quick learner or someone who you knew to be very dependable? Why? (Note that this question requires an either/or answer. Therefore, there is no possible neutral choice. If the employer refers to both being desirable, this indicates he/she is being very selective in that both quick learning and dependability are requirements. Thus, an employer indicating both would be a closed employer due to this restrictiveness.) Open: reference to trust, dependability, reliability; characteristics anyone could have not dependent on prior experience, education, etc. Neutral: none, this item must be rated open or closed Closed: Emphasis on ability to learn or indicating both are important (unable to choose one over the other) 10. Do you provide internships and/or apprenticeships? Note that this question requires an answer regarding what the company does at present, or might have done in the recent past. Indicating a willingness to provide these, without having done so would only merit a neutral rating. Open: Yes, offers these or has done so in recent past (within last two years) Neutral: Select this rating only if the company indicates willingness to do this even though they haven’t done so to this point. Closed: No or blank 11. Can supervisors flexibly assign job tasks based on an employee’s strengths and weaknesses? Open: Yes Neutral: No, but with a positive qualification as to why it is not possible, such as the jobs are too routine, or union issues, etc. Closed: No 12. Does your company have special recognition programs or activities for individual employees, teams, departments or other groups? (Note: This item refers to doing good things for employees like providing holiday parties, discounts, bonuses, etc. It does not pertain to employee benefits or anything that all employees get simply as a result of being employed.) Open: Listing of more than two that are related to personal or professional achievements, development and growth Score of four: a listing of two Neutral: Listing of one Closed: None 13. How do you communicate with, and get feedback from, your employees? Open: includes getting or giving information – requires four or more approaches Score of four: Three approaches Neutral: Two approaches or face-to-face meetings Closed: No formal or systematic approach Not scored – a placement information item 14. What kind of social activities does your company have and who participates? (Note: multiple events occurring during the year that are similar count as one activity) Open: Three or more activities Neutral: One or two Closed: None 15. If a new hire or current employee asks for an accommodation who would you go to for technical assistance? Open: Potentially could seek assistance from multiple sources, primarily external to the company, that could be local or from corporate headquarters Neutral: Tries something but only with internal resources, or one source Closed: No assistance sought or no acknowledgement this has ever been needed or shows a concern for litigation or legal issues 16. How do you handle it when an employee returns to work after an illness or injury? Open: Mention of some kind of accommodation with examples provided, or reference to a process to ensure smooth re-entry, such as inservice training Neutral: Need of doctor’s note only Closed: No formal approach or doesn’t know 17. What was the single most effective action your company took to promote diversity in the workplace? Open: A proactive approach or strategy; the company took deliberate action Neutral: Passive approach; it just happened; company not opposed to it but did not seek it out, emphasizes meeting legal or Equal Opportunity requirements Closed: No action, or flat statement that they don't discriminate 18. What are you most proud of at your place of employment? Open: Reference to giving back to the community; going beyond the company Neutral: Statements related to employees and/or customers Closed: References to product(s), services, productivity References Fry, R. (Ed.) (1997). Developing effective partnerships with employers as a service delivery mechanism. Menomonie, WI: Stout Vocational Rehabilitation Institute. Gilbride, D., & Stensrud, R. (2003). Job placement and employer consulting: Services and strategies. In E. Szymanski & R. Parker (Eds.) Work and Disability (2nd ed.) (pp. 407-439). Austin TX: ProEd. Gilbride, D., Stensrud, R., Vandergoot, D., & Golden, K. (2003). Identification of the charactreristics of work environment and employers open to hiring and accommodating people with disabilities. Rehabilitation Counseling Bulletin, 46 (3), 130-137. National Organization on Disability. (1998). NOD/Harris survey of Americans with disabilities. Washington DC: Author. National Organization on Disability (2000). NOD/Harris survey of Americans with Disabilities: Washington DC: Author.