Foundations for Evidence-Based Policymaking Act of 2018 - Program Evaluation Competencies and OPM Program Evaluation Workforce and Job Analysis Employee and Supervisor Surveys

September 20, 2021 Memorandum For: CXO Councils From: Kiran A. Ahuja, Director Subject: Foundations for Evidence-Based Policymaking Act of 2018 – Program Evaluation Competencies and OPM Program Evaluation Workforce and Job Analysis Employee and Supervisor Surveys The U.S. Office of Personnel Management (OPM) is working in consultation with the Office of Management and Budget (OMB) to implement specific requirements of Public Law 115-435, Foundations for Evidence-Based Policymaking Act of 2018. In accordance with the Act, OPM is required to identify key skills and competencies needed for program evaluation in an agency; establish a new occupational series, or update and improve an existing occupational series, for program evaluation within an agency; and establish a new career path for program evaluation within an agency. To fulfill these requirements, OPM is conducting an occupational study on program evaluation work. Additionally, this study supports OMB’s guidance, “Phase 4 Implementation of the Foundations for Evidence-Based Policymaking Act of 2018: Program Evaluation Standards and Practices” (March 10, 2020) by informing agency efforts to better track and build the program evaluation workforce. As part of the study, OPM is pleased to provide for immediate use the attached program evaluation competencies. These competencies were identified by Federal agency and human resources subject matter experts as needed for performing program evaluation work governmentwide. Agencies may use these program evaluation competencies to select, assess, and train program evaluation talent. The OPM Program Evaluation Workforce and Job Analysis Employee and Supervisor Surveys represent the next phase of the study and will be used to validate the program evaluation competencies and tasks identified by Federal agency and human resources subject matter experts as needed for performing program evaluation work governmentwide. OPM plans to issue the OPM Program Evaluation Workforce and Job Analysis Employee and Supervisor Surveys on September 20, 2021, the results of which will be used to develop a Program Evaluation Competency Model. Federal employees identified by their agencies as performing program evaluation work, and/or identified via eHRI data as being in the Management and Program Analysis Series, GS-0343, will be invited via email to complete the OPM Program Evaluation Workforce and Job Analysis Employee Survey; each employee will also receive a separate email with the OPM Program Evaluation Workforce and Job Analysis Supervisor Survey to forward to

their immediate supervisor (for their supervisor to complete). Employees and supervisors will have two weeks from the issuance date of this memo to submit their survey responses. We appreciate your assistance with developing policy in line with the requirements of the Foundations for Evidence-Based Policymaking Act of 2018. For additional information, please refer to the attached program evaluation occupational study frequently asked questions. Should you have any questions about OPM’s occupational study, please contact Classification and Assessment Policy at competency@opm.gov. Attachments (see below): Program Evaluation Competencies, and Program Evaluation Occupational Study Frequently Asked Questions cc: Chief Human Capital Officers (CHCOs), Deputy CHCOs, Human Resources Directors, Evaluation Officers, Chief Information Officers, and Chief Data Officers Program Evaluation Competencies The following tables present the 25 general competencies and nine technical competencies that have been identified on a governmentwide basis for program evaluation work. Agencies are responsible for conducting job analyses for work within their agency (5 CFR Part 300.103). Similarly, agencies must determine the applicability of these competencies to positions within their agency. Please refer to OPM's Delegated Examining Operations Handbook for more information on conducting a job analysis. Definitions of the competencies follow the tables. General Competencies • • • • • • • • • • • • • Accountability Conflict Management Continual Learning Creative Thinking Customer Service Decision Making Flexibility Influencing/Negotiating Integrity/Honesty Interpersonal Skills Leadership Legal, Government and Jurisprudence Manages Human Resources • • • • • • • • • • • •

Mathematical Reasoning Oral Communication Organizational Awareness Partnering Planning and Evaluating Political Savvy Problem Solving Reasoning Teamwork Technical Competence Technical Credibility Writing Technical Competencies • • • • • Data Analysis Evaluation Knowledge Management Organizational Performance Analysis Performance Measurement • • • • Project Management Quality Management Research Stakeholder Management

Program Evaluation General Competencies Accountability - Holds self and others accountable for measurable high-quality, timely, and cost-effective results. Determines objectives, sets priorities, and delegates work. Accepts responsibility for mistakes. Complies with established control systems and rules. Conflict Management - Manages and resolves conflicts, grievances, confrontations, or disagreements in a constructive manner to minimize negative personal impact. Continual Learning - Assesses and recognizes own strengths and weaknesses; pursues selfdevelopment. Creative Thinking - Uses imagination to develop new insights into situations and applies innovative solutions to problems; designs new methods where established methods and procedures are inapplicable or are unavailable. Customer Service - Works with clients and customers (that is, any individuals who use or receive the services or products that your work unit produces, including the general public, individuals who work in the agency, other agencies, or organizations outside the Government) to assess their needs, provide information or assistance, resolve their problems, or satisfy their expectations; knows about available products and services; is committed to providing quality products and services. Decision Making - Makes sound, well-informed, and objective decisions; perceives the impact and implications of decisions; commits to action, even in uncertain situations, to accomplish organizational goals; causes change. Flexibility - Is open to change and new information; adapts behavior or work methods in response to new information, changing conditions, or unexpected obstacles; effectively deals with ambiguity. Influencing/Negotiating - Persuades others to accept recommendations, cooperate, or change their behavior; works with others towards an agreement; negotiates to find mutually acceptable solutions. Integrity/Honesty - Contributes to maintaining the integrity of the organization; displays high standards of ethical conduct and understands the impact of violating these standards on an organization, self, and others; is trustworthy. Interpersonal Skills - Shows understanding, friendliness, courtesy, tact, empathy, concern, and politeness to others; develops and maintains effective relationships with others; may include effectively dealing with individuals who are difficult, hostile, or distressed; relates well to people from varied backgrounds and different situations; is sensitive to cultural diversity, race, gender, disabilities, and other individual differences. Leadership - Influences, motivates, and challenges others; adapts leadership styles to a variety of situations.

Legal, Government and Jurisprudence - Knowledge of laws, legal codes, court procedures, precedents, legal practices and documents, Government regulations, Executive orders, agency rules, Government organization and functions, and the democratic political process. Manages Human Resources - Plans, distributes, and monitors work assignments; evaluates work performance and provides feedback to others on their performance. Mathematical Reasoning - Solves practical problems by choosing appropriately from a variety of mathematical and statistical techniques. Oral Communication - Expresses information (for example, ideas or facts) to individuals or groups effectively, taking into account the audience and nature of the information (for example, technical, sensitive, controversial); makes clear and convincing oral presentations; listens to others, attends to nonverbal cues, and responds appropriately. Organizational Awareness - Knows the organization's mission and functions, and how its social, political, and technological systems work and operates effectively within them; this includes the programs, policies, procedures, rules, and regulations of the organization. Partnering - Develops networks and builds alliances; collaborates across boundaries to build strategic relationships and achieve common goals. Planning and Evaluating - Organizes work, sets priorities, and determines resource requirements; determines short- or long-term goals and strategies to achieve them; coordinates with other organizations or parts of the organization to accomplish goals; monitors progress and evaluates outcomes. Political Savvy - Identifies the internal and external politics that impact the work of the organization. Perceives organizational and political reality and acts accordingly. Problem Solving - Identifies problems; determines accuracy and relevance of information; uses sound judgment to generate and evaluate alternatives, and to make recommendations. Reasoning - Identifies rules, principles, or relationships that explain facts, data, or other information; analyzes information and makes correct inferences or draws accurate conclusions. Teamwork - Encourages and facilitates cooperation, pride, trust, and group identity; fosters commitment and team spirit; works with others to achieve goals. Technical Competence - Uses knowledge that is acquired through formal training or extensive on-the-job experience to perform one's job; works with, understands, and evaluates technical information related to the job; advises others on technical issues. Technical Credibility - Understands and appropriately applies principles, procedures, requirements, regulations, and policies related to specialized expertise. Writing - Recognizes or uses correct English grammar, punctuation, and spelling; communicates information (for example, facts, ideas, or messages) in a succinct and organized manner;

produces written information, which may include technical material, that is appropriate for the intended audience. Program Evaluation Technical Competencies Data Analysis - Knowledge of the principles, methods, and processes used to analyze data and datasets; including investigating trends or anomalies, establishing relationships, and identifying patterns. Evaluation - Knowledge of evaluation theory, concepts, standards, methods, and tools to assess the effectiveness and efficiency of programs, policies, and organizations. Knowledge Management - Knowledge of the value of collected information and the methods of sharing that information throughout an organization. Organizational Performance Analysis - Knowledge of the methods, techniques, and tools used to analyze program, organizational, and mission performance; includes methods that deliver key performance information (for example, comparative, trend, diagnostic, root cause, predictive) used to inform decisions, actions, communications, and accountability systems. Performance Measurement - Knowledge of the principles and methods for evaluating program or organizational performance using financial and nonfinancial measures, including identification of evaluation factors (for example, workload, personnel requirements), metrics, and outcomes. Project Management - Knowledge of the principles, methods, or tools for developing, scheduling, coordinating, and managing projects and resources, including monitoring and inspecting costs, work, and contractor performance. Quality Management - Knowledge of the principles, methods, and tools of quality assurance, quality control, and reliability used to ensure that a project, system, or product fulfills requirements and standards. Research - Knowledge of the scientific principles, methods, and processes used to conduct a systematic and objective inquiry; including study design, collection, analysis, and interpretation of data; and the reporting of results. Stakeholder Management - Knowledge of the concepts, practices, and techniques used to identify, engage, influence, and monitor relationships with individuals and groups connected to a work effort; including those actively involved, those who exert influence over the process and its results, and those who have a vested interest in the outcome (positive or negative).

Program Evaluation Occupational Study Frequently Asked Questions 1. What is OPM’s role under Public Law 115-435, Foundations for Evidence-Based Policymaking Act (Evidence Act) of 2018? The Evidence Act states the Director of the Office of Personnel Management, in consultation with the Director of the Office of Management and Budget, shall — (A) identify key skills and competencies needed for program evaluation in an agency; (B) establish a new occupational series, or update and improve an existing occupational series, for program evaluation within an agency; and (C) establish a new career path for program evaluation within an agency. Developing a competency model for program evaluation work is a key step towards ensuring Federal agencies can attract, recruit, and hire skilled employees to accomplish program evaluation work. 2. Why is OPM issuing employee and supervisor surveys for program evaluation? OPM is collecting information to identify the tasks and competencies that are most important for successful performance in Federal program evaluation positions at each grade level. Results will be used to inform Federal human resources policy, assist employees in career development and training, and help agencies with workforce planning, selection, and performance management. Ratings from employees and supervisors will paint a picture of the program evaluation work performed across the Federal Government. 3. Who is being asked to complete the OPM Program Evaluation Workforce and Job Analysis Employee and Supervisor Surveys? To obtain the best possible information, Federal employees performing program evaluation work are being asked to complete the employee survey; their supervisors are being asked to complete the supervisor survey. If you are a Federal employee who performs or supervises program evaluation work and you did not receive an email with a link to the designated survey, please email competency@opm.gov. 4. How were the participants for each survey identified? Agencies identified their own program evaluation workforce. On behalf of OPM, the Office of Management and Budget requested contact information for Federal employees in program evaluation positions from the Evaluation Officers and Federal evaluation community; these employees are found within many occupational series such as, but not limited to, Economist, GS-0110, Budget Analysis Series, GS-0560, and Operations Research, GS-1515. OPM also identified via eHRI data Federal employees in the Management and Program Analysis Series, GS-0343, for survey participation. Not all employees in the GS-0343 series perform program evaluation work; only employees in the GS-0343 series who perform program

evaluation work should complete their employee survey and forward their immediate supervisor the supervisor survey. 5. How will Federal employees performing program evaluation work receive the employee survey? Federal employees will receive an email from competency@opm.gov with a link and instructions to complete the OPM Program Evaluation Workforce and Job Analysis Employee Survey. They will also receive a separate email to forward to their supervisors with the supervisory version of the survey (for their supervisor to complete). 6. How will Federal employees supervising program evaluation work receive the supervisor survey? Federal employees who supervise one or more program evaluation employees will receive an email from their employees with a link and instructions to complete the OPM Program Evaluation Workforce and Job Analysis Supervisor Survey. 7. Should Federal employees supervising program evaluation work complete the supervisor survey for each of their employees? Federal employees who supervise one or more program evaluation employees should complete the supervisor survey once for each of their employees. Supervisors only need to rate competencies and tasks once for each grade level of program evaluation work they supervise. The first section (Occupational Background) has a question to ask supervisors if they previously completed a survey at the grade level they selected; if they select yes, the survey ends (remaining sections are skipped). 8. Should I share any of the survey links I receive with other Federal employees performing or supervising program evaluation work? The links to each survey are personalized and should not be shared. Once you submit either survey, you will not be able to view, change, or re-submit the survey. If other program evaluation employees or supervisors are interested in taking the designated survey, they may email competency@opm.gov to be provided a personalized survey link. 9. What information does the employee survey cover? The employee survey contains four sections: (I) Occupational Background, (II) Competencies, (III) Tasks, and (IV) Background Information. The Occupational Background section asks about your current job, including occupational series, grade, and agency. The Competencies section asks you to rate competencies that may be required to perform your job. The Tasks section asks you to rate tasks that may be performed as part of your job. The Background Information section asks for information about you, such as your years of Federal service.

10. What information does the supervisor survey cover? The supervisor survey contains four sections: (I) Occupational Background, (II) Competencies, (III) Tasks, and (IV) Background Information. The Occupational Background section asks for information about you and your employee, such as the department or agency for which you and your employee currently work. The Competencies section asks you to rate competencies that may be required to perform program evaluation work in your employee's job. The Tasks section asks you to rate program evaluation tasks that may be performed as part of your employee's job. The Background Information section asks for information about you, such as your years of Federal service. 11. What if I receive either survey, but I do not perform (or supervise others that perform) program evaluation work? If you receive the employee survey, but do not perform program evaluation work, you do not need to complete the employee survey; you also do not need to forward your immediate supervisor the supervisor survey. If you receive the supervisor survey, but do not supervise other Federal employees that perform program evaluation work, you do not need to complete the supervisor survey. 12. What competencies are included in the surveys? The surveys include the 25 general competencies and nine technical competencies that have been identified on a governmentwide basis for program evaluation work. 13. What is the timeline for the surveys? The surveys will be open for two weeks, starting from the issuance date of the surveys. For each survey, we will monitor the response rate, send reminder emails, and communicate with agency points of contact to encourage participation. 14. Who will see the responses to the surveys? The responses to each survey will be analyzed by OPM. All information provided will be treated in accordance with the Privacy Act by OPM. Only group data will be reported; in any public release of survey results, no data will be disclosed that could be used to identify specific individuals.