Foundations for Evidence-Based Policymaking Act of 2018 Updated Position Classification

May 06, 2024 Memorandum for Human Resources Directors From: Karen R. Jacobs Acting Deputy Associate Director Talent Acquisition, Classification and Veterans Programs Workforce Policy and Innovation Subject: Foundations for Evidence-Based Policymaking Act of 2018 Updated Position Classification The U.S. Office of Personnel Management (OPM) is working in consultation with the Office of Management and Budget (OMB) to implement specific requirements of Public Law 115-435, Foundations for Evidence-Based Policymaking Act of 2018 (Evidence Act). In accordance with the Evidence Act, OPM is required to identify key skills and competencies, establish or update an occupational series, and establish a new career path for program evaluation. OPM is pleased to release for agency use the updated Position Classification Flysheet for Management and Program Analysis Series, 0343 including a definition for program evaluation work. The program evaluation competency model and the career path guidance will be released through separate memo issuances. To fulfill the requirement set forth in Public Law 115-435 and support the President’s Management Agenda Priority 3, Managing the business of Government to build back better,

OPM conducted a governmentwide study to explore program evaluation work performed by agencies. OPM’s collected and evaluated data was provided by program evaluation and human resources technical subject matter experts. OPM found that program evaluation work is multidisciplinary, and the work outlined in the Evidence Act and OMB Guidance M-20-12 is covered by the Management and Program Analysis Series, GS-0343, and other series, including the Social Science Series, 0101 when supported by a job analysis and aligned with the agency’s mission. OPM defines multidisciplinary work as a position involving duties and responsibilities closely related to more than one discipline. As a result, the position could be classifiable into two or more occupational series. In this case, the position can be classified into series involving professional (0101 series) and/or nonprofessional (0343) work. The nature of the work is such that persons with training and experience in either of two or more occupations may be considered well-qualified to do the work (Interpretive Guidance for Cybersecurity Positions, p 21 -22).

In accordance with the Evidence Act and 5 U.S.C. § 311 (3) ‘evaluation’ is an assessment using systematic data collection and analysis of one or more programs, policies, and organizations intended to assess their effectiveness and efficiency. Furthermore, “evidence is broadly defined and includes the following foundational evaluation functions - fact finding, performance measurement, policy analysis, and program evaluation” (OMB Guidance M-19-23). These functions are found in the GS-343 series. The Management and Program Analysis Series, 0343, includes positions that primarily serve as analysts and advisors to management on the evaluation of the effectiveness of government programs and operations, the productivity and efficiency of the management of Federal agencies, or both using data collected and analyzed via qualitative and quantitative methods. As stated in the Handbook of Occupational Groups and Families, positions in the Social Sciences Series, GS-0101 involve professional work and advise, administer, supervise, or perform research or other professional and scientific work in one or any combination of the social sciences when such work is not classifiable in other series of the Social Science, Psychology, and Welfare Group, 0101. OPM’s completion of a comprehensive review of program evaluation work resulted in the update of the Position Classification Flysheet for Management and Program Analysis Series, 0343. Work classified in other series including the 0101 Social Science series, for example, based on the specialized knowledge required for performing the work should follow the appropriate classification guidance for the job series along with guidance found in the 343 Flysheet and OPM classification policy guidance (i.e., Introduction to Position Classification Standards). A new definition for program evaluation work is included in the 0343 Flysheet. OPM authorized the use of the title Program Evaluator for positions classified in the 0343 series. The parenthetical title (Program Evaluator) may be used for positions performing program evaluation work most of the time and not as a collateral duty as supported by the analysis of work. Agencies must apply updated OPM position classification standards to covered positions within 12 months of the date of issuance. If you have any questions regarding the Flysheet, please email the Classification and Assessment Policy team at FedClass@opm.gov. cc: Chief Human Capital Officers, Deputy Chief Human Capital Officers Attachment: Position Classification Flysheet for Management and Program Analysis Series, 0343 (see below)

Management and Program Analysis Series, 0343 Issued: May 2024 Revised: October 2024

Position Classification Flysheet for Management and Program Analysis Series, 0343 Table Of Contents Introduction ....................................................................................................................................2 Coverage .........................................................................................................................................2 Establishing the Occupational Series and Standard .................................................................2 General Series Determination Guidelines ..................................................................................3 Official Titling Provisions .............................................................................................................4 Series Definition ............................................................................................................................5 Titling ..............................................................................................................................................6 General Occupational Information .............................................................................................7 Impact of Automation .................................................................................................................10 Additional Occupation Considerations .....................................................................................11 Crosswalk to the Standard Occupational Classification .........................................................14 Grading Instructions ....................................................................................................................15 Appendix A – Historical Record and Explanatory Material .....................................................16

Management and Program Analysis Series, 0343 May 2024 Introduction This position classification flysheet updates the Management and Program Analysis Series, 0343, and provides the series definition and titling instructions. In the General Schedule position classification system established under chapter 51 of title 5, United States Code, the positions addressed here would be two-grade interval positions. The term “General Schedule” or “GS” denotes the major position classification system and pay structure for white collar work in the Federal Government. Agencies that are no longer subject to chapter 51 have replaced the GS pay plan indicator with agencyunique pay plan indicators. For that reason, reference to General Schedule or GS has been omitted from this flysheet. Coverage This position classification flysheet covers the following occupational series: Management and Program Analysis Series, 0343 Establishing The Occupational Series and Standard Issuance of this flysheet updates this occupational series as described in the following table. The table also indicates how to classify work covered by this series. New/Previous Series or Guidance Management and 0343 Program Analysis Series U.S. Office of Personnel Management Action Taken/How to Classify Work Previously Covered • Supersedes this Flysheet, last revised in August 1990. Management and Program Analysis Series, 0343 May 2024

General Series Determination Guidelines Determining the correct series for a position is usually apparent by reviewing its assigned duties and responsibilities and then comparing them to the series definitions and general occupational information the classification flysheet or standard provides. Generally, the classifier decides on the series for a position based on the primary work of the position, the highest level of work performed, and the paramount knowledge required to do the work of the position. In some situations, however, following this guidance may present difficulties. When the work of a position matches more than one occupation, then use the following guidelines to determine the appropriate series for classification purposes. • • • • Paramount knowledge required. Although there may be several different kinds of work in the position, most positions will have a paramount knowledge requirement. The paramount knowledge is the most important type of subject matter knowledge or experience required to do the work. Reason for existence. The primary purpose of the position or management’s intent in establishing the position is a positive indicator for determining the appropriate series. Organizational mission and/or function. Positions generally align with the mission and function of the organization to which they are assigned. The organization’s function is often mirrored in the organizational title and may influence the appropriate series. Recruitment source. Supervisors and managers can help by identifying the occupational series that provides the best qualified applicants to do the work. This is closely related to the paramount knowledge required.

The Additional Occupational Considerations section of this flysheet provides examples where the work may involve applying related knowledge and skills, but not to the extent that it warrants classification to this occupation. For further guidance, refer to The Classifier’s Handbook. U.S. Office of Personnel Management Management and Program Analysis Series, 0343 May 2024

Official Titling Provisions Title 5, United States Code, requires the U.S. Office of Personnel Management (OPM) to establish authorized official position titles to include a basic title may be appended with one or more prefixes and/or suffixes. Agencies must use the official position titles for human resources management, budget, and fiscal purposes. Instructions for assigning official position titles are provided in this section. Supervisors and Leaders Add the prefix “Supervisory” to the basic title when the agency classifies the position as supervisory. If the position is covered by the General Schedule, refer to the General Schedule Supervisory Guide for additional titling information. Add the prefix “Lead” to the basic title when the agency classifies the position as leader. If the position is covered by the General Schedule, refer to the General Schedule Leader Grade Evaluation Guide for additional titling information. Specialty or Parenthetical Titles Specialty titles are typically displayed in parentheses and referred to as parenthetical titles. Agencies may supplement the authorized title with agency established parenthetical titles if necessary for recruitment or other human resources needs. Organizational Titles Organizational and functional titles do not replace, but rather complement, official position titles. Agencies may establish organizational and functional titles for internal administration, public convenience, program management, or similar purposes. Examples of organizational titles are Branch Chief or Division Chief. Examples of functional titles are Chief of Policy Development and Chief of Operations. U.S. Office of Personnel Management Management and Program Analysis Series, 0343 Management and Program Analysis Series, 0343 May 2024

Qualification Standard Series Definition This series includes positions that primarily serve as analysts and advisors to management on the evaluation of the effectiveness of government programs and operations or the productivity and efficiency of the management of Federal agencies or both. Positions in this series require knowledge of: the substantive nature of agency programs and activities; agency missions, policies, and objectives; management principles and processes; and the analytical and evaluative methods and techniques for assessing program development or execution and improving organizational effectiveness and efficiency. Some positions also require an understanding of basic budgetary and financial management principles and techniques as they relate to long range planning of programs and objectives. The work requires knowledge of: • • • application of factfinding and investigative techniques; oral and written communications; and development of presentations and reports. U.S. Office of Personnel Management Management and Program Analysis Series, 0343 May 2024

Titling Nonsupervisory positions primarily concerned with analyzing, evaluating, and/or improving the efficiency of internal administrative operations, organizations, or management are titled Management Analyst. Nonsupervisory positions primarily involved in planning, analyzing and/ or evaluating the effectiveness of line or operating programs are titled Program Analyst. Positions which involve a mix of these functions, where neither is predominant are titled Management and Program Analyst. Positions primarily involved in performing, planning, implementing, executing, managing, and/or reporting of program evaluation activities overseen or conducted by evaluation staff in a Federal agency or external entities under agreement with a Federal agency are titled Program Evaluator. Agencies may supplement the basic position titles by adding parenthetical titles, where necessary, to identify duties and responsibilities which reflect specific knowledge and skills required in the work. Supervisory is prefixed to the title of positions which meet the criteria in the General Schedule Supervisory Guide. Management Analysis Officer, Program Analysis Officer, and Management and Program Analysis Officer titles are established for positions which have responsibility for establishing, planning, and directing programs in their respective functional specializations. U.S. Office of Personnel Management Management and Program Analysis Series, 0343 May 2024

General Occupational Information The intent in establishing this series is to cover staff administrative analytical and evaluative work related to program operations, and management and organizational efficiency and productivity. Positions which require full competence in a particular specialized or subject-matter field for satisfactory performance of the work are excluded from this series. The work of this occupation is typically performed in a staff capacity in that the results of the work support the accomplishment of the principal mission or line program(s) of the agency or organizational component in which the positions are located. In some cases, particularly in the larger agencies, the distinction may not always be readily apparent. For example, the mission or line work of an organizational component may be the development of staffing standards to be used throughout the agency. Positions involved in this work may be considered as performing the line work of the immediate organizational component. However, since the results of the work (i.e., the staffing standards) support accomplishment of the overall programs and mission of the agency, the positions are in fact still performing staff work for the agency. Line programs and positions directly affect the customer typically through delivering products and services. Staff work supports the organization with specialized advisory and support functions. Staff positions affect customers indirectly, to the extent that the support they provide helps line employees improve quality and customer satisfaction. Positions in this series serve as staff analysts, evaluators, and advisors to management on the effectiveness and efficiency with which agencies and their components carry out their assigned programs and functions. Such positions may be found at any organizational level within Federal agencies. The primary purpose of the work is to provide managers with objectively based information for making decisions on the administrative and programmatic aspects of agency operations and management. Positions in this series are concerned with a wide variety of assignments. The work encompasses program analysis, program evaluation, performance measurement, and/or foundational fact-finding. Listed below are some illustrations of the nature of the work and the intended coverage of this series. This list should not be considered as a definitive catalog of all the specific kinds or combinations of work performed by positions in this series. •

advising on the distribution of work among positions and organizations and the appropriate staffing levels and skills mix; U.S. Office of Personnel Management Management and Program Analysis Series, 0343 • • • • • • • • • • • • • • • • • • • May 2024

advising on the potential benefits/uses of automation or technology to improve the efficiency of support or program operations; analyzing and evaluating (on a quantitative or qualitative basis) the effectiveness of program operations or policy in meeting established goals and objectives; analyzing and evaluating functions and activities being considered for conversion to contract operations; analyzing and evaluating proposed changes in mission, operating procedures and delegations of authority; analyzing management information requirements to develop program or administrative reporting systems including the systems specifications, data gathering and analytical techniques, and systems evaluation methodology; analyzing new or proposed legislation or regulations to determine impact on program operations and management; assessing program evaluability; complying with the applicable mandates related to the management of data including Federal requirements for equipment, software, or interconnected systems used to collect, process, store, maintain, disseminate, disclose, and dispose of evaluative data or information; conducting program evaluations that are impartial, independent and credible and that meet Federal and professional standards; conducting studies of employee/organizational efficiency and productivity and recommending changes or improvements in organization, staffing, work methods, and procedures; consulting with technical work group/expert groups, to obtain input on the merit of the evaluation design, its implementation, planned analyses, and findings; coordinating evaluation processes and products; designing rigorous and feasible evaluations that address the identified research questions; determining resource requirements (for example, staffing, funding, equipment) based on program or project objectives or operational needs; developing comprehensive technical reports of findings providing sufficient evaluative detail and rigor; developing a stakeholder engagement strategy and plan to increase involvement of stakeholders throughout the evaluation lifecycle; developing life cycle cost analyses of projects or performing cost-benefit or economic evaluations of current or projected programs; developing management and/or program evaluation plans, procedures, and methodology; developing new or modified administrative program policies, regulations, goals, or objectives;

U.S. Office of Personnel Management Management and Program Analysis Series, 0343 • • • • • • • • • • • • • • • • • • May 2024

developing procedures and systems for establishing, operating, and assessing the effectiveness of administrative control systems such as those designed to prevent waste, loss, unauthorized use, or misappropriation of assets; developing workload based staffing standards to determine organizational staffing levels; engaging expert consultants throughout the evaluation lifecycle as part of technical work groups, expert groups, or peer reviews; evaluating the efficiency or effectiveness of programs, policies, or organizations; evaluating the impact of changes to laws, regulations, policies, standards, or procedures, including for example costs or benefits; evaluating and advising on the organization, methods, and procedures for providing administrative support systems such as records, communications, directives, forms, files, and documentation; identifying and developing data required for use in the management and direction of programs; identifying resources (staff, funding, equipment, of facilities) required to support varied levels of program operations; identifying strengths and limitations of the evaluation design and methods; managing evaluations to ensure they are impartial, independent, and credible and meet Federal and professional standards; performing management surveys to determine compliance with regulations, policies, procedures, sound management practices, and effective utilization of staff; planning, implementation, management, and reporting of activities; planning the design and methods used to conduct evaluation studies aligned to achieve intended goals of the studies; planning the dissemination strategy for sharing information and data timely using appropriate mechanisms and practices; planning, organizing, and overseeing the operation of a data collection survey; reporting systems including the systems specifications, data gathering and analytical techniques, and systems evaluation methodology; researching and investigating new or improved business and management practices for application to programs or operations; reviewing administrative audit and investigative reports to determine appropriate changes or corrective action required.

U.S. Office of Personnel Management Management and Program Analysis Series, 0343 May 2024 Impact of Automation Automation, computers, information technology (IT), and their widely varied applications are valuable tools for analytical and evaluative work related to program operations, and management and organizational efficiency and productivity. Automation increases the ability of Management and Program Analysts to perform a wide variety of tasks. Employees access files, initiate and track projects, analyze data, and generate reports. They input, store, and retrieve data in multiple formats. They also use the Internet to search for information pertaining to assignments. Although the incumbents use computers to perform basic work processes, knowledge of the rules and processes to perform the work remains the paramount subject-matter knowledge required. The kind of automation tools involved, and the skill required to use them, generally replace or supplement work methods and techniques previously performed through manual or machine-enhanced processes. Although computers are used to facilitate work within this series, the use of automation does not change the primary purpose of the work. Proper classification of positions is based on the relevant knowledge and skills required to perform the primary duties of the position. U.S. Office of Personnel Management Management and Program Analysis Series, 0343 May 2024 Additional Occupation Considerations

Some positions may include work requiring knowledge and skills typically associated with the Management and Program Analysis Series. However, a closer look at the work may reveal classification to this series may not be appropriate. The General Series Determination Guidelines section of this flysheet offers guidance on selecting the most appropriate series. The following table provides examples of work similar to that performed in the Management and Program Analysis Series, 0343, but not to the extent the paramount knowledge required, the reason for the position’s existence, the mission and/or function of the organization, and the recruitment sources for the best qualified candidates warrant classification to this series. For further guidance, refer to OPM’s publication The Classifier’s Handbook. If Work Involves… See This Standard or Series Definition: Positions that manage, supervise, administer, advise on, or deliver human resources management products or services. Work includes, but is not limited to the specialties of: Information Systems; Military; Classification; Recruitment & Placement or Recruitment or Placement; Employee Benefits; Human Resource Development; Performance Management; Employee & Labor Relations or Employee Relations or Labor Relations. Human Resources Specialist, 0201 Analytical positions which have as their paramount qualification requirement specialized subject-matter knowledge and skills equivalent to those required of a fully trained employee in the particular subject-matter occupations. Such positions should be classified in the appropriate specialized series (e.g., Social Science Series, 0101, Health Scientist, 0601, etc.), or if none is established, in the Miscellaneous Administration and Program Series, GS-0301.

Positions primarily engaged in conducting, supervising, or managing the line program activities or functions of the employing agency. Depending on the specific knowledges and skills required, such positions should be classified either in the appropriate subject-matter series, the Program Management Series, 0340, or the Miscellaneous Administration and Program Series, GS-0301. U.S. Office of Personnel Management Management and Program Analysis Series, 0343 May 2024 Additional Occupational Considerations Continued If Work Involves… See This Standard or Series Definition: Positions involved in supervising or performing clerical and technical work in support of management analysis and program analysis, the purposes of which are to evaluate and improve the efficiency, effectiveness, and productivity of organizations and programs. The work requires a practical knowledge of the purposes, methods, and techniques of management analysis and/or program analysis and the structures, functions, processes, objectives, products, services, resource requirements, and similar features of Government programs and organizations. Management and Program Clerical and Assistance Series, GS0344 Positions that perform professional accounting and auditing knowledge, standards, and principles when performing these duties: Auditing, 0511

• advising on, supervising, or performing work consisting of a systematic examination and appraisal of financial records, financial and management reports, management controls, policies and practices affecting or reflecting the financial condition and operating results of an activity; • analyzing work related to developing and executing audit policies and programs; • conducting performance audits; or • conducting activities related to the detection of fraud, waste and abuse. Positions that perform, advise, or supervise work in any of the phases of the budget administration process when such work requires knowledge of and skill in applying budget- related laws, regulations, policies, precedents, methods, and techniques. Budget Analysis, 0560 Positions managing, supervising, leading, and/or performing professional engineering and scientific work to determine, evaluate, predict, and advise on effective ways for an organization to use its production factors (i.e., people, equipment, materials, information, and energy) to make or process a product or provide a service. Industrial Engineering, 0896 U.S. Office of Personnel Management Management and Program Analysis Series, 0343 May 2024 Additional Occupational Considerations Continued If Work Involves… See This Standard or Series Definition:

Positions that direct or perform analytical and evaluative work requiring a comprehensive knowledge of (1) the theory and principles of finance applicable to the full range of financial operations and transactions involved in the general activities of the various types of business corporate organizations; (2) the financial and management organization, operations, and practices of such corporate organizations; (3) pertinent statutory or regulatory provisions; and (4) related basic economic, accounting, and legal principles. Financial Analysis Series, GS-1160 Positions that manage, supervise, lead, or perform scientific work that involves designing, developing, and adapting mathematical, statistical, econometric, and other scientific methods and techniques. The work also involves analyzing management problems and providing advice and insight about the probable effects of alternative solutions to these problems. The primary requirement of the work is competence in the rigorous methods of scientific inquiry and analysis. Operations Research, 1515 Positions that manage, supervise, lead, or perform scientific work or provide professional consultation in applying statistical theories, techniques, and methods to gather, analyze, interpret, and/or report quantified information. Statistics, 1530 Analytical positions for which the paramount requirement is knowledge of IT principles, concepts, and methods such as but not limited to: data storage, database management, system analysis, policy and planning, software applications, networking. Information Technology Management, 2210 U.S. Office of Personnel Management Management and Program Analysis Series, 0343 May 2024

Crosswalk to the Standard Occupational Classification The Office of Management and Budget requires that all Federal agencies that collect occupational data use the Standard Occupational Classification (SOC) system for statistical data reporting purposes. The Bureau of Labor Statistics uses SOC codes for the National Compensation Survey and other statistical reporting. OPM and other Federal agencies maintain a “crosswalk” between OPM authorized occupational series and the SOC codes to serve this need. This requirement and these SOC codes have no effect on the administration of any Federal human resources management system. The information in this table is for information only and has no direct impact on classifying positions covered by this series. The SOC codes shown here generally apply only to non-supervisory positions in this occupation. As changes occur to the SOC codes, OPM will update this table. More information about SOC is available at http://stats.bls.gov/soc. Federal Occupational Series and Position Title and The Related Standard Occupational Classification System Code Federal Occupational Series Standard Occupational Classification Code Based on Occupational Series Managemen 13-1110 t and Program Analysis Series, 0343 Management Analysts Position Title Standard Occupational Classification Code Based on Position Title Management Analyst 13-1111 Management Analysts Management and Program Analyst 13-1199 Business Operations Specialists, All Other Managemen 13-1080 t and Program Analysis Series, 0343 Logisticians and Project Management Specialists Program Analyst 13-1082 Project Management Specialists Managemen 15-2050 t and Program Analysis Series, 0343 Data Scientists All 15-2051 Data Scientists U.S. Office of Personnel Management Management and Program Analysis Series, 0343 May 2024

Grading Instructions This flysheet does not provide occupation-specific grading criteria. Specific grade level criteria for positions in this occupation have not been developed. As a general rule, positions included in this series should be evaluated by reference to classification standards for related kinds of work. (See the Introduction to the Position Classification Standards.) Nonsupervisory positions at grade GS-9 and above are evaluated by reference to the Administrative Analysis Grade Evaluation Guide. Due to the diversity of assignments in this occupation, users should not seek a one-to-one correspondence between the duties of a particular position and the factor level descriptions and work illustrations in the guide. Instead, users should strive to match the intent of the various factor levels and seek to locate concepts and examples which are comparable. For trainee and developmental positions GS-5 and GS-7, follow the guidance provided in the Administrative Analysis Grade Evaluation Guide. Evaluate leader positions using the General Schedule Leader Grade Evaluation Guide. Evaluate supervisory positions using the criteria in the General Schedule Supervisory Guide. U.S. Office of Personnel Management Management and Program Analysis Series, 0343 May 2024

Appendix A – Historical Record and Explanatory Material This appendix describes the development of this Position Classification Flysheet for the Management and Program Analysis Series. This section will highlight some key dates and milestones and provide information about the focus groups and addresses concerns expressed by reviewing agencies. Key Dates and Milestones The U.S. Office of Personnel Management (OPM) worked in consultation with the Office of Management and Budget (OMB) to implement specific requirements of Public Law 115-435, Foundations for Evidence-Based Policymaking Act of 2018 (Evidence Act). In accordance with the Evidence Act, the OPM conducted a study to identify key skills and competencies, establish or update an occupational series, and establish a new career path for program evaluation. Issuances for the skills and competencies and the career path for program evaluation work are separate from the classification policy guidance. The Office of Management and Budget (OMB) Memorandum M-20-12, “Phase 4 Implementation of the Foundations for Evidence-Based Policymaking Act of 2018: Program Evaluation Standards and Practices,” provided examples of leading practices for agencies to draw upon as they build evaluation capacity, develop policies and procedures, and carry out evaluations to support evidence-based policymaking. The practices were selected for their potential usefulness in supporting agencies’ implementation of the program evaluation standards described in OMB M-20-12 (Appendix C). This set of practices outlined in M-20-12 is not meant to be exhaustive of the efforts an agency, office, or program could, should, or must undertake to ensure the quality and integrity of evaluation or adherence to legal or other requirements. The Evidence Act established the role of the agency Evaluator Officer. Evaluation Officers have authority and responsibility for providing leadership over agencies' evaluation and Learning Agenda activities. As stated in the Evidence Act, the head of each CFO Act agency must designate a senior employee of the agency as the Evaluation Officer of the agency. This shall be done without regard to political affiliation and based on demonstrated expertise in evaluation methodology and practices and appropriate expertise to the disciplines of the agency. Non-CFO Act agencies, as well as sub-agencies, operational divisions, and bureaus of CFO Act Agencies are strongly encouraged to designate a qualified Evaluation Officer as appropriate. In the context of the Evidence Act “Program evaluation” and “evaluation” are synonymous according to OMB (see OMB M-20-12, p 11). According to 5 U.S.C. § 311(3), the term “evaluation” signifies “an assessment using systematic data collection and analysis of one or more programs, policies, and organizations intended to assess their

U.S. Office of Personnel Management Management and Program Analysis Series, 0343 May 2024 effectiveness and efficiency”. “Evaluation”, “program,” and other key terms are further described in OMB M-20-12, Appendix A. In addition, the types of evaluation and some of the methods, and analysis used in evaluation are provided below (OMB M-20-12). • Formative Evaluation is typically conducted to assess whether a program, policy, or organizational approach-or some aspect of these-is feasible, appropriate, and acceptable before it is fully implemented. It may include process and/or outcome measures. However, unlike outcome and impact evaluations, which seek to answer whether the program, policy, or organization met its intended goals or had the intended impacts, a formative evaluation focuses on learning and improvement and does not aim to answer questions of overall effectiveness. • Impact Evaluation assesses the causal impact of a program, policy, or organization, or aspect thereof, on outcomes relative to those of a counterfactual. In other words, this type of evaluation estimates and compares outcomes with and without the program, policy, or organization, or aspect thereof. Impact evaluations include both experimental (i.e., randomized controlled trials) and quasi-experimental designs. An impact evaluation can help answer the question, "does it work, or did the intervention lead to the observed outcomes?" • Intervention is a combination of program elements or strategies related to the design and implementation of programs and policies designed to produce specific results. •

Outcome Evaluation measures the extent to which a program, policy, or organization has achieved its intended outcome(s) and focuses on outputs and outcomes to assess effectiveness. Unlike impact evaluation above, it typically cannot discern causal attribution. Importantly, it is distinct from, but complementary to, performance measurement, as noted below. An outcome evaluation can help answer the question "were the intended outcomes of the program, policy, or organization achieved?" OMB M-20-12 Per Section 101(a) of the Evidence Act, program evaluation is defined as “an assessment using systematic data collection and analysis of one or more programs, policies, and organizations intended to assess their effectiveness and efficiency.” Program Evaluator work includes, but is not limited to: planning, managing, or conducting specific evaluation activities using complex and appropriate scientific methods; U.S. Office of Personnel Management Management and Program Analysis Series, 0343 May 2024

summarizing evaluation findings for agencywide/particular programs or policies; supporting other offices within an agency to interpret evaluation findings; and bringing evaluation-related evidence to bear in decision-making. The purpose of program evaluation is to assess the effectiveness or efficiency of a program, policy, organization or aspect of these. Evaluation can look beyond the program, policy, or organizational level to include assessment of particular projects or interventions within a program, for example, or particular aspects of a policy or functions or units within an organization. Evaluations may address questions related to the implementation or institution of a program, policy, or organization; the effectiveness or impact of specific strategies related to or used by a program, policy, or organization; and/or factors that relate to variability in the effectiveness of a program, policy, or organization or strategies of these. Evaluations can also examine questions related to understanding the contextual factors surrounding a program, as well as how to effectively target specific populations or groups for a particular intervention. They can provide critical information to inform decisions about current and future programming, policies, and organizational operations. Importantly, evaluation can be used for learning and improvement, as well as for accountability purposes.

As described above, there are different types of program evaluation, each of which answers different types of questions and requires different methodological approaches. Table A.1 in OMB Memorandum, M-21-27: Evidence-Based Policymaking: Learning Agendas and Annual Evaluation Plans lays out a series of evidence-building questions, indicates what type of evaluation may be most appropriate, and suggests methodological approaches. Program evaluators use their expertise to refine evidence-building questions, identify the appropriate evaluation type, and select the scientifically-sound method that aligns to that evaluation type and question. To do so, they bring understanding and awareness of a range of methodological approaches, including, but not limited to: pilot projects, ethnography, structured observations, qualitative interviews and focus groups, time studies, statistical analysis of program or participant data, community-based participatory research, randomized controlled trials, regression discontinuity design, propensity score matching, and difference-in-difference approaches. In support of the requirements of the Evidence Act, OPM conducted a comprehensive study including fact-finding activities, data analysis, and stakeholder engagement with Federal agencies. The focus of the study was the staff role supporting Evaluation Officers and creation of this role. An environmental scan was conducted to explore classification of, and competencies and tasks needed for the performance of, program evaluation work. To fulfill the requirement set forth in Public Law 115-435, OPM used an evidence-based approach to develop policy aligned with the requirements of the Evidence Act. A comprehensive review of the data and information collected from Federal agencies

U.S. Office of Personnel Management Management and Program Analysis Series, 0343 May 2024

and our environmental scan informed the classification policy determination and identified the skills needed to perform program evaluation work. The following information will outline the main tenets of the classification work performed for this study. The key skills, competencies, and duties were identified by Federal agency and human resources subject matter experts as needed for performing program evaluation work governmentwide along with the requirements outlined in the Evidence Act and corresponding OMB guidance. OPM reviewed position descriptions, job analysis, job announcements and other pertinent materials and information to collect information on program evaluation work. OPM held focus groups and stakeholder engagement activities with Federal agencies and key stakeholders including agency program evaluation and human resources subject matter experts identified by OMB and the occupational study. As a result of stakeholder engagement activities program evaluation work was identified as being performed as both a full-time and collateral duty by agencies. OPM included exclusion criteria based on the description of program evaluation work provided by OMB guidance to identify subject matter experts for the study along with the group of subject matter experts OMB identified with program evaluation expertise and knowledge. The subject matter experts identified by OMB served as a core reference group for this study. The work described in this standard to be titled “Program Evaluator” include full-time program evaluation work. Program Evaluation work was also described as including elements of multidisciplinary work based on OMB guidance (M-21-27). As further elaborated in OMB Memo M-20-12, program evaluator work includes the application of scientific principles and techniques to conduct and oversee the execution of evaluation studies to assess effectiveness and efficiency. The OPM study supported the multidisciplinary nature of program evaluation work. Agencies reported program evaluation work being performed in various occupational groups including positions in the following occupational groups and families: • • • • • • •

0100 – Social Science, Psychology and Welfare Group 0200 – Human Resources Management Group 0300 – General Administrative, Clerical and Office Services Group 0400 – Natural Resources Management and Biological Sciences Group 0500 – Accounting and Budget Group 0600 – Medical, Hospital, Dental, and Public Health Group 0800 – Engineering and Architecture Group OPM defines multidisciplinary work as a position involving duties and responsibilities closely related to more than one discipline. As a result, the position could be classifiable to two or more occupational series including in this case series involving professional (0101 series) and/or nonprofessional (0343) work based on the specific knowledge, skills and abilities required by the individual position. The nature of the U.S. Office of Personnel Management Management and Program Analysis Series, 0343 May 2024

work is such that persons with training and experience in either two or more occupations may often be considered well-qualified to do the work (Interpretive Guidance for Cybersecurity Positions, pp 21 -22). In accordance with OMB Memo M20-12 (p. 16), specialized program evaluation knowledge and expertise may come from advanced degrees in program evaluation or related fields with concentration studies in program evaluation or direct experience designing and conducting program evaluation may supplement or substitute for advanced degrees. Related fields may include public administration, public policy, implementation science, statistics, economics, social sciences, or other relevant academic disciplines. The duties and responsibilities assigned to most positions are covered by one occupational series, and the series determination is clear. For these positions, the series represents the primary work of the position, the highest level of work performed, and the paramount qualifications required. Some positions, however, are a mix of duties and responsibilities covered by two or more occupational series and classified by more than one standard or guide. Often the appropriate series for these positions is a general series for the occupational group covering the type of work performed. For positions whose duties fall in more than one occupational group, the most appropriate series for the position depends on consideration of a number of factors. For many of these positions the grade controlling duties will determine the series. Sometimes, however, the highest level of work performed does not represent the most appropriate series, and the series can be determined only after considering the paramount qualifications required, sources of recruitment and line of progression, the reason for establishing the position, and the background knowledge required. Classification of program evaluation work requires distinguishing between professional and administrative work. This requires the proper analysis of positions duties/responsibilities and the qualifications required to perform the work upon entry into the position. Professional and administrative work include specific characteristics as described below. See also the Introduction to the Position Classification Standards (p. 9) for additional information. Professional Work Professional work requires knowledge in a field of science or learning characteristically acquired through education or training equivalent to a bachelor’s or higher degree with major study in or pertinent to the specialized field, as distinguished from general education. Work is professional when it requires the exercise of discretion, judgment, and personal responsibility for the application of an organized body of knowledge that is constantly studied to make new discoveries and interpretations, and to improve data, materials, and methods. When professional work

U.S. Office of Personnel Management Management and Program Analysis Series, 0343 May 2024

is not inherent in a position, the position is classified in a nonprofessional series, based on the duties and responsibilities assigned and the qualifications required to do the work. For example, the Social Sciences Series, GS-0101 involve professional work. Positions classified in the Social Sciences GS-0101 series covers positions the duties of which are to advise on, administer, supervise, or perform research or other professional and scientific work in one or any combination of the social sciences when such work is not classifiable in other series of the Social Science, Psychology, and Welfare Group, 0101. Administrative Work Administrative work involves the exercise of analytical ability, judgment, discretion, and personal responsibility, and the application of a substantial body of knowledge of principles, concepts, and practices applicable to one or more fields of administration or management. While these positions do not require specialized education, they do involve the type of skills (e.g., analytical, research, writing, judgment) typically gained through a college level education, or through progressively responsible experience. Employees engaged in administrative work are concerned with analyzing, evaluating, modifying, and developing the basic programs, policies, and procedures which facilitate the work of Federal agencies and their programs. They apply a knowledge of administrative analysis, theory, and principles in adapting practice to the unique requirements of a particular program. For example, the Management and Program Analysis Series, GS-0343, includes administrative work and positions that primarily serve as analysts and advisors to management on the evaluation of the effectiveness of government programs and operations, the productivity and efficiency of the management of Federal agencies, or both. Positions in the 0343 series require knowledge of the substantive nature of agency programs and activities; agency missions, policies, and objectives; management principles and processes; and the analytical and evaluative methods and techniques for assessing program development or execution and improving organizational effectiveness and efficiency.

U.S. Office of Personnel Management Management and Program Analysis Series, 0343 May 2024 Below is an additional comparison of professional and administrative work. • Professional Work requires specialized educational preparation as a prerequisite to performing the work. There are situations in which an employee meets the formal education requirements for a particular professional field but does not perform professional work. This may be due to a lack of professional work to be done, or it may be because the organization and structure of the assignment does not require a professionally qualified employee. In such situations, the position is classified in an appropriate nonprofessional series, based on the duties and responsibilities assigned and the qualifications required to do the work. • Administrative Work does not require specialized educational preparation, it does require a high degree of qualitative and/or quantitative analytical skills, the ability to research problems and issues, written and oral communication skills, and the application of mature judgment in problem solving.

It is important to note that positions formerly classified to the Program Analysis Series, GS-0345 were merged with the 0343 in August 1990. The current 0343 series combined, in one occupation, positions which were found to perform similar duties and require many of the same, or closely related, knowledge, skills and abilities. In summary the series determination is informed by agency needs and missions as well as the knowledge, skills, and abilities (KSAs)/competencies needed to perform the work. The KSAs/competencies drive the path to skill acquisition. Agencies have reported challenges and successes using different occupational series to classify Program Evaluation work. As defined in the Evidence Act, program evaluation is “an assessment using systematic data collection and analysis of one or more programs, policies, and organizations intended to assess their effectiveness and efficiency.” Also, the OMB Guidance directed the appointment of Evaluation Officers with expertise in evaluation methods and practices and documented expertise to the disciplines of the agency.” These core factors were considered as well as specific agency needs. OPM collected data on the classification of program evaluation work directly from Federal agencies and external credible data and information as well. OMB Memorandum M-1923 identified the roles and responsibilities of the agency Evaluation Officer, while OMB Memo M-21-27 further elaborated on the role by emphasizing that “the role should be filled by a senior career employee with skills and expertise to maintain principles of scientific integrity throughout the evaluation process, ensure adherence to the agency evaluation policy, and maintain the standards in OMB M-20-12.” (p. 4). OPM’s role was U.S. Office of Personnel Management

Management and Program Analysis Series, 0343 May 2024

to extrapolate these functions to staff employees at the agency headquarters level and when appropriate at other levels within the organization to support the Evaluation Officer function and program evaluation work performed by the agency. Our study supported OMBs description of program evaluation work being aligned with disciplines of an agency. For example, agencies with healthcare missions may classify work in medical and healthcare occupational series and agencies with missions aligned with social sciences may classify their program evaluation in a social sciences’ occupational series. Therefore, the work is multidisciplinary in nature. Multidisciplinary includes specific characteristics that is not necessarily adaptable to one construct. Instead, the work is reflective of experienced gained through various means and represents a compilation of functions and work roles. These work roles include a mixture of educational and non-educational requirements or experiences that may be acquired academically or in a professional setting. The skills and qualification requirements for program evaluation work are driven by the classification series determination. OPM found that program evaluation work is multidisciplinary, and the work outlined in the Evidence Act and OMB Guidance M-2012 is covered by the Management and Program Analysis Series, GS-0343, and other series, including the Social Science Series, 0101. Other series may also be used to classify program evaluation positions when supported by a job analysis and aligned with the agency’s mission. Therefore, agencies must determine through a job analysis the qualification requirements for program evaluation work classified in the 0343 Series in conjunction with OPM General Schedule Qualifications Policy (i.e., General Schedule Qualifications Operating Manual and the Administrative Group Qualifications Standard). Agencies should follow qualification policy guidance for other series outside of the 0343 series, including the 0101 Social Science Series and others when used to classify program evaluation work. The work identified through the environmental scan, focus groups, surveys, review and analysis of 84 position descriptions supplied by Agencies, and 21 OPM appeal decisions for similar work discussed previously indicate program evaluation work is multidisciplinary and aligned with the main tenets of program evaluation work. Program evaluation work is Administrative when covered by the Management and Program Analysis Series, GS-0343 and Professional when subject matter knowledge is needed to perform the work when supported by a job analysis and aligned with the agency’s mission. The duties/responsibilities/tasks and qualifications required to perform program evaluation work identified as Administrative through the environmental scan, focus groups and surveys of typical Program Evaluator and Program Evaluation work definitions provided in Evidence-Based Policymaking Act of 2018 (5 U.S.C. 311 (3) and

U.S. Office of Personnel Management Management and Program Analysis Series, 0343 May 2024 OMB M-20-12 compare favorably to an existing series and are directly covered by the intended coverage of Management and Program Analysis Series, 0343. Positions performing work requiring specialized program subject matter knowledge are most appropriately classified in the respective specialized program subject matter series, for example, the Social Science Series, 0101, which would be driven by qualification requirements established by a job analysis in accordance with 5 CFR § 300.103 Basic requirements. U.S. Office of Personnel Management