SOUTH DAKOTA BOARD OF REGENTS Academic and Student Affairs AGENDA ITEM: 5 J DATE: December 4-6, 2018 ****************************************************************************** SUBJECT SDBOR Peer List Update Project CONTROLLING STATUTE, RULE, OR POLICY None BACKGROUND / DISCUSSION SDBOR central office staff have worked with campus staff over the last several months to assemble institutional peer sets for the Regental system. These peer sets will be used for general purposes in future SDBOR analyses where peer comparisons would be informative and appropriate. This item describes the methodology used in SDBOR s most recent peer selection process, and enumerates each institution s final peer set. This project constitutes an update of SDBOR s approved peer sets, which were originally established in 2014. All Regental campuses were given the option to participate in the current peer update project. BHSU, DSU, and NSU opted to participate, while SDSMT, SDSU, and USD opted to keep their existing (2014) peer sets. For an example use of SDBOR peer lists, visit the interactive SDBOR Peer Analysis Dashboard at https://www.sdbor.edu/dashboards. IMPACT AND RECOMMENDATION Using the updated parameters requested by BHSU, DSU and NSU a new set of peers have been identified which will be used moving forward for all peer related analyses. The matrix depicted in Attachment II highlights the set of proposed peers from 2014 and those identified for each institution in 2018. During the process implemented in 2014, institutions were provided a set of close and distant peers to select from. The Carnegie classification for each potential peer are also included as a reference point. (Continued) ****************************************************************************** DRAFT MOTION 20181204_5-J: I move to accept the updated peer institutions selected by BHSU, DSU and NSU and schedule additional reviews to update the current slate of peers in 2023 for all institutions.
SDBOR Peer List Update Project December 4-6, 2018 Page 2 of 2 If approved by the Board of Regents, the peer selection process will only be replicated every five years to ensure consistency in reporting of peer data with stakeholders across the state. Board staff recommends approval. ATTACHMENTS Attachment I Comparison of 2014 and 2018 Peers Attachment II Special Data Analysis SDBOR Peer List Update Project
ATTACHMENT I 3 *** Special Data Analysis *** SDBOR Peer List Update Project Some measures of institutional performance can be difficult to interpret without proper context. For example, knowing that University X spends $8,000 annually on instruction per FTE student means little in the absence of analogous information from other similar institutions. While SDBOR analyses often rely on inter-system comparisons, there are instances due to differences in institutional size, type, mission, etc. when these comparisons are not optimal. The following report summarizes the analytic process used to identify appropriate peer sets for each regental university. Purpose Most regental universities maintain their own lists of institutional peers for use in various internal analyses. Yet these peer groups sometimes are not suitable for use at the system level, given the variety of purposes and methods by which these lists were created. Some institutions, for example, have established multiple peer groups (with separate lists for current peers and aspirational peers ), while others have combined all peer types within a single list. In some cases, peers have been selected manually by expert judgment, and in other cases peer lists have emerged from rigorous statistical investigation. Because of this diversity of approaches, the SDBOR central office typically has relied on its own independent peer lists to ensure some degree of uniformity in the processes used to select these groups. Since the existing regental peer lists were established in 2014, the current project aimed to update these lists based on current data. 1 Moving forward, it is expected that these peer lists will continue be updated every five years. The current peer selection process was undertaken with several assumptions in mind: System-level peer groups should be established using a systematic methodology to help ensure that each institution is compared to a similarly-constituted group. System-level peer groups should reflect institutional priorities, and institutions should have an active voice in selecting their peers. System-level peer groups should be oriented toward present conditions, not future conditions. System-level peer groups serve a limited analytic role, and should not be seen to replace the peer groups campuses have developed on their own. 1 All regental campuses were given the option to participate in the current peer update project. BHSU, DSU, and NSU opted to participate, while SDSMT, SDSU, and USD opted to keep their existing (2014) peer sets.
ATTACHMENT I 4 Peer Selection Process Overview In keeping with the approach used in the original 2014 peer selection project, the current peer update project identified new campus peers through a hybrid approach combining: Phase 1: An initial screening conducted by the system office: By filtering and cluster analyzing IPEDS data, a full list of each institution s universe of potential peers would be culled to a short list of roughly 15-25 plausible peers. Phase 2: Final selection based on campus-level review: Each university would examine its short list of plausible peers and designate 5-10 of these institutions as final peers. Further information about each of these steps is provided on the following pages. Sidebar: Cluster Analysis As mentioned above, this project involved the use of a technique known as cluster analysis. Generally speaking, cluster analysis is a broad term for a family of procedures that are designed to reveal patterns within complex data. The main goal of cluster analysis is to identify homogenous groups based on multivariate measurements. In other words, cluster analysis is used to identify similar cases (in this instance, colleges and universities) based on those cases measured characteristics. Specifically, this analysis incorporated hierarchical agglomerative cluster analysis as its primary methodology. Hierarchical methods of cluster analysis are distinguished from other forms of cluster analysis by their focus on discovering taxonomic relationships between cases. As in biological classification, hierarchical cluster analysis attempts to group observations into descending ranks of resemblance. Consequently, this technique assisted in the process of narrowing down large populations of US colleges and universities into institutional groupings that increasingly resembled SDBOR institutions.
ATTACHMENT I 5 Process Details Phase 1: Initial Screening For each regental university, the screening process proceeded in three stages. The first stage was identical for all universities; stages two and three were customized for each campus. 2 Screening Stage (1) Gathering IPEDS data on the full universe of relevant postsecondary institutions (2) Filtering the above universe to a population of possible peers Description Institution-level IPEDS data were collected on all public, fouryear, degree-granting, non-tribal institutions in the United States. Data were gathered for a wide range of indicators, including institutional descriptors, student characteristics, enrollments, degrees conferred, tuition and fees, and financial data. All analyses were conducted using three-year-averaged data (from the three most recent years). For each SDBOR campus, the above universe of all US public, four-year, degree-granting, non-tribal institutions was filtered to a population of possible peers. Filtering was based on rough parameters that reflected the considerations recommended by campus staff. By institution, these criteria included: B D N M S U Enrollment Carnegie classification Academic emphasis (computers) Academic emphasis (STEM) Geography (western states) Following this procedure, each institution s list of possible peers contained roughly 75-150 institutions. (3) Clustering the above population to a short list of plausible peers In the final stage, lists of possible peers were cluster analyzed to identify groups of institutions that most closely resembled each SDBOR campus. Cluster analytic models were developed in collaboration with campuses in iterative fashion, meaning that campuses were given several opportunities to select the criteria that would be used in the peer search. Reflecting campus suggestions, final clustering models considered the following areas: (continued on next page) 2 As noted above, only BHSU, DSU, and NSU elected to participate in the current project. All information depicted in this report for SDSMT, SDSU, and USD is carried over from the 2014 selection summary report.
ATTACHMENT I 6 B D N M S U Total core revenues per FTE State appropriations per FTE Tuition revenue per FTE student Endowment per FTE Research spending per FTE Fall FTE enrollment UGs who are part-time (%) Total degrees awarded STEM degrees (%) Number of prof. programs Programs via distance (%) ACT scores Pell recipients (%) Following this procedure, each institution s list of plausible peers contained roughly 15-25 institutions. Phase 2: Final Selection In the second phase of the project, short lists of plausible peers were provided to AAC members at each campus. Each university was asked to select from its list of plausible peers a subset of 5-10 institutions that would serve as the institution s final peer set; each institution could select any number of peers within this range. Each institution was free to finalize its peer list by any method(s) it wished to use. Options included but were not limited to: Hand-selecting institutions based on qualitative impressions of similarity Cross-referencing with internal peer lists to identify overlapping peers Choosing peers based on additional data analysis Data used in the screening phase of the project were available to the universities on request in the event that campus staff wished to make use of them in the selection phase. Final peer lists were submitted by AAC members to the central office in April-May of 2018. These lists are shown in Appendix A.
ATTACHMENT I 7 Appendix A Final Peer Lists Table A1. Final Peer Lists BHSU Colorado State University-Pueblo, CO East Central University, OK Fort Lewis College, CO Louisiana State University-Shreveport, LA Shawnee State University, OH University of Wisconsin-Parkside, WI Western State Colorado University, CO DSU Arizona State University-Polytechnic, AZ University of Alabama in Huntsville, AL University of Illinois at Springfield, IL University of Maryland-Baltimore County, MD University of Michigan-Dearborn, MI West Virginia University Institute of Technology, WV NSU Bemidji State University, MN Black Hills State University, SD Dakota State University, SD Minnesota State University Moorhead, MN Minot State University, ND Northwestern Oklahoma State University, OK Oregon Institute of Technology, OR Southern Oregon University, OR The University of Texas of the Permian Basin, TX Western State Colorado University, CO SDSMT Michigan Technological University, MI Missouri University of Science and Technology, MO Montana Tech of the University of Montana, MT SDSU Montana State University (Bozeman), MT North Dakota State University, ND Oklahoma State University, OK University of Idaho, ID University of Nebraska Lincoln, NE University of Nevada Reno, NV University of North Dakota, ND Utah State University, UT USD Southern Illinois University at Carbondale, IL University of Idaho, ID University of Mississippi, MS University of Missouri Kansas City, MO University of Montana, MT University of North Dakota, ND University of Rhode Island, RI University of Vermont, VT
2014 Peers Carnegie Classification 2018 Peers Carnegie Classification BHSU Louisiana State University Shreveport, LA* Adams State University, CO Bemidji State University, MN Eastern Oregon University, OR Minot State University, ND* Montana State University Billings, MT Southwestern Oklahoma State University, OK Eastern New Mexico University, NM Fairmont State University, WV Georgia Southwestern State University, GA Auburn University at Montgomery, AL Delta State University, MS East Central University, OK Francis Marion University, SC Henderson State University, AR Indiana University Northwest, IN Mansfield University of Pennsylvania, PA Mississippi University for Women, MS Northwestern Oklahoma State University, OK Southeastern Oklahoma State University, OK Southern Arkansas University Main Campus, AR Bemidji State University, MN* Citadel Military College of South Carolina, SC* Shepherd University, WV* Southern Oregon University, OR* University of Montevallo, AL* University of South Florida- St. Petersburg, FL* University of Wisconsin, Superior, WI* DSU University of Illinois at Springfield, IL Eastern Oregon University, OR* Louisiana State University Shreveport, LA* Northwest Missouri State University, MO Oregon Institute of Technology, OR* Southern Polytechnic State University, GA SUNY Institute of Technology at Utica-Rome, NY California State University Channel Islands, CA Citadel Military College of South Carolina, SC University of Hawaii at Hila, HI The University of Texas of the Permian Basin, TX Vermont Technical College, VT Lake Superior State University, MI Minot State University, ND Northwest Missouri State University, MO Baccalaureate A&S Baccalaureate/Associates BHSU Louisiana State University-Shreveport, LA Colorado State University-Pueblo, CO East Central University, OK Fort Lewis College, CO Shawnee State University, OH University of Wisconsin-Parkside, WI Western State Colorado University, CO Fairmont State University, WV Indiana University Kokomo, IN Keene State College, NH Southwest Minnesota State University, MN DSU University of Illinois at Springfield, IL Arizona State University-Polytechnic, AZ University of Alabama in Huntsville, AL University of Maryland-Baltimore County, MD University of Michigan-Dearborn, MI WV University Institute of Technology, WV Christopher Newport University, VA St. Mary s College of Maryland, MD University of Mary Washington, VA University of Minnesota Morris, MN University of North Carolina at Asheville, NC Doctoral, Higher Research Doctoral, Higher Research ATTACHMENT II 8
Shepherd University, WV The University of West Florida, FL Fairmont State University, WV Lander University, SC Mansfield University of Pennsylvania, PA Massachusetts College of Liberal Arts, MA Morrisville State College, NY SUNY College at Old Westbury, NY SUNY College of Agriculture and Technology at Colbeskill, NY SUNY College of Technology at Alfred, NY Georgia Southwestern State University, GA Northwestern Oklahoma State University, OK Southeastern Oklahoma State University, OK Indiana University Kokomo, IN* Lewis-Clark State College, ID* Macon State College, GA* University of Houston Victoria, TX* NSU Concord University, WV Lake Superior State University, MN University of Minnesota Crookston, MN Indiana University East, IN* Indiana University Kokomo, IN* West Liberty University, WV* West Virginia State University, WV* Dickinson State University, ND Oregon Institute of Technology, OR Lander University, SC* Lewis-Clark State College, ID* Purdue University North Central Campus, IN* University of South Carolina Aiken, SC* Doctoral, Moderate Research Baccalaureate A&S Baccalaureate/Associates Baccalaureate A&S Baccalaureate/Associates Baccalaureate/Associates NSU Bemidji State University, MN Black Hills State University, SD Dakota State University, SD Minnesota State University Moorhead, MN Minot State University, ND Northwestern Oklahoma State University, OK Oregon Institute of Technology, OR Southern Oregon University, OR The University of Texas of the Permian Basin, TX Western State Colorado University, CO Mayville State University, ND Valley City State University, ND Institutions that are stricken through were given as options, but not selected by the institutions as peer institutions. *Denotes institutions who were on the distant peers list in 2014. In 2018, institutions were only given a list of close peers to choose from. ATTACHMENT II 9