Statistical Enhancements to FAEIS in

+ Statistical Enhancements to FAEIS in 2010-2011 NIFA and Expert Panel Meetings Washington, D.C. April 7-8, 2011 Eric Vance ervance@vt.edu Virginia Tech 1 From 2002 to 2009 the number of entries in FAEIS doubled. Four times as many institutions now submit 1994 1890 1862 data to FAEIS. of Hawaii D-Q of California HAWAII PACIFIC BASIN NORTHERN MARIANAS GUAM of Guam Northwest Indian Washington Oregon of Nevada Northern Marianas Tohono O Odham Community of Micronesia Univ. of Idaho Blackfeet Community Utah Salish Kootenai Montana Diné of Arizona Navajo Technical Ilisagvik Southwestern Indian Polytech. Institute ALASKA Fort Belknap Stone Child Little Big Horn Fort Peck Community Chief Dull Knife of Wyoming Colorado Institute of American Indian Arts New Mexico Turtle Mountain Community Fort Berthold Comm. United Tribes Technical Si Tanka/Huron Univ. Oglala Lakota Sitting Bull Coll. Sisseton Wahpeton Community Nebraska Indian Community Cankdeska Cikana Comm. North Dakota Univ. South Dakota Sinte Gleska Univ. Little Priest Tribal of Nebraska Kansas Haskell Indian Nations Langston Prairie View A&M White Earth Tribal & Community Leech Lake Tribal Fond du Lac Tribal & Comm. Oklahoma Texas A&M of Minnesota Iowa of Missouri of Menominee Nation Lincoln of Arkansas of Arkansas at Pine Bluff Louisiana Southern and A&M of Wisconsin of Illinois Alcorn Lac Courte Oreilles Ojibwa Community Tennessee Mississippi Kentucky Alabama A&M Univ. Auburn Tuskegee Bay Mills Community Michigan Purdue of Tennessee Saginaw Chippewa Tribal Ohio of Kentucky of Georgia Fort Valley Florida A&M West Virginia Univ. Pennsylvania West Virginia North Carolina A&T Clemson of Florida of Vermont of Massachusetts Cornell Virginia Tech South Carolina North Carolina of Maine of New Hampshire of Rhode Island of Connecticut Rutgers of Delaware Delaware of Maryland Park of Maryland Eastern Shore of the District of Columbia Virginia 1862 1890 1994 FEDERAL STATES OF MICRONESIA of Alaska PUERTO RICO / U.S. VIRGIN ISLANDS AMERICAN SAMOA American Samoa Community of Puerto Rico of the Virgin Islands map_lgu_all_front_12_9_09.ai! 2

With this increase in the quantity of data and the number of institutions reporting, new challenges have arisen: More data mean more outliers More institutions and programs mean more changes and transitions within the programs New institutions added mean more gaps in the data for previous years " 3 In an effort to improve the quality of the FAEIS data and prepare the data for rigorous statistical analysis, the FAEIS team asked LISA to collaborate and provide leadership on statistical issues from an independent, outside perspective. # 4

The FAEIS team and LISA are working together to incorporate statistical thinking into FAEIS processes, thereby adding long-term value to FAEIS. $ 5 What/Who is LISA? % 6

LISA helps VT researchers benefit from the use of Statistics Experimental Design Data Analysis Interpreting Results Grant Proposals Software (R, SAS, JMP, SPSS...) LISA s goal is to make statistics a strength of research at Virginia Tech, not a roadblock. 7 LISA helps VT researchers benefit from the use of Statistics Experimental Design Data Analysis Interpreting Results Grant Proposals Software (R, SAS, JMP, SPSS...) Collaboration Walk-In Consulting From our website request a meeting for personalized statistical advice Monday Friday* 12-2PM for questions requiring <30 mins Great advice right now: Meet with LISA before collecting your data Short Courses Designed to help graduate students apply statistics in their research 8

1948: The Statistical Laboratory was founded as a division of the Virginia Agricultural Experiment Station to serve the needs of agricultural and biological research in Virginia and VPI. 1973: The Statistical Laboratory was re-formed as the Statistical Consulting Center to assist with statistical analyses in every college of Virginia Polytechnic Institute & (VPI&SU). 2008: The Statistical Consulting Center was reorganized as the (LISA) to collaborate with researchers across the Virginia Tech. 9 1948: The Statistical Laboratory was founded as a division of the Virginia Agricultural Experiment Station to serve the needs of agricultural and biological research in Virginia and VPI. 1973: The Statistical Laboratory was re-formed as the Statistical Consulting Center to assist with statistical analyses in every college of Virginia Polytechnic Institute & (VPI&SU). 2008: The Statistical Consulting Center was reorganized as the (LISA) to collaborate with researchers across the Virginia Tech. 10

LISA collaborators meet weekly to discuss projects such as FAEIS and to learn from each other. 13 LISA supports the FAEIS team. We deal with their statistical issues. Albert Shen: 20 hours/wk Katie Griffin: 20 hours/wk Eric Smith: ~2 hours/wk Eric Vance: ~4 hours/wk Additional LISA collaborators as needed 14

Albert Shen, a statistics graduate student and LISA associate collaborator, was hired by FAEIS to: Create a SAS dataset from the FAEIS database currently in Oracle Develop algorithms and procedures for detecting outliers Deal with other statistical issues such as missing values and gaps in the data &$ 15 Katie Griffin, a statistics graduate student, was hired by FAEIS to: Support the FAEIS Help Desk with statistical issues to improve data accuracy Create reports to compare FAEIS data to IPEDS data Mine institutions data from their Institutional Research websites &% 16

Since being hired as a statistical analyst GRA, Albert Shen: Created SAS datasets Verified that SAS reports are identical to Report Builder Developed algorithms to identify outliers Developed algorithms to identify zeros and missing data &' 17 Creating a FAEIS dataset in SAS will provide many benefits: Flexibility to analyze the data using SAS (or any other statistical programs R, JMP, SPSS,...) Visualizing the data Detecting outliers Analyzing trends in the data, such as graduation rates Consistency of analyses Portability of reports &( 18

Since being hired as a GRA, Katie Griffin has: Compared FAEIS data to IPEDS data for a range of programs and institutions Assisted institutions with data collection and reporting Filled in missing FAEIS data with data from IR &) 19 Additional enhancements to FAEIS data using statistics: SAS algorithm to identify redundant/repeated data entries and misplaced CIP codes Automated identification of invalid/problematic data!* 20

Discussion: Your ideas for statistical enhancements to FAEIS?!& 21