Pursuing the Metrics and Analytics that Matter Most for Success in the Complete College Georgia Plan 2012 to 2020 Ed Rugg, 12-2-11 Complete College Georgia s goal of raising the higher education attainment of young adults in the state who have a college certificate, associate s degree or bachelor s degree from the current level of 42% to 60% by 2020 to meet the projected job demands for an educated workforce is strategically important to the growth of the Georgia economy. It is also highly ambitious. The Plan s success will require transformational reforms in the state s secondary and post-secondary educational systems and an intense focus on managing the metrics and analytics that matter most for goal attainment. A critical analysis of the metrics and analytics that are most needed to ensure that success is provided in this tactical report. Annual Certificate and Degree Production Matters Most at the Institutional Level for Achieving the Plan's Goal (Metric #1) If the principal goal of Complete College Georgia is to raise substantially the number and percentage of young adults (ages 25-34) in the state who hold postsecondary certificates, associate degrees, and bachelor s degrees by 2020, then the annual certificate and degree productivity of the state s higher education institutions and their system collectives is the most direct and important metric to manage and grow for the achievement of that goal. That metric tops the list of keys to success in the Complete College America national initiative. Indeed, the growth of annual certificate and degree production in higher education matters most. We should keep in mind that the numerous educational reforms described in the Complete College Georgia Plan and others that will evolve over the next eight years are not ends, but means they represent tactical initiatives intended to support achievement of what matters most. What matters most is producing 250,000 more certificate, associate degrees and bachelor s degrees in Georgia than current production levels will generate by 2020. That being the case, analytics that help steer the growth of certificate and degree production will be especially useful. Trend analysis by credential-level productivity by institution and system collectives will be vital for understanding the full range of post-secondary productivity that has occurred over the last decade and where it
2 is predicted to go over the next eight to ten years if major educational reforms are not forthcoming. Understanding the existing market share of post-secondary credential productivity by institution is essential for realistic goal setting for future productivity gains. That analysis should be completed at a highly disaggregated level of detail as well as at summary levels for the Georgia s higher education systems and the state as a whole. By highly disaggregated, I mean at the short term and long term certificate/diploma levels by career sector emphasis, at the A.A.S., A.A. and A.S. levels by career/transfer sector emphasis, and at the bachelor s degree levels by category of major emphasis. Predictive analytics should not only account for past productivity trends, but also for key input and process variables highly correlated with productivity outputs. Analytics for Nontraditional as well as Traditional College Students are Vital for Achieving What Matters Most (Metric #2) Since the new majority in higher education is comprised of nontraditional students, the Complete College Georgia Plan aptly recognizes that increasing the number of nontraditional college graduates will be as important as growth in the number of traditional-age college graduates in achieving its ambitious goal. In some respects as will become apparent below, attention to nontraditional students close to or in the 25-34 age range may be more important for reaching Complete College Georgia's goal by 2020. As the Plan indicates, three-fourths of today s college students are nontraditional, and therefore, new thinking is required to increase their college credential completions. Thinking traditionally and focusing primarily on increasing traditional first-time full-time freshmen cohorts and their retention and graduation rates would likely restrict and limit most institutions substantially in their abilities to contribute to the achievement of what matters most by 2020. Although there is much room for improvement in the pursuit of sophisticated analytics associated with the traditional college student population, there is, in comparison, a huge void in the application of analytics at any level for the new majority of nontraditional college students. Pursuing analytics for nontraditional students is an exceptionally challenging and complex task given that such students enter or re-enter college at all ages, at all student classifications, and in all terms of a calendar year and progress toward credential completion at highly variable as well as intermittent rates of speed, frequently attending several different colleges or universities before completing a long-term college credential. Clearly, new ways of thinking about the recruitment, enrollment, retention, progression and
3 graduation of nontraditional college students are needed to engage in meaningful analytics that help steer system and institutional success in achieving what matters most in the Complete College Georgia Plan. There are many important new ways of thinking about college students and their success in credential completion. For example, when nontraditional students are taken into account, new student intake analysis is best conducted in an annual context (i.e., unduplicated, combined Fall, Spring and Summer headcounts each year) rather than the traditional Fall Semester frame of reference since many new nontraditional students matriculate in the Spring and Summer terms. Likewise, the characteristics and attendance patterns of successful nontraditional college graduates are best revealed in a Credential Completion Analysis, starting with the identification of graduating student characteristics and tracing the graduates credential completion pathways backwards. A traditional analysis that focuses on the characteristics of a fall term s entering college student cohort and following its progression to degree completion forward is less practical or useful for nontraditional students than for traditional first-time full-time freshmen. Furthermore, refining the definition of a nontraditional-age college student at each student classification level and at credential completion is also needed to more accurately distinguish them from traditional college students than is the case when a commonly used single age cutoff is applied to all undergraduates. For example, undergraduates who do not enroll in college immediately after high school for several years to serve in the military or join the workforce, but who are under the age of 25 when they enter college, should be categorized as nontraditional students, but they are often counted as traditional students when a single age cutoff is used to distinguish between the two groups. All of these examples are just a few of the basic descriptive analytics that are needed to build a solid foundation for generating useful predictive analytics for nontraditional student populations that are likely to earn a college credential. Educational Reforms Intended to Increase Credential Completion Productivity Should Have Measurable and Monitored Impacts (Metric #3) Ideally, each major educational reform implemented as part of the Complete College Georgia Plan should have explicit, measurable and analytics-informed estimates of improved credential productivity attributable to the reform as well as a projected timeframe in which credential growth is expected to materialize. Without such clarifications, institutions could be expected to implement reforms that do not or cannot contribute to the 2020 goal. In fact, some of the proposed
4 reforms identified in the Plan are vital for ultimately increasing college credential completions and should be pursued, but their full impact is not likely to materialize until after 2020. If the Plan's goal of preparing an extra 250,000 young adults with college credentials must be reached by 2020, some educational reforms, some college student groups, and some institutions will be more centrally involved than others in the achievement of that particular goal. For example, the impacts of some reforms on most students beginning bachelor's degree programs over the next eight years are likely to be delayed beyond 2020 for two reasons. First, it may take several years before certain reforms, such as improved college readiness and remedial education, are fully implemented in secondary and post-secondary institutions. Second, the educational pipeline for bachelor's degree completion typically takes four to six years for beginning traditional students to pass through and often longer than that for beginning nontraditional freshmen. With only eight years left before 2020, there would not be enough time to fully implement the remedial education reforms and then also be able to assess their impact on increased bachelor's degree completions for beginning college students in the short span of the next eight years. Those educational reforms are worthy of pursuit, nonetheless, but their impact is not likely to be fully realized until later in the next decade beyond 2020. On the other hand, baccalaureate educational reforms that impact upper division undergraduate transfer students and nontraditional young adults returning for degree completion are among those that have a high likelihood of contributing to the Plan's goal attainment by 2020. In addition, there are likely to be delayed impacts of some educational reforms involving traditional-age students who enter the shorter educational pipelines in the pursuit of certificate or associate degrees, as well. For many of those students, they may have earned a credential which may not have been achieved prior to the reform over the next eight years, but they will not be old enough by the 2020 census to be counted in Georgia's 25-34 age group. Here too, pursuit of those educational reforms will be worthwhile, but their impact is not likely to be fully realized until later in the next decade beyond 2020. Here too, certificate and associate degree educational reforms that impact nontraditional-age young adult students may be among those that have a high likelihood of contributing to the Plan's goal attainment by 2020. It is also important to note that some institutions that currently have a large market share of the state's bachelor s degree productivity, but are highly selective
5 in admissions, cater largely to traditional students pursuing bachelor s degrees, and have high graduation rates for their first-time full-time freshman cohorts may not be able to squeeze too much more degree completion productivity out of their student niche and highly productive operations toward achievement of an extra 250,000 graduates with post-secondary credentials by 2020. Other institutions may have to take on more responsibility than those universities for achieving the Plan's goal by 2020. Beware the Limitations of an Over-Emphasis on Improving Retention and Graduation Rates (Metric #4) The national fascination with improving the retention and graduation rates of firsttime full-time beginning college student cohorts has some potential value for contributing to what matters most by 2020 under the Complete College Georgia Plan, but that value is very limited. After all, such graduation rates focus predominantly on traditional students who comprise only 25% of the students now served by colleges and universities. Those graduation rates also assume that traditional beginning college students will follow traditional patterns of enrollment to graduation, including entering college in the fall semester as a full-time student, maintaining full-time enrollment every term, not stopping out periodically, and not transferring to another institution. At most colleges and universities, nontraditional students following nontraditional patterns of enrollment to graduation and traditional college students who subsequently adopt nontraditional attendance patterns and become nontraditional in character are more prominent and earn the majority of the certificates and degrees awarded annually. Regrettably, the substantial credential productivity associated with nontraditional students goes unrecognized in traditional graduation rate statistics that pertain only to a shrinking segment of contemporary college students. Putting too much emphasis on educational reforms that improve the graduation rates of traditional college students while ignoring the greater reforms needed for improving the pathways to graduation for the new majority of nontraditional college students would be shortsighted in efforts to achieve what matters most by 2020 in Metric #1. Tracking the Expected Improvement of College Readiness and the Expansion of the College-Bound Pipeline from K-12 (Metric #5) Georgia is receiving high marks nationally for its progress in implementing a statewide student data system from pre-k to college. Nearly 90% of the state s school districts are now tied into the data system which has taken more than a decade to develop. Consequently, the capability to track student flows and the
6 college readiness of Georgia s high school graduates will soon be enriched substantially. Utilizing that capability to inform the ongoing planning and goal setting of Complete College Georgia will be a key metric of high value. However, it is not yet clear when sufficient data in the new tracking system will become available, limiting perhaps its impact on the achievement of what matters most by 2020. Its actual benefits could be delayed and realized more fully in the decade following 2020. Delayed impact beyond 2020 is also likely from K-12 educational reforms aimed at reducing the need for post-secondary remediation. The Transforming Remediation Work Group has only recently been established and is in an early stage of identifying appropriate actions to take in K-12 to ensure that high school graduates are college-ready. Much of the eight years remaining before 2020 is likely to be spent planning and implementing such K-12 reforms, precluding their impact on achieving what matters most in the current Complete College Georgia Plan. Frankly, K-12 remediation initiatives underway that may be targeted for the senior year in high school are likely to come into play too late to be of practical use to prospective college students who take their SATs in the fall of their senior year and receive their college admission decisions and learning support placement results soon thereafter. College readiness in mathematics is especially problematic, but college readiness in English composition is also a common weakness. In general, many high school graduates have not been prepared for the rigor of college courses and out-of-class study requirements and lack college-level critical thinking and study skills. Strengthening the alignment of the entire high school curriculum with college readiness standards should be prominent in K-12 reforms so that more high school graduates are college-ready as rising seniors. Tracking the Impacts of Post-Secondary Learning Support Reforms for all Nontraditional and Traditional Students (Metric #6) The Plan's important remedial education reforms that are currently underway in both post-secondary systems show promise of increased effectiveness, but are not scheduled to be fully implemented until 2013, leaving little time before 2020 for their impacts to be measured on college graduates, especially at the baccalaureate level. Pursuing those reforms and tracking their impact beyond 2020 will constitute an important metric, nonetheless.
7 Tracking the success of learning support program reforms on increasing the number of certificates and degrees completed by learning support students over current production levels (Metric #1) should be done for all beginning learning support students, nontraditional and traditional. Reliance solely on graduation rates would restrict those impacts to the fall cohorts of largely traditional learning support students and underestimate the impact of these important reforms on what matters most. It is also important to note that when graduation rates are calculated for learning support populations and are compared with the graduation rates of college-ready populations, the learning support populations are at a substantial disadvantage of not having a full three-year or six-year period to earn college credit toward certificate or degree completion since much of their first year or more is taken up with learning support courses that earn no degree credit. Consequently, learning support graduation rates will be suppressed simply as a function of actually having less than the full three or six years of the graduation rate period in question to complete college-level work. Tracking Prior Learning Assessment Credits and Boosted Degree Completion Initiatives. (Metric #7) Prior Learning Assessment (PLA) and its provision of a speedy pathway for millions of students who stopped out with a partially completed degree to return to college for degree completion is a key tactic in the Plan s efforts to shorten time to degree. PLA involves promoting some well-established options for credit by exam as well as the development of new assessment processes for validating knowledge and skills acquired in the workplace and other life experiences that are worthy of college credit. It also suggests that a major initiative will get underway to launch and promote degree completion plans for returning nontraditional college students. Metrics to be tracked for PLA activity and degree completion productivity at the institutional and system levels are indicated in the Plan and could have a positive impact on the Plan s goal attainment for 2020. Caution is advised to develop new PLA procedures that comply with SACS-COC accreditation requirements, especially when dealing with the award of credit for business credentials. Georgia s Workforce Requirements Matter in Setting System and Institutional Productivity Goals that Align with State Needs (Metric #8) The Complete College Georgia Plan recognizes the importance of monitoring the state s workforce requirements and needs for college-educated workers. That is a metric that ultimately validates the success of the Complete College Georgia
8 initiative, especially if projected workforce demands in the future are met with adequate supplies of well-educated personnel. Ideally, the analysis of workforce requirements would focus on identifying the preferred post-secondary credentials in different employment sectors at a level of detail and disaggregation that aligns well with the credential levels and preparation emphases of the trend analyses and predictive analytics described above in Metric #1. The extent to which credential productivities match workforce needs in the state should be assessed. When doing so, caution is advised in the treatment of some associate s and bachelor s degrees which often do not lead to career employment immediately, but are essential preparations for entry into professional and graduate degree programs that lead to career employment, typically in areas of high employer demand and compensation. In addition, many associate's and bachelor's degrees are not career-specific, but have value, regardless of their area of educational emphasis, as the preferred credential of employers in some job sectors. Furthermore, it is also important to keep in mind that no certificate, associate s degree or bachelor s degree should be considered a terminal credential anymore in a US economy that is constantly evolving and raising its educational standards, not only for job entry, but also for career advancement. Consequently, focusing on preferred credentials for future competitiveness in different employment sectors rather than on the minimum educational requirements for job entry in those sectors presently would provide stronger and more informative guidance for goal setting and decision-support in the implementation of the Complete College Georgia Plan. Associate Degree Productivity Could Be Raised Substantially by 2020 With Innovative Production Reforms (Metric # 9) Unlike many other states, Georgia has not developed or maintained a comprehensive community college system. Instead, post-secondary educational programs leading to technical certificates, diplomas, and A.A.S degrees are largely provided by the technical colleges in Georgia which have a strong focus on career education, especially at the certificate level. The A.A. and A.S. degrees are largely provided by the public two-year colleges which have had a strong focus on providing lower division undergraduate study that transfers into four-year bachelor's programs. Many two-year college students in Georgia transfer to fouryear colleges before completing an associate degree. That transfer should be counted as a success for those institutions with a transfer mission, but regrettably is not when graduation rates for associate degree completion are the metric of interest. In addition, many two-year colleges in the USG have been elevated to a four-year status in recent years and will be devoting substantial attention to four-
9 year program development at the expense of an associate degree emphasis. Consequently, Georgia's commitment to post-secondary credential production has been heavy at the technical certificate/diploma level and also at the bachelor's degree level and lighter at the associate degree level. Therefore, many college students who drop out before completing the baccalaureate walk away with no formal educational credential even though they may have completed enough credits to qualify for an associate s degree. Within that context, it is not surprising that a recent Complete College America study entitled, Certificates Count: An Analysis of Sub-baccalaureate Certificates (2010), found that Georgia's public two-year colleges were second to California's in the highest number of certificates awarded in 2007-08, accounting for 10% of the nation's total that year, thanks to its strong Technical College system. However, unlike California which has a large comprehensive community college system and which awarded 22.3 associate degrees per 10,000 in population, Georgia lagged the national average in associate degrees awarded by nearly half in 2007-08 (GA at 8.3 vs. the nation at 16.5; GA was 45th out of 50 states in annual production of associate degrees). Rather than build a comprehensive community college system, Georgia could become more competitive quickly in its associate degree production if all undergraduates at the four-year institutions who completed the 42-credit Core Curriculum (45 counting the HPE requirement) along with an additional 18 degree credits were automatically awarded an associate s degree on their way to baccalaureate completion. For those who do not subsequently earn the bachelor s degree, they would at least walk away with an associate s degree, and Georgia s annual associate degree productivity would rise substantially. Similarly, since there is often little difference in the credits required for a technical diploma and an A.A.S. degree in the same field of study, technical colleges could be rewarded for shifting some of their high long-term certificate productivity to associate degree productivity by advising students to invest in a few extra hours to complete what may be in the long run be a more valuable college credential for career and educational advancement in the future. Initiatives for Restructuring Delivery Have Numerous Metrics to be Tracked for their Impact on Achievement of What Matters Most. (Metric #10) The Plan s outline of potential reforms for restructuring delivery contain numerous metrics to be tracked for their impact on achievement of what matters most by
10 2010 (Metric #1). Several key tactics with great potential for achieving what matters most that are not yet mentioned in the Plan, but which could be pursued in the near term include: a) establishment of evening and weekend course and program offerings to accommodate working adult learners and other nontraditional students who prefer traditional or hybrid instructional delivery during times outside the typical workday; b) identification and rectification of course scheduling inefficiencies and bottlenecks for certificate and degree program completion; c) identification of weed-out or killer courses in the curriculum, especially in STEM fields, that require supportive interventions to promote student success and retention; and d) interventions to promote online course completion and success levels comparable to those for traditional course delivery. Summary Observations The title of a recent Complete College America web publication, New Thinking for a New Majority (2011) captures the essence of what it will take for the Complete College Georgia Plan to succeed. Traditional ways of delivering higher education to traditional students, using traditional metrics and analytics, need to give way to the inclusion of equally, if not more important, new ways of thinking about the new majority of college students. Innovative and nontraditional ways of delivering higher education to nontraditional and traditional students, using innovative and nontraditional metrics and analytics, is called for if the Complete College Georgia Plan is to optimize its potential to succeed. The content of this tactical report was prepared in the spirit of the Plan s commitment to be a living document with accessible updates, capable of modifications that ensure the Plan s goal attainment of what matters most. Hopefully, this report will stimulate new ways of thinking about the metrics and analytics that matter for preparing a well-educated workforce in Georgia. Regardless of whether what matters most can be achieved by 2020, or more likely by 2030, the educational reforms that the Complete College Georgia Plan will initiate and foster are on the right track to meeting future state needs for a college-educated workforce that will keep Georgia strong economically and competitive internationally.