eLearning Roadtrip

When I Think of Student Success

The tendency to link software with student success has accelerated since 2012, as the educational technology sub-sector known as "integrated planning and advising for student success" has gained prominence through a series of grants funded by the Bill & Melinda Gates Foundation to colleges and the tech partners they have selected to help the colleges improve student performance. These days, more than 100 companies offer integrated planning and advising technology platforms and tools for student success includes that serve a wide variety of functions, from deciding where to go to college, to facilities and course scheduling, from helping students select select majors to staying on track with program completion. Student success is a huge part of the current ed tech market scene. (Disclosure: In my current day job, I work for a company with a large part of its business dedicated to Student Success.)

EDUCAUSE has led efforts to help institutions explore both the promise and the risks of Integrated Planning and Advising for Student Success (iPASS). Per their website:  "iPASS is an integrative approach to student success that promotes shared ownership for educational progress among students, faculty, and staff. It encompasses services that help students formulate and advance toward educational goals, including advising, counseling, progress tracking, and academic early alerts. iPASS technologies can contribute by documenting and tracking students’ educational plans, improving data analysis, offering self-service resources that reduce advisor workloads, and triggering interventions based on student behavior or faculty input."

There have been two Gates Foundation-funded rounds of grant-making by EDUCAUSE in support of student success. The first was the Integrated Planning and Advising System project, designed to get schools and their tech partners to collaborate on getting platforms and practices to work together better. I was involved in the first IPAS round when the Predictive Analytics Reporting (PAR) Framework was contracted by the Gates Foundation to evaluate the 19 "student success + software initiatives" that were funded in that first go-round. We visited the 19 IPAS campuses that were working with specific tech company partners to further the implementation of a tech platform that could help campuses with student planing and advising platforms. PAR had used the PAR Student Success Matrix we had developed to help our own PAR members make the connection between predictions of  student risk with selection of interventions. We were able to determine and demonstrate measured effectiveness with particular groups of students under particular circumstances - sort of like getting the most effective prescription for dealing with a particular medical condition. We had been excited about our own results; the fact that we were going to be able to evaluate student success impact with our matrix of time-in-program and actionable predictors with a broader set of institutions was thrilling.

By the time the second round of grants were funded, the name of the initiative had become "integrated Planning and Advising for Student Success: iPASS". The connection between tech and advising (e.g. watchlists, early alerts), tech and planning (e.g. curriculum pathway planners, strategic scheduling), program fit and course navigation (e.g. obstacle course GPS), integrating predictive analytics with real-time student activity stream data.) As it happens, the iPASS initiative helped spark PAR Framework's acquisition by Hobsons in January, 2016, when two state systems of education proposed integrating PAR's predictives and intervention measurement with Starfish's real time student activity data stream used to build student watchlist/advisement capabilities as part of their iPASS grant proposal.

5 years, two major initiatives and several rounds of small planning and implementation grants later, EDUCAUSE has produced a library of student success resources for educational practitioners, on everything from "Things you should know about IPAS" (yes, that acronym is different than in the previous paragraph; more on that in a bit) to implementation handbooks, data tables and evaluation reports. and  from https://library.educause.edu/resources/2013/8/integrated-planning-and-advising-services-ipas-research.

It has been great to be a participant in this wave of innovation in education, where data is being used as currency for more effective decision-making in schools and colleges. No doubt, when we look back on how decisions were made in 10 years from now, we are going to wonder how we managed to live without platforms that help visualize patterns, red-flag student risks before they become problems, provide a 3D map of an academic transcript that can cross-walk among formal and informal learning experiences. 

Even so, I need to say something.

When I think of student success I never, ever think of software first. 

Software can make things easier to do, of course. It offers far better ways to keep track of searchable information requiring any manipulation. I'd rather build formulae in a spreadsheet, and have huge collections of data fueling explorations that report results in approachable visual reports and dashboards. Learning management and content management and curricular tracking, assessments and performance support are all more available, more accessible and more targeted at specific points in the educational experience than ever before.

Just as data associated with online and digital transactions help inform marketeers of our retail intentions even before we know for certain, ourselves, patterns of students behaviors can raise red flags before students are even aware that they may be drifting into dangerous territory for "students like them".

Technological inventions, applied to performance and learning problems, can help solve those problems. In doing so, the inventions become innovations, human ingenuity applied to solve performance problems in novel and compelling ways.

Innovative educational entrepreneurs  - "edu-preneurs', if you will - see an opportunity for tech in schools and are compelled to create products that promise to disrupt education as we know it. They point to deep data science and predictive analytics as if these analysis techniques are beyond the norm.

It's seems important to remind ourselves that in education, the most most important innovator in this value chain are not necessarily the ed tech entrepreneurs. Rather, the ones on the front lines of teaching and learning, supporting student achievement putting new tech tools to the test in the service of student success are todays true innovators. 

It's important to have shared articulated expectations of "the norm", so that one can identify those not yet achieving at that level, and how one inspire those working "above average" to continue to improve. Common expectations of meaning and outcomes are essential elements of student success that gives all students a shot at achieving their dreams. We need to include ALL students in our considerations of success, not just those attending college for the first time, going to school full time, or living on campus. We need to keep the student at the center.

With thanks to Kalah Burks and Rebecca Wheeler, University of South Alabama, for the great conversation that shook this post out of my head.

(tap, tap) Hey, is this thing still on?

During the past 6 years I have had the remarkable experience of taking an idea about predictive analytics all the way from brain-storming with smart, crazy people who wanted to try something new and different with big data techniques, to getting multiple grants from the Gates Foundation to see if our idea worked. It involved founding a not-for-profit organization to validate our ideas in the commercial ed tech marketplace, to being acquired by Hobsons, an international commercial software provider.  

Not bad for someone who had really just wanted a voice in the learning analytics conversation that was more focused on students than VC.

Now that I've had a chance to get settled in a new research job at Hobsons,  I've been reflecting on whether or not this work I've been doing in analytics still counts as elearning.  I'm thinking that it does.  I'm thinking that now, more than ever, using digital tools to extend out expectations of learning is exactly the next phase in learning innovation we elearning advocates have been waiting for.

There are some who continue to describe elearning as the activities utilizing electronic technologies to access educational curriculum outside of a traditional classroom. In these cases, people are typically referring to an online course, program or degree. I get that. It helps define elearning as something distinct from other cognitive or pedagogical pursuits. Some people think of elearning as authoring, or collections of webcasts.  I've seen some people say that elearning is dying because mobile learning is about to replace it. 

For me, elearning is about leveraging technologies in support of designing, creating, producing, evaluating, distributing and managing learning experiences that engage and inspire, regardless of physical location or time. It's a much bigger tent than content or courseware.

My new job at Hobsons is a research job. I'm exploring learning impact and efficacy in the P-20 learning systems we depend on in the US to prepare our citizenry for college and and career success. I've been working on reports describing the impact of dual credit for accelerating college completion, on the impact of upward transfers from community colleges to completion institutions, on online learning efficacy, on the value of exploration and engagement in early childhood, the things we can do to support learning success among non-traditional students. 

I'm amazed to see, in all these discussions, the degree to which elearning has evolved beyond content and courseware. Today elearning is more about personalized, interactive, responsive experience.  If ed tech is where we concentrate on better tech tools, these days elearning encapsulates all we have learned to do to extend digitalization into our teaching and learning, education and training practices, constructs and values.

All this reflection made me realize I need a place to reflect on the transformative evolution of our practice as tech changes our expectations of our learning lives.

Looks like it's time to get back on the eLearning Roadtrip.

 

 

 

Karen Swan and the Amazing Multi-institutional PAR Student Success Matrix

Back in June 2013 the PAR Framework openly published an intervention assessment framework called the Student Success Matrix to help PAR members figure out how to respond when confronted with predictions of which students were likely to drop out. "What should we do to respond to these predictions showing our students at risk?" our members asked.  At that time Dr. Karen Swan, Stukel Distinguished Professor of Educational Leadership at the University of Illinois Springfield, reminded us that there are already lots and lots and lots of things that academic advisors, faculty and other student success professional already do to help students succeed in school. Going back through the professional literature, she and Dr. Peter Shea, SUNY-Albany, helped us find models of student engagement and participation relevant to our work at hand.

Karen proposed a framework to map interventions using predictor behaviors of success and points in the academic completion cycle.  Over the past year Karen and Peter have worked closely with Sandy Daston, PAR Framework Director of Student Success and the entire PAR SSMx working group (Hae Okimoto, University of Hawaii System; Jennifer Freed, Dan Huston, and Fermin Ornelas, Rio Salado College; David Shulman, Broward College; Denise Nadasen, Jack Neill and Karen Vignare, UMUC; Patsy Moskal and DeLayne Priest, University of Central Florida; Michelle Wiley, Penn State World Campus; Mysti Reneau, Community Colleges of Spokane) to refine processes and tools for promoting intervention measurement. This group of experienced educational practitioners and decisionmaker has helped determine what can actually be done for student at risk - even down to knowing how to support those 37 students that are most likely to fail Math 101 this semester. 

PAR institutions have begun the process of inventorying, tagging, describing measuring and evaluating literally hundreds of interventions currenty in use on campuses to help student succeed. Thanks to the SSMx we have accelerated the process of connecting predictions of risk to the interventions that will mitigate risk. 

This post is to acknowledge Karen's leadership and contributions to PAR's thinking about using predictions as cataysts for taking action, and for linking interventions to predictions to underscore the importance of linking predictions to action.

 

The Big Data Landscape

If you post the term "Big Data" in your Google search window today, more than 2 billion google search results will appear. If you perform a similar search on Bing, you will see that more than 1,790,000 Big Data links have been added to the Internet in the past 24 hours alone. Clearly, there is big interest in big data.

This graphic by Matt Turck and Shivon Zilis is an exceedingly helpful image for understanding the breadth and complexity of the Big Data landscape. Matt Turck's blogpost  also provides great information for understanding some of where this emerging market is taking us. 

 

 

For those of us working in post-secondary education circles, the emergent interests in big data, analytics, learning analytics and related evidence-based decision-making endeavors have been inspired by possibilities of better experiences, more personalization and better behavioral targeting. Still most education stakeholders who are giving learning analytics some consideration may be surprised to see the depth, breath and sophistication of the big data world.  While WCET's work on the PAR Framework has certainly contributed to a growing awareness on the power of predictive analytics for academic risk identification and the value of collaborative, multi-institutional data mining to improve student success, we are mindful that our student success focus touches a relatively narrow, targeted part of the Big Data picture. This graphic offers a more complete picture of the total ecosystem.

NEW NSF REPORT: New Technology-based Models for Postsecondary Learning

In January 2013 I  participated in a workshop sponsored by the National Science Foundation to help establish a 10 year vision to reconceptualize postsecondary learning. This past week our co-convenors, Chris Dede, Harvard University and Eric Grimson, MIT, released the 47 page report that captured the results of that meeting. Entitled New Technology-based Models for Postsecondary Learning: Conceptual Frameworks and Research Agendas, this is a comprehensive overview of promises, opportunities and challenges that will keep all of we post-secondary learning stakeholders on our toes for the next decade.

You can download the working draft of this paper here.

The basic premises addressed include the following:

• Advances in technology and in knowledge about expertise, learning, and assessment have the potential to reshape the many forms of education and training past matriculation from high school.
• In the next decade, higher education, military and workplace training, and professional development must all transform to exploit the opportunities of a new era, leveraging emerging technology-based models that can make learning more efficient and possibly improve student support, all at lower cost for a broader range of learners. 
• Potential risks must be managed, including those arising from the disruption of established delivery economics in our current learning institutions, the variable quality of learning outcomes these new models offer today, and the technical and conceptual challenges of better understanding how to design, develop, and implement innovative capabilities in ways that reliably deliver on their promise.

Workshop participants developed a framework for understanding this sea change and sketched steps towards a research agenda for realizing its benefits while avoiding pitfalls.

Contributors to this report include Daniel E. Atkins, W.K. Kellogg Professor of Community Information, and Professor of Information and Computer Science, University of Michigan - Ann Arbor. Lori Breslow, Director, Teaching and Learning Laboratory, MIT. John Cherniavsky, Senior Advisor, Division of Research on Learning, NSF. J.D. Fletcher, Senior Research Staff Member, Institute for Defense Analyses. Diana Oblinger, President and CEO, EDUCAUSE. Roy Pea, David Jacks Professor of Education and Learning Sciences, Stanford University. James W. Pellegrino, Distinguished Professor and Co-Director, Learning Sciences Research Institute, University of Illinois at Chicago. Bror Saxberg, Chief Learning Officer, Kaplan, Inc. James H. Shelton III, Assistant Deputy Secretary for Innovation and Improvement,U.S. Department of Education. Russell Shilling, CAPT, MSC, USN, Program Manager, DARPA. Greg Tobin, President, English, Math, and Student Success, Pearson Higher Education and yours truly, Ellen Wagner, Executive Director, Western Interstate Commission for Higher Education Cooperative for Educational Technologies. Arthur Josephson, Harvard Graduate School of Education, was our researcher.

Since You've Asked: More About the PAR Framework

The December 15, 2011 Chronicle of Higher Education's Wired Campus blog by Marc Parry shared news about a groundbreaking project we have taken on at WCET.  It describes our efforts to create a single, federated database of de-identified student records contributed by multiple universities and colleges. Six forward-thinking WCET member institutions have worked together this past six months to see if it was possible to analyze variables common across these institutions to look for patterns that can inform us about student loss and momentum. I'm pleased to say that we have succeeded. Baby steps. But still.

We've received a lot of questions. Here are a few answers. There will be more.

This is a multi-phased project. We are concluding our work on Phase 1. The best is yet to be.

Our first phase focused on demonstrating that it is posssible to get multiple postsecondary institutions here in the US to contribute their data to create a single unified data set. We are now applying a variety of descriptive, inferential and predictive analyses to look for patterns showing variables likely to affect student success. 

Phase 1 = Proof of Concept. Creating a single federated data set from the de-identified student records from six unique institutions has never been done before. 

We've started with 6 schools. We are adding additional schools as we continue our efforts to build out the federated data set. While 640,000 students records and 3.1 million course records are a lot more than most of us are used to working with, it is still just a sample from 6 schools. 

We've started with 34 variables that are common across each of the institutional partners. Much of our current work has focused on establishing common definitions of these variables to ensure that comparisons and aggregations are valid, reliable and repeatable. There will be more variables.

We are currently focused on online programs. Just about every single transaction associated with an online program is digitally captured and stored somewhere. It seemed the obvious, self-evident place to begin.

We've focused on pattern strategy exploration, not null hypothesis testing (at least not in the conventional sense). 

We're really not trying to answer research questions using big data analysis techniques. Not yet. We'll get there. First things first. The journey of a thousand miles begings with the first steps.

More to the point...if we do our work right, then perhaps some day YOU will be answering YOUR research questions using the federated data set using big data analysis techniques.

The team working on this effort brings expertise from institutional research, statistical analysis, quant and qual research, cognition and instruction, online learning, higher ed administration, educational public policy, project management, and commercial software development. They come from the public and for-profit sectors. Then are from community colleges and 4 year comprehensives. Several of our institutional partners are already award-winning, acknowledged practice leaders in the hot, hot, hot domain of learning analytics. These are some of the smartest, most dedicated, creative, committed people I know. 

None of us - not WCET, not the core team of experts, not any of our fabulous institutional partners - are making any kinds of reccommendations from our findings yet. You may have noticed that my quote in the Chronicle was about as non-specific as one can possibly be. But be assured, there are going to be some very interesting implications for practice and policy that will emerge as the new patterns we are finding are more sharply focused in the coming weeks and months. 

Yes of course we are going to share our findings. Sharing is a core WCET value. We exist to share. That's what Cooperatives do.

 

20 "eLearnings" From Two Months on the Road(trip)

Been a busy couple of months of elearning road-trips. From EDUCAUSE to Online EDUCA, from WCET to WICHE, from DevLearn to AECT (with a little bit of virtual Sloan-C tossed in for good measure). Literally thousands of plenaries, presentations, and poster sessions from which to choose, along with hundreds of vendor stands and trade show booths. Here are some of my big take-aways:

• Hooray and congratulations! This distance/online/elearning thing we do has tipped into mainstream consciousness. Let's give a cheer for the researchers, the teachers, the authors, the artists, the technologists, the producers, the analysts and the evaluators who have made this happen. We rock.
• We matter enough that governments want to regulate us. Congressmembers rail against us. No more flying under the radar.
• Analytics are hot, hot, hot. Learning analytics, content analytics, web analytics, engagement analytics, to name a few.
• Speaking of analytics...mathematically speaking, a learning outcome is pretty much the same as a point of sale. (Thank you, Phil Ice.)
• Second Life is dead, dead, dead. Sorry.
• Mobile is the future.  Smartphones and tablets and readers, oh, my.
• Game based learning has a long way to go before people take it seriously.
• High quality video conferencing is making a comeback.
• We may get inspired by big ideas, but we are moved to action by fear, uncertainty and dread.
• On the commercial front...Pearson intends to own our universe. Adobe has pretty much given up on us. Just about everybody wants to sell us an "analytics solution". Or a mobile app.
• People really need to quit thinking that "open" is the same as "free".
• It's a lot easier to pontificate about the importance of change than it is to do things differently.
• There is a lot of research out there. Some of it is useful. 
• Empirical evidence doesn't need to be quantitative...but it does need to be a little more robust than simply pointing to people who are doing the same things.
• Quality is rarely absolute. It's contextual and situational. It may make more sense to think about "effective practices" rather than "best practice".
• Practitioners who make big sweeping statements without doing any background work on what others may have already been doing, saying or thinking end up sounding just a little bit naive and foolish. C'mon, people...use The Google.
• On the Internet, nobody knows you're a dog...but EVERYONE knows if you are not prepared.
• One can pack a lot of snark in 140 characters.
• Learning is as natural as breathing. Learning really isn't the problem. Enabling meaningful, intentional outcomes and creating experiences that engage and inspire without costing an arm and a leg and without killing our interest and enthusiam for learning is the problem. 
• Sometimes you just need to show up.

WCET Receives Grant for Groundbreaking Higher Education Analytic Research

I am pleased to share the following good news.

WICHE, the Western Interstate Commission for Higher Education, announced today that WCET, the WICHE Cooperative for Educational Technologies, has been awarded a $1,000,000 grant from the Bill & Melinda Gates Foundation to fund an initiative to unify student data from six U.S. institutions and demonstrate the effective use of predictive analytic methods for improving student outcomes. The goal is to identify variables that influence student retention and progression, and guide decision-making that improves postsecondary student completion in the U.S.

This grant for the Predictive Analytics Reporting (PAR) Framework will aggregate data representing more than 400,000 student records from across six WCET member institutions: American Public University System, Colorado Community College System, Rio Salado College, University of Hawaii System, University of Illinois Springfield, and the University of Phoenix. Each participating institution has been exploring or implementing descriptive, inferential or predictive analytics projects on their own student data; the PAR Framework expands on this work through exploration of patterns that can be derived when the six institutional datasets are considered as a single, unified sample.

The principal investigator is Phil Ice, Ed.D., American Public University System (APUS). The project will incorporate many of the best practices and award-winning predictive analytical methods for which Phil has been recognized by the Sloan Consortium and for which project partner Dr. Vernon Smith, Rio Salado College, has won the prestigious WCET Outstanding Work (WOW) award. WCET has appointed Beth Davis, WCET, to serve as the project director.

The WICHE Cooperative for Educational Technologies (WCET) is a cooperative, membership-driven, non-profit provider of solutions and services that accelerate the adoption of effective practices and policies, advancing excellence in technology-enhanced teaching and learning in higher education. More information about WCET’s institutional membership resources, services and common interest groups can be found on here.

In Case You Missed This Part (with thnx 2 @oldaily)

Picking up a thread from yesterday...

I continue to believe that the most immediate opportunity for demonstrating the value of learning analytics in institutions and enterprises comes from mining existing pools of data in creative and thoughtful ways. But Stephen Downes was absolutely right in his April 24th critique of my post Data Change Everything. He said that the emerging learning analytics paradigms will benefit not from mining old data new ways, but in looking for better ways to collect new information.

Doing old things in new ways is an important step toward change. Transformation means doing new things in new ways.

Gamification offers one very exciting new possibility for doing just exactly that. The mobile web opens lots of data collection opportunities and there are bound to be lots more.

Just think about it.

When Learning Analytics Meet Gamification

Great minds are already at work on a very big idea: What if we morph the emerging fascination with massive N predictive learning analytics with a mechanism for collecting massive transactional behavioral data where millions and millions of subjects already willingly volunteer their input?

Learning Analytics, meet Gamification.

I don't like the term gamificiation, either. But I'm not talking about the introduction of check-ins and badges to everything. THAT is something I would much prefer to call ... playing with games. Rather, I am talking about the benefits of considering the rich massive N data source for tracking the behavioral transactions of game play. And being able to subject that big pool of data to a battery of descriptive, inferential and predictive statistical methods to identify points of loss and momentum. 

Anne Derryberry, a 2010-2011 Fellow at the Reynolds Journalism Institute, University of Missouri School of Journalism, has been applying what she calls purpose-driven game design to journalism and journalism education. She has focused on games as an emerging medium and distribution channel for news-telling, particularly as a means to induce community engagement and civic participation.

Her research is also indicating ways to track and analyze behavioral transactions and player interactions of an online game to enable new ways to understand points of loss and momentum in communications- related endeavors. She's coined the term "journalytics" to describe how analytics and purpose-driven game design mesh to give journalism and news outlets the metrics they need to focus and adjust their interaction strategies.

Relevant for journalism. AND education. IMHO. Early and preliminary results. But still. We in the learning world can benefit from some of these ideas.

Anne will be quick to point out that America's Army and other MMOGs have been using game play for recruitment, community building and even into predictive performance advising and professional development guidance for a while now. She has been tracking serious games for learning, training and performance support for the past few years at her blog, I'm Serious.net. But she was fascinated at seeing if the things she saw in these game-based learning designs could be relevant in other disciplines. This past year she developed noozYou as a vehicle for testing some of her hypotheses about purpose-driven game design in journalism with Mizzou journalism students. 

She can tell you about this far, far better than I can. So I am going to suggest that you follow her on Twitter at @aderryberry and @noozYou. It's been fascinating stuff.

I WILL tell you she's been advising the Sage Road crew as we've been pushing our own predictive analytics horizons. We think this approach holds great promise and has immediate, direct positive implications for the new kinds of data collection opportunities.

I continue to believe the most immediate opportunity for demonstrating the value of learning analytics in institutions and enterprises is to mine existing pools of data in creative and thoughtful ways. But Stephen Downes was absolutely right in his April 24th critique of my post Data Change Everything. He said that some of the emerging learning analytics paradigms will benefit not from mining old data new ways, but in looking for better ways to collect new information. Here is one very exciting new possibility for doing just exactly that.