eLearning Roadtrip

Since You've Asked: More About the PAR Framework

The December 15, 2011 Chronicle of Higher Education's Wired Campus blog by Marc Parry shared news about a groundbreaking project we have taken on at WCET.  It describes our efforts to create a single, federated database of de-identified student records contributed by multiple universities and colleges. Six forward-thinking WCET member institutions have worked together this past six months to see if it was possible to analyze variables common across these institutions to look for patterns that can inform us about student loss and momentum. I'm pleased to say that we have succeeded. Baby steps. But still.

We've received a lot of questions. Here are a few answers. There will be more.

This is a multi-phased project. We are concluding our work on Phase 1. The best is yet to be.

Our first phase focused on demonstrating that it is posssible to get multiple postsecondary institutions here in the US to contribute their data to create a single unified data set. We are now applying a variety of descriptive, inferential and predictive analyses to look for patterns showing variables likely to affect student success. 

Phase 1 = Proof of Concept. Creating a single federated data set from the de-identified student records from six unique institutions has never been done before. 

We've started with 6 schools. We are adding additional schools as we continue our efforts to build out the federated data set. While 640,000 students records and 3.1 million course records are a lot more than most of us are used to working with, it is still just a sample from 6 schools. 

We've started with 34 variables that are common across each of the institutional partners. Much of our current work has focused on establishing common definitions of these variables to ensure that comparisons and aggregations are valid, reliable and repeatable. There will be more variables.

We are currently focused on online programs. Just about every single transaction associated with an online program is digitally captured and stored somewhere. It seemed the obvious, self-evident place to begin.

We've focused on pattern strategy exploration, not null hypothesis testing (at least not in the conventional sense). 

We're really not trying to answer research questions using big data analysis techniques. Not yet. We'll get there. First things first. The journey of a thousand miles begings with the first steps.

More to the point...if we do our work right, then perhaps some day YOU will be answering YOUR research questions using the federated data set using big data analysis techniques.

The team working on this effort brings expertise from institutional research, statistical analysis, quant and qual research, cognition and instruction, online learning, higher ed administration, educational public policy, project management, and commercial software development. They come from the public and for-profit sectors. Then are from community colleges and 4 year comprehensives. Several of our institutional partners are already award-winning, acknowledged practice leaders in the hot, hot, hot domain of learning analytics. These are some of the smartest, most dedicated, creative, committed people I know. 

None of us - not WCET, not the core team of experts, not any of our fabulous institutional partners - are making any kinds of reccommendations from our findings yet. You may have noticed that my quote in the Chronicle was about as non-specific as one can possibly be. But be assured, there are going to be some very interesting implications for practice and policy that will emerge as the new patterns we are finding are more sharply focused in the coming weeks and months. 

Yes of course we are going to share our findings. Sharing is a core WCET value. We exist to share. That's what Cooperatives do.

 

20 "eLearnings" From Two Months on the Road(trip)

Been a busy couple of months of elearning road-trips. From EDUCAUSE to Online EDUCA, from WCET to WICHE, from DevLearn to AECT (with a little bit of virtual Sloan-C tossed in for good measure). Literally thousands of plenaries, presentations, and poster sessions from which to choose, along with hundreds of vendor stands and trade show booths. Here are some of my big take-aways:

• Hooray and congratulations! This distance/online/elearning thing we do has tipped into mainstream consciousness. Let's give a cheer for the researchers, the teachers, the authors, the artists, the technologists, the producers, the analysts and the evaluators who have made this happen. We rock.
• We matter enough that governments want to regulate us. Congressmembers rail against us. No more flying under the radar.
• Analytics are hot, hot, hot. Learning analytics, content analytics, web analytics, engagement analytics, to name a few.
• Speaking of analytics...mathematically speaking, a learning outcome is pretty much the same as a point of sale. (Thank you, Phil Ice.)
• Second Life is dead, dead, dead. Sorry.
• Mobile is the future.  Smartphones and tablets and readers, oh, my.
• Game based learning has a long way to go before people take it seriously.
• High quality video conferencing is making a comeback.
• We may get inspired by big ideas, but we are moved to action by fear, uncertainty and dread.
• On the commercial front...Pearson intends to own our universe. Adobe has pretty much given up on us. Just about everybody wants to sell us an "analytics solution". Or a mobile app.
• People really need to quit thinking that "open" is the same as "free".
• It's a lot easier to pontificate about the importance of change than it is to do things differently.
• There is a lot of research out there. Some of it is useful. 
• Empirical evidence doesn't need to be quantitative...but it does need to be a little more robust than simply pointing to people who are doing the same things.
• Quality is rarely absolute. It's contextual and situational. It may make more sense to think about "effective practices" rather than "best practice".
• Practitioners who make big sweeping statements without doing any background work on what others may have already been doing, saying or thinking end up sounding just a little bit naive and foolish. C'mon, people...use The Google.
• On the Internet, nobody knows you're a dog...but EVERYONE knows if you are not prepared.
• One can pack a lot of snark in 140 characters.
• Learning is as natural as breathing. Learning really isn't the problem. Enabling meaningful, intentional outcomes and creating experiences that engage and inspire without costing an arm and a leg and without killing our interest and enthusiam for learning is the problem. 
• Sometimes you just need to show up.

WCET Receives Grant for Groundbreaking Higher Education Analytic Research

I am pleased to share the following good news.

WICHE, the Western Interstate Commission for Higher Education, announced today that WCET, the WICHE Cooperative for Educational Technologies, has been awarded a $1,000,000 grant from the Bill & Melinda Gates Foundation to fund an initiative to unify student data from six U.S. institutions and demonstrate the effective use of predictive analytic methods for improving student outcomes. The goal is to identify variables that influence student retention and progression, and guide decision-making that improves postsecondary student completion in the U.S.

This grant for the Predictive Analytics Reporting (PAR) Framework will aggregate data representing more than 400,000 student records from across six WCET member institutions: American Public University System, Colorado Community College System, Rio Salado College, University of Hawaii System, University of Illinois Springfield, and the University of Phoenix. Each participating institution has been exploring or implementing descriptive, inferential or predictive analytics projects on their own student data; the PAR Framework expands on this work through exploration of patterns that can be derived when the six institutional datasets are considered as a single, unified sample.

The principal investigator is Phil Ice, Ed.D., American Public University System (APUS). The project will incorporate many of the best practices and award-winning predictive analytical methods for which Phil has been recognized by the Sloan Consortium and for which project partner Dr. Vernon Smith, Rio Salado College, has won the prestigious WCET Outstanding Work (WOW) award. WCET has appointed Beth Davis, WCET, to serve as the project director.

The WICHE Cooperative for Educational Technologies (WCET) is a cooperative, membership-driven, non-profit provider of solutions and services that accelerate the adoption of effective practices and policies, advancing excellence in technology-enhanced teaching and learning in higher education. More information about WCET’s institutional membership resources, services and common interest groups can be found on here.

In Case You Missed This Part (with thnx 2 @oldaily)

Picking up a thread from yesterday...

I continue to believe that the most immediate opportunity for demonstrating the value of learning analytics in institutions and enterprises comes from mining existing pools of data in creative and thoughtful ways. But Stephen Downes was absolutely right in his April 24th critique of my post Data Change Everything. He said that the emerging learning analytics paradigms will benefit not from mining old data new ways, but in looking for better ways to collect new information.

Doing old things in new ways is an important step toward change. Transformation means doing new things in new ways.

Gamification offers one very exciting new possibility for doing just exactly that. The mobile web opens lots of data collection opportunities and there are bound to be lots more.

Just think about it.

When Learning Analytics Meet Gamification

Great minds are already at work on a very big idea: What if we morph the emerging fascination with massive N predictive learning analytics with a mechanism for collecting massive transactional behavioral data where millions and millions of subjects already willingly volunteer their input?

Learning Analytics, meet Gamification.

I don't like the term gamificiation, either. But I'm not talking about the introduction of check-ins and badges to everything. THAT is something I would much prefer to call ... playing with games. Rather, I am talking about the benefits of considering the rich massive N data source for tracking the behavioral transactions of game play. And being able to subject that big pool of data to a battery of descriptive, inferential and predictive statistical methods to identify points of loss and momentum. 

Anne Derryberry, a 2010-2011 Fellow at the Reynolds Journalism Institute, University of Missouri School of Journalism, has been applying what she calls purpose-driven game design to journalism and journalism education. She has focused on games as an emerging medium and distribution channel for news-telling, particularly as a means to induce community engagement and civic participation.

Her research is also indicating ways to track and analyze behavioral transactions and player interactions of an online game to enable new ways to understand points of loss and momentum in communications- related endeavors. She's coined the term "journalytics" to describe how analytics and purpose-driven game design mesh to give journalism and news outlets the metrics they need to focus and adjust their interaction strategies.

Relevant for journalism. AND education. IMHO. Early and preliminary results. But still. We in the learning world can benefit from some of these ideas.

Anne will be quick to point out that America's Army and other MMOGs have been using game play for recruitment, community building and even into predictive performance advising and professional development guidance for a while now. She has been tracking serious games for learning, training and performance support for the past few years at her blog, I'm Serious.net. But she was fascinated at seeing if the things she saw in these game-based learning designs could be relevant in other disciplines. This past year she developed noozYou as a vehicle for testing some of her hypotheses about purpose-driven game design in journalism with Mizzou journalism students. 

She can tell you about this far, far better than I can. So I am going to suggest that you follow her on Twitter at @aderryberry and @noozYou. It's been fascinating stuff.

I WILL tell you she's been advising the Sage Road crew as we've been pushing our own predictive analytics horizons. We think this approach holds great promise and has immediate, direct positive implications for the new kinds of data collection opportunities.

I continue to believe the most immediate opportunity for demonstrating the value of learning analytics in institutions and enterprises is to mine existing pools of data in creative and thoughtful ways. But Stephen Downes was absolutely right in his April 24th critique of my post Data Change Everything. He said that some of the emerging learning analytics paradigms will benefit not from mining old data new ways, but in looking for better ways to collect new information. Here is one very exciting new possibility for doing just exactly that.

 

Secret Handshakes of ID

I recently contributed an essay to the inaugural issue of the Journal of Applied Instructional Design. This is a new journal, sponsored by the Association for Educational Communications and Technology, aimed at giving the ID scholar - practitioner greater voice. I contributed an essay, called "In Search of Secret Handshakes of Instructional Design", that talked about the importance of finding the "know 'em when we seen 'em" attributes for recognizing fellow ID practitioners when we encounter them on the long and winding road to enterprise learning. I also observed that theoretical foundations guiding the study of the evolution of a field can fall awkwardly out of alignment with the evolution of a professional practice, particularly one so directly affected by the speed of technological change.

Here is a link to the full essay. 

And here is a 5 minute summary:

Whether we use a constructivistic approach, a social learning approach, a connectionistic approach or a behavioral approach, IDs produce value through the design, development and distribution of learning solutions. We used to look more like teachers, trainers and psychologists than artists, scripters or programmers, but that balance seems to have shifted a bit. Today's IDs work with technology tools, because so much of today's learning and performance support is enabled / managed / distributed via technology. IDs create learning content and courseware. But IDs are not just elearning content authors, either. IDs are also engaged in supporting and enabling distance learning's web collaborations and with mobile learning's apps and mediacasts, too. AND running the LMS.

IDs used to mostly come to the practice out of graduate instructional design and educational technology programs. Today, perhaps as many as two-thirds of today's ID practitioners come to the practice of ID from applied settings.  We come from creative professions (e.g. artists, designers, producers). We became IDs when assigned with learning and development responsibilities. We are the IT professional who is put in charge of the enterprise LMS. We are the training manager who gets put in charge of the new elearning – mobile learning – game based learning – virtual world learning – initiative that the enterprise wants to explore.

Regardless of our epistemological roots, our professional training, the number of or lack of our graduate degrees or the places where we work, we call ourselves IDs.

Do we know how to recognize each other when we see each other??

To be continued....

(Of course it will. You don't think I would get this all queued up and walk away do you??)

 

Data Change Everything

 
During the past several months there have been low background rumblings in the land of education and training, and the rumbling is starting to get louder. That is the sound of the learning world discovering what Internet professionals working in other vertical markets have known for years: The digital “breadcrumbs” that learners leave behind about their viewing, reading, engagement and assessment behaviors, interests and preferences provide massive amounts of data that can be mined to better personalize online experiences. 

Learning analytics are all about applying statistics and research methods in the service of better informed, more accountable decision-making. We're talking about using descriptive statistics, inferential statistics and predictive statistics on the data that we collect about, well, just about everything associated with Online Learning. eLearning. Distance Learning. Whatever you call it.

While predictive analytics are commonly used in consumer settings (e.g. to help generate recommendations for Amazon shoppers and select movies on Netflix), predictive analytics are not yet broadly used in educational settings for assisting with activities such as course selection, or for predicting points in a student’s academic experience where they may be at a point of increased risk. We're not even very good at keeping track of descriptive information in education, a la Google Analytics. at least not yet.

But we're getting there.

Here are five things I know for sure about the rising tide of learning analytics and what this is going to mean for helping people make better, more informed decisions about learning. Yours, mine, theirs, ours.

 

A vast number of today’s online learning transactions are captured in Content Management Systems, Learning Management Systems, Student Information Systems and Enterprise Resource Planning systems. We know A LOT about what happens when students - and faculty - are online. Given what we know about web analytics in Google, and personalized recommendation engines in Netflix and Amazon, and the power of hastags in Twitter, "thumbs up" in Pandora and Facebook...why WOULDN'T we want to personalize our learning experiences in the same way? 

I know for some that the most fun is going to be getting all crazy with the complex statistical modeling that this kind of work is going to require AND enable. But for a lot of us, what we are really going to care about is that learning analytics give ME a foundation for my own learning progress and completion. Set up the right way, these personalized predictively generated experiences will help ME stay on top of what I need to know. And THAT is very cool.

This isn't so much about collecting more information from people. It's more about getting smarter and  putting the data we already collect to better use.

 

This is the hard part. This is the "admitting we may have a problem" part. Because once you take a look at what is going on, you need to be prepared for the fact that you are going to need to respond. To SOMETHING, anything. Ignore at your peril.

Same basic idea, just a different way of saying it. I will tell you all that I've been watching people squirming about how to report results in ways that show the not-as-good as well as the good in a couple of projects recently - enough to tell me that the benefits of learning analytics are going to be balanced with some big cultural barriers that will require direct and focused attention.

Yup. These data we leave everywhere we go...they really are going to change everything. Because once we KNOW what we know, there will be no going back.

 

 

 

What Dave Longanecker Said About Alaska

During the WCET Annual Meeting Thursday November 11, 2010 Luncheon presentation, during the session the Debate  between Dr. Dave Longanecker and Dr. Peter Smith on the topic of Rethinking the Academy in times of Great Change, a question about higher education cost was presented to the debaters from the audience.

Peter responded first, talking about the importance of choice, and that students benefit when there are a variety of options from subsidized to completely tuition driven. Then it was Dave's turn. And this is part of what Dave said:

"If you look at National Center for Higher Education Management data, there's the 250% difference in the cost per degree from one state to ...from the lowest cost state to the highest cost state. And it's not a quality...if you looked at the states you will see that quality is not the determining factor. The most expensive cost per degree state in the country is Alaska. I work with Alaska, I love Alaska, but I wouldn't put it as the most, as the best system in higher education in the country."

If you click here you will be redirected to the Sonic Foundry Mediasite archived recording of the Debate. Move the slider bar to the time-stamp at 45:45. (That's 45 minutes and 45 seconds into the session, less than two minutes before the entire session was adjourned). You can see and hear what Dave said for yourself.

Here is  the NCHEMS data that Dave was referencing when he made his comment about most expensive cost per degree state.

This slide was taken from a presentation that Dave had recently given to the CCCCIO conference in San Diego CA on October 28th. You can download the entire presentation here  Download Statistics from 101028 - CCCCIO Presentation on Quality Improvement Pretty interesting stuff about higher education that most of us typically don't see about cost, graduation, tuition and so on. 

His comments upset several of the WCET attendees from Alaska, who talked to me, talked to him, have written letters and fired off tweets. I know Dave has already apologized. It's not worth pissing people off. Especially when absolutely no harm was intended.

As this issue had been raised to public awareness through the #wcet10 tweet-stream I wanted to be sure people had access to the links to take a look for themselves, and to see some of the data that Dave was referencing.

 

 

 

Management of Online Education Survey Completion Reminder

The 2010 Managing Online Education Survey went "live" this past week. This is the second annual survey, jointly sponsored by WCET and the Campus Computing Project, that documents the nuts and bolts of what it takes to successfully run online learning programs. Dr. Kenneth C. (Casey) Green, Founder of the Campus Computing Project, is the lead author for this important exploration. Invitations to participate in the survey were distributed last week.

IF YOU HAVE NOT YET RECEIVED YOUR INVITATION WE WANT TO HEAR FROM YOU!!!!! Now that online learning IS hot, hot hot it is really important that any and all of you working in higher education online learning make sure that your voice is heard. Please contact Casey directly at cgreen@campuscomputing.net. Tell him Ellen told you to get in touch with him about the MOE Survey. Don't be shy.

It is possible that your institution is not already part of WCET - possible that your invitation got spam filtered - perhaps the name we have for your institutional point of contact is wrong. So this is my personal invitation to please be sure that your institution is represented in this year's survey results. Get in touch with Casey at cgreen@campuscomputing.net. The deadline for completion is November 4th 2010.

Seriously. Don't be shy.

Thanks very much.

New from NCES: Postsecondary Institutions and Price of Attendance in US

The National Center for Education Statistics has just released a publication entitled ‘Postsecondary Institutions and Price of Attendance in the United States: Fall 2009 and Degrees and Other Awards Conferred: 2008-09, and 12-Month Enrollment 2008-09'. Great stuff if you are a statistics junkie. And even if you are not.

I learn about these reports from my clients and colleagues at WICHE, the Western Interstate Commission for Higher Education. WICHE publishes alerts of interest to education policy and research communities. You can SUBSCRIBE or UNSUBSCRIBE to the WICHE Policy Alerts and/or Stat Alerts distribution lists. WICHE also maintains a Policy Publication Clearinghouse, which includes past Policy Alerts and Stat Alerts, as well as other studies, reports, surveys, and policy briefs.

But back to the new NCES Report -  in 2009-10, there were 6,896 Title IV institutions in the United States (this is a classification that relates to one eligibility for federal financial aid) and other jurisdictions:

• 2,853 were classified as 4-year institutions
• 2,259 were 2-year institutions
• the remaining 1,784 were less-than-2-year institutions.

During 2009-10, 4-year public institutions reported average tuition and required fees of $6,400 for in-state students and $15,100 for out-of-state students.

4-year private not-for-profit institutions are charging an average of $21,100, while Private for-profit institutions reported charging an average of about $15,700

Between 2000-01 and 2009-10, 4-year public institutions reported a 46% increase in average inflation-adjusted in-state tuition and required fees and a 34% increase in average inflation-adjusted out-of-state tuition and required fees.

Over the same 10-year period, 4-year private not-for-profit institutions reported a 31% increase and private for-profit institutions reported a 20% increase in average inflation-adjusted tuition and required fees.

For more information on this topic please refer to the NCES and WICHE site links, above.