eLearning Roadtrip

Since You've Asked: More About the PAR Framework

The December 15, 2011 Chronicle of Higher Education's Wired Campus blog by Marc Parry shared news about a groundbreaking project we have taken on at WCET.  It describes our efforts to create a single, federated database of de-identified student records contributed by multiple universities and colleges. Six forward-thinking WCET member institutions have worked together this past six months to see if it was possible to analyze variables common across these institutions to look for patterns that can inform us about student loss and momentum. I'm pleased to say that we have succeeded. Baby steps. But still.

We've received a lot of questions. Here are a few answers. There will be more.

This is a multi-phased project. We are concluding our work on Phase 1. The best is yet to be.

Our first phase focused on demonstrating that it is posssible to get multiple postsecondary institutions here in the US to contribute their data to create a single unified data set. We are now applying a variety of descriptive, inferential and predictive analyses to look for patterns showing variables likely to affect student success. 

Phase 1 = Proof of Concept. Creating a single federated data set from the de-identified student records from six unique institutions has never been done before. 

We've started with 6 schools. We are adding additional schools as we continue our efforts to build out the federated data set. While 640,000 students records and 3.1 million course records are a lot more than most of us are used to working with, it is still just a sample from 6 schools. 

We've started with 34 variables that are common across each of the institutional partners. Much of our current work has focused on establishing common definitions of these variables to ensure that comparisons and aggregations are valid, reliable and repeatable. There will be more variables.

We are currently focused on online programs. Just about every single transaction associated with an online program is digitally captured and stored somewhere. It seemed the obvious, self-evident place to begin.

We've focused on pattern strategy exploration, not null hypothesis testing (at least not in the conventional sense). 

We're really not trying to answer research questions using big data analysis techniques. Not yet. We'll get there. First things first. The journey of a thousand miles begings with the first steps.

More to the point...if we do our work right, then perhaps some day YOU will be answering YOUR research questions using the federated data set using big data analysis techniques.

The team working on this effort brings expertise from institutional research, statistical analysis, quant and qual research, cognition and instruction, online learning, higher ed administration, educational public policy, project management, and commercial software development. They come from the public and for-profit sectors. Then are from community colleges and 4 year comprehensives. Several of our institutional partners are already award-winning, acknowledged practice leaders in the hot, hot, hot domain of learning analytics. These are some of the smartest, most dedicated, creative, committed people I know. 

None of us - not WCET, not the core team of experts, not any of our fabulous institutional partners - are making any kinds of reccommendations from our findings yet. You may have noticed that my quote in the Chronicle was about as non-specific as one can possibly be. But be assured, there are going to be some very interesting implications for practice and policy that will emerge as the new patterns we are finding are more sharply focused in the coming weeks and months. 

Yes of course we are going to share our findings. Sharing is a core WCET value. We exist to share. That's what Cooperatives do.

 

eLearning Isn't Software

One of the reasons I attend learning technology conferences is that they offer a highly efficient way to scan the horizon for emerging trends likely to affect learning technology adoption. Conferences also provide a chance to see which "idea seeds" took root from among last year's various trends du jour.  

In yesterday's summary of reflections from the past several weeks of of higher education, elearning, and learning technology conferences, I made the following observation: 

On the commercial front...Pearson intends to own our universe. Adobe has pretty much given up on us. Just about everybody wants to sell us an "analytics solution". Or a mobile app.

I seem to have gotten the attention of the Adobe elearning team. I think I may have hurt their feelings. Understandably. They will tell you that they are the market leader in elearning. There's Captivate. And Connect. And eLearning Suite.

I simply respond by acknowledging that those are fine products. But they are not elearning. Because no matter how much one might believe it to be so...elearning isn't software. eLearning is not a platform, unless one is speaking allegorically. eLearning is something people DO with software. It's a much bigger universe than rapid authoring, web conferencing, and content creation.

The fact that I am saying these words... well...

 

20 "eLearnings" From Two Months on the Road(trip)

Been a busy couple of months of elearning road-trips. From EDUCAUSE to Online EDUCA, from WCET to WICHE, from DevLearn to AECT (with a little bit of virtual Sloan-C tossed in for good measure). Literally thousands of plenaries, presentations, and poster sessions from which to choose, along with hundreds of vendor stands and trade show booths. Here are some of my big take-aways:

• Hooray and congratulations! This distance/online/elearning thing we do has tipped into mainstream consciousness. Let's give a cheer for the researchers, the teachers, the authors, the artists, the technologists, the producers, the analysts and the evaluators who have made this happen. We rock.
• We matter enough that governments want to regulate us. Congressmembers rail against us. No more flying under the radar.
• Analytics are hot, hot, hot. Learning analytics, content analytics, web analytics, engagement analytics, to name a few.
• Speaking of analytics...mathematically speaking, a learning outcome is pretty much the same as a point of sale. (Thank you, Phil Ice.)
• Second Life is dead, dead, dead. Sorry.
• Mobile is the future.  Smartphones and tablets and readers, oh, my.
• Game based learning has a long way to go before people take it seriously.
• High quality video conferencing is making a comeback.
• We may get inspired by big ideas, but we are moved to action by fear, uncertainty and dread.
• On the commercial front...Pearson intends to own our universe. Adobe has pretty much given up on us. Just about everybody wants to sell us an "analytics solution". Or a mobile app.
• People really need to quit thinking that "open" is the same as "free".
• It's a lot easier to pontificate about the importance of change than it is to do things differently.
• There is a lot of research out there. Some of it is useful. 
• Empirical evidence doesn't need to be quantitative...but it does need to be a little more robust than simply pointing to people who are doing the same things.
• Quality is rarely absolute. It's contextual and situational. It may make more sense to think about "effective practices" rather than "best practice".
• Practitioners who make big sweeping statements without doing any background work on what others may have already been doing, saying or thinking end up sounding just a little bit naive and foolish. C'mon, people...use The Google.
• On the Internet, nobody knows you're a dog...but EVERYONE knows if you are not prepared.
• One can pack a lot of snark in 140 characters.
• Learning is as natural as breathing. Learning really isn't the problem. Enabling meaningful, intentional outcomes and creating experiences that engage and inspire without costing an arm and a leg and without killing our interest and enthusiam for learning is the problem. 
• Sometimes you just need to show up.

Are YOU the Chief Online Education Officer at your Institution?

If so, we'd like to ask you to invite you to participate in a survey research project.

For the third year, WCET is partnering with the Campus Computing Project to conduct the Managing Online Education survey.  The survey focuses on the instructional, operational, and technology infrastructure of online education programs of institutions throughout the U.S. and Canada.

My colleague, Russ Poulin, has written an eloquent post on the WCET Frontiers blog, encouraging your particitation in this survey. To save you the hassle of clicking all the way through, I have excerpted part of his message, below. Please click here for the full post.

YOUR INSTITUTION NEEDS TO RESPOND TO THE SURVEY.

 <Sorry for yelling at you>

To avoid confusion, we would like one response from each institution. Given that some institutions are centralized, some are decentralized, and some laugh at the notion of organization, it is often difficult to know who is the correct person to invite to take the survey. There is no common position of “chief elearning officer” like there is with a CIO.

How Do You Participate?

Review the 2011 Managing Online Education survey to see what questions are being asked. If you are not what passes for a “chief Elearning officer,” ask around at your institution to see if someone has received an invitation and is completing the survey.  If you need an invitation or have other questions about the survey, contact Casey Green atcgreen@campuscomputing.net.  He will help you out and get you access to the survey.  Survey responses must be submitted through the online survey software and not by completing the .pdf linked earlier in this paragraph.

I’m looking forward to your participation, seeing the results, and sharing them with you!  NOTE:  THE SURVEY DEADLINE HAS BEEN EXTENDED TO DECEMBER 16.

Thanks!!

On Secret Handshakes and Making the World a Better Place with ID

I attended the 2011 annual conference of the Association for Educational Communications and Technology (AECT) in Jacksonville, FL this past week. AECT is a professional association focused on scholarship in the academic disciplines associated with instructional design. Its members are primarily college professors and their graduate students; members also include school librarians, media specialists, computer center directors, public television and publishing executives.

I spoke on the topic of Secret Handshakes of Instructional Design. My presentation was based on various blog posts I have written on this topic, as well as an essay published in the inaugural issue of the Journal of Applied Instructional Design on said same. It was intended as a gentle yet pointed look at this ID thing that we do. These may be the best of times for those who understand how to create experiences that inspire and enable learning, but our practice may be better served if we make our secret handshakes more inclusive, our practice less theoretical and abstract, and all our designs more actionable.

Here is the deck Download FINAL AECT Secret Handshakes for distrib I used in my presentation. I've removed copyrighted materials. All photos used in the deck come from iStockphoto. This presentation deck is distributed under a Creative Commons Attribution Share Alike, Non-commercial license.

Quick reflections - It was very nice to be back among the scholars. I enjoyed seeing former students and associates. It was great to talk with old friends, some I've known since my graduate school days. It was great to meet some new friends. I was inspired by the big ideas.

And yet, I was reminded that, regardless of our epistemological roots, our professional training, the number of or lack of our graduate degrees or the places where we work, many of us call ourselves IDs. And many of us in this wild and wacky ID practice universe continue to have a hard time recognizing one another. We come at this practice from diferent and various points in the ID value chain.

I was also reminded that big ideas and great intentions, while essential ingredients in successful recipes for change, are rarely sufficient for bringing about the kinds of systemic changes to culture and practice that are required to disrupt and transform.

Education transformation really is a team sport. Old ideas don't just fade away. Old ideas can be stubborn and relentless. Systemic change calls for many contributors. There are many variables to address, much practice and dedication required to develop expert skills. Transformation demands a clear understanding of what is to be changed, AND the will to see changed implemented. And with all due respect to everyone who claims the ID badge...We simply can't afford to all think the same way.

For real change - true transformation - we need all ID hands on deck. Scholars, analysts, artists, technologists, evaluators, and managers alike. Big thinkers. Expert practitioners. Skilled producers. People who can translate big ideas into actionable strategies and tactics driving real results about things that matter to the communities and stakeholders we purport to serve.

Secret handshakes DO help us recognize each other, help us evaluate what we and our fellow practitioners know, and what we don't. Maybe we DO need to make these "know 'em when you see 'em" attributes of our ID secret handshakes more explicit.  Maybe we really can use these secret handshakes as a conceptual checklist to find more of the expert practitioners who share ideas, information and practices that are similar or complementary to our own.

When you get right down to it, perhaps the most notable secret handshake all IDs share is the passion for finding solutions that make the world a better place for learning. And that is a great place to begin.

 

Lessons Revisited: RADAR, Pearl Harbor and Paying Attention to Evidence

One of our Sage Road Solutions LLC clients, the Western Interstate Commission for Higher Education, is holding its biannual meeting at the BYU Hawaii campus on the island of Oahu this week. Having spent the past several days in Denver hosting the annual meeting of the WICHE Cooperative for Educational Telecommunications, known as WCET), I'm still running on Mountain time. I woke up early, so I decided to take advantage of being in this glorious location to take a walk along the beach.

About 200 yards from the hotel an historical marker notes the location of the Opana mobile RADAR site. The story accompanying the marker offers an interesting object lesson for those of us trying to bring innovative applications of technology to the learning technology metaverse.

According to Wikipedia the term RADAR was coined in 1940 by the U.S. Navy standing for radio detection and ranging. It was brand new technology, showing enough promise that pilot programs were launched to test its utility. In 1941, a number of mobile radar trucks were deployed on Oahu to keep an eye on the skies near Hickam Field.

 

Early on the morning of December 7, 1941, two soldiers stationed at the mobile radar site near Opana Point saw electronic blips of what looked like many planes approaching the island. Reporting their findings, they were told that what they were seeing were *probably* planes expected from California, and not to worry about it.  Consequently this information was never reported to those in command. Shortly thereafter, the Japanese planes detected by the RADAR bombed Hickam Field and Pearl Harbor, destroying US ships and planes, leading to the US's declaration of war on Japan and igniting US involvement in World War II.

The historical information was certainly interesting in its own right. The narrative that accompanied the marker, however, was particularly striking:

"The failure to warn the Army or Navy command in Hawaii was not a failure of the technology as much as it was a failure of organization. While the technology of RADAR functioned as intended and detected the incoming planes, there was no way to accurately assess the information and communicate this knowledge to those in command." 

So, let's fast forward to October, 2011. On Thursday of this past week the WCET team working on our Predictive Analytics Reporting (PAR) Framework project revealed some of the first findings from our ground-breaking big data project. At the risk of sounding hyperbolic, the descriptive, inferential and predictive techniques being applied to this massive data set are providing some eye-opening perspectives on variables affecting student loss and momentum. As noted in a Forbes.com online article featuring Phil Ice, our project's principal investigator (who, by the way, really hates being called "the other Dr. Phil"...so please don't) these results are going to rock the way that educational decision-makers think about everything from college admissions, student services, technology investments and even "best practices" for online learning. 

Still, I have a feeling that these "big data" results will go through a period of being pooh-poohed by decision-makers who will have a hard time letting go of null hypothesis research methods. Traditional researchers' disdain for the so-called data mining techniques associated with pattern strategy and business intelligence used in other sectors are likely to get in the way. For many traditional researchers, educational data mining is a little bit like a stupid pet trick - even though these same folks really seem to like the convenience of Amazon shopping recomendations, Netflix movie recommendations and Pandora music recommendations.

I can't help but wonder if the findings about student loss and momentum that the cracker-jack research crew are sharing will be disregarded very much in the same way that Army and Navy command dismissed the evidence that Pearl Harbor was in harm's way because the techniques are too new to be taken seriously. I just hope that we can figure out how to help people pay attention to evidence before it is too late.

 

Big Data Comes to US Higher Education

WCET, the WICHE Cooperative for Educational Technologies, today announced the successful federation of datasets from the six institutions participating in the PAR (Predictive Analytics Reporting) Framework proof of concept project, just four months after the project’s early June launch. The goal of the PAR Framework is to identify variables that influence student retention and progression and to determine the impact of various demographic data on factors influencing loss and momentum. The data will be used to explore patterns that emerge when the datasets from considerably different institutions are analyzed as a single, unified sample. 

The dataset includes over 640,000 anonymized student records and over 3 million course level records, focusing on 33 common variables. The institutions represent public/private, two-year/four-year, and publicly-funded and proprietary institutions. 

“We have been using analytics as a tool for improving student retention and progression for several years,” said Dr. Phil Ice, vice president, research & development, American Public University System, and principal investigator of the PAR Framework Project. “Unifying records from multiple schools has the potential to expose generalizable patterns that can provide guidance to institutions across the board, regardless of their internal level of analytics expertise.” Ice continued, “In higher education, no predictive analytics project has been able to use a multi-institutional sample of this size. As a researcher, having a sample size of this number changes everything.” 

The WCET member institutions engaged in the PAR Framework project include American Public University System, Colorado Community College System, Rio Salado College, University of Hawaii System, University of Illinois Springfield, and the University of Phoenix. Each institution obtained IRB (Institutional Review Board) approval. All data was anonymized and encrypted to ensure that no data are personally identifiable. 

“We had been warned that we could never find six schools willing to collaborate on something as sensitive as an institutional student and course level data federation project,” noted Ellen Wagner, WCET Executive Director. “It’s exciting to see how forward-thinking WCET member institutions are bringing the power of “big data” to post-secondary education in the US. The magnitude of the data set unprecedented and the diversity of the participating institutions will provide us granular insight into similarities and differences across higher education institutions in the for profit, public and community college segments.” 

A summary of first findings are scheduled for presentations at the 23rd Annual WCET Conference in Denver, Colorado October 27th (https://wcetconference.wiche.edu/).

Badges?! We don’t need no stinkin’… Oh, wait, maybe we do.

Are you using badges yet?

via imserious.typepad.com

Great post from Anne Derryberry. Very pleased that she will be joining the Sage Road crew as a subject matter expert consultant as we take on the task of morphing learning analytics with gamification.

Honoring Steve

I have been asking friends if they had been startled at the strength of the emotions he or she has experienced at the passing of Steve Jobs. To a person, each has said yes.

I've wondered if some of the deep emotions are triggered when hearing and re-hearing his various inspirational speeches. They force a bit of self-reflection. As in, well, asking myself if MY attempts to think differently are workin' for me. Am I doing what I love? Uh....well...of course. But...THAT got me feeling all philosophical. And the more I reflected, the more I got all twisted up in a big emotional pretzel.

Then it dawned on me. It didn't have to be as hard as I was making it.

When I thought about what I really love (besides my spouse, colleagues and projects, of course), it was easy.

Cheese. I love cheese.

Once I said it out loud, it was super obvious what I could do to resolve this sadness. I signed up for a cheesemaking class. Something I've wanted to do for a bazillion years, but never made time to do it. 

My real lesson? Well, it's really not about the cheese, obviously. It's about making time to do the things that you love, want to love, or want to love more.

I've started daydreaming about "iChevre", "iFeta" and  "iPaneer". All of a sudden I feel just a little bitty bit less sad.

 

 

Thought You Might be Interested...

...in some of the thoughts I shared with my WCET crew today about learning analytics and lessons we can learn from Moneyball. It's an extension of my eLearning Roadtrip post on the same general topic.

I suppose this gives you an idea of what is top-of-mind for me these days. Well, that and wanting to raise the bar on online learning excellence. Simple pleasures.

If you are interested in taking a look at my WCET post, please click here. 

While there, please take a look around at what the fine folks at the WICHE Cooperative for Educational Technologies are doing these days. If you happen to care about accelerating the adoption of learning technologies innovation in practice you might find that WCET is engaged in work that matters to you.

As a matter of fact....WCET's Annual Meeting is being held in Denver October 26-29. We will be sharing some of the preliminary results from our Predictive Analytics Reporting Framework, along with lots of other information about learning analytics and other emerging topics of interest to online learning and learning technology professionals. Might be a great time to come see what the WCET buzz is all about.