eLearning Roadtrip

The Big Data Landscape

If you post the term "Big Data" in your Google search window today, more than 2 billion google search results will appear. If you perform a similar search on Bing, you will see that more than 1,790,000 Big Data links have been added to the Internet in the past 24 hours alone. Clearly, there is big interest in big data.

This graphic by Matt Turck and Shivon Zilis is an exceedingly helpful image for understanding the breadth and complexity of the Big Data landscape. Matt Turck's blogpost  also provides great information for understanding some of where this emerging market is taking us. 

 

 

For those of us working in post-secondary education circles, the emergent interests in big data, analytics, learning analytics and related evidence-based decision-making endeavors have been inspired by possibilities of better experiences, more personalization and better behavioral targeting. Still most education stakeholders who are giving learning analytics some consideration may be surprised to see the depth, breath and sophistication of the big data world.  While WCET's work on the PAR Framework has certainly contributed to a growing awareness on the power of predictive analytics for academic risk identification and the value of collaborative, multi-institutional data mining to improve student success, we are mindful that our student success focus touches a relatively narrow, targeted part of the Big Data picture. This graphic offers a more complete picture of the total ecosystem.

Since You've Asked: More About the PAR Framework

The December 15, 2011 Chronicle of Higher Education's Wired Campus blog by Marc Parry shared news about a groundbreaking project we have taken on at WCET.  It describes our efforts to create a single, federated database of de-identified student records contributed by multiple universities and colleges. Six forward-thinking WCET member institutions have worked together this past six months to see if it was possible to analyze variables common across these institutions to look for patterns that can inform us about student loss and momentum. I'm pleased to say that we have succeeded. Baby steps. But still.

We've received a lot of questions. Here are a few answers. There will be more.

This is a multi-phased project. We are concluding our work on Phase 1. The best is yet to be.

Our first phase focused on demonstrating that it is posssible to get multiple postsecondary institutions here in the US to contribute their data to create a single unified data set. We are now applying a variety of descriptive, inferential and predictive analyses to look for patterns showing variables likely to affect student success. 

Phase 1 = Proof of Concept. Creating a single federated data set from the de-identified student records from six unique institutions has never been done before. 

We've started with 6 schools. We are adding additional schools as we continue our efforts to build out the federated data set. While 640,000 students records and 3.1 million course records are a lot more than most of us are used to working with, it is still just a sample from 6 schools. 

We've started with 34 variables that are common across each of the institutional partners. Much of our current work has focused on establishing common definitions of these variables to ensure that comparisons and aggregations are valid, reliable and repeatable. There will be more variables.

We are currently focused on online programs. Just about every single transaction associated with an online program is digitally captured and stored somewhere. It seemed the obvious, self-evident place to begin.

We've focused on pattern strategy exploration, not null hypothesis testing (at least not in the conventional sense). 

We're really not trying to answer research questions using big data analysis techniques. Not yet. We'll get there. First things first. The journey of a thousand miles begings with the first steps.

More to the point...if we do our work right, then perhaps some day YOU will be answering YOUR research questions using the federated data set using big data analysis techniques.

The team working on this effort brings expertise from institutional research, statistical analysis, quant and qual research, cognition and instruction, online learning, higher ed administration, educational public policy, project management, and commercial software development. They come from the public and for-profit sectors. Then are from community colleges and 4 year comprehensives. Several of our institutional partners are already award-winning, acknowledged practice leaders in the hot, hot, hot domain of learning analytics. These are some of the smartest, most dedicated, creative, committed people I know. 

None of us - not WCET, not the core team of experts, not any of our fabulous institutional partners - are making any kinds of reccommendations from our findings yet. You may have noticed that my quote in the Chronicle was about as non-specific as one can possibly be. But be assured, there are going to be some very interesting implications for practice and policy that will emerge as the new patterns we are finding are more sharply focused in the coming weeks and months. 

Yes of course we are going to share our findings. Sharing is a core WCET value. We exist to share. That's what Cooperatives do.

 

Are YOU the Chief Online Education Officer at your Institution?

If so, we'd like to ask you to invite you to participate in a survey research project.

For the third year, WCET is partnering with the Campus Computing Project to conduct the Managing Online Education survey.  The survey focuses on the instructional, operational, and technology infrastructure of online education programs of institutions throughout the U.S. and Canada.

My colleague, Russ Poulin, has written an eloquent post on the WCET Frontiers blog, encouraging your particitation in this survey. To save you the hassle of clicking all the way through, I have excerpted part of his message, below. Please click here for the full post.

YOUR INSTITUTION NEEDS TO RESPOND TO THE SURVEY.

 <Sorry for yelling at you>

To avoid confusion, we would like one response from each institution. Given that some institutions are centralized, some are decentralized, and some laugh at the notion of organization, it is often difficult to know who is the correct person to invite to take the survey. There is no common position of “chief elearning officer” like there is with a CIO.

How Do You Participate?

Review the 2011 Managing Online Education survey to see what questions are being asked. If you are not what passes for a “chief Elearning officer,” ask around at your institution to see if someone has received an invitation and is completing the survey.  If you need an invitation or have other questions about the survey, contact Casey Green at[email protected].  He will help you out and get you access to the survey.  Survey responses must be submitted through the online survey software and not by completing the .pdf linked earlier in this paragraph.

I’m looking forward to your participation, seeing the results, and sharing them with you!  NOTE:  THE SURVEY DEADLINE HAS BEEN EXTENDED TO DECEMBER 16.

Thanks!!

Lessons Revisited: RADAR, Pearl Harbor and Paying Attention to Evidence

One of our Sage Road Solutions LLC clients, the Western Interstate Commission for Higher Education, is holding its biannual meeting at the BYU Hawaii campus on the island of Oahu this week. Having spent the past several days in Denver hosting the annual meeting of the WICHE Cooperative for Educational Telecommunications, known as WCET), I'm still running on Mountain time. I woke up early, so I decided to take advantage of being in this glorious location to take a walk along the beach.

About 200 yards from the hotel an historical marker notes the location of the Opana mobile RADAR site. The story accompanying the marker offers an interesting object lesson for those of us trying to bring innovative applications of technology to the learning technology metaverse.

According to Wikipedia the term RADAR was coined in 1940 by the U.S. Navy standing for radio detection and ranging. It was brand new technology, showing enough promise that pilot programs were launched to test its utility. In 1941, a number of mobile radar trucks were deployed on Oahu to keep an eye on the skies near Hickam Field.

 

Early on the morning of December 7, 1941, two soldiers stationed at the mobile radar site near Opana Point saw electronic blips of what looked like many planes approaching the island. Reporting their findings, they were told that what they were seeing were *probably* planes expected from California, and not to worry about it.  Consequently this information was never reported to those in command. Shortly thereafter, the Japanese planes detected by the RADAR bombed Hickam Field and Pearl Harbor, destroying US ships and planes, leading to the US's declaration of war on Japan and igniting US involvement in World War II.

The historical information was certainly interesting in its own right. The narrative that accompanied the marker, however, was particularly striking:

"The failure to warn the Army or Navy command in Hawaii was not a failure of the technology as much as it was a failure of organization. While the technology of RADAR functioned as intended and detected the incoming planes, there was no way to accurately assess the information and communicate this knowledge to those in command." 

So, let's fast forward to October, 2011. On Thursday of this past week the WCET team working on our Predictive Analytics Reporting (PAR) Framework project revealed some of the first findings from our ground-breaking big data project. At the risk of sounding hyperbolic, the descriptive, inferential and predictive techniques being applied to this massive data set are providing some eye-opening perspectives on variables affecting student loss and momentum. As noted in a Forbes.com online article featuring Phil Ice, our project's principal investigator (who, by the way, really hates being called "the other Dr. Phil"...so please don't) these results are going to rock the way that educational decision-makers think about everything from college admissions, student services, technology investments and even "best practices" for online learning. 

Still, I have a feeling that these "big data" results will go through a period of being pooh-poohed by decision-makers who will have a hard time letting go of null hypothesis research methods. Traditional researchers' disdain for the so-called data mining techniques associated with pattern strategy and business intelligence used in other sectors are likely to get in the way. For many traditional researchers, educational data mining is a little bit like a stupid pet trick - even though these same folks really seem to like the convenience of Amazon shopping recomendations, Netflix movie recommendations and Pandora music recommendations.

I can't help but wonder if the findings about student loss and momentum that the cracker-jack research crew are sharing will be disregarded very much in the same way that Army and Navy command dismissed the evidence that Pearl Harbor was in harm's way because the techniques are too new to be taken seriously. I just hope that we can figure out how to help people pay attention to evidence before it is too late.

 

Big Data Comes to US Higher Education

WCET, the WICHE Cooperative for Educational Technologies, today announced the successful federation of datasets from the six institutions participating in the PAR (Predictive Analytics Reporting) Framework proof of concept project, just four months after the project’s early June launch. The goal of the PAR Framework is to identify variables that influence student retention and progression and to determine the impact of various demographic data on factors influencing loss and momentum. The data will be used to explore patterns that emerge when the datasets from considerably different institutions are analyzed as a single, unified sample. 

The dataset includes over 640,000 anonymized student records and over 3 million course level records, focusing on 33 common variables. The institutions represent public/private, two-year/four-year, and publicly-funded and proprietary institutions. 

“We have been using analytics as a tool for improving student retention and progression for several years,” said Dr. Phil Ice, vice president, research & development, American Public University System, and principal investigator of the PAR Framework Project. “Unifying records from multiple schools has the potential to expose generalizable patterns that can provide guidance to institutions across the board, regardless of their internal level of analytics expertise.” Ice continued, “In higher education, no predictive analytics project has been able to use a multi-institutional sample of this size. As a researcher, having a sample size of this number changes everything.” 

The WCET member institutions engaged in the PAR Framework project include American Public University System, Colorado Community College System, Rio Salado College, University of Hawaii System, University of Illinois Springfield, and the University of Phoenix. Each institution obtained IRB (Institutional Review Board) approval. All data was anonymized and encrypted to ensure that no data are personally identifiable. 

“We had been warned that we could never find six schools willing to collaborate on something as sensitive as an institutional student and course level data federation project,” noted Ellen Wagner, WCET Executive Director. “It’s exciting to see how forward-thinking WCET member institutions are bringing the power of “big data” to post-secondary education in the US. The magnitude of the data set unprecedented and the diversity of the participating institutions will provide us granular insight into similarities and differences across higher education institutions in the for profit, public and community college segments.” 

A summary of first findings are scheduled for presentations at the 23rd Annual WCET Conference in Denver, Colorado October 27th (https://wcetconference.wiche.edu/).

Thought You Might be Interested...

...in some of the thoughts I shared with my WCET crew today about learning analytics and lessons we can learn from Moneyball. It's an extension of my eLearning Roadtrip post on the same general topic.

I suppose this gives you an idea of what is top-of-mind for me these days. Well, that and wanting to raise the bar on online learning excellence. Simple pleasures.

If you are interested in taking a look at my WCET post, please click here. 

While there, please take a look around at what the fine folks at the WICHE Cooperative for Educational Technologies are doing these days. If you happen to care about accelerating the adoption of learning technologies innovation in practice you might find that WCET is engaged in work that matters to you.

As a matter of fact....WCET's Annual Meeting is being held in Denver October 26-29. We will be sharing some of the preliminary results from our Predictive Analytics Reporting Framework, along with lots of other information about learning analytics and other emerging topics of interest to online learning and learning technology professionals. Might be a great time to come see what the WCET buzz is all about.

WCET Receives Grant for Groundbreaking Higher Education Analytic Research

I am pleased to share the following good news.

WICHE, the Western Interstate Commission for Higher Education, announced today that WCET, the WICHE Cooperative for Educational Technologies, has been awarded a $1,000,000 grant from the Bill & Melinda Gates Foundation to fund an initiative to unify student data from six U.S. institutions and demonstrate the effective use of predictive analytic methods for improving student outcomes. The goal is to identify variables that influence student retention and progression, and guide decision-making that improves postsecondary student completion in the U.S.

This grant for the Predictive Analytics Reporting (PAR) Framework will aggregate data representing more than 400,000 student records from across six WCET member institutions: American Public University System, Colorado Community College System, Rio Salado College, University of Hawaii System, University of Illinois Springfield, and the University of Phoenix. Each participating institution has been exploring or implementing descriptive, inferential or predictive analytics projects on their own student data; the PAR Framework expands on this work through exploration of patterns that can be derived when the six institutional datasets are considered as a single, unified sample.

The principal investigator is Phil Ice, Ed.D., American Public University System (APUS). The project will incorporate many of the best practices and award-winning predictive analytical methods for which Phil has been recognized by the Sloan Consortium and for which project partner Dr. Vernon Smith, Rio Salado College, has won the prestigious WCET Outstanding Work (WOW) award. WCET has appointed Beth Davis, WCET, to serve as the project director.

The WICHE Cooperative for Educational Technologies (WCET) is a cooperative, membership-driven, non-profit provider of solutions and services that accelerate the adoption of effective practices and policies, advancing excellence in technology-enhanced teaching and learning in higher education. More information about WCET’s institutional membership resources, services and common interest groups can be found on here.

There's a New Sheriff in (US Higher Education and Labor) Town

Dr. Darcy Hardy has been appointed to the Office of Assistant Secretary Jane Oats, Employment and Training Administration (ETA), United States Department of Labor. She will be supporting the recently announced Trade Adjustment Assistance Community College and Career Training (CCCT) Grant Program http://www.grants.gov ) that plans to award approximately $2 billion over the next four years.

She has taken a leave from her current position as Assistant Vice Provost for Technology Education Initiatives, University of Texas at San Antonio, Office of Information Technology, and has moved to Washington DC for her one year appointment.

I am very pleased (and relieved) that she will be able to continue on my WCET Executive Council during her time with the US Labor Department.

You many remember that, a few months back, Darcy and I had a bet over the World Series baseball championship. The loser of the bet was to present a webcast on a topic of the winner's choice. (There were a few other details of the bet, but we'll come back to those in a future post). Lucky me, I won the bet. I have asked Darcy to please plan for a WCET webcast in late February to tell us about the new CCCT Grant program she is heading up. 

Her first day on the new job is Monday January 24th. Let's give her a few weeks to get settled, then we'll get something scheduled. We'll be sure to announce the dates on the WCET website. And I'll post something here. 

January 24th is also Darcy's birthday. Isn't it great how the stars aligned for that to happen. Happy Birthday my friend!

Your online learning minions are here to serve. Sign us up.

 

 

What Dave Longanecker Said About Alaska

During the WCET Annual Meeting Thursday November 11, 2010 Luncheon presentation, during the session the Debate  between Dr. Dave Longanecker and Dr. Peter Smith on the topic of Rethinking the Academy in times of Great Change, a question about higher education cost was presented to the debaters from the audience.

Peter responded first, talking about the importance of choice, and that students benefit when there are a variety of options from subsidized to completely tuition driven. Then it was Dave's turn. And this is part of what Dave said:

"If you look at National Center for Higher Education Management data, there's the 250% difference in the cost per degree from one state to ...from the lowest cost state to the highest cost state. And it's not a quality...if you looked at the states you will see that quality is not the determining factor. The most expensive cost per degree state in the country is Alaska. I work with Alaska, I love Alaska, but I wouldn't put it as the most, as the best system in higher education in the country."

If you click here you will be redirected to the Sonic Foundry Mediasite archived recording of the Debate. Move the slider bar to the time-stamp at 45:45. (That's 45 minutes and 45 seconds into the session, less than two minutes before the entire session was adjourned). You can see and hear what Dave said for yourself.

Here is  the NCHEMS data that Dave was referencing when he made his comment about most expensive cost per degree state.

This slide was taken from a presentation that Dave had recently given to the CCCCIO conference in San Diego CA on October 28th. You can download the entire presentation here  Download Statistics from 101028 - CCCCIO Presentation on Quality Improvement Pretty interesting stuff about higher education that most of us typically don't see about cost, graduation, tuition and so on. 

His comments upset several of the WCET attendees from Alaska, who talked to me, talked to him, have written letters and fired off tweets. I know Dave has already apologized. It's not worth pissing people off. Especially when absolutely no harm was intended.

As this issue had been raised to public awareness through the #wcet10 tweet-stream I wanted to be sure people had access to the links to take a look for themselves, and to see some of the data that Dave was referencing.

 

 

 

Management of Online Education Survey Completion Reminder

The 2010 Managing Online Education Survey went "live" this past week. This is the second annual survey, jointly sponsored by WCET and the Campus Computing Project, that documents the nuts and bolts of what it takes to successfully run online learning programs. Dr. Kenneth C. (Casey) Green, Founder of the Campus Computing Project, is the lead author for this important exploration. Invitations to participate in the survey were distributed last week.

IF YOU HAVE NOT YET RECEIVED YOUR INVITATION WE WANT TO HEAR FROM YOU!!!!! Now that online learning IS hot, hot hot it is really important that any and all of you working in higher education online learning make sure that your voice is heard. Please contact Casey directly at [email protected]. Tell him Ellen told you to get in touch with him about the MOE Survey. Don't be shy.

It is possible that your institution is not already part of WCET - possible that your invitation got spam filtered - perhaps the name we have for your institutional point of contact is wrong. So this is my personal invitation to please be sure that your institution is represented in this year's survey results. Get in touch with Casey at [email protected]. The deadline for completion is November 4th 2010.

Seriously. Don't be shy.

Thanks very much.