eLearning Roadtrip

When I Think of Student Success

The tendency to link software with student success has accelerated since 2012, as the educational technology sub-sector known as "integrated planning and advising for student success" has gained prominence through a series of grants funded by the Bill & Melinda Gates Foundation to colleges and the tech partners they have selected to help the colleges improve student performance. These days, more than 100 companies offer integrated planning and advising technology platforms and tools for student success includes that serve a wide variety of functions, from deciding where to go to college, to facilities and course scheduling, from helping students select select majors to staying on track with program completion. Student success is a huge part of the current ed tech market scene. (Disclosure: In my current day job, I work for a company with a large part of its business dedicated to Student Success.)

EDUCAUSE has led efforts to help institutions explore both the promise and the risks of Integrated Planning and Advising for Student Success (iPASS). Per their website:  "iPASS is an integrative approach to student success that promotes shared ownership for educational progress among students, faculty, and staff. It encompasses services that help students formulate and advance toward educational goals, including advising, counseling, progress tracking, and academic early alerts. iPASS technologies can contribute by documenting and tracking students’ educational plans, improving data analysis, offering self-service resources that reduce advisor workloads, and triggering interventions based on student behavior or faculty input."

There have been two Gates Foundation-funded rounds of grant-making by EDUCAUSE in support of student success. The first was the Integrated Planning and Advising System project, designed to get schools and their tech partners to collaborate on getting platforms and practices to work together better. I was involved in the first IPAS round when the Predictive Analytics Reporting (PAR) Framework was contracted by the Gates Foundation to evaluate the 19 "student success + software initiatives" that were funded in that first go-round. We visited the 19 IPAS campuses that were working with specific tech company partners to further the implementation of a tech platform that could help campuses with student planing and advising platforms. PAR had used the PAR Student Success Matrix we had developed to help our own PAR members make the connection between predictions of  student risk with selection of interventions. We were able to determine and demonstrate measured effectiveness with particular groups of students under particular circumstances - sort of like getting the most effective prescription for dealing with a particular medical condition. We had been excited about our own results; the fact that we were going to be able to evaluate student success impact with our matrix of time-in-program and actionable predictors with a broader set of institutions was thrilling.

By the time the second round of grants were funded, the name of the initiative had become "integrated Planning and Advising for Student Success: iPASS". The connection between tech and advising (e.g. watchlists, early alerts), tech and planning (e.g. curriculum pathway planners, strategic scheduling), program fit and course navigation (e.g. obstacle course GPS), integrating predictive analytics with real-time student activity stream data.) As it happens, the iPASS initiative helped spark PAR Framework's acquisition by Hobsons in January, 2016, when two state systems of education proposed integrating PAR's predictives and intervention measurement with Starfish's real time student activity data stream used to build student watchlist/advisement capabilities as part of their iPASS grant proposal.

5 years, two major initiatives and several rounds of small planning and implementation grants later, EDUCAUSE has produced a library of student success resources for educational practitioners, on everything from "Things you should know about IPAS" (yes, that acronym is different than in the previous paragraph; more on that in a bit) to implementation handbooks, data tables and evaluation reports. and  from https://library.educause.edu/resources/2013/8/integrated-planning-and-advising-services-ipas-research.

It has been great to be a participant in this wave of innovation in education, where data is being used as currency for more effective decision-making in schools and colleges. No doubt, when we look back on how decisions were made in 10 years from now, we are going to wonder how we managed to live without platforms that help visualize patterns, red-flag student risks before they become problems, provide a 3D map of an academic transcript that can cross-walk among formal and informal learning experiences. 

Even so, I need to say something.

When I think of student success I never, ever think of software first. 

Software can make things easier to do, of course. It offers far better ways to keep track of searchable information requiring any manipulation. I'd rather build formulae in a spreadsheet, and have huge collections of data fueling explorations that report results in approachable visual reports and dashboards. Learning management and content management and curricular tracking, assessments and performance support are all more available, more accessible and more targeted at specific points in the educational experience than ever before.

Just as data associated with online and digital transactions help inform marketeers of our retail intentions even before we know for certain, ourselves, patterns of students behaviors can raise red flags before students are even aware that they may be drifting into dangerous territory for "students like them".

Technological inventions, applied to performance and learning problems, can help solve those problems. In doing so, the inventions become innovations, human ingenuity applied to solve performance problems in novel and compelling ways.

Innovative educational entrepreneurs  - "edu-preneurs', if you will - see an opportunity for tech in schools and are compelled to create products that promise to disrupt education as we know it. They point to deep data science and predictive analytics as if these analysis techniques are beyond the norm.

It's seems important to remind ourselves that in education, the most most important innovator in this value chain are not necessarily the ed tech entrepreneurs. Rather, the ones on the front lines of teaching and learning, supporting student achievement putting new tech tools to the test in the service of student success are todays true innovators. 

It's important to have shared articulated expectations of "the norm", so that one can identify those not yet achieving at that level, and how one inspire those working "above average" to continue to improve. Common expectations of meaning and outcomes are essential elements of student success that gives all students a shot at achieving their dreams. We need to include ALL students in our considerations of success, not just those attending college for the first time, going to school full time, or living on campus. We need to keep the student at the center.

With thanks to Kalah Burks and Rebecca Wheeler, University of South Alabama, for the great conversation that shook this post out of my head.

(tap, tap) Hey, is this thing still on?

During the past 6 years I have had the remarkable experience of taking an idea about predictive analytics all the way from brain-storming with smart, crazy people who wanted to try something new and different with big data techniques, to getting multiple grants from the Gates Foundation to see if our idea worked. It involved founding a not-for-profit organization to validate our ideas in the commercial ed tech marketplace, to being acquired by Hobsons, an international commercial software provider.  

Not bad for someone who had really just wanted a voice in the learning analytics conversation that was more focused on students than VC.

Now that I've had a chance to get settled in a new research job at Hobsons,  I've been reflecting on whether or not this work I've been doing in analytics still counts as elearning.  I'm thinking that it does.  I'm thinking that now, more than ever, using digital tools to extend out expectations of learning is exactly the next phase in learning innovation we elearning advocates have been waiting for.

There are some who continue to describe elearning as the activities utilizing electronic technologies to access educational curriculum outside of a traditional classroom. In these cases, people are typically referring to an online course, program or degree. I get that. It helps define elearning as something distinct from other cognitive or pedagogical pursuits. Some people think of elearning as authoring, or collections of webcasts.  I've seen some people say that elearning is dying because mobile learning is about to replace it. 

For me, elearning is about leveraging technologies in support of designing, creating, producing, evaluating, distributing and managing learning experiences that engage and inspire, regardless of physical location or time. It's a much bigger tent than content or courseware.

My new job at Hobsons is a research job. I'm exploring learning impact and efficacy in the P-20 learning systems we depend on in the US to prepare our citizenry for college and and career success. I've been working on reports describing the impact of dual credit for accelerating college completion, on the impact of upward transfers from community colleges to completion institutions, on online learning efficacy, on the value of exploration and engagement in early childhood, the things we can do to support learning success among non-traditional students. 

I'm amazed to see, in all these discussions, the degree to which elearning has evolved beyond content and courseware. Today elearning is more about personalized, interactive, responsive experience.  If ed tech is where we concentrate on better tech tools, these days elearning encapsulates all we have learned to do to extend digitalization into our teaching and learning, education and training practices, constructs and values.

All this reflection made me realize I need a place to reflect on the transformative evolution of our practice as tech changes our expectations of our learning lives.

Looks like it's time to get back on the eLearning Roadtrip.

 

 

 

The #OEB14 Debate: Is Data Corrupting Education?

Online Educa Berlin is a major international gathering of online/elearning/distance learning/mobilized/datified learning innovators. This year I was a participant in the Online Educa Debate, an annual featured event of the conference. The house motion for 2014 was "Data is Corrupting Education". Inge deWaard and I were asked to speak in favor of the house motion; Dr. Viktor Mayer-Schönberger and Dr. George Siemens spoke in opposition. I have provided the full motion below, in italics, for your reading pleasure.

MOTION: The House believes that data is corrupting education.

• Is the invasive extensive profiling and rigorous achievement test, backed by for-profit organizations, skewing the means of gathering information and determining relevance?

• Will a focus on data corrupt the processes of learning, training and development or will it reveal new unimagined insights that will make way for streamlines, personalized and effective learning

• Is big data destined to be in the hand of a few, or will openness prevail for the benefit of all?

At first, I was completely nonplussed by this challenge. But the more I got my head wrapped around ways to support it, the more I stretched my thinking, which of course is the whole reason one does these things. Collaborating with Inge was delightful; having the opportunity to present with Viktor and George was a real pleasure. I am very pleased to have participated.

The video of the Debate will be available soon, and I will share it when available. It was lively discussion, among we debaters and with participants from the audience. For a while I even thought the crowd was with Inge and me,  but I have to admit, even I would have had a tough time voting in favor of the motion as written.

While we lost the vote, I think we helped remind people that we all need to be mindful about the implications for unexpected consequences that innovative work may bring to practice. In the sprit of sharing have posted my notes of my opening argument, below:

Higher education has typically been organized around authority-driven models of decision-making; that is, the more senior among us are generally afforded positions of authority based on experience and expertise. The emergence of analytics tools and platforms are making it very clear that data can provide an even more complete, and certainly a more nuanced picture than expertise alone. 

Educators are just beginning to understand the IMPLICATIONS of data analytics, where mining data for insight to support decision-making is the focus of exploration. While it is clear that there are great opportunities for gleaning heretofore unimagined insights, the fascination with data makes many uncomfortable. We have come to realize that the patterns of our data generation, our online engagements, transactions and interactions, tell people things about us or things we are doing that that we never could have anticipated or expected.  It means that intelligence about us can be gleaned by those who have access to our various data streams. Are you aware of all the stories that your data are telling other people about you?

Our current fascination seems to be about data as a meme – an idea that spreads from person to person within a culture. Data-as-meme, especially BIG data, has come to stand for something bigger that the data themselves. It stands all of our excitement about the possibilities that they offer for risk mitigation and personalization. However it also stands for the doubts and fears about how all these data are going to affect how we do our work, how we go to school, how we live our lives, how we engage as citizens. 

Data without context has marginal value. However, data WITH context is information. Information is power.  And as the British historian Lord Acton reminded us, Power corrupts. Absolute power corrupts absolutely.  http://www.acton.org/pub/religion-liberty/volume-2-number-6/power-corrupts

Simply stated, corruption is the self-serving deviation from an ideal. If one accepts this simple description, then one CAN make a case that data may very well be used to less-than-ideal ends. It might mean that we actively exclude some learners at the expense of others. We may push people of varying predicted skill levels into programs of study based upon test results, rather than on personal passions and interest. Will data have an effect the way that we approach our pedagogies? Absolutely. It this going to change how educational professional approach our work? Absolutely. How do YOU feel about instructional design as a data driven algorithm? How about licensing digital content where the price point is determined in part to how well students do in the class where the content is used? How do faculty feel about the prospect of expectations that we will be lurking in the lives of our students to see what and how they are doing at every single moment? How do YOU feel knowing that students aren’t going to be the only ones who are going to be tracked and measured in these ways? 

I don’t think that data, by themselves, have corrupted, are corrupting or will corrupt education.  I do worry about uninformed, naïve or even nefarious uses of data. I worry about data being used to punish, to limit access, constrain or restrict participation.

While I am excited about the possibilities for personalized, customized learning experiences, I am also fairly certain that educational practitioners who use data badly, non-transparently, or naively are very likely going to give data a very bad name.

Who Owns Your Data?

Who actually owns your data?  At first the answer to this question may seem obvious - of COURSE we own our personal health records, education records, financial records... 

But do we, really? Data ownership presumes use rights, who can share what, with whom. What about those education records that reside in someone else's learning management system? Judging from the recent news story about student records being sold as part of ConnectED's bankrupcy settlement, it is clear that some companies believed these records were theirs to sell.

Stories like this, and of InBloom's implosion over the prospect of sharing student data with commercial service providers, have even resulted in bans on data sharing that have emerged in some states. But data sharing according to what?

What about the data captured on our mobile devices? Does it belong to the individual, the carrier or the device manufacturer? What about our social media engagements and interactions? Despite the number of well-intentioned postings of declarations of data ownership I've seen on Facebook, it seems clear that if one posts items on a free internet services site page to share, that we may not be the only one sharing them.What about all those data left behind when we complete a transaction on Amazon? Don't you ever wonder about how those companies from whom you buy your various consumer products know enough about you to send you a birthday card???

Have you every seen how many companies are sucking up the digital crumbs of your online interactions?  (Hint: Download the Ghostery, Inc. client to your browser. You might be very surprised to see just how many analytics, advertising and beacons links are paying very close attention to what you are doing.)

This brings me to two big observations:

(1) Privacy, as many of us have known it, is a thing of the past. It is extinct. When one considers the number of sensors, transponders, transmitters and cameras we encounter on a daily basis, it is very clear how easy it can be to reconstruct what one is up to, should one be a "person of interest". On the positive side, it will hopefully keep people honest in their dealings and transactions, since we can all check up on each other. On the continued positive side, it represents opportunities for convenience and customization, with services tailored to our uses and preferences.

(2) Nobody owns all our data. Not even us. It is data anarchy out there. While we can hope, we can't expect that some intra-global agency will make overarching rules that everyone will follow and all will live happily ever after. We are all going to need to know enough about the benefits and possible risks that sharing data brings to make smart decisions about the data we are willing to share, and what we expect to get in return. Most of us accept some exposure in exchange for the convenience that data sharing affords us; but most of us also get very uncomfortable at the prospect of a bunch of code jockeys deciding to hack a cloud service hosting our digital stuff and sharing our private information with the world.

Data-readiness means knowing enough to pay attention  - to security settings, to filters, to groups, to passwords and shared information - so that we know what is really going on with our digital data, knowing what and how we are sharing, and with whom. THEN we can make decisions based on evidence, because THAT is the whole point.

 

For my .EDU friends: Here's How Venture Capital Works

The education sector's current fascination with innovation, distruption and entrepreneurship is being fueled by the emergence of start-up commercial ventures that create new products and services as solutions for big hairy audacious problems plaguing today's schools and students. 

An April, 2014 post by CB Insights noted that investments in education technology hit a record high in Q1 2014 as investors poured more than $559 million across 103 deals in the first three months of this year.  Of note, Ed Tech funding in Q1 2014 represented nearly 45% of total funding to the sector in all of 2013. The record quarter comes after Ed Tech investments hit $1.25B across 378 deals in in 2013, the second-straight year of over $1B invested across the Ed Tech sector.  With everything from MOOCs to tutoring marketplaces to standardized APIs for K-12 schools getting funding, investors are clearly bullish on the education industry.

So what could possibly be wrong with getting rounds of other people's money to help you grow your big educational idea into a product that becomes the NEXT BIG THING?

Well, nothing...except you need to pay back all the money that everyone had given you. With interest and at a prenegotiated rate of return. That means that once you take someone else's money as an investment in your hot shot edu-preneurial company, EVERYTHING you do is going to be focused on paying your investors back. 

Aral Balkan recently blogged about his experience with a new social media platform called Ello. His post caught my eye because it included a pointed summary of how venture capital works. Having just spend the past several months at various educational technology meetings and trade shows where the buzz over acqusitions and investments is ever-present, I found myself sharing the following paragraphs with my friends from education who may not have had the experience of working with investors:

"Here’s how venture capital works: you go to an investor, before you’ve even built the thing you’re building and you tell them how you’re going to exit. It’s called an exit plan or exit strategy. You tell them, for example: “Hey, we’re going to get 100 million people using our new platform in two years time, how much will you give me for 100 million people?” And they go “Umm, we’ll give you this much for 100 million people because we’re pretty sure we can get that amount back several times over when we sell those 100 million people in an exit either to another company or in an IPO.”

"When you take venture capital, it is not a matter of if you’re going to sell your users, you already have. It’s called an exit plan. And no investor will give you venture capital without one. In the myopic and upside-down world of venture capital, exits precede the building of the actual thing itself. It would be a comedy if the repercussions of this toxic system were not so tragic.

"Let me put it bluntly: if a company has taken venture capital, you have already been sold. It’s not a matter of if, it’s simply a matter of when. (Unless the company goes under before it can exit, that is.)

"A venture-capital funded startup is a temporary company that has to convince enough people into using their platform so that they can make good on the exit they promised their investors at the very beginning. It is the opposite of a long-term, sustainable business." (Italics are mine).

You can read Aral Balkan's complete "Ello, Godbye" blogpost by clicking here.

I draw your attention to this article because every time I hear someone confide that their platform provider is a rising VC funded firm that is going for its next round of money, I wonder if these education customers understand the degree to which EVERYTHING their vendor provider must now do is focused on meeting their investors' expectations.

Alan Greenberg: Blackboard Reboots

I admire and respect Wainhouse Research analyst Alan Greenberg, so was quite interested to read his recent post in which he reflects on changes that Blackboard introduced at this year's Blackboard World. I, too, was impressed with the "new Blackboard" while at Blackboard World this year, and am really looking forward to diving into some of the work we (the PAR team) will be doing with the Blackboard team.

Please refer to Alan's newletter for the complete article and especially for his take on why Blackboard's changes are a big deal.

Here are a few "nuggets" from his post to whet your appetite for more......

Blackboard Reboots @ Blackboard World
Alan D. Greenberg 

In an effort to give institutions and schools more value and greater exposure to its technology sets, Blackboard has completely revamped its product line, now integrating together its product portfolio with a totally new user experience (UX) and a declared focus on learners. All teaching and learning solutions will now include Blackboard’s “base” learning management system (Blackboard Learn) and provide a full-featured online environment with course delivery, synchronous collaboration tools, mobile apps, and integration with and access to content. Technology adoption services are also included.  

A tiered set of solutions are aimed at helping the educational user community address challenges such as retention, student engagement, learning analytics, communications and standards preparedness. There are four higher education teaching and learning solutions, four higher education campus commerce and security solutions, and five K-12 solutions that center on teaching and learning and community engagement. The company is considering adding more solution sets for international markets.

Blackboard also plans to launch a new version of its learning environment that is delivered via the public cloud. Now the company is providing customers a full set of options: self-hosted, hosted in Blackboard’s private cloud, or via the public cloud. Word is this public cloud offering will be fully available sometime this month or in September. Being piloted in the public cloud is Blackboard’s Open Education powered by Blackboard, also announced at Blackboard World. This is a course platform that allows teachers and faculty to run public online courses and/or Massive Open Online Courses (MOOCs).

The company has created a customized learning and analytics offering available to the 259 higher education Internet2 members. The Internet2 NET+ offering will not only provide access to Blackboard’s solutions, but also will include robust analytic and learning outcomes tools, a cloud-based, cross-institutional content repository that supports most competitive LMS’s, and help desk and adoption services. Cornell University; the University of Nebraska — Lincoln; the University of Maryland, Baltimore County; the University of Texas at Arlington; and Virginia Commonwealth University had representatives who guided and shaped the Internet2 NET+ certified solution.

At the core of the offering, which is fully deployable now, is Blackboard’s Learning Insight solution, which is a full-featured online and mobile version of Blackboard Learn LMS and Blackboard Collaborate.

(Ellen's aside: yes, this means that Collaborate works in the browser. NO more Java downloads!)

FOR MORE, click here and scroll down for Alan's full article.

PAR Framework: Learning Analytics as a Service for Higher Ed

Thanks to Libby Nelson's great article Big Data 101: Colleges are hoping predictive analytics can fix their dismal graduation rates, we have been fielding a lot of questions about the PAR Framework. Here is an update on who we are and the work in which we are engaged.

MEET THE PAR FRAMEWORK - The Predictive Analytics Reporting (PAR) Framework is a national, non-profit provider of learning analytics as a service. PAR brings 2-year, 4-year, public, proprietary, traditional and progressive colleges and universities together to collaborate on mitigating student loss by identifying the effective practices that support student progress toward their academic goals. The PAR Framework offers a unique multi-institutional perspective for examining dimensions of student success that will help improve retention in US higher education.  PAR is distinguished among the many data analytics solutions emerging in the education domain by its common, openly published data definitions and student success frameworks.

HOW PAR WORKS - PAR uses predictive analytics to deliver tools and reports aimed at improving academic success for students in online, blended and on-ground programs. PAR members receive flexible predictive models, cross-institutional benchmarks, and an intervention insight toolkit supporting a student success methodology that links predictions with interventions and student supports.

• PAR institutions provide a full set of anonymized undergraduate student data that allows for comparative investigation of student success trends over time, at the individual student, course and degree level. (PAR members provide periodic updates of their data ensuring the ability to measure changes over time, the impact of student interventions and enabling the predictive models that PAR produces to be adjusted and tuned for current data.)

• In return, PAR institutions receive actionable reports and student watch lists developed using predictive analytics.  Once institutions deliver their student-anonymized and institutionally de-identified data, PAR researchers and data scientists use statistical and data mining techniques to identify and reveal the factors that most impact student success in the combined dataset and the individual institution level.  Members quickly receive institutionally-specific models that predict the likelihood of each student achieving institutional specific milestones. These include but are not limited to passing a course or courses and being retained at the institution.  Member institutions also receive academic models for their institution that include detailed student level data to create watch lists, structure interventions and inform local decisions about the factors identified as having the greatest impact on student success. They also receive institutionally anonymized and de-identified findings from the entire dataset that yield over-arching insights about retention patterns in U.S. higher education to inform practice and policy.

PAR TOOLS FOR BENCHMARKING:

• The PAR Student Success Matrix (SSM^X) helps institutions comprehensively assess their student success policies, interventions and programs. The SSM^X makes it possible to organize the wide variety of student supports – from orientation to mentoring to advising – into a systematic validated framework designed to quantify the impact of student success practices and determine the best support for students at the point of need.  The SSM^x also reveals gaps and overlaps in student support programs and gives institutions the tools to evaluate the efficacy of their investments at the program level.  
•  Quantified intervention effectiveness results. The common PAR measures for assessing and predicting risk and the validated frameworks categorizing student support services create the mechanism to effectively measure the impact of student supports within and across institutions.  PAR member institutions are encouraged to take part in quantitative, closed loop field tests, measuring the effect of targeted student supports (e.g. tutoring, student services, email, text messages alerts) against at-risk students.
• Institutional benchmarks. PAR Framework members receive institutionally-reflective and cross-institutional benchmark reports of student level outcomes such as retention, successful credit accumulation, academic performance indicators, and earning a credential.

COMPREHENSIVE INSIGHTS BASED ON VARIATIONS IN:

• Student demographics
• Financial aid factors
• Traditional vs Non-traditional status
• Full-time/Part-time status
• Major/Program pursued
• Credential pursued
• Online vs. On-ground vs. Other modalities of course taking
• Prior academic history

PARTICIPATION IN A COLLABORATIVE COMMUNITY OF EXPERTS: PAR Framework members represent an engaged community of forward thinking institutions, each with perspective and practices around student retention, progression, and completion. PAR membership provides institutional leaders with the models, technical support, services, and guidance needed to evaluate techniques, tools and solutions used to improve student outcomes. PAR provides the collaborative environment needed to support policy implementation and effective practice guidance necessary to capitalize on the power of data-driven decision-making for improving student success in reliable, repeatable, scalable ways.

 For more information about joining PAR, please contact us at [email protected].

 

Now that Predictive Analytics are Hotter than Ever

Now that predictive analytics are hotter than ever in higher education, it's probably a good idea to review a few fundamentals for those who are just starting to sip from the cup of possibilities.

WHY PREDICTIVES? Predictive analytics have been used in a wide variety of settings, including higher education, to manage finances, inventory, operations, assets and resources. Increasingly, higher ed institutions are turning to business intelligence for enrollment management and student recruitment. The next great wave for predictive analytics adoption in higher education is focusing on institutional performance outcomes and individualized student success. 

WHY COLLEGES NEED TO CARE: The more that colleges and universities are being held accountable for achieving specific performance outcomes (e.g., improved student retention, better completion rates for college students), the more predictive analytics will be touted as a solution for anticipating risks likely to be encountered while trying to achieve those outcomes. 

SIMPLY KNOWING WHO IS AT RISK ISN'T ENOUGH. Knowing how to mitigate risks and how different students can be better served with targeted interventions and support makes predictions actionable. Predictions without action don't really matter very much to anyone.

BEWARE THE BRIGHT AND SHINY: It's going to be a little bit like the early days of the first dot.com out there, with lots of  solutions available and even more promises about all the great things that those interventions and solutions can do (!!!). How will you know who to believe? How will you know what's right for you? You owe it to yourself to be informed.

THERE IS NO SINGLE "RIGHT WAY" TO CONDUCT PREDICTIVE ANALYSES. There are many techniques used to conduct predictive analyses. Typically one chooses the technique or techniques likely to yield results for the kinds of predictions one wants to make.

For example, if you are interested in predicting a discrete attribute you might be likely to use techniques including Logistic Regression, Decision Trees (CHAID, CART, Random Forest), Naïve Bayes, Support Vector Machine, Survival Analysis or Neural Networks. If you were more interested in predicting a continuous attribute, you'd look toward techniques such as Multiple Linear Regression, Time series or Decision Trees (CHAID, CART, Random Forest). If you were most interested in common groups, you might consider Hierarchical k-nearest neighbors, Neural Networks or Decision Trees (CHAID, CART, Random Forest).

Selecting the right techniques used for conducting predictive analyses have a lot to do with knowing the questions that the predictions will help answer, or the performance problems that are likely to be solved. Speaking of which...

START WITH A PROBLEM TO SOLVE: Sorry to burst your bubble but predictive analytics don't work quite like Magic 8-Ball. Not even Hadoop technologies do that. Answers simply do not emerge fully formed from the mists of your analytic techniques.  It's easier to focus on solving a problem (e.g. what causes students to drop out? Are these causes common in all settings?) or finding a new opportunity (e.g. what motivates students to complete courses faster?). Your problem statements and queries will help you focus on finding data sources and selecting techniques for analyses that are likely to reveal the patterns you seek.

Living Under the Sword of Data, on #Lrnchat

On May 9, 2013 I will have the pleasure of joining the crew at #Lrnchat for a session on the topic of learning analytics. You will find us on Twitter, Thursday, 8:30 pm ET/5:30 PT. Look for us at #lrnchat, follow along at @lrnchat.

 #Lrnchat is a regularly scheduled event that takes place every Thursday evening.  Participants using any number of tweeting tools (such as Tweetdeck and Tweedgrid) to keep track of the messages that fly around among the session participants. It is hosted and facilitated by a number of learning and development social media experts who shall remain nameless until I get their permission to drop their names. Questions get asked, responses get posted, conversations get started, friendships get launched. It's amazing how much one can pack into 140 characters! 

The name of this particular post, in anticipation of tomorrow's #Lrnchat, comes from one of my favorite analytics parables. In March of 2009, the lead designer from Google resigned in protest to what he described as an engineering culture that was obsessed by numbers. Douglas Bowman offered his reasons for resigning  in a blogpost entitled "Goodbye Google". In his explanation he gave the following examples of how data-obsessed Google had become:

"Yes, it’s true that a team at Google couldn’t decide between two blues, so they’re testing 41 shades between each blue to see which one performs better. I had a recent debate over whether a border should be 3, 4 or 5 pixels wide, and was asked to prove my case. I can’t operate in an environment like that. I’ve grown tired of debating such minuscule design decisions. There are more exciting design problems in this world to tackle."

In closing, he wrote that "I’ll miss working with the incredibly smart and talented people I got to know there. But I won’t miss a design philosophy that lives or dies strictly by the sword of data."

I remember being struck by that phrase as the perfect way to describe the double-edged benefits and risks that analytics bring to discussions of learning, achievement, progress, success. I offer this as something that each of us should keep in mind as we gravitate toward a greater use of enterprise data in our decision-making. 

I know we will have 90 minutes to talk about this. In anticpation of the conversation can I just offer this observation - It helps to have an idea about the problem you are trying to solve before you dive into an analytics campaign. And once you DO know, you need to be prepared to respond. Problem identification, without taking action, is almost worse than not knowing at all.

Here are a few references that I hope you will find to be useful. Happy reading! See you on Twitter.

Background References

Goodbye, Google The blogpost written by designer Douglas Bowman when he resigned from Google in protest to the data culture around decisionmaking. http://stopdesign.com/archive/2009/03/20/goodbye-google.html

The Gawker story that picked up the story of event leading to Bowman’s resignation from Google http://gawker.com/5177144/googles-data-fetish-drives-away-its-top-designer

Articles I've written on learning analytics:

http://www.educause.edu/ero/article/data-changes-everything-delivering-promise-learning-analytics-higher-education

http://www.astd.org/Publications/Magazines/TD/TD-Archive/2012/08/8-Realities-Learning-Professionals-Need-to-Know-About-Analytics

Recent tweets on Big Data –related topics by @edwsonoma

The Big Data Landscape http://bit.ly/12KXpvA 

11 Big Data myths http://bit.ly/15CD26B 

The Surprising Predictive Power of Analytics http://lnkd.in/M2VS9j 

The Rising Costs Of Misunderstanding Big Data http://lnkd.in/fBGWVP 

Ten things you need to know about Big Data http://tinyurl.com/cabxeam 

What's the skillset of a talented #BigData Scientist? http://ow.ly/kNKw7 

 

NEW NSF REPORT: New Technology-based Models for Postsecondary Learning

In January 2013 I  participated in a workshop sponsored by the National Science Foundation to help establish a 10 year vision to reconceptualize postsecondary learning. This past week our co-convenors, Chris Dede, Harvard University and Eric Grimson, MIT, released the 47 page report that captured the results of that meeting. Entitled New Technology-based Models for Postsecondary Learning: Conceptual Frameworks and Research Agendas, this is a comprehensive overview of promises, opportunities and challenges that will keep all of we post-secondary learning stakeholders on our toes for the next decade.

You can download the working draft of this paper here.

The basic premises addressed include the following:

• Advances in technology and in knowledge about expertise, learning, and assessment have the potential to reshape the many forms of education and training past matriculation from high school.
• In the next decade, higher education, military and workplace training, and professional development must all transform to exploit the opportunities of a new era, leveraging emerging technology-based models that can make learning more efficient and possibly improve student support, all at lower cost for a broader range of learners. 
• Potential risks must be managed, including those arising from the disruption of established delivery economics in our current learning institutions, the variable quality of learning outcomes these new models offer today, and the technical and conceptual challenges of better understanding how to design, develop, and implement innovative capabilities in ways that reliably deliver on their promise.

Workshop participants developed a framework for understanding this sea change and sketched steps towards a research agenda for realizing its benefits while avoiding pitfalls.

Contributors to this report include Daniel E. Atkins, W.K. Kellogg Professor of Community Information, and Professor of Information and Computer Science, University of Michigan - Ann Arbor. Lori Breslow, Director, Teaching and Learning Laboratory, MIT. John Cherniavsky, Senior Advisor, Division of Research on Learning, NSF. J.D. Fletcher, Senior Research Staff Member, Institute for Defense Analyses. Diana Oblinger, President and CEO, EDUCAUSE. Roy Pea, David Jacks Professor of Education and Learning Sciences, Stanford University. James W. Pellegrino, Distinguished Professor and Co-Director, Learning Sciences Research Institute, University of Illinois at Chicago. Bror Saxberg, Chief Learning Officer, Kaplan, Inc. James H. Shelton III, Assistant Deputy Secretary for Innovation and Improvement,U.S. Department of Education. Russell Shilling, CAPT, MSC, USN, Program Manager, DARPA. Greg Tobin, President, English, Math, and Student Success, Pearson Higher Education and yours truly, Ellen Wagner, Executive Director, Western Interstate Commission for Higher Education Cooperative for Educational Technologies. Arthur Josephson, Harvard Graduate School of Education, was our researcher.