eLearning Roadtrip

The #OEB14 Debate: Is Data Corrupting Education?

Online Educa Berlin is a major international gathering of online/elearning/distance learning/mobilized/datified learning innovators. This year I was a participant in the Online Educa Debate, an annual featured event of the conference. The house motion for 2014 was "Data is Corrupting Education". Inge deWaard and I were asked to speak in favor of the house motion; Dr. Viktor Mayer-Schönberger and Dr. George Siemens spoke in opposition. I have provided the full motion below, in italics, for your reading pleasure.

MOTION: The House believes that data is corrupting education.

• Is the invasive extensive profiling and rigorous achievement test, backed by for-profit organizations, skewing the means of gathering information and determining relevance?

• Will a focus on data corrupt the processes of learning, training and development or will it reveal new unimagined insights that will make way for streamlines, personalized and effective learning

• Is big data destined to be in the hand of a few, or will openness prevail for the benefit of all?

At first, I was completely nonplussed by this challenge. But the more I got my head wrapped around ways to support it, the more I stretched my thinking, which of course is the whole reason one does these things. Collaborating with Inge was delightful; having the opportunity to present with Viktor and George was a real pleasure. I am very pleased to have participated.

The video of the Debate will be available soon, and I will share it when available. It was lively discussion, among we debaters and with participants from the audience. For a while I even thought the crowd was with Inge and me,  but I have to admit, even I would have had a tough time voting in favor of the motion as written.

While we lost the vote, I think we helped remind people that we all need to be mindful about the implications for unexpected consequences that innovative work may bring to practice. In the sprit of sharing have posted my notes of my opening argument, below:

Higher education has typically been organized around authority-driven models of decision-making; that is, the more senior among us are generally afforded positions of authority based on experience and expertise. The emergence of analytics tools and platforms are making it very clear that data can provide an even more complete, and certainly a more nuanced picture than expertise alone. 

Educators are just beginning to understand the IMPLICATIONS of data analytics, where mining data for insight to support decision-making is the focus of exploration. While it is clear that there are great opportunities for gleaning heretofore unimagined insights, the fascination with data makes many uncomfortable. We have come to realize that the patterns of our data generation, our online engagements, transactions and interactions, tell people things about us or things we are doing that that we never could have anticipated or expected.  It means that intelligence about us can be gleaned by those who have access to our various data streams. Are you aware of all the stories that your data are telling other people about you?

Our current fascination seems to be about data as a meme – an idea that spreads from person to person within a culture. Data-as-meme, especially BIG data, has come to stand for something bigger that the data themselves. It stands all of our excitement about the possibilities that they offer for risk mitigation and personalization. However it also stands for the doubts and fears about how all these data are going to affect how we do our work, how we go to school, how we live our lives, how we engage as citizens. 

Data without context has marginal value. However, data WITH context is information. Information is power.  And as the British historian Lord Acton reminded us, Power corrupts. Absolute power corrupts absolutely.  http://www.acton.org/pub/religion-liberty/volume-2-number-6/power-corrupts

Simply stated, corruption is the self-serving deviation from an ideal. If one accepts this simple description, then one CAN make a case that data may very well be used to less-than-ideal ends. It might mean that we actively exclude some learners at the expense of others. We may push people of varying predicted skill levels into programs of study based upon test results, rather than on personal passions and interest. Will data have an effect the way that we approach our pedagogies? Absolutely. It this going to change how educational professional approach our work? Absolutely. How do YOU feel about instructional design as a data driven algorithm? How about licensing digital content where the price point is determined in part to how well students do in the class where the content is used? How do faculty feel about the prospect of expectations that we will be lurking in the lives of our students to see what and how they are doing at every single moment? How do YOU feel knowing that students aren’t going to be the only ones who are going to be tracked and measured in these ways? 

I don’t think that data, by themselves, have corrupted, are corrupting or will corrupt education.  I do worry about uninformed, naïve or even nefarious uses of data. I worry about data being used to punish, to limit access, constrain or restrict participation.

While I am excited about the possibilities for personalized, customized learning experiences, I am also fairly certain that educational practitioners who use data badly, non-transparently, or naively are very likely going to give data a very bad name.

Who Owns Your Data?

Who actually owns your data?  At first the answer to this question may seem obvious - of COURSE we own our personal health records, education records, financial records... 

But do we, really? Data ownership presumes use rights, who can share what, with whom. What about those education records that reside in someone else's learning management system? Judging from the recent news story about student records being sold as part of ConnectED's bankrupcy settlement, it is clear that some companies believed these records were theirs to sell.

Stories like this, and of InBloom's implosion over the prospect of sharing student data with commercial service providers, have even resulted in bans on data sharing that have emerged in some states. But data sharing according to what?

What about the data captured on our mobile devices? Does it belong to the individual, the carrier or the device manufacturer? What about our social media engagements and interactions? Despite the number of well-intentioned postings of declarations of data ownership I've seen on Facebook, it seems clear that if one posts items on a free internet services site page to share, that we may not be the only one sharing them.What about all those data left behind when we complete a transaction on Amazon? Don't you ever wonder about how those companies from whom you buy your various consumer products know enough about you to send you a birthday card???

Have you every seen how many companies are sucking up the digital crumbs of your online interactions?  (Hint: Download the Ghostery, Inc. client to your browser. You might be very surprised to see just how many analytics, advertising and beacons links are paying very close attention to what you are doing.)

This brings me to two big observations:

(1) Privacy, as many of us have known it, is a thing of the past. It is extinct. When one considers the number of sensors, transponders, transmitters and cameras we encounter on a daily basis, it is very clear how easy it can be to reconstruct what one is up to, should one be a "person of interest". On the positive side, it will hopefully keep people honest in their dealings and transactions, since we can all check up on each other. On the continued positive side, it represents opportunities for convenience and customization, with services tailored to our uses and preferences.

(2) Nobody owns all our data. Not even us. It is data anarchy out there. While we can hope, we can't expect that some intra-global agency will make overarching rules that everyone will follow and all will live happily ever after. We are all going to need to know enough about the benefits and possible risks that sharing data brings to make smart decisions about the data we are willing to share, and what we expect to get in return. Most of us accept some exposure in exchange for the convenience that data sharing affords us; but most of us also get very uncomfortable at the prospect of a bunch of code jockeys deciding to hack a cloud service hosting our digital stuff and sharing our private information with the world.

Data-readiness means knowing enough to pay attention  - to security settings, to filters, to groups, to passwords and shared information - so that we know what is really going on with our digital data, knowing what and how we are sharing, and with whom. THEN we can make decisions based on evidence, because THAT is the whole point.

 

For my .EDU friends: Here's How Venture Capital Works

The education sector's current fascination with innovation, distruption and entrepreneurship is being fueled by the emergence of start-up commercial ventures that create new products and services as solutions for big hairy audacious problems plaguing today's schools and students. 

An April, 2014 post by CB Insights noted that investments in education technology hit a record high in Q1 2014 as investors poured more than $559 million across 103 deals in the first three months of this year.  Of note, Ed Tech funding in Q1 2014 represented nearly 45% of total funding to the sector in all of 2013. The record quarter comes after Ed Tech investments hit $1.25B across 378 deals in in 2013, the second-straight year of over $1B invested across the Ed Tech sector.  With everything from MOOCs to tutoring marketplaces to standardized APIs for K-12 schools getting funding, investors are clearly bullish on the education industry.

So what could possibly be wrong with getting rounds of other people's money to help you grow your big educational idea into a product that becomes the NEXT BIG THING?

Well, nothing...except you need to pay back all the money that everyone had given you. With interest and at a prenegotiated rate of return. That means that once you take someone else's money as an investment in your hot shot edu-preneurial company, EVERYTHING you do is going to be focused on paying your investors back. 

Aral Balkan recently blogged about his experience with a new social media platform called Ello. His post caught my eye because it included a pointed summary of how venture capital works. Having just spend the past several months at various educational technology meetings and trade shows where the buzz over acqusitions and investments is ever-present, I found myself sharing the following paragraphs with my friends from education who may not have had the experience of working with investors:

"Here’s how venture capital works: you go to an investor, before you’ve even built the thing you’re building and you tell them how you’re going to exit. It’s called an exit plan or exit strategy. You tell them, for example: “Hey, we’re going to get 100 million people using our new platform in two years time, how much will you give me for 100 million people?” And they go “Umm, we’ll give you this much for 100 million people because we’re pretty sure we can get that amount back several times over when we sell those 100 million people in an exit either to another company or in an IPO.”

"When you take venture capital, it is not a matter of if you’re going to sell your users, you already have. It’s called an exit plan. And no investor will give you venture capital without one. In the myopic and upside-down world of venture capital, exits precede the building of the actual thing itself. It would be a comedy if the repercussions of this toxic system were not so tragic.

"Let me put it bluntly: if a company has taken venture capital, you have already been sold. It’s not a matter of if, it’s simply a matter of when. (Unless the company goes under before it can exit, that is.)

"A venture-capital funded startup is a temporary company that has to convince enough people into using their platform so that they can make good on the exit they promised their investors at the very beginning. It is the opposite of a long-term, sustainable business." (Italics are mine).

You can read Aral Balkan's complete "Ello, Godbye" blogpost by clicking here.

I draw your attention to this article because every time I hear someone confide that their platform provider is a rising VC funded firm that is going for its next round of money, I wonder if these education customers understand the degree to which EVERYTHING their vendor provider must now do is focused on meeting their investors' expectations.

What an earthquake taught me about Twitter

We were rudely awakened this past Sunday morning when an earthquake threw us out of bed at 3:20 am. It was a big one. The roar of the quake, the smashing glasses, dishes, and pottery, accompanied by the sound of exploding transformers and the splash of hundreds of gallons of water being thrown out of the pool, was deafening.

And then the shaking ceased, the noise stopped, and it was all over.

Of course, we knew what had happened. We live in Northern California. People who live in "earthquake country" are prepared for earthquakes, or at least we like to think we are. Flashlights in every room, with shoes at bedside (to avoid bare feet meeting broken glass), 5 gallons of water and food/shelter for 72 hours. 

What we were NOT prepared for is the shock of sitting in the silence and the dark in the immediate aftermath of an earthquake that throws you out of bed at 3:20 in the morning.

The wifi and landline phone were dead; we had no power.  But I had a fully charged mobile phone and working cell service. I turned to Twitter and within about 15 seconds I learned that we had just experienced an 6.1-on-the-scale earthquake, the epicenter of which was just 9 miles to the southeast of where we live. I saw that friends in San Francisco and San Jose had been awakened by the quake, too, and had already checked in to see if we were okay. We saw that our friends in Napa had been slammed, and were advised not to drive over to help because the roads had been damaged. 

As the morning went on, we eventually posted pictures on Instagram and messages on Facebook to let friends and family know that we were alright. But what I learned in those first moments, when immediacy mattered most, was that in case of emergency, when WE needed targeted information, Twitter was where we turned first. 

 

 

 

 

Alan Greenberg: Blackboard Reboots

I admire and respect Wainhouse Research analyst Alan Greenberg, so was quite interested to read his recent post in which he reflects on changes that Blackboard introduced at this year's Blackboard World. I, too, was impressed with the "new Blackboard" while at Blackboard World this year, and am really looking forward to diving into some of the work we (the PAR team) will be doing with the Blackboard team.

Please refer to Alan's newletter for the complete article and especially for his take on why Blackboard's changes are a big deal.

Here are a few "nuggets" from his post to whet your appetite for more......

Blackboard Reboots @ Blackboard World
Alan D. Greenberg 

In an effort to give institutions and schools more value and greater exposure to its technology sets, Blackboard has completely revamped its product line, now integrating together its product portfolio with a totally new user experience (UX) and a declared focus on learners. All teaching and learning solutions will now include Blackboard’s “base” learning management system (Blackboard Learn) and provide a full-featured online environment with course delivery, synchronous collaboration tools, mobile apps, and integration with and access to content. Technology adoption services are also included.  

A tiered set of solutions are aimed at helping the educational user community address challenges such as retention, student engagement, learning analytics, communications and standards preparedness. There are four higher education teaching and learning solutions, four higher education campus commerce and security solutions, and five K-12 solutions that center on teaching and learning and community engagement. The company is considering adding more solution sets for international markets.

Blackboard also plans to launch a new version of its learning environment that is delivered via the public cloud. Now the company is providing customers a full set of options: self-hosted, hosted in Blackboard’s private cloud, or via the public cloud. Word is this public cloud offering will be fully available sometime this month or in September. Being piloted in the public cloud is Blackboard’s Open Education powered by Blackboard, also announced at Blackboard World. This is a course platform that allows teachers and faculty to run public online courses and/or Massive Open Online Courses (MOOCs).

The company has created a customized learning and analytics offering available to the 259 higher education Internet2 members. The Internet2 NET+ offering will not only provide access to Blackboard’s solutions, but also will include robust analytic and learning outcomes tools, a cloud-based, cross-institutional content repository that supports most competitive LMS’s, and help desk and adoption services. Cornell University; the University of Nebraska — Lincoln; the University of Maryland, Baltimore County; the University of Texas at Arlington; and Virginia Commonwealth University had representatives who guided and shaped the Internet2 NET+ certified solution.

At the core of the offering, which is fully deployable now, is Blackboard’s Learning Insight solution, which is a full-featured online and mobile version of Blackboard Learn LMS and Blackboard Collaborate.

(Ellen's aside: yes, this means that Collaborate works in the browser. NO more Java downloads!)

FOR MORE, click here and scroll down for Alan's full article.

When I realized how easy is was to get started with learning analytics, I laughed out loud

I've always wanted to write a headline like that.  

If you are like me, any link suggesting that learning analytics can be easy is worth a quick look. Even if only to sneer at the temerity of whoever would say that. So, thank you for being curious enough to click. Now that I have your attention...

Getting started with a learning analytics initiatve doesn't need to be hard. What's hard is sustaining a learning analytics initiative IF one is only doing something, anything, as a pet project, an exploration, or a trial run because someone gave you free software. 

To paraphrase the incomparable Tina Turner, we're gonna start this learning anaytics conversation nice and easy (don't worry, no dancing required). Here we go. Five quick easy steps to get started:

(1) Start with a real problem to solve. Scope your problem statement so that you maximize your probabiities of  success. Do you have market conditions to address, sneaking suspicions that there might be trouble in River City that you'd better find out about before someone else does? Maybe places where you are pretty sure you can do a better job? You might want to think of something that will help you test the waters of change without putting yourself at risk of total humiliation if it doesn't work.

(2) Figure out who needs to be involved in creating the business case to get this problem solved. How much of a priority is solving the problem you identified going to be for others in your organization? Who need to be engaged and on-board to get this done? 

(3) Determine what you need to answer questions related to solving this problem. What information do you need? Who owns it? Where does it live? How are you going to secure it? How will it be analyzed? Who can see it, work with it, publish it? How will results be used? How long will the results live, and who will have access?

(4) Determine the necessary project talent to make sure you can get the work done. It's not just about having access to people who understand how to work with data. It's about having the right team in place across the board who can execute to plan and deliver results, and making sure that you have planned for enough of their time to be effective. How will you find them? What will you need to attract their attention and keep them engaged?

(5) Develop a bottom-up budget to see if you can afford to do what you want to do. Odds are, you will need more money than you have on hand. (Well... odds are that you will end up needing more money than you think you do, too, but that's a conversation for a later time). So, where might you find the funding to do what you want to do? Who else might be interested in getting involved in your great adventure?

Obviously each one of these steps can take on a life of its own once you get going, but as a place to begin these are a great way to get your arms around your big idea and see if its possible to move ahead.

Now that Predictive Analytics are Hotter than Ever

Now that predictive analytics are hotter than ever in higher education, it's probably a good idea to review a few fundamentals for those who are just starting to sip from the cup of possibilities.

WHY PREDICTIVES? Predictive analytics have been used in a wide variety of settings, including higher education, to manage finances, inventory, operations, assets and resources. Increasingly, higher ed institutions are turning to business intelligence for enrollment management and student recruitment. The next great wave for predictive analytics adoption in higher education is focusing on institutional performance outcomes and individualized student success. 

WHY COLLEGES NEED TO CARE: The more that colleges and universities are being held accountable for achieving specific performance outcomes (e.g., improved student retention, better completion rates for college students), the more predictive analytics will be touted as a solution for anticipating risks likely to be encountered while trying to achieve those outcomes. 

SIMPLY KNOWING WHO IS AT RISK ISN'T ENOUGH. Knowing how to mitigate risks and how different students can be better served with targeted interventions and support makes predictions actionable. Predictions without action don't really matter very much to anyone.

BEWARE THE BRIGHT AND SHINY: It's going to be a little bit like the early days of the first dot.com out there, with lots of  solutions available and even more promises about all the great things that those interventions and solutions can do (!!!). How will you know who to believe? How will you know what's right for you? You owe it to yourself to be informed.

THERE IS NO SINGLE "RIGHT WAY" TO CONDUCT PREDICTIVE ANALYSES. There are many techniques used to conduct predictive analyses. Typically one chooses the technique or techniques likely to yield results for the kinds of predictions one wants to make.

For example, if you are interested in predicting a discrete attribute you might be likely to use techniques including Logistic Regression, Decision Trees (CHAID, CART, Random Forest), Naïve Bayes, Support Vector Machine, Survival Analysis or Neural Networks. If you were more interested in predicting a continuous attribute, you'd look toward techniques such as Multiple Linear Regression, Time series or Decision Trees (CHAID, CART, Random Forest). If you were most interested in common groups, you might consider Hierarchical k-nearest neighbors, Neural Networks or Decision Trees (CHAID, CART, Random Forest).

Selecting the right techniques used for conducting predictive analyses have a lot to do with knowing the questions that the predictions will help answer, or the performance problems that are likely to be solved. Speaking of which...

START WITH A PROBLEM TO SOLVE: Sorry to burst your bubble but predictive analytics don't work quite like Magic 8-Ball. Not even Hadoop technologies do that. Answers simply do not emerge fully formed from the mists of your analytic techniques.  It's easier to focus on solving a problem (e.g. what causes students to drop out? Are these causes common in all settings?) or finding a new opportunity (e.g. what motivates students to complete courses faster?). Your problem statements and queries will help you focus on finding data sources and selecting techniques for analyses that are likely to reveal the patterns you seek.

Living Under the Sword of Data, on #Lrnchat

On May 9, 2013 I will have the pleasure of joining the crew at #Lrnchat for a session on the topic of learning analytics. You will find us on Twitter, Thursday, 8:30 pm ET/5:30 PT. Look for us at #lrnchat, follow along at @lrnchat.

 #Lrnchat is a regularly scheduled event that takes place every Thursday evening.  Participants using any number of tweeting tools (such as Tweetdeck and Tweedgrid) to keep track of the messages that fly around among the session participants. It is hosted and facilitated by a number of learning and development social media experts who shall remain nameless until I get their permission to drop their names. Questions get asked, responses get posted, conversations get started, friendships get launched. It's amazing how much one can pack into 140 characters! 

The name of this particular post, in anticipation of tomorrow's #Lrnchat, comes from one of my favorite analytics parables. In March of 2009, the lead designer from Google resigned in protest to what he described as an engineering culture that was obsessed by numbers. Douglas Bowman offered his reasons for resigning  in a blogpost entitled "Goodbye Google". In his explanation he gave the following examples of how data-obsessed Google had become:

"Yes, it’s true that a team at Google couldn’t decide between two blues, so they’re testing 41 shades between each blue to see which one performs better. I had a recent debate over whether a border should be 3, 4 or 5 pixels wide, and was asked to prove my case. I can’t operate in an environment like that. I’ve grown tired of debating such minuscule design decisions. There are more exciting design problems in this world to tackle."

In closing, he wrote that "I’ll miss working with the incredibly smart and talented people I got to know there. But I won’t miss a design philosophy that lives or dies strictly by the sword of data."

I remember being struck by that phrase as the perfect way to describe the double-edged benefits and risks that analytics bring to discussions of learning, achievement, progress, success. I offer this as something that each of us should keep in mind as we gravitate toward a greater use of enterprise data in our decision-making. 

I know we will have 90 minutes to talk about this. In anticpation of the conversation can I just offer this observation - It helps to have an idea about the problem you are trying to solve before you dive into an analytics campaign. And once you DO know, you need to be prepared to respond. Problem identification, without taking action, is almost worse than not knowing at all.

Here are a few references that I hope you will find to be useful. Happy reading! See you on Twitter.

Background References

Goodbye, Google The blogpost written by designer Douglas Bowman when he resigned from Google in protest to the data culture around decisionmaking. http://stopdesign.com/archive/2009/03/20/goodbye-google.html

The Gawker story that picked up the story of event leading to Bowman’s resignation from Google http://gawker.com/5177144/googles-data-fetish-drives-away-its-top-designer

Articles I've written on learning analytics:

http://www.educause.edu/ero/article/data-changes-everything-delivering-promise-learning-analytics-higher-education

http://www.astd.org/Publications/Magazines/TD/TD-Archive/2012/08/8-Realities-Learning-Professionals-Need-to-Know-About-Analytics

Recent tweets on Big Data –related topics by @edwsonoma

The Big Data Landscape http://bit.ly/12KXpvA 

11 Big Data myths http://bit.ly/15CD26B 

The Surprising Predictive Power of Analytics http://lnkd.in/M2VS9j 

The Rising Costs Of Misunderstanding Big Data http://lnkd.in/fBGWVP 

Ten things you need to know about Big Data http://tinyurl.com/cabxeam 

What's the skillset of a talented #BigData Scientist? http://ow.ly/kNKw7 

 

The Big Data Landscape

If you post the term "Big Data" in your Google search window today, more than 2 billion google search results will appear. If you perform a similar search on Bing, you will see that more than 1,790,000 Big Data links have been added to the Internet in the past 24 hours alone. Clearly, there is big interest in big data.

This graphic by Matt Turck and Shivon Zilis is an exceedingly helpful image for understanding the breadth and complexity of the Big Data landscape. Matt Turck's blogpost  also provides great information for understanding some of where this emerging market is taking us. 

 

 

For those of us working in post-secondary education circles, the emergent interests in big data, analytics, learning analytics and related evidence-based decision-making endeavors have been inspired by possibilities of better experiences, more personalization and better behavioral targeting. Still most education stakeholders who are giving learning analytics some consideration may be surprised to see the depth, breath and sophistication of the big data world.  While WCET's work on the PAR Framework has certainly contributed to a growing awareness on the power of predictive analytics for academic risk identification and the value of collaborative, multi-institutional data mining to improve student success, we are mindful that our student success focus touches a relatively narrow, targeted part of the Big Data picture. This graphic offers a more complete picture of the total ecosystem.

NEW NSF REPORT: New Technology-based Models for Postsecondary Learning

In January 2013 I  participated in a workshop sponsored by the National Science Foundation to help establish a 10 year vision to reconceptualize postsecondary learning. This past week our co-convenors, Chris Dede, Harvard University and Eric Grimson, MIT, released the 47 page report that captured the results of that meeting. Entitled New Technology-based Models for Postsecondary Learning: Conceptual Frameworks and Research Agendas, this is a comprehensive overview of promises, opportunities and challenges that will keep all of we post-secondary learning stakeholders on our toes for the next decade.

You can download the working draft of this paper here.

The basic premises addressed include the following:

• Advances in technology and in knowledge about expertise, learning, and assessment have the potential to reshape the many forms of education and training past matriculation from high school.
• In the next decade, higher education, military and workplace training, and professional development must all transform to exploit the opportunities of a new era, leveraging emerging technology-based models that can make learning more efficient and possibly improve student support, all at lower cost for a broader range of learners. 
• Potential risks must be managed, including those arising from the disruption of established delivery economics in our current learning institutions, the variable quality of learning outcomes these new models offer today, and the technical and conceptual challenges of better understanding how to design, develop, and implement innovative capabilities in ways that reliably deliver on their promise.

Workshop participants developed a framework for understanding this sea change and sketched steps towards a research agenda for realizing its benefits while avoiding pitfalls.

Contributors to this report include Daniel E. Atkins, W.K. Kellogg Professor of Community Information, and Professor of Information and Computer Science, University of Michigan - Ann Arbor. Lori Breslow, Director, Teaching and Learning Laboratory, MIT. John Cherniavsky, Senior Advisor, Division of Research on Learning, NSF. J.D. Fletcher, Senior Research Staff Member, Institute for Defense Analyses. Diana Oblinger, President and CEO, EDUCAUSE. Roy Pea, David Jacks Professor of Education and Learning Sciences, Stanford University. James W. Pellegrino, Distinguished Professor and Co-Director, Learning Sciences Research Institute, University of Illinois at Chicago. Bror Saxberg, Chief Learning Officer, Kaplan, Inc. James H. Shelton III, Assistant Deputy Secretary for Innovation and Improvement,U.S. Department of Education. Russell Shilling, CAPT, MSC, USN, Program Manager, DARPA. Greg Tobin, President, English, Math, and Student Success, Pearson Higher Education and yours truly, Ellen Wagner, Executive Director, Western Interstate Commission for Higher Education Cooperative for Educational Technologies. Arthur Josephson, Harvard Graduate School of Education, was our researcher.