On December 8th, Dr. Anthony Picciano, Professor and Executive Officer of the PhD program in Urban Education, joined us for a Futures Initiative Open Session.

This was a very topical kick-off to our Open Sessions as we are gearing up for our inaugural course, Mapping the Futures of Higher Education, being co-taught by Dr. Cathy Davidson and Dr. William Kelly in the Spring 2015 semester.

Image Credit: Lisa Tagliaferri

Dr. Picciano, who attended Hunter College as an undergraduate and began his career as an adjunct at BMCC, is currently a professor at Hunter College and the Graduate Center. He will be entering his 45th year at CUNY in January. He first wrote an article warning about the Educational-Industrial Complex in 1964, and has written prolifically about education and technology ever since (here is a selection of his publications). His deep familiarity with CUNY and with the intersection of education and technology (teaching COBOL programming was among his first positions) has provided us with a very beneficial perspective as we think through evaluation and assessment as part of our upcoming course.

“You want technology to respond to you, you don’t want to respond to technology.” – Anthony Picciano

Tony described four waves of change that have transformed online teaching and learning over the last thirty years.

First Wave – The Beginnings of Online Learning (1990s)

In the 1990s, when Tony taught his first class, he said the leaders in the market were distance learning education leaders such as the University of Maryland. The problem during this time was that the software systems were not in place yet to maximize the online learning experience. Because there were no software platforms that catered to the online education market at the time, educators built their own websites with rudimentary tools and had to have some knowledge of HTML code in order to program the sites.

Listservs were often the standard for communication and online discussion, though this provided a somewhat limited form of Interaction between the students and other students and between the students and the teacher. A program would allow the listserv to be displayed on a website in a threaded list. The benefit was that everyone could see it and interact with it, but the drawback was that it was difficult to navigate.

Teaching his first blended online course at Hunter College in 1996 and first fully online course in 1997, Tony spoke of technology as a convenience, especially for CUNY students who are often carrying a heavy load that includes full- or part-time employment in addition to childcare obligations.

Rather than adapting to technology, he emphasized that pedagogical goals should drive technology, explaining, “You want technology to respond to you, you don’t want to respond to technology.”

Image Credit: Lisa Tagliaferri

Second Wave – Pedagogical / Blended Model (2000-2008)

The second wave of online learning started around 2000, and included colleges and institutions that were not traditionally involved with distance based learning. Tony characterized this wave as a pedagogical / blended model for online education which included face-to-face learning and utilization of online components and tools to enhance the interaction and learning experience.

The blended model for online learning was faculty driven rather than administratively driven. At the same time that more faculty were discovering how online interaction could enhance courses, there was a blossoming of online degree programs (for profit) that were growing across the country. While blended learning was mostly based on pedagogical purposes to enhance and supplement face-to-face learning, the for profit operations often used online components to reach a bottom line.

No one knows exactly how many students are now enrolled in blended learning courses. Colleges and universities have been weak in trying to gather this information. By 2000, most colleges and universities had started subscribing to course management systems like Blackboard, but the administrators do not have an adequate metric of which courses and professors use these systems.

Third Wave – MOOCs and the Access-Cost Benefit Model

The blended / pedagogical model was eclipsed in 2008 when George Siemens offered the first MOOC. The media hype around MOOCs began when Sebastian Thrun, co-founder of Udacity, followed a few years later with his own version of a Massive Open Online Course. A lot of politicians and administrators became excited about MOOCs as the savior for education around the world–lofty claims were made that MOOCs would decrease the cost of education and provide opportunities for education to whoever wanted it worldwide.

By 2013, MOOCs had already started to demonstrate that they could not live up to the goals of reduced cost and education for all. Costs were sometimes higher for MOOCs to run effectively or learners did not achieve the same outcomes as traditional face-to-face learners. Following a less-than-successful MOOC experiment, Cal State put its Udacity-affiliated project on hold. Their MOOC courses did not do well when compared to traditional face-to-face and blended courses.Sebastian Thrun of Udacity said they need to rethink the MOOC concept and applicability. Some of this has translated in a move of MOOC providers out of higher education and into corporate education programs instead.

Fourth Wave – Reconciliation of Pedagogical / Blended and MOOC Models

Following the decline in popularity of MOOCs, a fourth wave of online learning is now reconciling the role of online learning in education with a return in focus to the pedagogical / blended learning model that is more faculty driven. Educators are asking themselves what are the goals and objectives of a course? And, what are the pedagogical reasons for including or not an online component in a course? This is the first time that many in higher education have been exposed to pedagogical deliberations. Outside of education, fewer faculty members have had a background in pedagogical methods and teaching.

Challenges for Evaluation

Once a faculty member decides to include an online component for a course, the next question becomes how do you know if you have achieved your pedagogical and learning outcomes? How do you evaluate the course?

One basic problem is how to ensure that students are the ones actually doing the work. Less than graceful methods have been to have students go to computer facilities for proctored exams in online courses. Other examples include video recording students as they take exams and having proctors monitor the video feeds.

There are now 7 million students in online courses, equivalent to 1/3 of the college student population. Online learning presents unique challenges for educators. How do you differentiate what is a substantive posting versus a less than substantive posting? In online discussions, postings may become the most valuable contributions to the course and learning process. Evaluating those contributions can be difficult.

A lot of faculty members have moved toward incorporating self evaluation into the course structure. Students are responsible for grading their own contributions as well as those of the other students in the course. Tony argued that this is fine with mature students. He said that it is the mature, self-directed student that does better in an online, blended-learning environment anyways.

In the collaborative learning environment, and important kind of skill many educators have been anxious to engage students with, there are other problems for evaluation such as increased time requirements, more meetings, how work is distributed amongst groups, etc. Wikis, blogs, and programs like Google docs have solved some of the logistical problems. But again, evaluating those contributions may be problematic.

Digital Divide and Access to Technology

In addition to evaluation problems, another issue is that not all students have equal access to technology. About 80% of homes in the U.S. have access to the Internet at home. Not everyone has high-speed access either. For those that do not have home access, public access is available at libraries, but this may be difficult. The convenience of readily available access gives and edge up over students lacking access. Research conducted by Hunter College Library, for example, observed that some students could not afford broadband access at home and have to rely on their mobile device for access to online components of coursework.

Big Data, Learning Analytics, and Education Research

Online course management programs like Blackboard track nearly every interaction on the site, but how can that information best be used for pedagogical purposes and research? Some research has already been done doing discourse analysis on student responses. Software programs can search for which words were used most frequently amongst the students or other participants in the class

Learning analytics programs are also a hot topic of discussion right now. Many think analytics will have a significant impact on teaching methods and learning outcomes in the future. How can “big data” be used on students in the online environment? Predictive models are being discussed that would help predict what grade a student will receive in the course based on how they are performing and progressing in the online environment. These are based usually not on tests and assessments but on the interactions. Every keystroke can be tracked and assessed. While these types of analytics will likely be useful in the fully online course environment, they may be less useful in the blended-learning environment where they are only capturing one facet of student engagement.

_________________________________________________

Links to additional information:

Handout: https://apicciano.commons.gc.cuny.edu/2014/12/09/rethinking-evaluation-and-assessment-in-online-and-blended-learning-environments/

Tony’s Blog about the event: https://apicciano.commons.gc.cuny.edu/2014/12/09/rethinking-evaluation-and-assessment-in-online-and-blended-learning-environments/

Storify: https://storify.com/danicasavonick/rethinking-evaluation-and-assessment-in-online-and