Approaches to assessing students

Approaches to Assessing Students
While Ensuring Academic Integrity
and Student Convenience

WCET 2011 Conference, October 27, 2011
Denver, Colorado

Panelists:
April Cognato, Michigan State University (MI)
Deb Gearhart, Troy University (AL)
Mark Sarver, EduKan (KS)

Moderator:
Teresa Theisen, Minnesota State Colleges and Universities (MN)

Facilitator:
Scott Woods, University of Phoenix (AZ)

Overview
Various methods are used in distance education to
achieve a higher level of academic integrity.

In the presentations, you will learn from several
administrators who are striving to improve academic
integrity, avoid implementation obstacles, and provide
convenient methods for authenticating distance-learning
students.

Materials and activities will focus on lessons learned,
success stories, and best practices in academic integrity.

April Cognato
April Cognato has been an assistant professor of biology at
Michigan State University since 2006 where she is involved
in the development and implementation of traditional and
online curricula.

Concurrently, she is a contributing author to two non-
majors biology textbooks, and author of online digital
assets for McGraw-Hill Higher Education.

She has been an undergraduate biology instructor since
1996.

Cognato received her B.S. from University of California,
Davis and an M.S. and Ph.D. from Texas A & M University.

One Falsehood Spoils a
Thousand Truths
--Ashanti Proverb

April Cognato, Ph.D.
Michigan State University

Academic Integrity…

…is the very principle upon which
colleges and universities are built

Academic Integrity…
…is “a commitment, even in the face
of adversity, to five fundamental
values: honesty, trust, fairness, respect,
and responsibility”
Center for Academic Integrity

Academic Integrity Matters
• Higher Education Opportunity Act
2008 requires that distance learning
meets the same expectations as in-
class instruction

• Requires student authentication in
distance learning

Academic Integrity Matters

“Without academic integrity, there can
be no trust or reliance on the effectiveness,
accuracy, or value of a University's
teaching, learning, research, or public
service activities.” –UC Davis Code of Academic Conduct

The credibility of our
academic system is at stake

Academic Integrity is in Crisis
Admitted to cheating at Perceived severity of
least once in the last behavior (moderate to
Dishonest behavior
year (2002-2005) serious infraction)
n = 71,071
Learning what is on a 33% 64%
test from someone who
took it
Copying from another
student
Helping someone cheat
63% 20%

10%
90% 92%

89%
Using unauthorized 8% 90%
notes
Using an 5% 90%
electronic/digital device
False excuse to delay 16% 58%
test
McCabe, D. (2005) Cheating Among College and University Students: A North
American Perspective. International Journal for Academic Integrity Volume 1, No.

“If it should ever come to that, and you
…”Cheating is considered dishonest. It
know no one has no proof whatsoever
counts as stealing and lying. There are
that you cheated: deny, deny, deny!
some cases, however, where cheating on
(Convince yourself as well be your
a test might be argued to as
adversaries you did not cheat,are tests
acceptable. Sometimes there remaining
firm and the result of As long as theythan
that are confident.) politics, rather can
not prove it but can only wonder or hear
practicality.”
someone else's words against your own,
you're in the clear”.

Our Immoral Brains

Opportunity to lie Opportunity to lie No opportunity to lie
Decision: LIE Deciding whether Decision: Honest
or not to lie

Greene, JD and JM Paxton (2009) Patterns of neural activity associated with
honest and dishonest moral decisions. PNAS 106 (30) 12506-12511

Moral decisions…

…depend more on the absence of
temptation than on the active resistance
of temptation.

Greene, JD and JM Paxton (2009) Patterns of neural activity associated with
honest and dishonest moral decisions. PNAS 106 (30) 12506-12511

Our Responsibility
• Remove temptation
• Provide a secure testing environment

Case 1
Michigan State
University

Remote Proctor Pro
A Comprehensive Proctoring Solution

Dr. April Cognato, Ph.D.
Michigan State University
ac@msu.edu

Online Course
• Fundamentals of Genetics
o Fully online
o 150 students
o Single summer session (7 weeks)
o Not proctored, remote

• Assessments
o Problem Sets, short answer, word problems
o Summative assessments, weekly “mid-term” like
exams

Academic Dishonesty
Infractions included:

• Plagiarism from internet sources
• Copying answer key (verbatim!!) provided
by other students
• Submitting, obtaining the key, resubmitting
with the key (LMS bug)

Suspected many; failed 6 students in 2009, 2010

A Comprehensive Remote
Proctoring System
Desired End User Desired System
Features Features
1. Flexible 1. Authenticates student
2. Cost-effective identity
3. Easy to employ for 2. Monitors student
faculty and students activity
4. Tech support 3. Secures computer
access
5. Customer support
4. Compact, portable
5. LMS compatible

Remote Proctor Pro Solution
 Authenticates
o Biometric data
 Secures computer
o Lockdown browser
 Monitors students
o 360° camera
 Compact & portable
o Desktop, USB, disassembles
 Records
o Video record of exam session
 Reports
o Exam policy violations

Remote Proctor Pro
How it Works
1. Assembly
2. Enrollment
3. Authentication
4. Assessment
5. Post-assessment Review

Remote Proctoring Solutions
Provided students with two options:

• Remote Proctor Pro (RPP)

• National College Testing Association (NCTA),
Consortium of College Testing Centers
(CCTC) In-person Proctoring

RPP vs. Testing Center
Feature RPP CCTC
Authentication X X
System
Monitoring X X

Browser security X Human

Compact/portable X

Convenient X
End-user
Cost-effective X

User friendly X X

Tech support X

Customer support X X

Cost Comparison
Testing Center Hours Fee CTCC 2011 RPP
MSU M 8AM-5PM $30/exam
$180
W 12PM-5PM
Ferris State M 9AM-4PM $20/hour
University
$240
W 9AM-4PM
Grand Rapids CC M 7:30AM-7:30PM $10/hour
W7:30AM-7:30PM
$120
Kalamazoo CC M 8:30AM-9PM $20/test
$120 $100
W 8:30AM-9PM
Central Michigan M 9AM-1AM $30/test
University
$180
W 9AM-1AM
Lake Superior State M 8:30AM-4PM $15/hour
$180
W 8AM-12PM
Southwestern $20/exam
Michigan College
M 8AM-5PM $120
W 8AM-5PM

Remote Proctored Exams
• 86 students opted to use the RPP
• 6 exams over 7 weeks
• Semi-synchronous exam periods
• 2 hour time limit
• Multiple choice, short answer, essay (word
problems)
• Survey of proctoring experience delivered for credit
at the end of the course

Remote Proctor Pro
Protects Integrity
LMS Bug
4% 4%
8% Impossible/Diffic
ult to Cheat
Easy to Cheat

Knows someone
84%
who cheated
No reponse

Student Testimony
“It’s a great way to have the freedom to take the exam on
your own time in whatever setting you want”

“It’s easy to use and you can take [the test] when you want”
“The remote proctoring device was easy and convenient…you could
take your exam in your own home and at whatever time you wanted.”

“You couldn’t cheat and I took my test in the comfort of my own home.”

“Remote proctoring tech support was VERY NICE and HELPFUL.”

“The lowered cost compared to the in-person proctoring exam was a plus.”

“It looks like something out of Star Wars!”

Take Home Points
• Remote Proctor Pro system effectively protects
academic integrity
• RPP is a comprehensive proctoring system
o Authenticates student identity
o Monitors student activity
o Secures computer
o Records exam session
o Reviews and Reports
• RPP meets the needs of the end user
o Easy to use (faculty and students)
o Convenient
o Cost effective
o Responsive tech support
o Attentive customer service

Future Best Practices
• Inform students of proctoring requirement (and
associated costs) at the time of registration.
• Deploy device at least 2 weeks prior to beginning
of course.
• Require a practice assessment the first week.
• Institute a backup exam plan.
• Establish good communication with your CMS IT.
• Provide students with clear instructions for
Technical Support.
• Anticipate technically challenged students!

Pedagogical Best Practices
• Tutorial on academic
integrity
• Synchronized
assessments
• Question pools
• Random delivery of
questions and answers
• Exam review controlled

Testing center at the University • Required exam
of Central Florida
NY Times July 5, 2010
proctoring

Image Sources

Logo: Center for
Academic Integrity http://dougleschan.co http://nonsensepics.blogspot.co http://catholicno
(http://www.academici m/dougleschan/wp- m/2010/11/low-tech-way-to- tebook.blogspot.
ntegrity.org/) content/uploads/2011/0 prevent-cheating-on.html com/2007_07_01
8/integrity-150x150.gif _archive.html

Deb Gearhart
Deb Gearhart is the director of eCampus for Troy University.

Previously Gearhart served as the founding director of E-
Education Services at Dakota State University in
Madison, South Dakota and was there for the 11 years.

Before joining Dakota State she spent 10 years with the
Department of Distance Education at Penn State.

She earned a M.Ed. in Adult Education with a distance
education emphasis and an M.P.A. in Public
Administration, both from Penn State. Gearhart completed
her Ph.D. program in Education, with a certificate in
distance education, from Capella University.

• On average eTROY proctors over 7500
course exams a term
• A need to assist students with
proctoring options
• Adopted an academic operating
procedure for online course proctoring in
2008.
• Needed both human and technology
based proctoring options.

The educational-technology firm is
trusted by over 100
colleges, universities and
corporations as the best way to
reduce incidents of academic
dishonesty by connecting students
face-to-face with a human proctor.

See the student See what they are doing Know who they are

• Students interact with a live proctor in
real time.

• Our identity authentication method is
the strongest in the industry.

• ProctorU is easy to use for students
and faculty.

• Students enjoy the convenience of
testing at home and instructors rest
easy knowing the integrity of their
exam is secure.

Approaches to assessing students

Mark Sarver
Mark Sarver has served in many capacities in higher
education. He is the CEO of EduKan, the online
education consortium for the six western community
colleges in Kansas. Sarver has taught courses in
leadership, organizational
behavior, accounting, international business, franchise
development, marketing and management. Sarver's
experience in higher education strategy and
assessment spans virtually all aspects of institutional
management from admissions and marketing, to
financial analysis, to program and course assessment.

Troy University
Northwestern University
The University of Florida
The University of Arizona
The University of Illinois
Bob Jones University
Chaminade University of Honolulu
Kansas State University
Lawson Learning
National American University
Oregon State University
Saint Louis University
Thomas Edison State College
The University of West Florida
The University of Louisiana at Lafayette
The University of Mississippi
The University of New England
The University of Tennessee Health Sciences
Washington State University
Northern Virginia Community College (ELI)
Others…

“I have completed countless online
courses. Students provide me with
passwords and user names so I can
access key documents and online exams.
In some cases, I have even contributed to
weekly online discussions with other
students in the class”.

“IDENTITY THEFT ON DEVICES AND OVER NETWORKS CREATES THE FASTEST
GROWING WHITE COLLAR CRIME. BIOMETRIC SIGNATURE ID HAS AN ANSWER.”

Patents: 7663614, 8004491

Copyright ©2011 All Rights Reserved - Confidential
Biometric Signature ID www.biosig-id.com

Agenda
• About EduKan
• BSI Company Background
• Briefing on Biometrics and Multi factor Authentication
• Edukan case study – Dr Mark Sarver CEO
• Q&A

47

Biometric Signature ID Corporation
 Dallas-based privately held software development company
 The way you move your mouse (pointing device) is unique and can
make a positive ID using dynamic (behavioral) biometrics
 Gesture (dynamic) biometrics capture
speed, length, angle, height direction of how we write not what
we write
 “3 identity proofing software products” that require no hardware
or installation unlike other static biometrics (fingerprint etc…)
 Satisfies security need in multiple sectors for remote identity
proofing and access to the PC and smart devices
 DEA, SSA, HHS, DE, Homeland Security, financial services, white
house, cloud computing, e-commerce, FFIEC…
48

Static versus Dynamic Biometrics
• Static biometrics are physiological or anatomical i.e.
fingerprints, iris scans, facial recognition, DNA swabs etc.
• Static biometrics require “hardware”.

• Dynamic biometrics measure behavioral characteristics and do
not require the use of hardware

 In our case we capture a students movements as they draw with
their mouse, finger, stylus, touch screen (pointing device).
 Movements such as speed, length, angle, height, direction are
unique to each student.

49

Multi-Factor Authentication
Is the new “Gold Standard”

Something You Have Something You Know
Who you Are
Credit card, Tokens, Unique characteristics - static PINS, Passwords, Images
Smart Card, devices OR Dynamic Biometrics
Weak Strong Very Weak
50

Closed Loop Technology – Always 2 Levels
BioSig-ID Click-ID CSQ – Complex Questions

+ +

-

Enrollment – Two levels are always required for enrollment in BioSig-ID and Click-ID. In rare
circumstances Click-ID and CSQ’s are used only when user cannot enroll in BioSig-ID.
Validation - If user exceeds the number of attempts to validate with BioSig-ID, they are directed
to Click-ID to validate. If successful, user creates a new enrollment with BioSig-ID. This is the
same process if the user has enrolled with Click-ID and CSQ’s
Cost Savings - Can be used for password resets – reducing help desk calls and overhead.

51

EduKan Case Study - Primary Goals
 EduKan worked with Biometric Signature ID to run a proof-of-technology pilot
because:

 10,000 hours for proctoring and related costs

 Proctoring may create a competitive disadvantage

 Proctoring may create physical presence in some states
 Meets HEOA and HLC requirements
 Wanted to assess their biometric technology as a cost-effective way of addressing
online student security and reduction of proctoring
 Measure student acceptance of new identity verification software
 Uses a biometric system where no hardware is required
 Tests a system where students create their unique password by just drawing
shapes/letters/numbers with their mouse


EduKan Methodology
 During March-April, 2011, 174 students from multiple classes volunteered
to verify their identity from their own computers using BioSig-ID

 Test subjects/students were asked to enroll, create a profile and verify
their identity at various times for a total of 6 times before the final exam.
At the final exam they were asked to validate their identity and receive a
password to access the exam

 Full audit trails were analyzed of all activity and an online survey was
administered

 Nearly 6,300 enrollments/verifications were completed by the participants
in 60 days


Current Verification Methods


EduKan Overall Findings

Based on the results of this technology pilot, the BioSig-ID verification software
using just a mouse was a useful and practical tool for remote identity proofing.

During the pilot study, it required little administration (only 9 help desk calls from
174 students) and will help EduKan comply with newer, more stringent
recommendations of the HEOA and HLC.

EduKan intent is to replace proctored exams with BioSig-ID and have started full
deployment with over 1,600 students


97% Expressed Ease of Use


16% did not view the video?


For Testing: 97% of Students Prefer BioSig-ID


Typical Responses


Features Students Liked from EduKan


Going Forward

 Full implementation with deep integration into
LMS (Pearson Learning Studio – eCollege)

 Must change the way we teach, evaluate and
assess

 All 4,000 students

 Implementing U-Sign

Contact information
Stephanie Piche, Director Marketing stephanie.piche@biosig-id.com 310-729-2997
Jeff Maynard, President jeff@biosig-id.com 972-436-6862
Dr. Mark Sarver, CEO EduKan marks@edukan.org 620-204-0374

Test drive the software
http://www.biosig-id.com/products/test-drive/

Biometric Signature ID completes very successful proof of technology with Houston Community
College District the 4th largest in the nation with 98% satisfaction
• http://www.biosig-id.com/about/people-are-talking/
Biometric Signature ID white paper on How to reduce Academic Dishonesty (highlights uSignOnline)
• http://www.biosig-id.com/products/edu/
Independent third party test results announced proving BioSig-ID software exceeds the
national NIST standard for accuracy by a 3 fold factor;
• http://www.biosig-id.com/biometric-signature-id-scores-an-outstanding-99-
97-accuracy-against-identity-fraud-from-the-tolly-group/

62

User Acceptance + 3rd Party Testing Accuracy Studies

Summary of Results

100.0% 99.78%
100 97.3% 99.97%
94.0%

75

2 minutes
50

19 Seconds
25

Enrollment *Ease of Ease of Stop Imposters Access Granted Average Average
Attainment Enrolling Validation With Passwords To Legitimate Users Enrollment Validation

V2.0 Validations – University Pilots (multiple campuses), other users, Independent 3rd party testing subjects by The
Tolly Group =32,000 total enrollment and validation transactions


BSI: Recognition, Validation and Acceptance
 Awards: BSI awarded the “2010 New
Product Innovation of the Year Award Frost & Sullivan employs 1,800
analysts in 40 global offices…
for North America Signature
Biometrics” from Frost and Sullivan
 Third Party Testing: Software denies
>99.9% of imposters from successful
log in, results exceed NIST standards
 Rigorous Evaluation: State of Texas
recognition and investment as
“breakthrough technology”
 User Acceptance 98% of first time
users performing over 50,000 actions
had a positive experience
 Patented Technology: BSI has
University of Texas Systems –
received two US patents for their User Acceptance Studies
gesture biometrics technologies 9 campuses, grad/undergrad, 46 classes, 167
students, 6 weeks, 10 authentications at
(Patent -7663614,8004491) random, different ages, locations, disciplines

64

Approaches to assessing students

More Related Content

Similar to Approaches to assessing students (20)

More from WCET (20)

Recently uploaded (20)

Approaches to assessing students

Editor's Notes