How ProctorU and Examity make cheating on college exams easier than ever

Updates

April 1, 2018 I worked with ProctorU for the last three months to improve virtual machine detection, and in that time we have made a big difference in detecting and stopping virtual machine users and other cheating methods. ProctorU has also hired a full-time Security Engineer. We will continue to work together to improve the integrity and security of their testing platform. Still no response from Examity, and their platform still does not detect virtual machine use as of this update.

Jan 19, 2018 My blogpost was read by ProctorU CTO Matt Jaeh after it popped up in his Google Alerts, and he reached out to me via email. I met with him and his team over video conference and discussed my concerns with having not been caught while using a virtual machine to take my test. From my conversations with them it is clear that PrcotorU is dedicated to stopping cheaters, and is open to feedback for improvement, even though it did take some time to get the attention of the right people. They offered to hire me as a consultant to help their team improve the virtual machine detection scripts and overall testing security. They are aware of the threat posed by virtual machines being used by cheaters, and have also corrected the confusing statement about the use of virtual machines on their technical requirements page. I will be keeping the details of those scripts and cheating prevention methods secret, and hopefully we will be stopping cheaters who use virtual machines on ProctorU exams soon.

No response from Examity so far.

Disclaimer

It is not my intention to help students cheat by sharing this information. The opposite is actually true, I am advocating for these companies and those which provide a similar service to tighten up test taking security by preventing use of virtual machines. Schools need to put pressure on these companies also, because the value of their college degrees is in question when cheating is this easy.

The problem

ProctorU and Examity do not take any steps to prevent cheating by using a virtual machine. Cheating is as easy as loading up a virtual machine, and taking the test inside of it, while you do anything you want on the host machine.

Using a virtual machine while taking an exam gives the test taker the ability to:

• Connect a screen sharing app like join.me to the Host machine. This means someone can remotely control the mouse and take the test while it appears from the Proctor’s perspective via the webcam that the student is the one taking the test.
• Use Google, textbook, or notes on the Host machine.

I have reported it to both companies. In the last two months neither has added any kind of basic checking for virtual machines.

Background

It seems to me that neither company even has anyone working there who has a clue what a virtual machine is. Just look at what ProctorU's website has to say about virtual machines, and count how many things are wrong just in this brief statement:

Note: We do not proctor computers running virtual machines. If you are running a virtual machine when you come in, you will be asked to close your virtual machine and boot into your actual operating system to take your test.

...boot into your actual operating system to take your test?

Whoever wrote this clearly doesn’t understand what a virtual machine is. It’s not a dualboot, you don’t have to reboot to get out of it.

And a quick Google search of ‘site:examity.com “virtual machine”’ yields zero results, so they might not even be aware of the problem.

Examity might even care less about cheaters, just look what their CEO had to say in his response to this blog post

Michael London, CEO of Examity on July 8th, 2015, 11:11 am Entertaining, but absurd. Not one of them would work with our solution.

His statement is wrong of course, virtual machines are not detected by Examity. No cheating methods should be written off as being absurd without thorough investigation and testing.

Addenda

The demand for online college courses has risen significantly in the past decade, and now just about every major college is offering at least some entirely online courses. In addition, there are now many respectable, fully accredited online-only colleges. Both cases have created the need for students to take tests remotely in a secure manner. To meet the demand, companies like ProctorU and Examity have stepped forward to solve the problem.

ProctorU and Examity work nearly identically from the test taker’s perspective. You basically schedule your college exam at any time you want, and at that time you log in and get connected to your proctor. The proctor is someone in a third world country, who you’ve never met before. You will be trusting this person with installing programs and gaining remote access to every file and program on your computer while you take your exam. They will watch you through your webcam to make sure you do not look at notes, or talk to anyone during the test.

I am not against the concept of ProctorU or Examity’s services. I enjoy the convenience of remote testing. I have no problem sharing my screen or being watched by a stranger while I take my test, that doesn’t bother me or creep me out at all. After all, that part is no different from a test at a school. The problem is giving away complete control of my personal computer.

After they connect, I noticed that the proctor runs several scripts and executables. I haven’t analyzed them yet, and I’m sure they are well intentioned, but those could do anything - install malware, steal banking info. Not to mention if one of those scripts does some accidental damage which prevents me from being able to work the next day because I have to restore from backup.

So for those reasons I decided to use an old laptop with a fresh install of Windows 10. And that worked fine. I had no problem with giving the proctor complete control of my old laptop which didn’t have any personal files on it.

But then one day I wanted to take an exam while traveling for work. There was no way I was going to give full access to my personal laptop, so I considered my options. I could dualboot into another install of Windows 10, or I could try using a Windows 10 virtual machine and risk getting caught. I decided it was going to be too much work to setup the dualboot just for the occasion, so instead I tried using the virtual machine. I figured that I would get caught and then just play dumb and reschedule the exam for another day.

But what I found out is that ProctorU didn’t have a script that checked if the exam was being run in a virtual machine, nor did it properly train employees to do basic checks. Even when I connected my webcam with the name of “Virtualbox Webcam”, the proctor saw it and didn’t ask any questions. It’s not the proctor’s fault, they were just not trained properly. Right after that, the proctor opened up Task Manager to check if I had Virtualbox running inside my virtual machine. It wasn’t, so I was cleared to start the exam.

Later in the semester I also tried taking a test through Examity in a virtual machine, with the same result. No checks were in place whatsoever.

Emails with ProctorU

From: Reilly Chase [mailto:rchase@rchase.com] 
Sent: Thursday, January 18, 2018 10:59 AM
To: 'Darrett Stevenson' <dstevenson@proctoru.com>
Subject: RE: ProctorU Exam Security
Darrett,

Also, I’ve noticed that ProctorU checks to see if Virtualbox/Vmware is running before you start the test, but that is pointless.. If you install the ProctorU client INSIDE the virtual machine, checking the processes will not show it running, because Virtualbox is running on the HOST, not on the virtual machine that you are checking… 

If you want me to show you what I mean I don’t mind doing a demo for you guys

This kind of fundamental misunderstanding of the problem is on the tech-requirement website:

https://www.proctoru.com/tech-requirements/

“Note: We do not proctor computers running virtual machines. If you are running a virtual machine when you come in, you will be asked to close your virtual machine and boot into your actual operating system to take your test.”


Reilly Chase
760 622-7756

From: Reilly Chase [mailto:rchase@rchase.com] 
Sent: Thursday, January 18, 2018 10:35 AM
To: 'Darrett Stevenson' <dstevenson@proctoru.com>
Subject: RE: ProctorU Exam Security

Hi Darrett,

It is still possible to use a virtual machine to take ProctorU exam, allowing the student to cheat by using the host machine to do anything they want … search Google for answers, record the exam questions to give to a friend, or even invite a friend to share their entire screen and take the exam for them.

It's a serious issue that damages the integrity of online test taking and even online degree programs as a whole.

Are you guys still working on a fix? It shouldn’t be difficult to write a quick script that checks for a few of the most common virtualization softwares like Virtualbox, VMware, etc. Of course I would like to put it to the test when you are done to see how it could possibly be bypassed still, but that would just about get rid of the problem.

I recently took a test with Examity and saw the same problem there so just let them know about it too.

Thanks
Reilly Chase
760 622-7756



From: Reilly Chase [mailto:rchase@rchase.com] 
Sent: Friday, November 10, 2017 4:16 PM
To: 'Darrett Stevenson' <dstevenson@proctoru.com>
Subject: RE: ProctorU Exam Security

Thanks Darrett! If they have any questions feel free to reach out



Reilly Chase
760 622-7756

From: Darrett Stevenson [mailto:dstevenson@proctoru.com] 
Sent: Friday, November 10, 2017 4:11 PM
To: Reilly Chase <rchase@rchase.com>
Subject: Re: ProctorU Exam Security

Hi Reilly,

Thank you very much for the detailed summary and suggestions. This has been provided to our development as well as operational leadership, and they're very appreciate of the suggestions.

While we work rigorously to prevent unpermitted activity without compromising our user experience, there is always room for improvement; and feedback like this helps make sure we continue to do that.

Once again, thanks for the help. We look forward to seeing you for your next exam!

-Darrett


On Fri, Nov 10, 2017 at 9:19 AM, Reilly Chase <rchase@rchase.com> wrote:
Hi Darrett,
Has your team had time to look at this? 



Thanks,
Reilly Chase
760 622-7756


From: Reilly Chase [mailto:rchase@rchase.com] 
Sent: Tuesday, November 7, 2017 5:10 PM
To: 'Darrett Stevenson' <dstevenson@proctoru.com>
Subject: RE: ProctorU Exam Security

Hi Darrett,
I have taken several exams through ProctorU as part of my Bachelor of Science in Cybersecurity and Information Assurance program at WGU. When I take exams, I usually use my extra laptop which does not have any of my personal files on it, just as a precaution to protect my personal data and prevent any accidental damage that could occur with any programs ProctorU might run (even though I’ve never had a problem).

However, there’s been a few times that I didn’t have my extra laptop handy, and so I used Windows 10 virtual machine in Virtualbox so that I wouldn’t have to give ProctorU full access to my personal/work laptop OS. 

I expected that your software would detect this, but it didn’t. I didn’t try to hide it in any way to trick either the Proctor or the software into believing I was not using a virtual machine. I also never cheated on any exam.

It appears your software does not check if the student is taking the test from within a virtual machine. Also, it appears that your Proctors have been trained (incorrectly) to check Task Manager for any processes that show VirtualBox or VMware and kill them, however, those processes only run on the Host (not the virtualmachine itself) and can’t be seen from within the guest (the virtual machine that the Proctor has control of). Not to mention that even if it were the case, the processes could be easily renamed by the student beforehand, so it’s not a good method to check.

Because ProctorU is not stopping students from using virtual machines, the students can easily cheat (the technical barrier is very low to setup a virtual machine). The student can take the test in the virtual machine and give the Proctor full access, while at the same time share his Host machine with a friend over TeamViewer/LogMeIn etc. His friend can see the virtual machine and send him messages on the host machine for the answers (all of this invisible to the Proctor)

In addition, students are able to copy all of the test questions by recording the Host PC screen to share with others or sell

Your team does a very good job at other cheating methods already. They search the room, require ID and verification questions, and they really do watch and listen. I have been stopped before because of music in the background, and I’ve heard of people being stopped for looking out the window. That is a good thing. Proctors also check to make sure you don’t have a second monitor showing the test to someone else. That is smart.

The worst part about the VirtualMachine cheat is that is totally undetectable by all of those checks. With the virtualmachine, the student appears to be looking at the test, but he could actually be looking at the answers being sent to him on chat window in his Host PC.

Suggestions:
1.  You guys already run lots of scripts to check for things, just make a new script which checks for virtual machine settings. There is lots of info online how to do this. Make sure you check for different types (VirtualBox, VMware, etc)
2.  Train Proctors to spot Webcams with names that start with “Virtual …”, it was surprising to me that a Proctor never asked why my camera was named “VirtualBox Cam 1” for example, but it’s not their fault if they haven’t been trained to watch for that
3.  I can help test for you guys to make your product more secure if you want

Like I said, I’ve never cheated before, I am sharing this information to help better your product and make tests more secure. I think you guys provide a great service, and outsourcing proctoring is the future, something that will only become more needed, but also will require great security.

Thanks,
Reilly Chase
760 622-7756


From: Darrett Stevenson [mailto:dstevenson@proctoru.com] 
Sent: Tuesday, November 7, 2017 4:24 PM
To: Reilly Chase <rchase@rchase.com>
Subject: Re: ProctorU Exam Security

Hi Reilly,

Thanks for reaching out. My name is Darrett, I serve as our main point-of-contact for WGU.

We appreciate you reaching out with this feedback. Would you please let me know more details about the cheating method you've referenced? 

Thank you,
-Darrett

On Tue, Nov 7, 2017 at 9:44 AM, Reilly Chase <rchase@rchase.com> wrote:
Hi there,
I am a security researcher and student at Western Governor’s University.

Can you put me in touch with your security/development team? I would like to let them know about a cheating method which should be prevented by your software but isn’t.

Thanks,
Reilly Chase
760 622-7756





-- 

Darrett Stevenson ● Client Success Department Manager
352-505-9239 (Office) ● 209-728-4547 (Cell)
www.proctoru.com ● ProctorU Proven.





-- 

Darrett Stevenson ● Client Success Department Manager
352-505-9239 (Office) ● 209-728-4547 (Cell)
www.proctoru.com ● ProctorU Proven.

Emails with Examity

From: rchase@rchase.com
Sent: Thursday, January 18, 2018 1:50 PM
To: Matt Keough [mailto:Mkeough@examity.com] 
Cc: Kurt Grabner <kgrabner@examity.com>
Subject: Examity- 
Hi,
I recently took an exam with Examity and discovered it is possible (easy) to take the test from within a virtual machine. There were no checks to prevent this. As you can see in the screenshot (which I created just as an example, not during an actual test), by connecting to the proctor from inside the virtual machine (right), the test taker would be able to have full control of his Host PC, while the proctor only has view and control of the virtual machine.

With full control of the host PC, the test taker can then search Google for answers, or even start a screen sharing session with a friend. The friend could take the whole test for him and the proctor would be none the wiser because it would appear through the webcam that the student is taking the test, and the screenshare would only see the virtual machine.

It is a serious issue which has the potential to devalue the degree that honest students like myself are receiving from online courses, and something I would expect to be a top priority to resolve by Examity and similar companies.

I reported the issue to ProctorU as well 2 months ago, and although they seemed appreciative and eager to correct it, they still haven’t fixed the problem.