We Got Phished
by Paul Dancstep • October 20, 2016
Last month the Exploratorium was the target of a cyber attack. This is an account of what we think happened and how we dealt with it.
The story begins on September 6, when an Exploratorium staff member received an email from a familiar mailing list. It contained a link to a shared document, which the employee clicked on, sending her to a page like this:
She logged into her account but couldn’t find the document and, with other more urgent emails to deal with, she quickly moved on and put this brief event out of mind.
This staff member will henceforth be known as PZ, or “patient zero.”
The login page wasn’t really a login page. It was a decoy webpage, designed to look legitimate in order to trick unsuspecting recipients into typing in their private login credentials. Having fallen for the ruse, PZ had effectively handed over her email username and password to an unknown party outside the Exploratorium.
This type of attack is known as “phishing.” Much like putting a lure into a lake and waiting to see what bites, a phishing attack puts out phony prompts, such as a fake login page, hoping that unwitting recipients can be manipulated into giving up personal information.
The period following a successful phishing attempt is known as “the pivot.” This is when the attacker steps back and asks, “Now that I have access, how can I exploit this place from the inside out?” Our attackers spent three days planning what to do next.
At around 10:30 a.m. on September 9, three things happened to PZ’s email account. First, her contact list was deleted. Second, a filter on the account was changed sending all incoming mail to the trash folder. And third, the account began spewing out emails to the entire museum (and all of PZ’s other contacts), each with a link to a phishing login page. The emails had even been customized for our institution; the subject line was “Explratorium Report.”
Most of us could tell right away that PZ’s email seemed phishy. Unfortunately the barrage of replies PZ began receiving (eg “Are you spamming me?,” “I think you’ve been hacked,” etc.) weren’t showing up in the inbox. By diverting all incoming email to the trash, the attackers had essentially blinded PZ to the fact that anything was amiss. Moreover, once it became apparent that the account had been hacked (people began showing up at PZ’s desk with the bad news), PZ had no way to warn anyone. You can’t alert your contacts not to open a previous email from you if your contact list has been deleted. PZ’s account was erupting with fraudulent emails and there was no way to shut it off or mitigate the damage it was surely causing.
One of our IT specialists received the email and responded with the following message:
Hi PZ,
Did you mean to add me to this?
Two minutes later came this reply:
Just sharing some of out document for the year 2016 with our partners you might be interested in it, I made it secured.
Thank you.
-PZ
This is not at all how PZ talks. Someone, somewhere, was monitoring PZ’s trash folder and responding to her messages, trying to maintain the illusion of normalcy for as long as possible.
Our IT department began investigating. Where was the account being controlled from? If you log in to a Gmail account on a web browser and scroll to the bottom right you’ll see a link that says ‘Details.’
Clicking on this brings up a window that shows all of the devices logged into that email account (this may include smartphones, a work computer, a home computer, etc.). When our IT department brought up PZ’s account activity they saw two suspicious-looking logins, one from a server in Fargo, North Dakota, and another from an IP address located in Nigeria.
IT also tracked down the fake login page that the emails linked to. This required some care: the webpage could have been set up to load malware onto visiting computers, which would mean that clicking on the link would lead to immediate infection. The address of this webpage had been disguised in the link using a URL shortener, which IT reverse engineered with a URL expander in order to run a virus scan. The page appeared clean; its only apparent intention was to collect passwords. This webpage was being hosted on another server, this one at a server farm in Texas.
From these and other clues collected using forensic tools in Google Apps and our firewalls, our overall impression of the attack is as follows:
By spreading the attack across many devices, the attackers were hoping to avoid Gmail’s fraud detection system. Emails coming from a domestic server are inherently less suspicious than emails coming in from overseas and, by splitting the parts of the attack between two different hosts, the scheme was made harder to detect.
Using our Intrusion Detection System, IT determined that 54 people at the Exploratorium had clicked on the malicious link. As one of these 54 all I can say is that–as paranoid as I usually am–this just didn’t trigger an alarm for me. An email from PZ has high social credibility within the museum and the presence of our museum name in the subject line (even if misspelled) was enough to successfully fake me out.
“What makes an attack like this so effective is that you never expect to see something as convincing as this,” said Miles Reed, one of our IT systems administrators, “The vast majority of the phishing emails received by the Exploratorium are rejected outright or filtered into spam.” A convincingly personalized phishing attack like this is known as “spearphishing.”
Around the time that PZ’s account was sending its 200th email, Google flagged the activity as suspicious and blocked any further outgoing email from the account. IT sent out a museumwide alert, restored PZ’s account to normal, and had everyone who’d visited the phishing site change their password. After attending to our basic network hygiene, IT set our alerts to a higher threshold to monitor for suspicious login attempts. Indeed, throughout the next several weeks our email accounts were being periodically probed for password weaknesses. These alerts had a chilling quality to them, a bit a like if you changed the locks on your house and then received a note every time someone tried to break in with the old key.
We can only speculate about what these attackers intended to do with our information. Login credentials can be bought and sold on the black market. Networks of compromised computers can be operated as “botnets” to send out spam or take out webpages. Perhaps they intended to capture credit card data from our online ticketing system. In 2013, Target had 110 million customer credit card records stolen by hackers, a breach that originated with a simple phishing attack on one of its subcontractors.
Knock on wood, but it seems like this attack was diverted without any serious negative impacts. This one was more like catching a bad cold then getting seriously knee-capped, although it does leave us in the institutionally sheepish position of having probably given this stupid cold to some of our friends and partners.
October is National Cybersecurity Awareness Month and, as always, it’s flu season on the internet. Maybe this is as good a time as any for you to switch to two-step verification.
Be safe out there.
