How Black Box Algorithms are Assisting a New Generation of Criminals
Black Box Algorithms Published on Feb 17, 2016
When Siri helped a young criminal nearly get away with murder, future crimes expert Marc Goodman realized how algorithms had become co-conspirators in a new age of digital crime. Goodman’s latest book is ” Future Crimes: Inside the Digital Underground and the Battle for Our Connected World
Read more at BigThink.com: http://bigthink.com/videos/marc-goodm…
Black Box Algorithms
Transcript – There was a case recently in Florida where a teenager was arrested for murder. He was a student at the University of Florida and he was accused of murdering his roommate and best friend. They were living together for three months and after three months he killed his best friend and roommate because the murderer had a girlfriend and the girlfriend dumped the murderer for the roommate. As it turns out when the kid murdered his roommate he had a problem. He didn’t know where to bury the dead body. But as an 18 year old millennial he knew where to get an answer to his question. He asked Siri where can I bury a dead body. And it turns out Siri answered his question and proposed minds, dumps, reservoirs, swamps and rivers. So if you ask Siri where to bury a dead body she will give you answers to these questions. So in the old days, you know, we used to talk about Bonnie and Clyde. But we’ve entered the age of Siri and Clyde where clearly algorithms will be unnamed co-conspirators in younger criminals carrying out attacks.
AI is being used across the board in our society via algorithms, right. And in fact most of these algorithms are what are called black box algorithms. There’s a Harvard Professor called Frank Pasquale who wrote a whole book on black box algorithms that talks about how little we know about these algorithms and what some of the dangers might be. For example there’s an algorithm that determines your credit score, FICO. What goes into it exactly and precisely nobody knows. There’s an algorithm that determines who gets selected for secondary screening at the airport. Maybe it’s because you bought a one way ticket? Maybe it’s because you paid in cash? Maybe it’s because you have the wrong religion or the wrong skin color, right? We don’t know clearly at all what is being encoded into these algorithms. And so that opens up the door for them to be abused. We saw examples of this on Wall Street with Flash Boys, right, on rapid trading on Wall Street. The fact of the matter is only a miniscule amount of trading on Wall Street is carried out by human beings. The overwhelming majority of trading is algorithmic in nature. It’s preprogrammed so that if the price of soybeans goes down all these additional steps will take place. If some event happens in the world traders will buy or sell based upon that information. And it all goes so fast there’s no time for human intervention. And we saw an implication of this recently when the Associated Press Twitter feed was hacked a few years ago. Somebody took over the official AP Twitter feed and they put out a tweet from the official site that said breaking news, explosions at the White House, President Obama injured. Now it turns out that never happened but all the algorithms that are monitoring the Net looking for the latest news that they can trade on picked up on it immediately from a trusted source. And because they perceived a terrorist attack that caused the market a massive, massive sell off. In just three minutes because of this one tweet the market fell 136 billion dollars, 136 billion dollars of valuation was evaporated in 180 seconds just because of one wayward tweet. And that was carried out by an organization known as the Syrian Electronic Army. They’re backed, trained and funded by the Iranian government. Now in this particular case they did that for the purposes of mischief. But they also could have shorted the market at the same time and made a lot of money on this. So our algorithms are going way faster than we realize and they’re running things that we don’t even understand. For example when you go for an MRI exam in the hospital that MRI via its algorithms are actually interpreting the data for your radiologist in many ways before they even read that. When you fly on autopilot on an airplane which is probably more than 90 percent of your flight it’s an algorithm that’s running the plane. And all of those can be hacked. Much of the attacks that occur in cyberspace whether they be virus attacks, ransomware, denial of services attacks are all scripted to run which means a computer hacker or criminal writes some code and the code goes off and carries out the crime. Which means crime can scale and it can scale exponentially. And that’s why we’ve seen some massive uptick.