Fighting AI Scams with Patrick Coughlin of Savi Security

Ryo Koyama Hi, this is Ryo Koyama. Welcome to another episode of Remotely Local. Mike Johnson and I are very, very excited to welcome Patrick Coughlin today, CEO of Savi Security and also author of Dark Side of the Boom. And so, very excited to have you. Thank you, Patrick, for joining us today.

Patrick Coughlin Super happy to be here. This is going to be fun.

Ryo Koyama So, let's first, for those that don't know, because Savi Security is brand new, why don't you give us a little backdrop about that, and then we'll dig into the whole space about your book and why we are excited to talk to you.

Patrick Coughlin Yeah, absolutely. I'll give you the quick background. So, I'm a cybersecurity guy by trade. I've been working in the industry for about 20 years, done work on the government side and on the enterprise side. I had a company called TruSTAR that was a threat intelligence platform for large Fortune 500 companies. That was acquired by Splunk, and then Splunk, in a very Silicon Valley story, was acquired by Cisco. There's an image of little fish getting eaten by bigger fish here. But a couple years ago — and we can get into the story later — my mom was the target of a fake kidnapping scam that was powered by AI. And that sent me down a rabbit hole of looking at the huge challenge that is facing consumers today coming from AI- powered fraud. And the data is off the charts. Americans are losing close to $200 billion a year, and it's being driven largely by imposter scams. And it's hitting all demographics. It's hitting senior citizens. It's hitting young people on gaming platforms and social media. It's hitting the sandwich generation, our spouses, our partners, our parents, our kids. And so, you know, I recognized that we need a whole new level of innovation to protect families from scams and fraud in this AI era. And so I partnered with my brother, in the long list of questionable decisions. We're making this a real family affair. My brother is an amazing product entrepreneur. He's been at places like Apple and Spotify. And so we teamed up, my cybersecurity knowledge and his consumer product experience, and that's what started Savi about 10 months ago. We've got a great team, and we're bringing products to consumers every day to help protect them.

Ryo Koyama Well, fantastic. Plus, at least this way, if your brother calls you and he's sitting next to you, then it's actually —

Patrick Coughlin That's right. Exactly.

Ryo Koyama We sadly have some firsthand experience in the whole voice scamming kind of thing. So, Mike, why don't you tell our little story?

[laughter]

Mike Johnson Yeah. So, we had a new employee, and we don't go in the office often, but once a week we go to our little office in Petaluma. And, you know, I roll into the office and I go, "Where's Stuart? He's supposed to be here." And he goes, "Whoa, Ryo called him up and put him on a task." And I go, "A task? I don't know anything about it." So I started asking questions, and then Stuart calls and asks some weird question, and then we're discussing what's going on, and I guess what had happened is he said he was doing a task for Ryo. Well, someone with Ryo's voice, probably cloned off of one of the podcasts, called Stuart up and dropped some stuff about people that might work for Remote.it, which is our company, and said that he had an important task for him to do. And so we didn't really know what was going on, and then finally he's texting one of the other employees about some stuff, and everything is not right. So we were all like, "Hey, let's figure out what's going on." And so I actually texted Ryo. I go, "What do you have Stuart doing?" And he goes, "Nothing." So I go, "Holy — it's a scam." And I'm like, call him up. I call him up. I'm like, ring him, ring him, ring him until he answers. I go, "Stuart, whatever you're doing, it's a scam."

Patrick Coughlin Was he at a Bitcoin ATM or —

Mike Johnson No, he had actually bought four Apple cards, man.

Patrick Coughlin Of course. Totally a red flag, right? But it happens all the time.

Mike Johnson Yeah. Yeah. But we actually stopped him just in time, because the guy was pressuring him to scrape the numbers off and give him the PINs, right?

Patrick Coughlin Wow. So you stopped the scam.

Mike Johnson We stopped the scam.

Patrick Coughlin So that puts you in the minority, I would say. And you're talking about it, which is something people don't often do either. I mean, 85% of victims don't ever even report or talk about or share with anybody what actually happened to them. But just think about how far we've come. There was a moment, it doesn't feel like that long ago, when scams in the internet era were like a random email from a Nigerian prince claiming to have some fortune that he was willing to share.

Ryo Koyama Wait, wait, that's not true. I just sent him some bank information.

Patrick Coughlin Or there was phishing and business email compromise, and masquerading as the CEO on email, and maybe trying to spoof an email address or make it look like somebody that you worked with. And now we're talking about a perfect clone of your voice that rang the phone number of an employee and asked them to do something. I mean, that's a huge leap, and I think it's indicative of where this threat is going, how far it's already come, and ultimately where it's going to take us.

Ryo Koyama So, Patrick, why don't we take a second here. Dark Side of the Boom, the book. You wrote that and it covers a lot of this. I think a lot of people would be very interested in understanding what that is, because there's probably a lot of information. Like you said, we're not so brave as to sit there and face the embarrassment of telling people we almost got scammed. But I think it's important for people to have that narrative. So why don't you talk a little bit about the book and what it covers, for people who might be interested in downloading a copy?

Patrick Coughlin That's exactly it. And so, after our mom got hit with this scam, I basically went down the rabbit hole and I was kind of Beautiful Mind-ing my office, putting up: who are these groups, what are the tactics and techniques that they're using? Trying to understand what had fundamentally changed in the underlying economy of these scams, that now all of a sudden little old Beth Coughlin from Prairie Village, Kansas, would be a target of a sophisticated social engineering scam, the likes of which I used to see pointed at juicy national security targets or big Fortune 500 enterprises. But something had changed, that it was now being targeted at an everyday consumer. And who else was being targeted by this? And how big was this problem? And what I found was exactly what the data bears out, which is digital fraud is now the fastest growing crime in this country. It is the largest economic crime in this country. It is the most under-reported economic crime in this country. And it's being turbocharged by artificial intelligence. And so, in a past life when I worked for the government, maybe this kind of research would have culminated in some sort of briefing in a windowless conference room in DC, to people with the authority, hopefully, to act on it. But my clearance expired more than a decade ago. So my choices were, what am I going to do with all this? And ultimately I started a company and figured I'd publish all this stuff in a book. And I think it's an interesting way to come over the shoulder of these scams as they're happening and really understand the tactics that are being used. So we try to bring readers into a series of different imposter scams, the ones that are really wreaking havoc across consumers and families today. And those are things like what you just experienced. We call those workplace imposter scams — somebody pretending to be your CEO or a colleague or somebody from the finance department. But there are others, like the one we experienced in my family. We call those family emergency scams. Somebody's been kidnapped. There's been an accident. You need to wire money. Somebody got arrested. There's all kinds of versions of this. Government imposter scams are a big one. Oh, this is the IRS. Or you didn't show up for jury duty and there's a huge fine. So there's a series of these imposter scams that are driving the majority of the growth in digital fraud today. These imposter scams now make up over a third of all digital fraud in this country. And if you think about why — well, what is AI really good at? I mean, the promise of AI is that it can speak the language of humanity, right? It can talk like us. If you play with these tools, you can chat with them and they seem to engage with us like a human. They can sound like us. They can create visuals that are impossible for us to detect as real or not with the naked eye. And those are really powerful tools for productivity for humans, but they are an industrial-grade weapon in the hands of somebody who wants to deceive us, right? And so I was seeing all of this debate that is still going on, by the way, this week: will AI save us or destroy us or disappoint us? And there's all this sort of prophesying about the future and what it's going to look like. And what I was seeing in the data and what I was experiencing is, well, hold on, there's something real that is happening right now that we don't have to guess at, and it's visible in the data, and everybody has a story. I mean, you all have a story right here. I've got a story about my mom. Everybody in this country now has a story, because tens of millions of Americans are impacted by this every year. And so I wanted to write a book that demystified what is going on here, that helps everyday people understand what these scams look like, and do it hopefully in an entertaining and engaging way that brings you into the narrative of the victims, and then also looks at the real challenges, which is: how are these scams happening? Who's behind them? Why is it so hard to stop them? And then not just be all pessimistic, but hopefully make some recommendations about how we as a society can make some improvements to start to turn the tide against the scammers.

Ryo Koyama I think it's very important that you touch on it. And it's interesting, because I was at a big open-source event recently and the discussion was about AI, and one of the things I tried to point out — I was talking about general human intelligence — and I pointed out, I go, well, throughout history almost all the great leaders have essentially been liars or manipulative. [laughter] And you emphasize that point, because that is something that AI is good at. Mike and I were having a discussion not too long ago where we specifically talked about how social media is already the best propaganda machine in history, and combined with AI, it's probably the best war machine in history. And to your point, it's a criminal activity. Now, one of the things I think is interesting in your book is that you touch on the psychology of how these scams happen. But I'm kind of curious, because you brought up something else: it's very, very under-reported. What do you think is the psychology about why it doesn't get reported? For example, in the past, as an Asian living here, a lot of times Asians have been targeted in certain communities because they tend to not report things to the police. So what do you think is behind the reason a lot of these crimes don't get reported?

Patrick Coughlin It all comes down to shame. I'll give you an analogy. If your house gets broken into and something's been stolen from your house, and the cops show up, and maybe your neighbors come out and they say, "Oh my god, I'm so sorry that happened to you. How can we help?" If you've been scammed, often what happens, unfortunately, is somebody says, "How the hell, Stuart, could you have fallen for that?" And so there's this failure to give credit and blame to the actual perpetrators of the crime. And we look at this particular type of crime as something that we did wrong. We got conned. Man, I fell for that. We got duped. Instead of: they stole. And so that shame that surrounds these incidents keeps people from talking about them. I mean, there's crazy data. Something like half of people that have ever been scammed don't actually tell another living soul about it.

Ryo Koyama Wow. That's sad and horrifying. I always thought that one of the failures of social media is — we all grew up, maybe not you, because I think you're younger than me —

Patrick Coughlin I've got some gray there, man.

Ryo Koyama But we all know the Constitution because of Schoolhouse Rock, and we can all sing the Constitution. And I always felt like 15% of social media has to be public service announcements, and it's not. And I think this would be a wonderful thing, if every fifth thing that I saw on TikTok was, this is the scam that's going around. It normalizes it and it creates an awareness. I think that would be pretty critical. And so I'm a big believer that as this AI boom is hitting, something bad is going to happen, and it's going to probably happen pretty soon. So, you're in an interesting position where I think you have a front row seat. What do you think are the big things that could happen that could be somewhat catastrophic out there?

Patrick Coughlin So, I'm not sure when this is coming out, but all the rage in the cybersecurity community right now is this bit of a "me too" game going on across the foundation AI model companies saying, "Ooh, our AI model just went rogue and attacked another company." And first it started with OpenAI saying it, then Anthropic the next week came out, "Oh, so did ours." And this week Meta came out and said, "Oh, ours did too." So there's this concern around enterprise security and autonomous models that have these incredible capabilities in coding assistance. But if you're really good at coding, you know what else you're really good at? Identifying vulnerabilities in code. And so it's not a huge surprise that these models are incredibly efficient and effective at identifying and exploiting vulnerabilities in code. And so I think we're worried about how these models can autonomously seek out and create disruption across anything that runs on code. And I think that's — by the way, that's most of our society, certainly all of our payment rails. Every company has software now, our data, everything like that. And so I think that is a huge concern, that we'll have these autonomous cyber models that don't even necessarily need to be in the hands of bad guys to do bad things. What I'm actually more concerned about is the consumer fraud version of that. Because enterprises, government agencies, which I've worked with for decades now, are used to playing this cat-and-mouse game with cyber attackers, right? You know, they tool up, they create something, they exploit some vector, and then the enterprise responds to it, and you have this cat-and-mouse game, and there's budgets, and there's tooling, and there's talent associated with that. And when I started my career, companies like McAfee and Norton — these are great companies, but they were the icons of the early internet. They were everywhere. They were like my AAA card that gave me confidence that I could go out on the information superhighway and explore, regardless of how good their products were. They were there. I think what happened over the last 20 years is, as enterprise and government started to digitally transform, the market for enterprise security took off — the Palo Alto Networks and the CrowdStrikes — and so a lot of the investment and innovation and talent flowed to those problems, and it wasn't sexy to work in consumer cybersecurity for most of my career, because that was like eat your cyber vegetables: have strong passwords, turn on multi-factor authentication, like that stops everything. But what we're seeing now, and what's reflected in what you all experienced and what my mom experienced and what tens of millions of people are experiencing, is there's a sophistication coming at consumers that is being powered by AI that I think is going to facilitate the greatest transfer of wealth from this country to potentially nefarious criminal organizations aligned with our national security enemies. And that, I think, is what needs to be elevated in this conversation, because that impacts everyday people and it impacts our perceptions of AI. Like, we in Silicon Valley and Silicon Valley adjacent, we look at AI and we're obsessed with it and we are excited about it and we see the potential and the opportunity. But if you go to where I'm from, in Kansas City, most people's experience with AI is in the form of their phones blowing up with more scams now. And so if we want to get the country behind this technology wave that we all believe is transformative and important, we've got to do a way better job of managing the downside, or the dark side, and addressing it in real time rather than sitting in these ivory towers and prophesying about what it might be in the future.

Mike Johnson So this might be in your book, and it probably is. But we had to do a little decompression after our incident and go over what's good and what's bad, right? So we came up with our little company ISO prep and family ISO prep stuff, so we could have these passwords and how you do things and call-backs. What should the regular consumer do? What can they do? I'm sure this is in your book, right?

Patrick Coughlin It is. We talk about — but I would just say, I think you nailed it. One of the first things we do is say, create a family password, just to get you talking about it, and then talk about scenarios where you might use it. We talk about these new rituals that we need in the family, which maybe it's me leading by example and telling my kids and my wife, look, I'm never going to give money to a random stranger online. And I think that should be a family rule. If we haven't met them in person, we don't send money unless we talk about it. So a lot of this — and Ryo, you brought this up — a lot of this is talking about these things in the open, making these kitchen table and living room issues so that we feel comfortable that when something doesn't look right, we're willing to get a second opinion, instead of retracting or feeling shameful that we don't know. So start by talking about it. I think the second thing is there's a little more responsibility — and you hinted at this, Ryo, with TikTok. We need to demand a little bit more responsibility from these platform players in terms of how they show up in our society. If you look at Meta — and again, I'm just sharing facts — there was an incredible story that Reuters broke last year around the amount of fraud that is trafficked through Meta's platform.

Ryo Koyama Oh, even just on the marketplace. I mean, without a doubt, that's — no, I'm just going to back up real quick, just so that everybody's clear, because you both talked about family passwords, but what you really mean is a safe word. And so I want to make sure no one walks away and says we should all have the same password as a family. What you're saying is, if I'm talking to some family member, I want to say something like, if I don't say "kangaroo" or something, then it's not me.

Patrick Coughlin Yes. Yeah. And we say pick something that means something to you all as a family. It could be a color, it could be a phrase, it could be a place you used to live. And encourage people to use it. I tell my mom, if she asks me for the family password, or she hangs up on me because she's not sure it's me, we should celebrate that moment. We shouldn't be mad about that, which is kind of hard to do. But as important — the safe word is a mechanism to begin the conversation. It's as important as picking the right safe word, or whatever it is, is getting everybody together, because now you're talking about why. And that's just as important as anything else.

Ryo Koyama No, no. I think that's such an important thing, because like you said, we have to normalize some of this stuff, but the most important thing is normalizing the conversation.

Patrick Coughlin Yes. Yes.

Ryo Koyama And I think all crime is this way. It appeals to psychology. And so you have to overcome that psychology. And one of the most powerful emotions is embarrassment, because it is the thing that's likely to let people get away with stuff.

Patrick Coughlin It's exactly it. And it isolates people, which is a scammer's dream, because then they're not communicating. And in fact, there's insane data about people who've been scammed once who then get scammed again, because they are more likely to be isolated. But I will just do a soft plug here. One of the reasons why we started Savi is because we saw this huge gap in the tool landscape that was available to everyday consumers. So we created a free scam checker that requires no email signup. It's available at scamwise.com. You could forward suspicious emails to it. You can ask it a question. You could send a screenshot of a text. You could take a picture of physical mail. We get a lot of somebody who gets a Medicare bill who's like, wait a minute, I don't know if this is real or not. And it will give you a quick second opinion and recommend ways that you could verify it. So we're creating some products and trying to use AI to help make it easier to talk about these issues, to get second opinions, and ultimately, where we're going with Savi and with our mobile apps, to block these things before they actually even get to you.

Ryo Koyama I have an interesting question for you, because I was thinking about who's going to cover the cost of these expenses, because something's going to happen. Do you think it'll almost get to the point where it's a national disaster, where we have a hurricane or something hits and a disaster relief fund comes in, but it could be at that scale? I mean, you're talking about something that could be pretty broad-based, and it's not like it's going to be the credit card companies, which are really insurance companies. It's going to be something broader than that. So what do you think is going to be the remedy? Because clearly something bad could happen.

Patrick Coughlin I think there's two options. I think the national disaster analogy is unfortunately a likely one, and often in any country, and in this country, it takes something of that scale to mobilize public opinion and legislative action around doing the right things. So I could see autonomous or rogue models that create a massive fraud event across banks or wallets and drain them, creating this huge issue of economic loss that the government has to step in on, almost in a too-big-to-fail way, but at a very huge consumer distribution level. But I could also see a slower burn. And I think this is what concerns me a little bit, and part of the point in the book is, for a couple generations now, we kind of let social media run with our kids, and here we are now, and there's a series of lawsuits and action happening, starting to come to terms with that, and there's all this news about the Meta lawsuits on social media with kids and all that. My hope is that we don't burn a couple generations of loss and pain when there are real things that we can actually start fixing incrementally now, without hamstringing innovation. But we just need to focus on them and talk about them. And so that's what I think we're on the path for right now: more of the social media outcome, where we spent 20 years growing gangbusters and there's this seedy underbelly of byproduct and dark side that is creating real havoc, but we don't address it until far later down the road, and until far too many people have already been impacted.

Ryo Koyama Do you think regulation is going to have to happen? For example, you talk about how many of the attacks come from foreign agencies, foreign countries. Do you think a situation might arise where the United States effectively has the equivalent of the Great Firewall of China, where it's like, okay, we're just not going to allow a lot of things to come in? Because one of the things that obviously allowed the internet to become so great is that it was sort of untethered, but it sounds like untethered may be exactly the wrong recipe in this case.

Patrick Coughlin Yeah. You know, the R word is a difficult one sometimes. I find it hard to believe that if this is a technology as transformative as we all believe it is, then why wouldn't it require at least incremental regulatory changes? And so my position is yes, but it doesn't necessarily mean that there aren't nuanced ways to do that, or incremental ways that we can start doing that. And you're right about the international nature, and this is what concerns me a little bit. Like, right now there are 1,500 bills moving through state legislatures to try to stop scams and fraud. The state has no jurisdiction over how these things are executed. If you know anything about the global telecommunications infrastructure or VoIP providers and how these things work, a state law doesn't matter. And in fact, it only creates a patchwork of regulation, and then makes it easier for scammers to maneuver through them, because they're very good at that. So I do think we need national and international cooperation on what is an international problem. But it could start with things like KYC, know your customer, a little bit. It's harder for a consumer in this country to buy Sudafed — the real Sudafed, the good stuff — than it is to have unlimited API access to a model that its own creators claim is capable of creating anything from bioweapons to all these terrifying things. So we can do better. We can start small. We can collaborate internationally. And I think we just have to agree that this is a problem that matters first.

Mike Johnson Yeah. I personally think the regulation or the laws, it's not going to work, because that's a global thing, right? And you're basically handing the keys off to your competitors when you lock down your stuff. What I do like about what you're talking about is that it's education. Everybody — the more you can talk about it, the more you can get the information out there, the more you can open up people's eyes to it, to me, is the solution.

Patrick Coughlin Yeah.

Mike Johnson Right. Because I don't want to regulate my industry or somebody else's industry and stifle it till we lose, right?

Patrick Coughlin No. And that's competition. So I agree with that, but — this will be fun, we can disagree a little bit, maybe that will help with viewer engagement, to be controversial and polarizing. So I will agree with you in one breath and say, yes, education. If you think about why that Nigerian prince scam really died, it's because it became a joke, a meme. It was like all the late night comedians were using it, and everybody just got it, and that's why it stopped working. Yeah, Microsoft did some things on their email exchange that helped, but the reality is everybody learned about it. I think the concern now is that this is evolving very quickly. So education will be part of the solution, but it can't be the whole solution. I think there are other ways to approach regulation. So if we took an innovation mindset: okay, well, the regulation could be, hey, social media, open your platforms up to allow consumers to police the content better that their kids see. Or device makers, make it easier for third party companies to filter out bad things. This technology exists, but it struggles to innovate within the walled gardens, where the scammers currently have free rein but security companies or others who may want to try to intercept it don't. And so regulation doesn't have to be stop our AI, let other people's AI flourish. It could be a little bit more nuanced, so that we find the right protective balance, so that at least we can play this cat-and-mouse game, which right now I think we're the mouse, and I think we're getting our butts handed to us.

Mike Johnson Well, I think you're right. That would actually be probably a very workable solution. I just worry about companies. I don't want to say Facebook, but they're making so much money off the scammers too.

Patrick Coughlin That's exactly right. And so you can't ask them. I have some of these conversations and people get angry. They're like, well, Meta is making too much money off this fraud. And I'm like, well, give them a business reason not to. At the end of the day, we've all been in business, they have a responsibility to their shareholders, and there is no moral responsibility to the shareholders. As much as we all believe we would act differently and do these things, we have to create some business reasons that'll either allow innovative solutions to help deliver protection, or make it more painful for them when they deliver crime. [laughter] Those are the only two options.

Ryo Koyama Well, like I said, it's very interesting, because we have a whole discussion just about civil liberties and who you are and do you really have to authenticate yourself. But at the same time, it may be fundamentally important to be able to do that, at a minimum, so that if someone calls you in your family, you know it's them. All right. So, one of the things we love to close with is your hot take. So what's the non-conventional thing that you think might be happening in this whole space that isn't conventional wisdom, that people aren't thinking? How do you think the world might shake up in a different way than people are thinking right now?

Patrick Coughlin I'll give you two answers to that. If you can't tell from some of my comments, I'm not really buying this rogue AI model narrative that's coming out of the foundation labs today. I think it's an incredible — I call it rogue stunting, where you frame the power of your AI model to free itself from containment as a benchmark, that like any other sort of F1 score, it should be part of the F1 score: did it get out? If you've ever been in a cybersecurity incident, one of the last things you do is call up a reporter from The Information or any media outlet and say, hey, I've got a story for you, our model got out, we had a huge cyber attack. It doesn't really work like that. And so I look at a lot of this as incredible marketing. And it doesn't necessarily mean that it's nefarious, it doesn't mean that these models aren't powerful, but I think there's a little bit of playing into a narrative that is dangerous. I think playing into the narrative is dangerous because, again, it focuses the conversation on a bad thing that for them is really a good thing, but it kind of prevents the conversation about the actual bad things from being elevated and talked about. So that's my hot take. I would say my future hot take is the world you all live in. If you think about what's happening with robotics, what's happening on shop floors of large manufacturers, if you look at the entire IoT space and where that is going with the advancement in these technologies — every big advancement of technology comes with risk. It happened with the internet, new kinds of risk, like virtualization, the move to cloud. It created all these new surface areas that need to be protected. Mobile, social media. We just talked about these big things. I think we have just barely seen what's going to be happening with AI plus robotics in our lives, and what that does to the attack surface area of enterprises, governments, and everyday people. And I'm excited about that, not in a freaky way, but in a — I think there's going to be a whole new wave of transformation and innovation that's required, and I think that the world that you all work in is super important.

Ryo Koyama Well, no, I think both of those things make a tremendous amount of sense. And I think the difficult reality is that every time there's an evolution, or really a reinvention like is happening today, there are casualties. Like you talked about in the early days of the internet, lots of casualties, there was a lot of identity theft. I had identity theft multiple times. And we're facing a situation where we're definitely going to run into that. Well, I think this is fascinating. I think we could probably have this conversation a lot, and we probably are going to have to have it a lot — although we'll have to create a safe word, because I'm still not sure that I'm not actually talking to a digital twin right now. But we'll have to figure that out. And thank you so much for being on the episode. I'm pretty sure we're going to bug you to come on again, because next time some sort of hot button comes on. So thanks a lot for being on the show.

Patrick Coughlin Thank you so much for having me.