Protecting Your Business from Cyber Threats in 2019 and Beyond
Webinar Transcript
This is a transcript of a webinar recorded on March 28, 2019. View recorded webinar here.
Tyler Snyder: Alright good morning everyone, thank you all very much for making time to join us today. My name is Tyler Snyder, I'm the Vice President of Operations for Innovative Incorporated. We are a strategic service provider located in Hagerstown, MD.
We decided to hold this educational webinar just to help end users get a better grasp on some very real threats that are targeting businesses today. We see it all the time and felt that education is one of the best strategies to combat these issues.
The topic of cybersecurity and specifically ransomware comes up a lot in our strategy discussions with businesses. It is an extremely frustrating problem, but one that every single employee in your business needs to be aware of. We all need to be operating within a safe and secure environment so it's really important to understand what solutions and strategies exist to help keep data safe.
Cybersecurity is a massive topic, but the goal of today's session is to make everyone aware of some key cyber strategies and ultimately understand how you can reduce downtime in your business.
So, with that being said, I am very pleased to introduce you to Mike DePalma, who will be our presenter today, and with that, I'm going to kick it over to Mike.
Mike DePalma: Thank you very much, Tyler. So, my name is Mike DePalma, Channel Development Manager at Datto. Very quick just brief overview of Datto and who we are. We are a backup, disaster recovery company. We're based out of Norwalk, CT, so about 40 miles east of Manhattan.
Founded in 2007, for the entrepreneurs on the call it's a pretty cool story. Our founder started the company when he was 22. Started building these backup devices by hand in his parents' basement and selling them online. We fast forward 8 years to 2015 and the Wall Street Journal gave us a billion-dollar valuation, so we're actually the only unicorn company in the state of Connecticut. We're up over 1,500 employees worldwide, we have 22 offices, and we protect over 460 petabytes of data in our 9 data centers.
We have 2 data centers here in the States. We've got one in Reading, PA and we've got one out in Salt Lake City, UT, and I'll get to the importance of those locations in a little bit.
I do a lot of presentations really all over North America and you'll be surprised when I'm up in Canada the first question I always get asked is, “are you sending our data down to the States?”
And I tell them, "no."
For some reason, they're not too trustworthy of us or our government looking at their stuff. So, we do have two data centers up there. The idea is to keep the data within the country it originates in, so any US based company here is going to have their data kept in the States.
One of the cool things about Datto is really the team that I'm on and the reason our CEO created us. I'm not on the sales team at all, none of my compensation is based on any of that. He wanted to create a team that just went around and did presentations, and sat on panels, and really learned real-life stories. So, I get the opportunity really to travel all around, and I've done presentations alongside folks from the FBI and from the Department of Homeland Security.
So, I'm going to be telling a lot of stories throughout this webinar and basically, I stole them from some of these guys who have seen it in the real world.
Before we get started I want to get a quick sense of who the audience is here. So, I want to just run a quick poll or two just to see if I need to get into more technology-based stuff or, you know if you want more of high-level, story-level stuff.
So, the poll is now open now. Are you in the IT space or is your role IT-based? So, if anyone could answer that, that would be fantastic. That will give me kind of a sense of where we should take some of these conversations.
Looks like it's a lot of folks not in the IT space, which is great. Looks like it's kind of split.
So, we just want to do one more poll here, just to kind of get a sense of what your role is within the organization. Are you the owner, president of the company, are you C-level status, are you an individual contributor just looking to learn some more? Really, regardless of what your current title is a lot of what we're trying to do on these educational webinars is to give you some data that you can bring back to the rest of the team.
It's great for you individually to get up to speed with some of these threats and ways to protect yourself, but it's just as important for you to go back and let in everyone in your office because everyone really should have this culture of security based around them. So, we'll get that launched.
So yeah, we'll get right into some of these threats that are out there, and we'll get into who's behind them, and really, ways that we're seeing folks, you know, really combat some of these things.
So, with that, I'm going to give it a few more seconds and then we're going to get started.
Alright, perfect.
So there are a lot of things that these criminals are trying to exploit and unfortunately, some of the best and the brightest minds in the IT space are moving over to the dark side because there's so much money to be made there.
And obviously it's a multi-layered approach you want to do as much as you can through education, your people are your greatest weakness, you want to make sure you have intrusion protection and your firewalls and all of that up to date so that you're protecting a lot of these things from getting through, but ultimately your last line of defense is going to be your backup and the criminals are very well aware of this.
And they know for the most part that most businesses are backing up their data in some form or fashion, you're not going to find too many companies that have this big database of data and they're not backing it up. The problem is a lot of these folks are just using traditional forms of backup, you know technology has evolved obviously greatly over the last decade or so, but a lot of folks are still using the same backup solutions they were 10 years ago, maybe 20 years ago. And the issues with these traditional forms of back up, the kind of commoditized forms of backup are first and foremost the higher failure rates.
You know I mentioned I get to go all around the country [and] I'm surprised to hear how many folks are still using tape backup despite studies that show the higher failure rates of those. Same goes for a lot of the traditional forms of backup: the external hard drives; USB; even just cloud backups. Sending your data offsite, which is great having that second copy of your data in a different geographical location, but there’s still some limitations: manual testing; who’s taking those backups.
For the most part, when I ask those questions to business owners, they can answer that question. How often are you taking your backups? Who’s taking them? Who’s kind of responsible for them?
But when I ask how often are you testing these backups and is that an automated process, or is somebody responsible for it, you start to see a little bit of finger-pointing.
“Well, um, I think Jim down the hall does that.”
“I don’t know how often he does it.”
“I’ve never seen any reports on that.”
And criminals love this. They know OK, you’re going to go restore from your backup. I’ve got your data. Alright, I’m going to rely on the fact that: A, it might have failed, and B, nobody knows that it failed until a disaster strikes or an attack has occurred.
Same goes for just the risk of infection or theft. A lot of folks just have one copy of their data, and it’s either sitting on-premise or their in-house IT guy drives it down to their home every weekend.
Obviously, you’re running a lot of risk here. Even if it’s encrypted on-site if that data is compromised and is now stolen there’s nothing you’re going to do to get that data back. So, it’s important to have multiple copies of your data.
But the biggest piece of leverage that these criminals have is this slow recovery time. Again, most people could tell me how often they’re backing up their data, but when you start to ask how long is it going to take you to get back up and running into a true production environment, a lot of folks can’t answer that question.
And that is where the criminals really have you. They’ve got your data, you want to go get it back, OK, let’s see how long it’s going to take you.
You’ve got cloud backup, fantastic, go try and bring 4-5 terabytes of data down from the cloud and give me a call when all of that data’s down and you’re back up and running.
I mentioned I go around, and I tell a lot of stories and I listen to a lot of stories. I’ve been giving a lot of ransomware-specific studies for years now, probably 3-4 years I’ve been doing presentations and panels about ransomware. You know probably 3-4 years ago people would be listening but kind of looking at their phones. You know ransomware, I don’t’ really know what it is. As the years have gone on we’ve seen more and more head nods, whether that company has been attacked or has a client or vendor who’s been attacked or a friend, and now people start to understand a little bit of what that threat looks like.
And then, back in May of 2017, we saw WannaCry, which now made the front page of every paper, the lead story on every website. This was a single attack by a single virus that impacted 220,000 networks worldwide in a single day. 150 countries were impacted. And what was unique about this, beyond the scale, is that they only asked for $300 across the board, which is different for them because normally these criminals try to do some research about the company they’re attacking. So, if they’re attacking a Fortune 500 Company, they’re going to ask for 6 or 7 figures. If they’re attacking a small business or a single man shop, maybe they’ll ask for $800. They want to make sure they can get as much money as possible without overpricing.
It this was $300 across the board. So, in the IT space, what the consensus really was that this was just a test. This was phase 1, they wanted to see how many computers they could attack in a single day. Phase 2 was going to be, OK now we’re going to figure out how to monetize it because they only netted about $300,000 worldwide, which is really a small about.
About a billion dollars changed hands in 2018 from the business community over to these criminals with ransomware alone so $300,000 wasn’t a lot of money.
Now I was doing a presentation down in Allentown, PA about two weeks after this attack and the keynote was a gentleman from the FBI head of the cybersecurity unit in Philadelphia. So, I do my presentation and I end with WannaCry because it was the hot button topic of the day and you know I told them the theory, right, this was a test, we’ve all got to prepare ourselves.
Now the FBI agent gets up there and he starts telling some really scary stories: cyberwarfare; hacktivism; all of these things, but he also finishes with ransomware. He says, OK, we’ve got two theories, one is exactly what Mike just said, phase 1. So, we’re going out, we’re trying to figure out what vulnerabilities were exposed, we’re trying to educate as much as possible, that’s why we’re out here right now in the field.
We have a second theory, which at the time when I listened to it, I thought was pretty far-fetched, but as the months went on I think there’s some real validity to it. And the second theory was that their entire goal was to make this worldwide news. [It] was to make everybody see this, that this attack can happen because what they wanted to do was actually do one thing, which was to increase the cost of bitcoin.
Remember this is May of 2017, we all got bitcoined to death towards the end of 2017. But the reason why this makes a lot of sense was that these attacks had been occurring for years, and 99% of them ask for money in bitcoin because it’s anonymous cryptocurrency that they know can’t get tracked back to them. So, they’re not going to ask for your credit card. They’re not going to ask you to wire some money, because now their risk of getting caught increases. So, they’ve been collecting these bitcoins in the millions of dollars per year when a bitcoin was $50, $100, and you can see on this chart that it opened up in 2016 as just over $440 dollars, 2017 just under $1,000. You see that first spike, and when’s that first spike, that was May-June of 2017, because what was happening was you had companies all over the world, a lot of large companies gobbling up bitcoin, because they were preparing for this second phase that everyone was telling them it was coming and they wanted to make sure it obviously didn’t hit them.
But what if it did?
What is [the] worst case scenario?
You can’t just go into a Wells Fargo and say I want to fund $100,000 in bitcoin. It doesn’t work that way, you’ve got to set up and account, you’ve got to find an exchange and you have to fund that.
Now at Datto, there’s four of us on my team. The finance department all set up bitcoin accounts for us so that we could talk intelligently about it. I talk about bitcoin all the time. And so, we’re [Datto] going to fund $100. I get the email, the $100 is on its way. It took 6 days for that money to actually hit that account and by the time it got there, it was only worth $87.
And so these companies were gobbling up bitcoin and just like anything else the markets followed it and we saw that crazy bonanza in Q4 of 2017. And it [bitcoin] actually opened up at $1,500 for a single bitcoin in 2018. So you saw the stories of the 20-year-old kid who bought some bitcoin a few years back and became a millionaire or the savvy investor, and they’re all feel good stories, but the folks who were making hundreds of millions of dollars during that crazy Q4 bonanza were these criminals. Criminals that had been collecting bitcoin for the last 5 or 6 years when a bitcoin was under $100.
So, whether that theory that the FBI gave was true or false, the bottom line is that these criminals did make hundreds of millions of dollars. And you saw this drop and now its around $3,000 and the theory is, well because the criminals owned most of these bitcoins they sold off and made most of their profits. And so this is a scary proposal and its just one way that ransomware is really impacting the business community.
I mentioned that billion-dollar number, that’s the actual number that’s estimated by the FBI that actually changed hands. The impact in North America alone was around 75 billion dollars last year and what makes up that 75 billion dollars, well that includes the cost of data loss and the cost of downtime.
Now you could sit through a webinar like this and get just hammered with all these acronyms but I’m going to leave you two, and these are two that hopefully by the end of the call you can go, and you could answer these questions about your business, and that’s your RPO and your RTO. Criminals are very well aware of these two acronyms.
Your RPO over on the left-hand side of your screen is your Recovery Point Objective. How much data are you at risk for losing at any given time, simply put how often are you taking your backups.
Are you taking daily backups?
OK, are you OK with losing a days’ worth of data?
And you very well might be able to. Every company on this webinar is going to be different, different industry, different shape and size, and all the rest. For the most part, folks feel pretty confident about answering that question.
Now the caveat is that if you’re not testing those backups there’s no way you can answer that question truthfully because you don’t really know how often those backups are truly being taken and that data is really valid, but for the most part companies feel pretty comfortable about that, and criminals know that.
With ransomware they’re just encrypting your data, they’re not stealing it. And so, they know that OK if you’re not backing up your data that they’ve got you, you’ve got to pay, but for the most part they feel like OK companies are doing that, and for the most part, traditional backups are working and all the rest.
The righthand side of the screen is the piece of leverage that they have. And I mentioned slow recovery time, and this is when this comes into play. Your RTO is your Recovery Time Objective. How much downtime would it incur if your systems were to go down?
Whether it’s an attack, whether it’s a server failure, whether it’s a fire or flood, how long would it take your business to get back up and running?
Have you ever done one of those disaster tests?
Is it going to be one day, one week, and is that aggressive enough for you?
Criminals know that for the most part companies can’t answer that question so what they try to do is they try to price that ransom payment at a more attractive price than the cost of downtime.
They might say pay me $5,000, the other option is to go restore and get back up and running. You might be down a week.
How much is that week of downtime going to cost you?
Most of the time, far greater than the $5,000 they’re asking for, and that’s why people pay.
And the sophistication of these guys is pretty crazy. It’s not the guy in his mother’s basement anymore doing these attacks because he doesn’t want to get a real job, or he wants to go brag to his friends on the dark web about hey did you read that story, that was me.
This is what organized crime has shifted to. Here in the States, Eastern Europe, China, Russia, all around the world, North Korea this is where organized crime has shifted to because there’s so much money to be made with so little risk of getting caught. And I mentioned that the best and the brightest are moving to this. It’s pretty scary stuff.
I don’t know if you saw the story last year but actually the greatest casino heist of all time was perpetrated last year, and it wasn’t the kind of heist that you’d see in the movies, right the, “I’m going to sneak through the HVAC duct and get down,” or, “I’m going to have a costume and get through.”
That’s not how crime is done anymore. What these criminals were able to do is that they were able to find a vulnerability, and that vulnerability was in the internet-connected fish tank thermometer. We’ve all heard the phrase internet of things, everything’s connected, well these guys found a vulnerability and through that internet-connected fish tank thermometer, they were able to hack into this system and steal all of the information for all of the high rollers for this particular casino.
That information can be bought and sold on the dark web for hundreds of millions of dollars. If you ever go to the dark web, and hopefully you’ve never been there, but if you go down there it’s like an exchange.
You can go on there and say I have a 55-year-old man. He’s got 7 million dollars in his bank account. Here’s his social security number. Here’s the company he works for. I’ll sell this to you for 20 grand.
And that person who did the attack, they wipe their hands clean and they move on to the next. And then it’s another phase of OK, I bought this information for twenty thousand dollars, what am I going to do with it?
It’s really a corporate-type structure in that dark web and it’s pretty scary stuff.
Now another presentation that I gave with a gentleman from the FBI was in Manhattan and he’s giving this scary presentation, again cyber warfare, all these things, and he shows a picture of a house.
“Does anybody recognize this house?”
And there’s about 200 people in this room, and everybody kind of looks around at each other for a second, and a guy in the back of the room says, “that’s my house.”
And [the presenter] goes, ”you have a beautiful house sir, and you just got a promotion at work a few weeks ago, that’s fantastic,” and he starts rattling off all of these facts about this gentleman.
So now he’s got their attention.
And he says, “I bet you thought I used all the tools at my disposal to gather this information about this gentleman, right?”
“Nope, what I did was I went to the organizer of the event and I got a list of everyone. And I knew everyone’s name and their company, that’s all I had. What I did was I looked you up on LinkedIn, I picked one of you at random and I said, ‘OK I’m going to find them on LinkedIn,’ now I know what your face looks like so from there I can go and I can find you on Facebook, maybe Twitter, maybe Instagram, and within 10 minutes all the information that you readily give to me on the internet through all of these social media sites. I have a pretty good sense of who you are, so what I’m going to do, this particular gentleman he graduated from Penn State, that’s a great university, I also know it’s got one of the best alumni associations and one of the largest alumni associations in the world. You probably get emails from them all the time. So, I’m now going to create an email with the same template, the same look and feel as every other email you’ve gotten from them. I’m going to bury a hyperlink in there that says ‘click here’ to download all the information you need about your 20-year reunion festivities. And you’re going to click on it because it looks like every other email in your inbox. And if that doesn’t’ work I’m going to go down the list and I’m going to see who some of your coworkers are, who some of your vendors are, maybe a previous employer, and I’m going to create some emails like that. And one of these days it’s going to be a Friday afternoon, you’re going to be trying to clean up your inbox before you go home for the weekend, and I’m going to catch you asleep at the wheel. You’re going to click on the wrong link and I’m going to infect you.”
And by the way, the best time to infect you if you’re a criminal is to infect you on a Friday afternoon, because a lot of times you won’t even know you’ve been hit until Monday morning.
And that’s what we’re up against here. When you hear the term, socially engineered spearfishing attacks, this is what they mean. They gather some information about you, this email is going to look like every other email in your inbox, not the ones that come out from the Nigerian Prince asking you for some money or your long-lost Uncle. None of that stuff, nobody’s falling for that for the most part. This is what they’re doing.
So, when you see ransomware stories, unfortunately, the small business community feels like this is never going to happen to me because you’ve seen these large stories.
You might have seen the city of Atlanta was hit with a ransomware attack last year. Well, the reason I tell the story is because it shows that cost of downtime, those ripple effects, what makes up that 75 billion dollars.
Now this particular attack, they asked for $52,000. Which when you think about it and you take a step back for the City of Atlanta that’s not a crazy amount of money, but really the policy for the government is not to pay these guys. So, we don’t know whether they paid them or not, they say that they did not.
So, they decided OK, so we’ve got a backup solution, we’re going to get back up and running. Within a couple [of] weeks an article came out that stated from the city that they had already paid over 2.7 million dollars in additional fees and services and outside consultants and software upgrades that they needed to make. This was in the first two weeks.
So now that $52,000 ransomware ask is already impacted them to the tune of 2.7 million dollars, but it doesn’t stop there. They had to go back to the city for another 9.5 million dollars because they had their police department [that] couldn’t get access to their database and they had folks, you know, writing down handwritten citations and filing away in folders. And then after their systems got back up and running, somebody had to go and now upload all of that stuff.
So now the total impact from this $52,000 ransomware attack was over 12 million dollars. So, when you want to know where that 75 billion comes from, it’s from that gap right there on the screen.
And I know these are large numbers and they’re kind of hard to relate to your business, but I can tell you that the total IT budget for the City of Atlanta was 35 million dollars. So, a third of that budget went to this single attack.
So at Datto, we’ve done now three years running, we’ve done the largest ransomware report in the industry where we’ve interviewed two buckets of people, the MSP community (the managed services community), and the small business community. And what we found didn’t really surprise us, what we found was that every single vertical, every industry is susceptible to these attacks.
The reason being, I mean the genius behind these ransomware attacks is that they’re not stealing your data. You know 5-10 years ago the folks that were getting attacked were the businesses that had information that was really valuable on the dark web, the personal information, financial information. With ransomware they don’t care what your data is, they just know that regardless of the size of your business or the industry that you’re in, you can’t operate anymore without access to your data. That allows them to cast this very huge net and allows them to attack anyone from the one-man nonprofit all the way up to the municipalities or the hospitals.
And there’s really, I mentioned the corporate structure, there are two different ways about going about your business. You could be the homerun hitter and go after folks like a hospital where you might get 6 or 7 figures out of them, or you could be the single hitter who just tries to attack hundreds or thousands of small businesses in a single week knowing that not only are they most vulnerable, they can’t survive the cost of downtime. PricewaterhouseCoopers put out a study last year, and you can google all kinds of searches on this, their study showed that 71% of small businesses that were down for a week or more closed their doors within 12 months. And so criminals know that. That’s why so many people pay.
And you know one of the issues is that less than a quarter of these attacks in the small business community are being reported. Mostly because they know the FBI’s not going to investigate anything really under one hundred thousand dollars, probably closer to a quarter of a million dollars. They don’t have the resources to look at every 5-10-thousand-dollar ransomware attack, but beyond that small businesses don’t want this to make the news. They don’t want to be known as the company in their circle, in their industry that’s been attacked. Because they know their competitors are going to use that against them. So, they try to just handle it in-house, maybe they have an outsourced IT company that’s helping them, but they don’t really want to be out there letting folks know that I’ve been attacked.
And again, within that study what we saw was that the cost of downtime was 10 times higher. The average ransom payment from the small businesses that we heard about, $4,300. The actual impact, close to $50,000. Again, that math is always going to be there, and it’s scalable. Regardless of what you’re paying in ransom that cost of downtime far exceeds it.
So, what’s the government doing about it?
Well, they will investigate some of the larger ones. They want you to report it.
They’re really not catching a lot of these guys, or preventing them from getting in.
So, their next step is to educate. That’s why you will see a lot of FBI agents [and] folks in the Department of Homeland Security on the ground doing seminars, doing presentations, sitting on panels trying to educate the business community.
But ultimately, what does the government do best? They regulate. Right? They want to be able to go back to their constituents and say, “yes we’re addressing this problem through these regulations.”
And really, Europe is kind of at the forefront of this. If you have any companies that you work with over in Europe, you already have to abide by GDPR. It’s really the broadest and the most powerful real data protection plan that’s out there. And we’re seeing that sweep through the States as well, and here it’s mostly state-by-state and industry-by-industry, but really, I mean California already enacted basically what amounts to GDPR 2.0.
I mentioned we’re up here in Connecticut, Connecticut, New York, New Jersey, we get regulated to death, and so what are these regulations really doing? They’re not really helping to catch any of these guys, they’re really not helping to prevent any of these attacks from coming in, what they’re doing is they’re shifting that burden over to the small business community and saying it’s your job to do this.
Because not only do you have to worry about these attacks and the cost of downtime and the fees, and all the rest, but afterwards, OK, if you can’t show that you’ve done your due diligence to protect your data and your clients’ data, well there’s fines and all the rest coming your way as well. And that’s why we’re seeing a lot of businesses saying, “You know I can’t handle this in-house, I need to be working with an outsourced IT company to make sure that I’m compliant and protected and all the rest.”
So, let’s take a look lastly before we wrap up and open up to some Q and A, what are we talking about with that cost of downtime, how do you compete against that?
You know that right-hand side of the screen, how do you get an RTO that’s aggressive enough to really do what your company needs to do. Well, that’s where business continuity defines itself differently than traditional backup.
And here’s a couple of those differentiators. One is a hybrid approach. So, you have that on-premise backup which is easy for restores and virtualization, which I’ll get to in a minute, but you also have multiple copies of your data. And I mentioned our two data centers, Pennsylvania and Utah, the reason why we selected those is because they’re on opposite sides of the country. We want to make sure that regardless of what’s going on around you whether it’s something in your office, whether it’s a true disaster, regardless of what’s happening, you’ve got a safe copy of your data because they’re on opposite sides of the country.
So, you have that on-premise backup and automatically replicating to those data centers. You also want to take the user error out of it. User error is the leading cause of downtime, leading cause of data loss. We all love technology, but it relies on us doing the right thing. A lot of these attacks occur because somebody clicked on that email and opened the door.
And so, if that were to occur you want to make sure that those backups are safe and that they’re being tested.
So, you want to be able to set up that schedule that meets your needs, your RPO needs.
How much data can you lose?
Is it an hour’s worth, is it a day’s worth is it less than that? Make sure that that’s aggressive enough.
And then set up your schedule. I back up my data here with a Datto device and I back up hourly during business hours and twice a day on the weekends. That’s enough for me.
Now, you also want to be automatically testing these backups, not relying on somebody to go do that.
So, testing those backups both locally, and making sure that offsite sync occurred. Now you might get an X here, that says hey that backup failed, but now you’re able to be proactive about that. Tyler and his team can go and figure out why did this backup fail, what can we do to fix it, is it maybe a false negative and it really did occur?
You can be proactive and not find out that your backups failed three weeks ago and then you get attacked and now you’ve lost three weeks’ worth of data.
Now the way the backup is actually taken works differently as well. Most of those traditional forms of backup, for years they backup the files and the folders that you’re telling it to. And for the most part, they do what you’re telling it to. Here’s the file, here’s the folder, this is where it’s going. That’s safe. Now, obviously you run the risk of missing a volume, but for the most part, you’ve got your files and folders, but at the end of the day that’s all you have.
What image-based does is it really looks at backup in a different way. It takes a snapshot of that entire server, so it captures not only files and folders, but the operating system, the applications, every single thing that lived on that server, now lives on that on-premise device and is subsequently sent up to those two data centers.
And so just on the restoration process, when you talk about that downtime, this is just on restoration, think about it, if you’ve got to replace your server before you’ve got to bring those files over you’ve got to put your operating systems on there, you’ve got to put your applications and now you’re ready to start that restoration process, and which I mentioned you’re looking at days or weeks.
With an image-based backup, all of that information already lives on there. So, when you do that restore, we’re bringing everything over, the operating systems, the applications, everything. And you could even do this to dissimilar hardware, maybe you’re just operating off an old server that finally died and you were planning on moving to a different vendor anyway. Well, this can occur to a different server and it will look and feel just like your last one.
And what it gives us the ability to do is to set up what’s called inverse chains. So, once we set up that initial base image and it makes it to that device and it goes up to the cloud, from there you set up that schedule that I showed you earlier and every one of those backup points are not independent of each other. All we’re backing up now is the block level incremental changes. So, if you get hit with ransomware at 11:00, you take that time machine back to 10:00 and there’s no risk of those backups being impacted. They’re all independent of one another, whereas a lot of traditional backups are layered upon previous backups.
What it also gives us the ability to do, and what really eliminates that downtime, is the virtualization capabilities. And what I mean by this is since everything that lives on that server, lives on that device, you get that scary red screen that says pay me some ransom, because everything lives there, you would call up Tyler and his team and he would spin up what’s called a virtual machine meaning that now this off-premise device would be operating as your server.
For the most part, folks in your office wouldn’t even know this occurred. They’re going to be operating as business as usual. So, before you even start that restoration process, let’s get you up and running and make sure that your business continues to operate. That downtime now that we’re looking at instead of days or weeks, you’re looking at minutes or hours that we could have you back up and running. Right, you’re eliminating that.
And what’s really good about this as well is that in the ransomware space, which is even scarier, is that a lot of times they are doing an attack where they’re stealing your data and they’re compromising your data and they’re taking over control of your server and once they’ve got what they’ve want they detonate one of these ransomware attacks.
And then, for the most part, they know that companies will say, “Oh man I got hit with ransomware, what’s the first thing I do? Well, I’m going to wipe that server clean because it’s infected.”
Well, now you’ve just eliminated all of the evidence from that attack. By going into a virtual state that allows Tyler and his team to come in and do some forensics on that impacted server and figure out was there another attack, and then once that’s all there we do that bare metal restore that I mentioned earlier and get all of that data back onto that new piece of hardware.
So, we’ve been taking active backups when you’re in this state, so when you’re ready to do that restoration it’s not like you’re getting back to where you were the moment that scary red screen popped up, you’re actually getting back to where you were the minute that you’re ready to do that restoration. You can actually start that restoration process while you’re in this virtualized state. The benefits are pretty clear. That downtime that we’ve been talking about, you’re going to reduce that again, minutes or hours instead of days or weeks, to get back up and running.
Those RTO and RPO questions, you could answer with confidence, say here’s my RPO schedule, I’ve set up that schedule that was aggressive enough for me, we’ve been testing those backups and we know what our restoration looks like because we could run a test on how long it would take to get those servers up and running.
And ultimately you could do either one of these things from either locally or from the cloud, and these virtualization capabilities are all still being tested every single night. The virtual machine is spun up in the background all the way up and to the boot screen, a snapshot is taken of the boot screen, it gets taken and sent off to Tyler and his team to say all these cool virtualization capabilities as well, they have been tested. And you have the ability to do this either locally or from the cloud.
I mean what really put Datto on the map was Hurricane Sandy. We’re right on Long Island Sound, and I can tell you my house was out of power for eight days. At the time a lot of our clients were based in the northeast, and so we had thousands of businesses up and running out of our data center from remote sites as their server rooms were being drained out and new racks were being put in.
How many of those businesses would not have been able to survive those two or three weeks they were down?
And at the end of the day, after all the restorations occurred, we were able to put out a statement that said we didn’t lose a single piece of data for a single client that was operating out of our cloud, and that really propelled us to where we are.
So again, what business continuity does is in terms of how it differentiates itself from traditional backup is, yeah, we can back up the data and test it, and that piece of leverage that they have is taken care of. We can also take care of that downtime as well, and make sure that second piece of leverage, the real reason why most companies pay is taken care of as well.
And so, with that, I want to open it up to some questions, and we’ll see if some come in, and we’ll see if Tyler has any real-life stories.
So, there’s a chat box that you can enter, everybody’s muted, so you’ll have to put this in manually.
As some questions start coming in I’ll turn this over to Tyler, and I guess to start off Tyler, do you have any real-life experiences of clients or maybe some folks that you’ve talked to that maybe came to you after an attack that you have dealt with, especially on the ransomware front?
Tyler Snyder: Yeah, we’ve actually got a couple of good examples. Unfortunately, we see this a good bit in our industry.
We have a long-time customer, they’ve probably been with us for about 10 years, and they have about a terabyte of data on their file server I’d say, and at the time they had a cloud file-based backup that was backing up about three times a day.
So, they did get hit with some form of ransomware and we were able to get in, figure out where it came from, rectify that issue, but like Mike shared, once the damage is done, people are on their screens and they simply see that this ransom is requested before you can do anything. So, your only strategy at that point is that you have to go back from backup.
So, once we got to that point, it took three full days to restore a terabyte of data back down from the cloud. And it wasn’t that the data wasn’t available for them, or that we didn’t have a good backup or a recent backup, the issue was the amount of data and where it was housed.
So, that strategy didn’t line up with their Recovery Time Objective. So we implemented Datto for them, and then unfortunately almost a week, maybe a week and a half later, they got hit again.
Extremely unfortunate, but the recovery this time was a completely different story, and I remember actually mapping it out because I wanted to show him a good before and after picture, but it took us 1.2 hours. 1.2 hours to get them completely back up and running again.
The owner actually made a point to call me and thank us for forcing him to improve his BDR [Backup and Disaster Recovery] strategy. And that was really cool.
And you know, one other that does jump out at me, Mike, it doesn’t’ have to do with malware necessarily, but it does showcase why we switched almost all of our customers over to Datto. And it’s because we have a customer in Daytona Beach, FL who was protected with Datto, and Hurricane Irma came through.
I believe it started building intensity in early September of 2017, I want to say, and we held a strategy call with them to decide what could we do proactively to help protect the business. So we decided to help shut down all of their servers and equipment ahead of time so they could run their business in the cloud, and because Datto protects them in a hybrid approach, meaning there’s an onsite component and an off-site component, we simply took that off-site component, turned it on in the Datto datacenter, which by the way is just part of their value proposition, there’s no additional charge for that, and then we could very easily repoint users to those servers in the cloud, and that customer’s business was able to continue running safely even though they couldn’t physically be in their building for a while.
And I was reading that Irma did about $64 billion in damage, which likely affected a lot of businesses in that area, but these guys are still around, still going strong, so it really showcases the power of the Datto solution and why we choose to protect our customers with it.
Mike DePalma: Yeah, those are two great examples.
And actually, I want to add to the first example that you gave, and actually we have a few other questions coming in, but it was funny you talked about them getting attacked twice. And that is something that is very prevalent where somebody’s attacked, and they’re like “OK I’m through, I made it safe,” [but] they will continue to attack.
And I was actually doing a presentation to a group of law firms up and Washington, D.C. and you know it was all specific to about ransomware and I said, you know, “show of hands, how many people have been attacked with ransomware?”
And a bunch of hands go up, and afterward, I’m sitting down having lunch and I’m sitting next to one of the gentlemen who raised their hands and I said, “Oh you guys have been attacked by ransomware?”
And he said, “yeah, we’ve been hit three times in the last six months.”
And I said, “wow did you ever figure out how these guys are getting in? I mean I’m assuming it was some sort of email spam or who were they attacking?”
And the guy kind of put his head down and shrugged his shoulders and he said, “yeah, we figured it out, it was me, I fell for it all three times. I’m actually here as punishment for that.”
So, I guess its punishment to listen to me talk about ransomware, but it illustrates the point that you know, getting attacked once doesn’t mean it’s not going to happen again, and actually, you’re probably more likely to get attacked.
So, another question coming in, “can you talk about the security of the cloud?”
Which is a great question, and I’ll address that first, Tyler.
So, I mentioned we have private datacenters, one of the largest private datacenters in the word. So, we don’t use Azure or AWS and there really isn’t a risk of the cloud data being infected, and the reason is actually pretty simple. Data intrinsically can’t be encrypted twice. So, if something’s encrypted granted that backup on that device, that specific backup point is encrypted, but it won’t be able to make it to our data center, we actually encrypt in transit and at rest 256-bit military grade encryption. So, there isn’t that risk of it getting up to our cloud.
If anybody wants to see any of our SOC 2 reports they’re available to get out there, but we take our cloud very seriously. We have a team of folks that are on site at both of those data centers, and again I mentioned that we actually built the one in Reading ourselves, so all that data lives within our datacenters.
So, let’s see there’s one more, this is kind of repetitive, alright here’s a good one, “how do I know if I’m using a different form of backup how long my recovery time is?”
And I guess that’s a really hard question to answer not knowing what that is, but if somebody comes to you Tyler and says, “OK this is what I’m using can you help me try to figure out my RTO,” how do you even try to go about addressing that?
Tyler Snyder: Yeah, our first step is we would do a network analysis.
We would want to get on-site and we would want to get a good understanding of what product, how much data, how many servers is it running on, is it local, is it off-site, is it both, and once we have some of those figures we can better outline what a recovery would actually look like.
You know for the customer that I mentioned that had a file backup, those backups worked perfectly fine. The amount of data, the bandwidth speeds that were available in that area, that really dictated how long it took them to recover and again it wasn’t because those backups weren’t’ there.
So, I think you have to go in and sort of understand that first before you can really have some good strategic discussions on what it is, and then what it could be and what it needs to be.
Mike DePalma: Yeah, I think that’s a great point, and then I would say the takeaway from this is first and foremost is just take it back to your office all the things that you’ve learned and try to create this culture of security within the office through some education.
I recommend doing at least quarterly reviews, but beyond that having Tyler and his team come out and do a network assessment is pretty critical just to get a sense of are you protected, are those two that RPO and RTO goals being met with whatever you’re using now. It’s just good to have another set of eyes on there to make sure that whatever you’re using does protect you and does give you those RPO and RTO schedules that are aggressive enough for you.
Alright well, I’m not seeing any more questions coming in. We did record the webinar, so we can send that out so that you can share it with anyone else in your team or with anyone else you think would want to take a listen to it.
So, before we wrap up Tyler do you have any parting words to say to the group?
Tyler Snyder: Well, I mean I definitely hope everyone learned something today. Obviously, we’d love the opportunity to sit down and help develop a strategic technology plan for your business that can help prevent and recover from threats like what we just talked about.
I think my contact information is still there, I’d be more than happy to speak with you and you know, get you connected to the proper resources. So, we really appreciate everyone making the time to join us today and I hope everyone has a great rest of their day.