Transcript

[00:00:00] Ned: That was a couple of years ago. And his hearing seems to be fine. Selective, but otherwise fine.

[00:00:06] Chris: Yeah. I don’t think they make a pill for that.

[00:00:08] Ned: No, no. He tells me to take a chill pill all the time. He does not say that because he is not 40.

[00:00:19] Chris: And then he told me to eat my shorts.

[00:00:22] Ned: He might on the 90. These are back, baby. And it is disturbing the degree to which people have embraced the mid 90s.

[00:00:32] Chris: Right.

[00:00:33] Ned: And since both you and I lived through that and that was formative years, I don’t know if I like it.

[00:00:39] Chris: It’s not great.

[00:00:40] Ned: It’s definitely not 100% positive. Can we can we say that?

[00:00:45] Chris: I think that’s fair. Okay, that’s fair.

[00:00:48] Ned: I will say I just got tickets to go see Life of Agony perform River Runs Red in its entirety, an album that came out in 1993. Nice. So it is 30 years old.

[00:01:02] Chris: I mean, the one thing I will say for the 90s, we did have the only good Jurassic Park.

[00:01:10] Ned: Accurate. I haven’t seen any of the other ones, but accurate for the best. With the exception of Parks and Rec. I don’t really like Chris Pratt. That’s not true. Guardians of the Galaxy.

[00:01:25] Chris: He’s okay in the first one. The first one, which is the only one. No, that’s not fair.

[00:01:31] Ned: I’ve rewritten the second one and it’s ruined me.

[00:01:34] Chris: Fine.

[00:01:35] Ned: There’s a really good analysis of Guardians of the Galaxy Two that makes the case that it actually is a good movie viewed through the proper lens.

[00:01:47] Chris: So, like, slight concussion, maybe a little inebriated. Sound is not all the way on. So you’re kind of making up your own dialogue.

[00:01:55] Ned: You’re watching it on an error plane and you have nothing.

[00:01:58] Chris: You’re watching it on someone else’s seat.

[00:02:01] Ned: I won’t say that the argument was entirely convincing, but it was a good argument. So kudos to that person. If I could remember which podcast or video series it was that made that argument, I would tell you the big.

[00:02:17] Chris: Feeling I had was there was a lot of great ideas there. It needed another edit.

[00:02:23] Ned: Yeah. I think someone was doing an analysis of James Gunn, and I think that’s where it fell into. It might have been Patrick H. Williams, because that’s the sort of thing he does, and he does well.

[00:02:34] Chris: Yeah.

[00:02:34] Ned: So if you’re not already watching him, but I’m sure you are, of course. Okay. We’re basically neighbors, you and I. We’re like YouTube neighbors. I don’t know. Hello, alleged human, and welcome to the Chaos Lever podcast. My name is Ned. I’m definitely not a robot. I am definitely composed of living tissue made up of trillions of tiny cells that constantly divide, live and die, all without me noticing. I definitely do not find that prospect troubling, as well as a philosophical conundrum in the vein of Theseus’s ship. It’s fine. I’m fine, Chris.

[00:03:17] Chris: Yeah, you seem fine.

[00:03:22] Ned: I am not paralyzed. With fear and indescribable existential TerraForm all the time.

[00:03:31] Chris: Especially at 330 in the morning when the carnival starts.

[00:03:37] Ned: That would be less funny if my little one true. If my middle one hadn’t come down three times last night because she couldn’t sleep. Leap. My mind won’t stop going, honey, welcome to never will.

[00:03:52] Chris: Never seen someone hit a dolphin so quickly.

[00:03:54] Ned: He really just slammed her right in the face. I felt terrible for it. I was like, oh, God, I wish I had some words of comfort or advice for you, but really just sit next to me and ride it out. Yeah, colo. Be a long night.

[00:04:10] Chris: Pat her on the head and be like, all of life is misery. It’s all downhill from here.

[00:04:13] Ned: Right. You’ve peaked ten. Oh, tow to the listeners. I did not tell her any of that yet. That’s when they turn eleven. Start turning the screws. Now go get a job, because nothing means anything.

[00:04:32] Chris: That podcast is called Hard Conversations with Ned Beloved.

[00:04:37] Ned: Oh, let’s talk about some tech garbage. We got a doozy this week.

[00:04:40] Chris: Yeah. And as is tradition, I kept it short. And you used a lot of words.

[00:04:44] Ned: Actually, I don’t think that’s tradition. I think the inverse is usually 2023. In fact, you’ve complained about that.

[00:04:50] Chris: 2023.

[00:04:50] Ned: Okay, fine. In 2023. Apparently I’m the word, you one. This is sort of an expansion of something you brought up during the predictions episode, which was we’re going to see more security incidences in 2023.

[00:05:07] Chris: Incidents.

[00:05:08] Ned: Incidents.

[00:05:09] Chris: There we are.

[00:05:10] Ned: Incitements investments.

[00:05:12] Chris: No, no, no. The opposite of that.

[00:05:15] Ned: No. And I think the the corollary. Corollary. Wow, this is going to be a good one. Corolla to that.

[00:05:23] Chris: Buckle up, everybody.

[00:05:25] Ned: Was that we said people would care less about outages. I don’t know if they’ll care less about security incidents.

[00:05:36] Chris: Oh, you got at that time.

[00:05:37] Ned: And as we shall see, some of them are fairly disruptive.

[00:05:41] Chris: Yeah. This is going to be a fun little like a survey of the topic, shall we say? Yes.

[00:05:46] Ned: So let’s start with a few different incidents that we had over. We’ll start with one that began in 2022, in December. And we’ll expand out from there. But yes, one has already occurred in 2023. Yay. So let’s start with rackspace. And they’re very good. Very bad. No good. Exchange day. So as we mentioned back in episode 37 yeah, I actually looked it up. Also. It wasn’t very long ago, so it was super easy. And I also discovered there’s no search functionality on our website. So now I have another project. Quite a static site.

[00:06:26] Chris: I would totally help you with that, though. But I’m like, busy that day.

[00:06:29] Ned: Which day?

[00:06:31] Chris: That day.

[00:06:31] Ned: Okay. So Rackspace’s managed exchange service went mysteriously dark on December 2, with nary an explanation from the company. As hours dragged on today’s, without a satisfactory explanation from Rackspace, rumors started flying. Our own Chris Heiner hey, heard of him?

[00:06:54] Chris: That’s me.

[00:06:55] Ned: Oh, he floated the idea that they’d been hit by a ransomware attack, a claim that turned out to be startingly accurate.

[00:07:03] Chris: I really need to hire somebody to pat me on the back. My arm is getting so tired.

[00:07:09] Ned: I have an eleven year old that needs a job. So rackspace has since released the results of a forensic examination into the incident. It is not complete in any way, but it does draw some conclusions and makes some things more clear. So the speculation prior to the release of this examination was that the attack vector was the Proxy Not Shell exploit. That exploit is composed of two separate VCE that Microsoft had registered. One was 41 40, and the other one is 41 82. You don’t have to remember those. Just remember there were two of them. Now, the first one allowed an authenticated user to gain access to an endpoint and an authenticated user to gain access to a back end remote shell endpoint through a front end endpoint. In this case, the Auto Discover service on the front end of the Exchange server.

[00:08:12] Chris: Right?

[00:08:12] Ned: And then the second vulnerability allowed them to run pretty much any command they wanted against that remote endpoint. In this case, it was the remote shell access endpoint. So proxy. Not shell. That’s kind of where the name came from. So that’s what the two vulnerabilities allowed for. The first one uses a technique called path confusion, where you start out with a valid path in the request, and then you tack on a bunch of other stuff and it somehow gets forwarded to the back end service, the remote PowerShell service. And that second vulnerability allowed for arbitrary command execution. So you can do things like tell it to run a PowerShell command on the Exchange server to install a persistent threat. Fun. The theory was that both vulnerabilities needed to exist to successfully execute the Proxy Not Shell attack prior to releasing patches for Exchange. Microsoft released mitigation instructions to prevent the exploit, specifically blocking requests that use path confusion against Auto Discover. Further research by CrowdStrike determined that the mitigations were insufficient and that there was a previously undisclosed method for intrusion that did not require the Auto Discover attack. The attack instead used the OA endpoint Outlook web access I know what it means for those out in the listening community.

[00:09:46] Ned: So it used that endpoint instead and then proceeded down the same path of using remote shell to pull down and install a persistent process on the Exchange server for a few further exploits. Further research by the Crowd strike team determined that the vulnerability was likely 41 80, which was patched as part of the November 8 Exchange security update. It was not previously identified as part of the Proxy Not Shell attack, but all three vulnerabilities in play were all patched by that November 8 Exchange update.

[00:10:20] Chris: Now, do you mean that they were patched in terms of a patch was available or one was applied by Rackspace, the former.

[00:10:28] Ned: Right. Now, because the endpoint in question was different, it was OA instead of Auto Discover. The mitigation that Microsoft recommended was not sufficient. Now, the timeline here is pretty important, so stick with me for a minute. The initial proxy nutshell VCE were announced in September, along with the mitigation recommendations from Microsoft. Patches for both the VCE and the additional OA based VCE were released in early November, november 8. Rackspace’s hosted Exchange went down on December 2, although we don’t know when the attack actually started, and CrowdStrike published their expanded analysis on December 22, they don’t ever explicitly say that they used the incident at Rackspace as the basis for that research, but it’s heavily implied. Now, although Rackspace has published a post mortem identifying the OA based CBE as the likely attack vector for their intrusion, the attackers apparently wiped out the logs on the Exchange servers to cover their tracks. They have not provided any information regarding when the attack happened and what mitigations they had in place to prevent it. Given that the knowledge of the vulnerability and the mitigation options had been available for three months at the time of the attack, I see why they would.

[00:11:55] Chris: Choose to omit that information.

[00:11:57] Ned: Right. Rackspace has also announced that they will not be rebuilding the hosted Exchange environment, and they will work to move all existing customers to Microsoft 365 or help them recover their data to another platform. It seems that Rackspace had planned to sunset their offering anyway, and this was simply the straw that encrypted all of your data. That’s how it goes, right?

[00:12:22] Chris: Yeah.

[00:12:23] Ned: Okay, so what we have here is a service that was likely not super important to Rackspace to begin with. And I’m not saying they skimped on security, but I’m also certain it wasn’t their priority. Should they have patched sooner? Probably. Should they have had other tools in place to block such an attack? Yeah. Should they have had a dr plan that took ransomware into account?

[00:12:54] Chris: Yeah.

[00:12:56] Ned: But at the end of the day, it wasn’t a priority for them. And if you were their customer, I guess that was the risk you took.

[00:13:03] Chris: Whether you knew it or not.

[00:13:05] Ned: That’s right.

[00:13:07] Chris: So that was Rackspace in a nutshell.

[00:13:10] Ned: I think we can some proxy nutshell if you’re right there.

[00:13:15] Chris: Right there.

[00:13:16] Ned: I think we can wait till the end to come up with some general recommendations and advice. So let’s move on to the next one. A favorite of yours.

[00:13:24] Chris: I am so angry about this. LastPass.

[00:13:29] Ned: LastPass.

[00:13:31] Chris: For fuck’s sake. LastPass. You used to be great. I used to recommend LastPass to basically everybody. I even did it on other podcasts on an episode last year of Backup Central’s Restore at all podcast. I talked extensively about password managers in general and recommending LastPass specifically, going so far as to say that because of LastPass’s encryption and security model, and I quote myself it doesn’t matter if LastPass gets hacked. Your data would still be anonymous and safe. Yeah, because even if your vault was leaked, it would be like I said, anonymous. And even if your hash was stolen, surely it wasn’t crackable.

[00:14:18] Ned: Surely.

[00:14:19] Chris: Now why would I say that?

[00:14:21] Ned: Why?

[00:14:22] Chris: That’s what LastPass said. Remember LastPass private company? LastPass is code, not open source. Oh, and of course, why would a multibillion dollar company lie?

[00:14:37] Ned: What’s the incentive?

[00:14:38] Chris: None that I can think of. So in conclusion, as you’ll see by the end of this article lit I don’t know what to call it. I was bit wrong on that.

[00:14:48] Ned: Okay, so what happened?

[00:14:51] Chris: Short version, since we’re hitting a lot of company fails today.

[00:14:54] Ned: Indeed we are.

[00:14:55] Chris: Earlier in 2022, LastPass suffered a breach. Internal credentials were fished from some employee and they employees credentials were used to steal LastPass source code. This was an incomplete attack. It was a big embarrassment. There were a lot of failures. But LastPass claims that they fixed it. And most importantly, LastPass claimed that no customer data was leaked.

[00:15:22] Ned: If they’d been open source to begin.

[00:15:24] Chris: With, this would not be an issue.

[00:15:26] Ned: Indeed.

[00:15:27] Chris: So that was let’s call that an Oopsie.

[00:15:30] Ned: Oopsie daisy.

[00:15:32] Chris: So here’s the problem with that Oopsie that they claimed that they had mitigated.

[00:15:37] Ned: So this is known Oopsy Crapsy.

[00:15:39] Chris: We’re getting there.

[00:15:40] Ned: Okay.

[00:15:41] Chris: In August and then this was from the announcement that came in late December. But in August, the same attack from earlier was used to breach LastPass again. This time the attackers were able to move laterally inside of LastPass’s network and eventually steal all customer data from their improperly secured cloud storage environment. Now it is an open question as to whether or not they had an advanced persistent attack in place that never got cleared. Or did they do the same thing again? Neither of these is good.

[00:16:21] Ned: Right. And ultimately doesn’t matter.

[00:16:24] Chris: Right.

[00:16:24] Ned: The fact was they were able to move laterally through the environment and get to customer data.

[00:16:31] Chris: Correct. The only thing that was not lost in this exfiltration was customers credit card information. Because credit card information is tightly regulated. So ostensibly it was stored in an area of the infrastructure that was actually secure.

[00:16:48] Ned: Is this one of those situations where like if the black box always survives the plane crashing, why don’t we build the whole plane out of the black box?

[00:16:56] Chris: Yeah, I mean I assume that this was a financial decision. Well, this is what’s covered by PCI DSS, so we’re going to spare no expense. Everything else that’s fine.

[00:17:07] Ned: I’m sure.

[00:17:08] Chris: Yeah. So what was the impact of this? Everything else you had in LastPass was exfiltrated for everyone.

[00:17:18] Ned: Seems bad.

[00:17:19] Chris: And I’m going to quote LastPass’s own press release here. Quote the threat actor was able to copy a backup of customer vault data from the encrypted storage container which is stored in a proprietor great, now I’m doing it. Thanks a lot.

[00:17:33] Ned: You’re welcome.

[00:17:35] Chris: Which was stored in a proprietary binary format that contains both unencrypted data, such as website URLs, as well as fully encrypted sensitive fields such as website kubernetes, and passwords secure notes, and form filled data. Unquote. I need to emphasize that this is every customer they have. They lost vaults, they lost master password hashes, and my personal favorite, they lost unencrypted metadata. They went a little soft by just saying it was website URLs. What they lost that metadata is something that hackers can use to identify an individual user’s vault, something they said was impossible.

[00:18:23] Ned: Ah.

[00:18:24] Chris: Then if they find a target of high value, say a CEO or CFO or a politician or a celebrity, they can then focus on that one account run. Extremely complex, expensive and time consuming, but still possible brute force attacks against the master password hash to open that vault.

[00:18:48] Ned: That’s assuming that the person who created the master password hash actually used something that was sufficiently complex.

[00:18:56] Chris: Yes.

[00:18:56] Ned: Have you met CEOs?

[00:18:59] Chris: So here’s the thing about LastPass, in terms of what you just said, LastPass has been around for a long time, 15 odd years. Over that time, they have gotten more rigid about master password, security length, complexity requirements and such. What they didn’t do was backport that stuff. So right now, if you try to create a master password in LastPass, it will require complexity and twelve characters. So even that is leaps and bounds better than the password being LastPass. Lol exactly.

[00:19:36] Ned: Yes.

[00:19:37] Chris: If your password was that, you would never have to change it. They never mandated that you update your master password. They would recommend it with a little yellow exclamation point, but all you had to do was click. Nah. Second thing, the complexity and the level of encryption and the number of times that it was encrypted by their AES 256 implementation, which was roll your own, by the way.

[00:20:03] Ned: Never roll your own encryption.

[00:20:06] Chris: It started out at a low level. Now you could go into the settings and change that to a higher level of encryption if you wanted to.

[00:20:14] Ned: And new to guess who’s got two.

[00:20:15] Chris: Thumbs and had no idea that was even an option.

[00:20:19] Ned: You have two thumbs, I have two middle fingers. I can see those.

[00:20:25] Chris: So, I mean, you’re right about that. In terms of the complexity of your master password greatly influences how long it would take to crack. There are certain ones you if you had a 90 character, completely randomized master password, realistically, right now, the chances of that being cracked by brute force methods are it would take infinity time.

[00:20:46] Ned: Right? Heat. Heat, death of the universe, et cetera, et cetera.

[00:20:49] Chris: Right. Two things to remember one, computers are getting faster every day. Two, those vaults are going to be out there forever. And three, not everybody has a 90 character, completely randomized master password.

[00:21:00] Ned: Mine is only 16.

[00:21:01] Chris: I think mine’s in the 20s, but I can’t count that fast. That wasn’t a math major. Cut me off. Cut me a break.

[00:21:09] Ned: Okay.

[00:21:10] Chris: Why are we talking?

[00:21:11] Ned: I don’t know. Who are you?

[00:21:12] Chris: I’m Ron Burgundy. Anyway, so this means that in the future, like the indefinable future, those vaults and the identifying information to tell you them exactly who that vault is owned by is just out there forever. Which means you have to assume that eventually it’s going to be cracked. Right.

[00:21:35] Ned: So anything that you had in that vault, you need to change.

[00:21:39] Chris: Yeah. Every solitary password in there needs to be changed. Basically by everybody. Right now. It does come down to a risk management issue. If you don’t think that you’re a, quote, person of interest, the chances are they’re not going to bother. They’re going to go after Brad Pitt before they go after Brad Smith. Look, Brad doesn’t even rake his leaves.

[00:21:58] Ned: Fucking Brad.

[00:21:59] Chris: You know, it’s not a trustworthy name. That’s all I’m saying.

[00:22:03] Ned: No.

[00:22:04] Chris: And if you have significant accounts, such as social media, bank accounts, school entries, credit cards, make sure that they have MFA.

[00:22:15] Ned: Right?

[00:22:17] Chris: And if they don’t, maybe turn that on. Yeah. And if they don’t offer it, maybe get a better service, maybe.

[00:22:24] Ned: Who knows?

[00:22:25] Chris: So I’ll put a link in the show notes. I actually went into some significant detail about this on my own site. But, yeah, the TLDR change all your passwords, make sure your accounts are protected by MFA, and probably you’re going to want to cycle off of LastPass. Better options exist for both individuals and companies.

[00:22:45] Ned: Indeed. Speaking of cycling everything, our next one, Circle CI. And this is the one that happened this year.

[00:22:56] Chris: Yay.

[00:22:56] Ned: We’re off to a great start, everybody. And just like LastPass, this one hurts. Unlike LastPass, this one primarily affects businesses and their CI CD processes. For those who are not aware of Circle CI, it’s a SaaS business focused on continuous integration and delivery processes. That’s the manner by which software is built and deployed onto infrastructure. Now, I’m grossly oversimplifying here, but the important thing to know about all that is that as part of that build and deploy process, you need to store some sensitive information that is used by the process. Stuff like API keys, database passwords, access credentials.

[00:23:42] Chris: These all sound important.

[00:23:44] Ned: Additionally, in order for Circle CI to integrate with whatever repository software you’re using, it needs a token to interact with that software. An OAuth token, usually. And you also need a token from CircleCI to authenticate who you are and that you’re allowed to interact with the project you’ve developed on Circle CI. So, a lot of secrets, a lot of important information all being stored as part of Circle CI’s offering. They stored in a vault for you. I don’t know if they call it a vault, but that’s essentially the idea. Where I ever heard that vault thing before?

[00:24:21] Chris: Batman.

[00:24:22] Ned: That’s it. Thank you, Alfred. The information available on Circle CI’s website is frustratingly vague to the point of parity on what happened. So let me first read you this sentence and see what you get out of it. We wanted to make you aware that we are currently investigating a security incident and that our investigation is ongoing.

[00:24:48] Chris: Wow.

[00:24:49] Ned: So that seems like maybe bad, but not catastrophic, right? So what should I do? Because you’re investigating the security incidents out of an abundance of caution, we strongly recommend that all customers take the following actions immediately rotate any and all secrets stored in Circle CI. These may be stored in project environment variables or in contexts. We also recommend customers review internal logs for their systems for any unauthorized access starting from December 21, 2022, through today, January 4, 2023, or upon completion of your secrets rotation. Additionally, if your project uses Project API tokens, we have invalidated those and you will need to replace them.

[00:25:37] Chris: So that seems worse than the first thing you said.

[00:25:40] Ned: Yes. Yes. So that person probably should have read, holy shit, someone has gained access to all of our Secrets data. We have now intentionally broken all of your build processes, and your sensitive information is likely in the hands of attackers. Happy New Year, asshole.

[00:25:59] Chris: So, basically, they opened with there might be a wound, and we’re investigating Band AIDS and they ended with flesh eating disease. Is it really that bad?

[00:26:09] Ned: You know the bullet holes will get you first. Yeah, so wow, that seems bad. And I don’t think I can overstate how disruptive this is for any organization that’s using Circle CI. The company revoked all Project API tokens, which was probably necessary prudent, really?

[00:26:33] Chris: Yeah.

[00:26:33] Ned: I mean, they probably had to do that, but that breaks a lot of the automation that customers use and forces them to provision new tokens. What’s infinitely worse is that anything you stored in Circle CI now needs to be revoked on the source system. AWS access keys, database accounts, API keys, whatever, all of it needs to be revoked immediately. And you need to comb through your logs to verify no one has used this information to attack your infrastructure. As far as I can tell, Circle CI hasn’t provided any clarification on which customers were affected, all what level of access the attacker had, all how systems, how many systems were compromised, all how those systems were compromised, you don’t want to know. And how in the flying fuck an attacker was able to access all customer secrets across the entire company.

[00:27:32] Chris: He said please.

[00:27:33] Ned: It would seem that Circle CI really shit the bed on this multitenancy and separation of concerns thing. Worse yet, they have provided no details on how long the intruder was present, only stating that users should check their logs starting on December 21 forward. So while you are sitting at Christmas dinner listening to your Uncle Jerry yammer on about snowflake liberals who are ruining democracy for the true patriots for the sake of their precious Avocado toast. Something which is quite delicious.

[00:28:03] Chris: Don’t get me started.

[00:28:04] Ned: I swear to God, your company’s precious data could have been siphoned off to some godless heathen country like China or Andora they know what they did. Considerably unimpressed. Consider me suitably unimpressed by Circle CI’s flippant attitude to the whole mess. This is the closest they have to an apology, and it was in the first post. We apologize for any disruption to your work.

[00:28:35] Chris: Our thoughts and prayers are with you during this challenging no, you don’t get.

[00:28:39] Ned: Any fucking thoughts and prayers, Chris. We apologize for any disruption. Feels like a gross understatement to there are alleged 1 million developers and 30,000 organizations, doesn’t it?

[00:28:53] Chris: A bit. A bit, yeah.

[00:28:55] Ned: Between LastPass and Circle CI, I’m starting to feel pretty goddamn twitchy about storing secrets anywhere besides under a mattress, inside a Chinese puzzle box protected by Cthulhu and an army of cenobites. Yes, and I did actually spell Cthulhu right on the first try, so bonus points.

[00:29:15] Chris: You can’t give yourself bonus points, but.

[00:29:17] Ned: I did, so that is that was a good one. We’re doing great with the security thing.

[00:29:27] Chris: Yeah, yeah. Crushing it.

[00:29:28] Ned: All right, so you had one more, I think.

[00:29:31] Chris: Yeah, I guess in this case, I should say, hashtag crushing it. Twitter joke. I got jokes. Twitter. This place cannot do anything right now. I am not a conspiratorial thinker in general. Ned, why are you crying?

[00:29:58] Ned: That’s so funny.

[00:30:02] Chris: Is it the 5G nanobots giving you COVID?

[00:30:06] Ned: No, your tin foil hat has shifted a little to the right. You might want to straighten it before we continue.

[00:30:14] Chris: I am just thinking out loud here.

[00:30:17] Ned: Okay?

[00:30:18] Chris: But I’m starting to think that Elon is doing this on purpose. Wait, what was that sound? Was that the sound of Elon firing another dozen high ranking people on the security and content moderation teams with no reason and no notice? Just this weekend.

[00:30:33] Ned: You know, every time he does that, another hacker gets their wings.

[00:30:37] Chris: Amazing. Anyway, as we’ve talked about and I actually should have linked back to our not the Twitter files. What was that guy’s name? The guy with the thing?

[00:30:49] Ned: The one with the stuff?

[00:30:50] Chris: Yeah.

[00:30:51] Ned: All right. We’ll link back to it. Yes, mudge.

[00:30:53] Chris: The Mudge story made it clear that Twitter has had a lot of security breaches over the years. To be fair, predating Elon’s interest, let alone ownership. But this and this most recent one seems to have followed a familiar pattern. A crappy. Twitter API. Now, this breach appears to have happened in 2021. Thereabouts. Although the timeline is still not 100% clear.

[00:31:23] Ned: Okay.

[00:31:24] Chris: And really, the term breach might not necessarily apply what it seems happened? Looks more like slow motion scraping that the API allowed indefinitely.

[00:31:39] Ned: It’s like an ASMR thing.

[00:31:42] Chris: Avocado toast. I think I just scared everyone. So what happened is, if a user had a valid Twitter account and they had an email address or a phone number for another user. They could access the API and look up all the information about all the accounts that were associated with it, either the email address or the phone number.

[00:32:11] Ned: You could branch out.

[00:32:12] Chris: Yes. And these requests could happen. An unlimited amount of time for an unlimited amount of emails or phone numbers. Great work.

[00:32:25] Ned: Good job, everybody.

[00:32:26] Chris: Apparently, there were no monitors to set up to watch this API activity, and the hacker got info on at least 235,000,000 accounts. Now, a lot of what I just said is frustratingly vague, and that’s because, one, you can’t take a hacker at their word. They’re a hacker. And two, Twitter has not really been upfront about what happened either. Probably, like I said, since there’s no monitoring, they don’t know.

[00:32:53] Ned: Yes. And they have no security team to check.

[00:32:56] Chris: So this happened over a period of months, and the hacker, who goes by the name of Ryushi, attempted to sell the database back to Twitter. Twitter did not respond.

[00:33:06] Ned: There’s no one there.

[00:33:09] Chris: So Ryushi just released the entire 63 gigabyte data file on the Dark Web for free. If you would like to know if your information is in that database, the answer is probably yes.

[00:33:23] Ned: Mine is hooray.

[00:33:26] Chris: If you would like to confirm for yourself, though, Have Ibinpone.com has already integrated the info from the database so you can safely and reliably check your email and phone number there to see if it was in the breach. But like I said, if you had an account in or around the middle of 2021, the answer is probably yes.

[00:33:48] Ned: The good news is that the data does not appear to include your password, encrypted, or otherwise correct. The bad news is it probably has enough information to help someone impersonate you or just be a general nuisance or docs you. Oddly enough, last week, someone tried to add me to a Google Meet group using my email address and video. Called me three times. Nice coincidence.

[00:34:15] Chris: No, because nobody willingly calls you three.

[00:34:18] Ned: Times, especially over video. That’s the reason we podcast. Chris okay, so that is by no means a complete overview of the security incidents we’ve had.

[00:34:32] Chris: These are just the first ones we could think of.

[00:34:34] Ned: There’s also an ongoing one with slack that we hadn’t had a chance to get into. But I think I want to step back here and talk a little bit about what you as the listener, could potentially do personally and from a business perspective, right, given all of what’s happening with these security problems. So aside from acknowledging that computers were a mistake and so was agriculture and leaving the trees to begin with, chris, what do you have?

[00:35:03] Chris: So I think the first thing that we should have been doing for years, but we definitely need to do start going forward, is do not take companies at their word.

[00:35:15] Ned: Yes.

[00:35:16] Chris: I’m thinking of LastPass in particular, because I’m the most mad about that. But this is a situation where, like I said, LastPass has been around for a while. It’s a known brand. If you talk to someone about password management, they’re probably going to know LastPass is they probably will not know what Bit Warden is. That does not mean that Bit Warden.

[00:35:36] Ned: Is worse fears it might be better.

[00:35:40] Chris: Right. So one of the biggest things that burn people here is they took LastPass at their word, and LastPass was less than honest.

[00:35:52] Ned: So would you recommend going with companies that have had an independent third party security audit to verify their claims?

[00:35:59] Chris: LastPass had some of them, too. They just used a lot of weasel words and kind of talked around them. So, as in many things, it’s going to come down to your acceptance of risk and your attitude and taste for it. One thing that we should have known about LastPass also is they’ve had many security breaches over the years. This is not the first one. That should have been a red flag enough to say, you know what? Maybe this is not for us.

[00:36:26] Ned: Right.

[00:36:27] Chris: I know I’m in that camp. I took them at their word and was just like, well, they did have breaches, but my information is safe. And that was a false sense of security that I put upon myself.

[00:36:39] Ned: So what are you saying? It’s all your fault?

[00:36:41] Chris: Yeah, that’s what I’m getting at.

[00:36:43] Ned: All right, I have a couple of things. One, if it’s really that important, host your own shit. There are any number of good open source projects out there that are pretty secure that will allow you to host your own secrets. Yeah, do that. At least then, if it’s really a priority to you, you can verify the way in which it is secured and stored. You don’t have to take any other company at their word. And since it’s an open source project, at least it will be vetted by a lot of people.

[00:37:18] Chris: Right.

[00:37:19] Ned: So pick any popular or open source project for storing secrets and give that a try. If that had been the case, if you’d done that with something like Circle CI, let’s say, just pulling something out of the air, not that you’re upset.

[00:37:36] Chris: About it or anything, it would have.

[00:37:38] Ned: Been less of an issue, especially if you weren’t storing your secrets in Circle CI. You were only using them for automation and then hooking into some other system you were hosting for that secret information. You would avoid a lot of work. Yeah, you got to regenerate those Project API tokens or whatever they were. Everything else should pretty much continue to work fine. And you don’t have to rotate things. I mean, you should still rotate things. Which gets me to my second point. Make stuff dynamic. The less secrets you have to store, the better.

[00:38:08] Chris: That’s also true.

[00:38:10] Ned: So using things like Open ID Connect allows you to make the generation of credentials short term and ephemeral, I need access to my AWS account. I’ll authenticate using OIDC. It’ll generate a token that’s good for an hour. If that gets leaked, it’s already invalid by the time someone can try to use it.

[00:38:32] Chris: Right. One thing I would add here also, in terms of the secrets that you use, is whatever you write and put out into the world, document the daylights out of it. Know what secrets you have, know what you need, know where they come from. Because if you have an instance like this circle CI situation, and you’re not documented, you’re going to be in trouble when it comes to even when it comes to rebuilding your own environment, let alone migrating to something else.

[00:39:02] Ned: Indeed, circle CI, to their credit, did release a tool that will show you all of the secrets you have stored on their system to make it somewhat easier. That’s helpful, but that’s like bare minimum, right?

[00:39:15] Chris: And that’s only half the equation.

[00:39:16] Ned: It’s less than half the equation because now you need to take that information and use it against the systems that actually use those secrets.

[00:39:23] Chris: Right.

[00:39:24] Ned: Which is fun. So wherever you can make it dynamic, don’t store a secret. And the last one is, and this is more for personal, but it’s trickling the business as well. Go passwordless wherever you can for your accounts. This is still challenging. Not all websites support it, but many websites are starting to support it, especially with 502 and its adoption. So if you can go without a password, highly recommend that you do.

[00:39:54] Chris: Right. And also, companies 2023. Just a thought. Just throwing it out there.

[00:40:02] Ned: Whoa.

[00:40:02] Chris: Maybe do better.

[00:40:03] Ned: Whoa.

[00:40:04] Chris: I know, I know. Sounds crazy.

[00:40:07] Ned: It’s the year of the security professional. Yeah.

[00:40:11] Chris: I actually haven’t looked at the Zodiac. This might be the year of the face palm.

[00:40:15] Ned: Interesting. Lightning round.

[00:40:19] Chris: Lightning round.

[00:40:20] Ned: All right.

[00:40:23] Chris: Apparently Gen Z thinks that the coolest technology is the technology that is as old as they are. What is old is new again. It’s happening already, apparently with fashion. With many important people telling me that the 90s look is back in. I can’t say that I know that for sure because I wasn’t part of the cool crowd when the 90s were actually happening around the first time either. But it seems like they’re doing it again, this time with cameras. Yay laughing in the face of Apple and Samsung’s pocket phone megapixel monsters. According to the Ticker talkers, the new hip thing to do is go out and, quote, buy the cheapest digital camera you can find so that you can, quote, get that overexposure look out on the gram. The youths seem to think that we’re becoming a bit too techy and so are rolling into parties with old Canon Power shots from 2011. I know. Now, this trend, of course, ignores the fact that the pictures accompanying the article are perfectly fine. And most of the time, the massive amounts of megapixels on consumer cameras are totally useless, aside from marketing a big number and have been for at least 15 years.

[00:41:45] Chris: But in researching this article, I actually started digging into my physical archives, aka the Attic whoa. And was actually able to find a working Sony Mavica from the 90s that has a whopping 1.2 megapixels and stores its images and I swear to God this is true on a floppy disk. So am I cool now, too? Anyone? Bueller?

[00:42:14] Ned: It’s not going to happen.

[00:42:15] Chris: Fine. I’ll just go find a wall to moodily. Lean on you’re not cool enough for that. Damn it.

[00:42:21] Ned: Android without the arm. Sounds like a risky proposition.

[00:42:26] Chris: I feel good about that.

[00:42:28] Ned: See myself out. So, at the recent Risk Five summit, android director of engineering Lars Bergstrom got on stage and clarified Google’s position on Risk Five adoption for Android. And that position is 100% in favor.

[00:42:45] Chris: Yay.

[00:42:46] Ned: With a detailed checklist of what needs to happen for the operating system to be ported over to the alternative architecture. And he also talked about the availability of a 64 bit branch for Risk Five in the Android open Source project repository. But why? Android and Arm are like peas and carrots, right? Why would Google spend the money on supporting an alternate compute architecture? Choice. The answer is one of choice. Arm licenses its technology out to designers and chipmakers, and at this point, mobile chipmakers don’t have much of a choice. Thank you. Intel either pay the licensing fee or GTFO. Arm is a for profit company, so they are incentivized to keep prices up. Sue anybody who thinks that they are going to get away with using that license without paying. And as long as those license fees are less than X 86, they’re going to keep doing it. Risk Five, on the other hand, is an open standard managed by the nonprofit Risk Five Foundation. Anyone can use their standards to build out a chip design without worrying about licensing. Or more importantly, for the Chinese potential trade embargoes from the west. Now, you won’t be holding a Risk Five Android phone tomorrow.

[00:44:14] Ned: The development process is likely to take several years, plus the need for someone to develop the hardware. But Risk Five has been taking has had some serious uptake recently, and I would not be surprised to have a Risk Five based phone hit the market in 2026. Add that to the predictions sheet, assuming that we’re still here.

[00:44:36] Chris: Speaking of processors, apple made a mistake designing one.

[00:44:42] Ned: That seems bad.

[00:44:43] Chris: So you’ve heard of Apple, right? They make real expensive stuff that’s real good and lasts a real long time. One of these reasons is that they have this reputation. Is that it’s true. Wait, that’s a tautology.

[00:45:00] Ned: And a banal one at that.

[00:45:01] Chris: Silence, penance. What I meant to say is one of the reasons they have this reputation is because their R and D standards are preposterously high and almost always met. Except this time. Reports have surfaced that show a recently released iPhone Pro 14 was supposed to have an even more powerful processor. The testing on this processor was done entirely in simulation and appeared very promising, but when it was prototyped, it failed in unexpected ways that overheated the device.

[00:45:36] Ned: That’s bad.

[00:45:37] Chris: This is generally considered bad not only for battery life, but also not great for what we in the business call not exploding. Oh, Apple did catch this late, but it’s important to note that they did catch it. End result was the iPhone 14 Pro that came out still has a top of the line chip, but just not AWS tippy top of the lineiest as they were hoping it would.

[00:46:03] Ned: Non compete is Bullshit and soon also Illegal when you are looking to get a job, the potential employer holds a lot of power, especially if you really need that job to like, have health care or afford rent. You know, silly things like that. As such, companies have long abused that power to enforce restrictions on your employment that don’t seem entirely moral and in some cases are blatantly illegal. One important example of the latter is wage information sharing. You have the right to consent to disclose your compensation to anyone you want, and your employer cannot take punitive action if you do. An equally important example of the former immoral, but not illegal, is the non compete agreement. In fact, you may already have one of these in your contract. About one in five people do. The noncompete says that you cannot quit and go work for a competitor within a certain time frame. Who that competitor is, that’s kind of up to the company. While not always invoked, the noncompete can serve as a way to suppress worker wages by preventing mobility and wage competition and reducing opportunity. Just knowing that you might be sued if you go work for a competitor is usually enough to make most people think twice.

[00:47:25] Ned: The Federal Trade Commission has proposed a rule that would prohibit noncompete clauses on workers for both new and existing contracts, also forcing companies to inform their employees that the existing non compete is now null and void. Several states, including California, have already banned noncompetes with no obvious catastrophic impact. So I’d say this will be a net good for the workers. Of course, companies and lobbies will be more than happy to challenge the FTC in court saying they are overreaching. And with our current SCOTUS, I don’t hold out a ton of hope that they would side with the little guy. Still, it’s a small beacon of light in an otherwise dark horizon, and I choose to celebrate it. Noncompetes can suck it.

[00:48:13] Chris: Graphic artist who did real work accused of using AI because, quote, it looks artificially generated.

[00:48:20] Ned: We did it everybody. Congratulations.

[00:48:23] Chris: So this is an interesting twist on the whole AI generated image controversy. A real life professional artist who goes by the name Ben Moran was contracted to create a book cover for an upcoming fantasy book titled Mandate of Heaven. The writer of Mandate has a preexisting relationship with Ben from many previous contracts. Ben worked to create this book cover, which was accepted, and then decided to share it on that bastion of fairness and objectivity known as Reddit. Here’s where the fun begins. So Ben posted the image on Art as a reasonable bit of self promotion. 100 hours of work went into the COVID Why not share it and see if more business can come? A Reddit moderator looked at the image and said, ah, fake took down the post and banned Ben from the subnet entirely. I’m leaving it. So this is a laughable case, right? Reddit mods are frequently known as bad actors, and the one who banned Ben is a historically awful one. But this is probably the beginning of a trend. With AI becoming so prevalent and so capable, it is going to start to be very hard for people to tell real from fake or fake from real.

[00:49:45] Ned: That’s another tautology you’re despicable. Thank you. A Fond Farewell to 3G in 2008, I purchased my one and only iPhone. The iPhone 3G powered by at and T, and it changed my mind on what a smartphone could be. I think we can safely say that between the App Store and modern smartphone really arrived in 2008. Before that point, I had been happy to carry my BlackBerry running on what seemed like a perfectly capable GPRS and Edge cellular connection. I had access to email, text messaging, and a chronic brick breaker habit. 3g made the web accessible on a smart device. It made cellular modems on laptops actually useful and very expensive. It made cellular a viable backup connection for remote offices. The 3G standard allowed for data transfers of up to 40 42 megabits per second, compared to 170 kbps on 2G.

[00:50:56] Chris: That’s like ten times faster.

[00:50:57] Ned: At least you could stream data, have video calls, and do all the things on mobile we take for granted now. And 15 years later, time has come calling for three G to take its place in the radio technology volhala. At the end of 2022, Verizon shut down its 3G CDMA network, being the last major carrier in the US to still support Three G. The technology has long since been supplanted by Four G and now even 5G. But there were still some people and machines using the Erstwhile 3G network. Now, unless you’re running a private 3G installation, that option is closed in the US. Thank you to 3G for bringing us to the modern mobile computing era. I’ll pour out some RF spectrum for you on my Oscilloscope. Hey, thanks for listening or something. I guess you found it worthwhile enough if you made it all the way to the end. So congratulations to you, friend. You accomplished something today. Now go and contemplate the assemblage of organic material you used to listen to the show and marvel at the magic of pressure waves, delivering information to your brain. You’ve earned it. You can find me or Chris on Twitter at ned 1313 and at heiner 80, respectively, or follow the show at chaos underscore lever if that’s the kind of thing you’re into.

[00:52:21] Ned: Show notes are available@chaoslever.com if you like reading things. You can also sign up for our newsletter, helpfully located on that site as well. We’ll be back next week to see what fresh hell is upon us. Tata for now.

[00:52:37] Chris: I like the hardware. This entire show, absolutely nothing about CBS.

[00:52:42] Ned: I studiously avoided it. I thought about it. I saw a few things.

[00:52:47] Chris: Everything looks very insecure and cheap.

[00:52:50] Ned: Lots of stuff around. VR. I was actually going to bring that up in Lightning around. And I didn’t. Because I respect us. I respect me.

[00:52:59] Chris: There we go. Okay.