So, this actually was first detected on Friday July 25, escalated all the way up to the Emergency Operations Center on July 28 (Monday), state of emergency / near total intranet shut down (they are quarantineing the whole system) on July 29 (Tuesday).
It seems to me that some kind of rather sophisticated threat actor managed to get into the core … this techxplore article calls it a ‘VPN’, but it isn’t technically a VPN, its a secure access tunnel system that city-gov systems and employees use to talk to each other, it almost certainly is not intended to be geared toward broad internet access/usage, beyond accepting user input from public facing government web portals, such as say, people paying their utliity bills online or trying to submit a business liscense application online, things like that.
This system is sounding like it got fully compromised (as in, low level/high privilege level access was secured), and was either sending data out/in through improper IP addresses, and/or was possibly being hijacked to do some kind of DOS attack … on itself?
I am having a really hard time finding any exact details on this, but this is my best guess.
Given that the EOC essentially immediately shutdown everything and called in a National Guard Cybersecurity team, it seems to me that there is a high chance this was done by basically a nation-state level threat actor.
It also at least seems like the systems, the data, the hardware, have at least not yet been locked down in a ransomware style move, which… could be largely due to their just quickly pulling the whole thing offline, or could be because that wasn’t the goal of the attackers… or some combination of both.
It is. Others have given some details, but I’ll keep it simple.
A VPN makes remote devices seem like they’re on the same network. You can have all traffic be routed through that virtual network, or just some of it. Common use cases:
consumers - make yourself appear to be somewhere else; basically replaces old SOCKS proxies (all traffic routed)
workplace - provide access to internal, protected resources to those that need them (only relevant traffic is routed)
home lab - expose internal services publicly (reverse of workplace use case)
Those are all VPNs, though the first is acting more like a proxy than the others.
National Guard Cybersecurity team
This isn’t some crack team of experts, it’s mostly part-time soldiers who likely have a relevant day job. My brother-in-law is a mechanic at the National Guard, not because he’s an expert, but because they paid for his 4-year degree and only expect a few hours of work each month. A lot of people join for inexpensive medical insurance.
This cybersecurity team is probably just a handful of locals who work in IT locally and have had training on systems commonly used by the military.
If this was a high profile attack by a state actor or something, they wouldn’t call the National Guard, they’d call the NSA, CIA, or something similar, as in an actual crack team. The National Guard is mostly there to provide structure in emergencies, like organizing rescue efforts in a flood or help firefighters with labor in fighting wildfires. They’re just weekend warriors, not experts.
I guess my confusion here comes from trying to reconcile the broad, colloquial understanding of a VPN, and the actual, precise, technical definition.
When a news article runs with VPN in a wide audience usage… 95% of people think SurfShark or Nord or PIA or whatever, something that is consumer oriented, that accesses/fancy proxies the broad internet, as you give in your first example, where it basically functions as a more elaborate set of proxies than what most people could probably manage on their own.
So… yes, it technically is a type 2 VPN as you’ve listed, but it technically isn’t a type 1 VPN, which is what 95% of people think a VPN is.
I’ve worked remote for a decently long while, and most other remote workers I’ve known… they do not have really any understanding at all that their work login thing… is fundamentally the same kind of VPN as Surfshark, just configured differently.
My goal was to emphasize this difference, but yeah, I could have used better wording.
And yes, I know as well that Nat Guard CyberSec are by no means the creme de la creme of cybersec specialists, but the fact that a top level Municipal agency went ‘oh fuck’ and basically escalated the issue to the next level of IT support, the State Nat. Guard… that means they got pretty fucking spooked.
Also, the FBI is involved as well, they’d be the ones to pass it up to NSA and/or Homeland Security, I think… and the Nat Guard would be the ones capable of passing it up to… Army CyberCom… and I think if it makes it up to either Army CyberCom or the NSA or Homeland Sec, well at that point, its theoretically possible that any member of the alphabet soup could be called upon, or at the very least, have it come up on someone’s desk.
I am not exactly sure what the CoC of escalation pathways is here, but it seems like this got escalated to as many people as the Municipal Emergency Response Team could, quite rapidly.
Its ‘the emergency response team looked at this for 24 hours and then called in another emergency response team’.
So… yes, it technically is a type 2 VPN as you’ve listed, but it technically isn’t a type 1 VPN, which is what 95% of people think a VPN is.
Sure. But VPNs were around long before the consumer-oriented VPNs were a thing.
spooked
Or they just had one person handling their IT and needed help, and didn’t want to pay an outside contractor.
I’m honestly surprised the National Guard was called at all. If anything, that shows how backwards Minnesota is, or at least the mayor of St. Paul. I’d expect that if my state government got hacked, they’d call in a local cyber security firm to come audit things, and we have plenty of them here (I’m in Utah, so not even a big state). This isn’t a National Guard situation, it’s an independent cyber security audit and FBI situation.
Here’s how I expect this happened:
St. Paul’s small IT team escalated the issue to the mayor because they were overwhelmed
Minnesota Governor (Tim Walz) didn’t know what to do, so he called everyone, including the National Guard
Sure. But VPNs were around long before the consumer-oriented VPNs were a thing.
No argument there, you’re right.
(technically =P)
Or they just had one person handling their IT and needed help, and didn’t want to pay an outside contractor.
Nah, read the links I provided.
It went from the normal IT department, to the city level Emergency Response Team, to the Nat Guard and FBI.
Cities, larger ones anyway … often have their own sort of local mini-FEMA, who have their own capacities to order around other local agencies, but also have a whole bunch of protocols for… who to contact when something exceeds the capacity of everything they can more or less order around with their own authority.
I’m honestly surprised the National Guard was called at all. If If anything, that shows how backwards Minnesota is, or at least the mayor of St. Paul.
I am not in particular familiar with St.Paul specifically… but …
It could overall make sense given the capacities of the city (the Twin Cities, St. Paul + Minneapolis), and them knowing their own constraints.
It could also make sense if they rather rapidly at least suspected a very sophisticated, foreign threat actor.
That second half is kinda most of my argument:
Why would you start up the Military chain of escalation unless you either suspected a potential foreign nation state actor, and/or, critical infrastructure systems were breached, so critical that they’d been previously deemed an actual national security risk, should that happen?
I am not certain of what happened, nor certain of the validity of this logic… but this is my logic, from the original comment.
Sure, they could have just panicked. I don’t know that they did or did not.
But I have worked with people who’ve been employed by, led things like FEMA and DHS and City level emergency response teams, their specialities being the cybersec/netsec variety, and… this seems like actually following a previously outlined set of steps to me.
I’d expect that if my state government got hacked, they’d call in a local cyber security firm to come audit things, and we have plenty of them here (I’m in Utah, so not even a big state).
Ahahah, two things here:
Basically, see what I just wrote above.
Really? Utah, prime recruiting ground for the CIA, Utah, with the largest NSA data center complex in the country, possibly the world, that is archiving essentially all US internal communications they can so they can search through them later if need be, Utah, with more and more corporate datacenters all the time… you don’t class Utah as a big state, in terms of the tech sector?
Perhaps I am misunderstanding you, but I just find that silly.
The primary purpose of a VPN is to create a tunnel between two networks, hence the name “virtual private network”. I’m very familiar with them as I work with these systems for a living.
I’m guessing some people don’t know (or forgot) that site-to-site and remote access VPN’s are a thing, and was the initial purpose of VPN’s. Masking or hiding your location became a benefit after the fact, and todays more common client VPN is technically a remote access VPN with a new purpose.
Remote access VPN’s are a very common attack vector for companies, look up companies compromised with Fortinet gear and its typically through the firewalls VPN.
In fact, a primary purpose of a VPN, spoofing your IP/geolocation, pretending you are someone you aren’t… is pretty much antithetical to a highly controlled system of users with varying levels of access to specific, private areas of that system.
Most modern remote access VPN’s do exactly that, so it is not antithetical at all and is how most client VPN’s keep you from accessing other users data. I would encourage you to read up on WireGuard and the like, they are fun to learn about and awesome tools when configured properly.
Also, we removed the above comment because the last sentence was fairly rude and violates rule 3 @sp3ctr4l@lemmy.dbzer0.com
https://techxplore.com/news/2025-07-fbi-national-st-paul-cyber.html
https://www.reuters.com/world/us/minnesota-calls-national-guard-after-st-paul-slammed-by-digital-attack-2025-07-29/
https://techcrunch.com/2025/07/30/minnesota-activates-national-guard-as-cyberattack-on-saint-paul-disrupts-public-services/
So, this actually was first detected on Friday July 25, escalated all the way up to the Emergency Operations Center on July 28 (Monday), state of emergency / near total intranet shut down (they are quarantineing the whole system) on July 29 (Tuesday).
It seems to me that some kind of rather sophisticated threat actor managed to get into the core … this techxplore article calls it a ‘VPN’, but it isn’t technically a VPN, its a secure access tunnel system that city-gov systems and employees use to talk to each other, it almost certainly is not intended to be geared toward broad internet access/usage, beyond accepting user input from public facing government web portals, such as say, people paying their utliity bills online or trying to submit a business liscense application online, things like that.
This system is sounding like it got fully compromised (as in, low level/high privilege level access was secured), and was either sending data out/in through improper IP addresses, and/or was possibly being hijacked to do some kind of DOS attack … on itself?
I am having a really hard time finding any exact details on this, but this is my best guess.
Given that the EOC essentially immediately shutdown everything and called in a National Guard Cybersecurity team, it seems to me that there is a high chance this was done by basically a nation-state level threat actor.
It also at least seems like the systems, the data, the hardware, have at least not yet been locked down in a ransomware style move, which… could be largely due to their just quickly pulling the whole thing offline, or could be because that wasn’t the goal of the attackers… or some combination of both.
It is. Others have given some details, but I’ll keep it simple.
A VPN makes remote devices seem like they’re on the same network. You can have all traffic be routed through that virtual network, or just some of it. Common use cases:
Those are all VPNs, though the first is acting more like a proxy than the others.
This isn’t some crack team of experts, it’s mostly part-time soldiers who likely have a relevant day job. My brother-in-law is a mechanic at the National Guard, not because he’s an expert, but because they paid for his 4-year degree and only expect a few hours of work each month. A lot of people join for inexpensive medical insurance.
This cybersecurity team is probably just a handful of locals who work in IT locally and have had training on systems commonly used by the military.
If this was a high profile attack by a state actor or something, they wouldn’t call the National Guard, they’d call the NSA, CIA, or something similar, as in an actual crack team. The National Guard is mostly there to provide structure in emergencies, like organizing rescue efforts in a flood or help firefighters with labor in fighting wildfires. They’re just weekend warriors, not experts.
I guess my confusion here comes from trying to reconcile the broad, colloquial understanding of a VPN, and the actual, precise, technical definition.
When a news article runs with VPN in a wide audience usage… 95% of people think SurfShark or Nord or PIA or whatever, something that is consumer oriented, that accesses/fancy proxies the broad internet, as you give in your first example, where it basically functions as a more elaborate set of proxies than what most people could probably manage on their own.
So… yes, it technically is a type 2 VPN as you’ve listed, but it technically isn’t a type 1 VPN, which is what 95% of people think a VPN is.
I’ve worked remote for a decently long while, and most other remote workers I’ve known… they do not have really any understanding at all that their work login thing… is fundamentally the same kind of VPN as Surfshark, just configured differently.
My goal was to emphasize this difference, but yeah, I could have used better wording.
And yes, I know as well that Nat Guard CyberSec are by no means the creme de la creme of cybersec specialists, but the fact that a top level Municipal agency went ‘oh fuck’ and basically escalated the issue to the next level of IT support, the State Nat. Guard… that means they got pretty fucking spooked.
Also, the FBI is involved as well, they’d be the ones to pass it up to NSA and/or Homeland Security, I think… and the Nat Guard would be the ones capable of passing it up to… Army CyberCom… and I think if it makes it up to either Army CyberCom or the NSA or Homeland Sec, well at that point, its theoretically possible that any member of the alphabet soup could be called upon, or at the very least, have it come up on someone’s desk.
I am not exactly sure what the CoC of escalation pathways is here, but it seems like this got escalated to as many people as the Municipal Emergency Response Team could, quite rapidly.
Its ‘the emergency response team looked at this for 24 hours and then called in another emergency response team’.
Sure. But VPNs were around long before the consumer-oriented VPNs were a thing.
Or they just had one person handling their IT and needed help, and didn’t want to pay an outside contractor.
I’m honestly surprised the National Guard was called at all. If anything, that shows how backwards Minnesota is, or at least the mayor of St. Paul. I’d expect that if my state government got hacked, they’d call in a local cyber security firm to come audit things, and we have plenty of them here (I’m in Utah, so not even a big state). This isn’t a National Guard situation, it’s an independent cyber security audit and FBI situation.
Here’s how I expect this happened:
No argument there, you’re right.
(technically =P)
Nah, read the links I provided.
It went from the normal IT department, to the city level Emergency Response Team, to the Nat Guard and FBI.
Cities, larger ones anyway … often have their own sort of local mini-FEMA, who have their own capacities to order around other local agencies, but also have a whole bunch of protocols for… who to contact when something exceeds the capacity of everything they can more or less order around with their own authority.
I am not in particular familiar with St.Paul specifically… but …
It could overall make sense given the capacities of the city (the Twin Cities, St. Paul + Minneapolis), and them knowing their own constraints.
It could also make sense if they rather rapidly at least suspected a very sophisticated, foreign threat actor.
That second half is kinda most of my argument:
Why would you start up the Military chain of escalation unless you either suspected a potential foreign nation state actor, and/or, critical infrastructure systems were breached, so critical that they’d been previously deemed an actual national security risk, should that happen?
I am not certain of what happened, nor certain of the validity of this logic… but this is my logic, from the original comment.
Sure, they could have just panicked. I don’t know that they did or did not.
But I have worked with people who’ve been employed by, led things like FEMA and DHS and City level emergency response teams, their specialities being the cybersec/netsec variety, and… this seems like actually following a previously outlined set of steps to me.
Ahahah, two things here:
Basically, see what I just wrote above.
Really? Utah, prime recruiting ground for the CIA, Utah, with the largest NSA data center complex in the country, possibly the world, that is archiving essentially all US internal communications they can so they can search through them later if need be, Utah, with more and more corporate datacenters all the time… you don’t class Utah as a big state, in terms of the tech sector?
Perhaps I am misunderstanding you, but I just find that silly.
Yeah that’s a vpn
Removed by mod
The primary purpose of a VPN is to create a tunnel between two networks, hence the name “virtual private network”. I’m very familiar with them as I work with these systems for a living.
I’m guessing some people don’t know (or forgot) that site-to-site and remote access VPN’s are a thing, and was the initial purpose of VPN’s. Masking or hiding your location became a benefit after the fact, and todays more common client VPN is technically a remote access VPN with a new purpose.
Remote access VPN’s are a very common attack vector for companies, look up companies compromised with Fortinet gear and its typically through the firewalls VPN.
Most modern remote access VPN’s do exactly that, so it is not antithetical at all and is how most client VPN’s keep you from accessing other users data. I would encourage you to read up on WireGuard and the like, they are fun to learn about and awesome tools when configured properly.
Also, we removed the above comment because the last sentence was fairly rude and violates rule 3 @sp3ctr4l@lemmy.dbzer0.com