Scan the whole internet while drinking coffee
Netz 🚀
Netz lets you run internet-wide misconfigurations research easily with all the things you need for that research continuously. It contains the infrastructure-as-code as part of it so you can put your plan in a config file, run the CLI, and then wait for results. It also contains some more advanced features like making the scan BLAZING fast by using the PF_RING ZC (Zero Copy) kernel module.
What Netz gives you in a nutshell:
- Massive scan for internal and the public internet
- Expands your penetration testing toolbox easily with one CLI tool
- Easy setup to PF_RING ZC (Zero Copy) kernel module
- A tool both for Red Team & Blue Team
It’s Fully Open Source!
You can find the project here Netz, and you are more than welcome to jump into it and make it even better!💃
WIIFM ($> What’s in it for me?)
In this post, you will find the macro and the micro of internet misconfigurations problem, and you will learn how Netz helped us to understand the problem landscape as one of our research tools @ SpectralOps.
Now that Netz is open source— you can do such research as well!
Soon we will release more data from our research, and we will release more tools from our cybersecurity research arsenal.
The Common ways to Scan Scan Scan
The quickest way to detect a network asset that is publicly exposed to the internet is to use one of search engine services like Shodan, Censys or Zoomeye, and use the query language to see wide internet components. If you would like to do it manually instead of using an online service, or if you want to do a scan on internal networks, there are multiple ways to do it. The different types of scanning are between a small network or a big network, where a big network can scale up to the whole internet.
For scanning on a small scale, you can use the popular command nmap (which is available in the most advanced penetration testing distribution OS — kali). For larger-scale networks, the most popular are ZMap and masscan.
So how am I going to do it?
So, you are on a research mission, and you want to scan the whole internet (or some of it) — how are you going to use those tools? Great question!
If you will try to scan the whole internet with ZMap or masscan from your own computer, and you are curious about the results and can’t leave the computer, you should prepare some food for a long time because it’s going to take weeks (while risking the Netflix bandwidth for other family members in your house). Why is that? because those tools are aggressive by design to be able to scan the whole internet in a minimum of time. You could control the bandwidth capacity those tools are using and decrease it to be less aggressive, but then you will need even more food…
Improve the time to wait
To improve the time to wait, you can use a public cloud machine, but even then, the NIC (network interface controller) on a basic machine is limited for PPS (packets per second).
So what can you do? You can use a stronger machine with a stronger NIC and much more PPS, and then it will be much faster. How much faster? it depends on the machine type and the NIC type — better will be less time, but even that could take hours to days. The issue is that if you want to do a lot of tests in minimum time, even the strongest machine with the best NIC type is limited, so what now? We can use 2 different approaches to split the workload:
- Distribute the scan to multiple machines, each machine scanning a dedicated CIDR subnet, so for instance we can split 0.0.0.0/0 into 4 subnets, so each machine will scan its own subnet, and then merge the results.
- Use one strong machine with multiple NICs, and let the machine use the whole subnet.
Just network, or applicative scan as well?
Until now we talked about network scanning, but like I mentioned earlier if you would like to do more than just know what are the open ports and the metadata, and you want to do applicative actions against those ports while you scan, you need an application scanner. One of the most popular is ZGrab2.
Execution plan
So here is the execution plan workflow we want to achieve:
- Scan some subnet as fast as possible
- Pipe those IP/ports tuples to the applicative scanner to test various security issues in those ports
- Goto beach :)
Get more than just Shodan view
Here are a few examples for application security scanning:
- You want to test if an Elasticsearch server is not configured with admin access — we can test it by doing an HTTP call to `/_cat/indices`, and in case it returns 200 OK — that’s a problem.
- You want to test if a web server is configured by mistake with the whole ‘.git’ directory, so someone can read it and expose all the good stuff you have there. A simple HTTP call to `/.git`, and in case it returns 200 OK — that’s a problem.
- You want to test if Redis / Memcached / PostgreSQL / MySQL is configured without credentials — there are concrete applicative scanners in ZGrab2 so you can test it as well.
In any case of missing protocol or technology stack you want to add, ZGrab2 is pluggable, so you can just write one small function in Golang, and you easily extend the tool abilities.
Why should I look for internet-wide misconfigurations
I would like to start with a game: How would you feel if I told you that your home router is open to the internet for internal access..? Sounds a bit scary, but the first thing that pops into your head is that: do I have something I need to protect? Most of the time the answer will be — nahhh, nothing special. For those that answer nahhh, I will challenge you with the famous story of hackers breaching baby monitors, and I will let you imagine what happened next 😅
Now for a new challenge: How would you feel if I told you that your company’s internal data, or even worse — your company’s customers data — are open to the internet? If you feel your heartbeat start rising — congratulations! you passed the challenge successfully! You can now continue reading.
How do we get to that problem?
The world has changed: we are in the fifth wave of technology, and as usual, the world of software is moving first, while cyberspace solutions follow. Organizations are moving faster than ever to stay competitive, adding software automation in different areas, and adopting different tools and systems by all teams. Companies are moving fast thanks to the rise of the public cloud and the explosion of “Everything-as-a-Service’’, and the fact that everywhere, everything is being controlled by software automation.
Down to earth
Today almost any company in the world uses different 3rd party SAAS, PAAS, and IAAS. On top of those “As-A-Service” solutions are installed data-pipelines tools; data-science frameworks; different open-source projects; and DevOps infrastructure & observability tools. All these different solutions are using different configuration & secrets/credentials, including database connection strings; API keys; asymmetric keys; tokens; username/password combinations; admin, security, and privacy settings; and much much more. Even the infrastructure today is not how it used to be — the infrastructure is controlled by code, which again contains a lot of config types, and with it — more complexity. In some organizations, the complexity is even higher with multiple infrastructures in multiple public cloud vendors.
Ironically, these cutting-edge technologies allow us to move faster and faster to a better place, but at the same time — to a cyberspace risk. With such great power comes great responsibility: how do you make sure all of your company assets are secured?
So what now?
Now that we are on the same page, and the complexity is crystal clear, it’s easy to understand how misconfigurations or mishandling of any critical part of the supply chain could lead to a breach. The size of the mistakes is disproportionate to the damage, with one relatively small mistake that could cause a company to lose millions of dollars in the “best case”, or out of business in the worst case.
What is the meaning of the project name:: Netz
The word has a dual meaning, first, when you are saying this word, you said in Hebrew the word Hawk 🦅. Hawk is a predator that can scan the ground for meals from miles up in the sky, and then goes to catch it — which is an analogy for the way bad hackers act in the world: they are looking for open and free holes from miles away on the whole internet, and go to catch it. Second, the word contains Net which is the network scanning part, and Z which are the most critical security issue called Zero-day attacks.
Final Disclaimer
Our main drive in life is to make the world a better and safer place. If you would like to use this information to harm someone, you are doing the opposite, and at your own risk.
