• Mateslab
    • About
  • Team
  • Projects
    • Security awareness>
      • Global statistics scan
      • Bluedriving
      • Unanomaly
      • Darm. Sniffer in python
    • Social projects>
      • La videoquemadora (Copyleft)
      • Videos Translation
    • Arduino related projects>
      • Toti robot
      • Home-made sensor Arduino shield
      • Propeller driven car with Arduino
      • AM radio broadcasting with arduino
      • Home-made gsm-controlled weather station
      • Brushless motor based arduino boat
      • Keypad
      • Home-made game console
      • Pepi Robot
      • Theremin
      • Xylophone
    • Software related projects>
      • Domain Analyzer: Security Domain Analysis Tool
      • Web Crawler Security Tool
      • Darm. Sniffer in python
      • dnmap, the distributed nmap
      • blueblue, distributing files via bluetooth!
      • NetPlotter
      • Unanomaly
      • SnUp!
      • Bluedriving
      • CuiCui
      • xBasic
      • Python-easy-api
      • MorseTalk
      • Steganoroute
      • ShiOSan
      • OIP for OS X
    • Hardware hacking>
      • Charging motorola phones with a solar cell
      • Home-made battery-less AM radio receiver
      • DIY Microscope for mobile phones
    • Wireless related projects>
      • Analysis of the state of wireless networks>
        • Wardriving Mar del Plata (Argentina)
        • Wardriving Tandil (Argentina)
        • Wardriving Ciudad Autónoma de Buenos Aires (Argentina)
      • Ekoparty2011
      • Ekoparty2012
      • Biquad wireless antenna creation
      • PirateBox
    • Lockpicking>
      • Lockpicking village
    • Darkturito
    • hackaton
    • Courses>
      • curso-pen-test
      • BioHacking
    • Crafts>
      • Homemade coffee table with recycled wooden box
  • Blog
  • Files
  • Multimedia
    • Photos>
      • Biohacking (Mar 2013)
      • FLISoL 2012
      • Hackaton
      • MatesLab: meetings
      • MatesLab: HQ inauguration
      • MatesLab: the beginning
    • Videos
  • Education
  • Contact
    • Blackout. STOP CISPA!
  • Donate!

The global statistics scan

This project born some time ago in a join meeting between mateslab members and a security company that was looking for a methodological way of legally obtaining statistics about services open to Internet. They needed to find out how many of certain types of services were up and running around the globe.

Goal

The goal of this project is to legally find out the amount of hosts that have a certain group of ports open.

The scanning will be service-oriented, meaning that we only look for one group of services at a time. We will not indiscriminately scan every open port.

Challenges

The challenge of the project was not to scan ports, but to scan a very large group of hosts in a timely and accurate manner. We had to design the methodology to properly generate and control the scans done and the results achieved.
Also, the scans must be legal in the countries were they are done. This means that only SYN packets were sent and only to those countries were this is not illegal.

Methodology proposed

We proposed to use a methodology that takes advantage of our hackspace group and speeds up the scanning process. In order to control the process and to have a good feeling about how we were going, we started with only one country.

  1. Find out the networks assigned to one country. In our case, they were more than 46,309,640 subnets.
  2. Find out the subnets that were worth scanning. To reduce the amount of networks to scan.
  3. Find out the active hosts in those networks
  4. Scan the specified ports in those hosts.
  5. Verify the performance metrics, that is, try to reduce the false positives.


Step 1 is straightforward using some web sites online. 

Step 2, subnets worth scanning

To find out which subnets were worth scanning, we developed a novel technique we called NetPing. Its purpose its to find out if some network has active hosts inside or not without scanning them all. It can be considered analogous to the techniques used to find out if one host is active.
In a network with a /24 CIDR, we propose to search for some small amount of hosts. The main idea is that in any /24 network, the first hosts that are assigned are those.

To support this idea we conducted a quick 5 day test to verify this assumption. We randomly scanned 4350 networks (/24). Results are:
  • Total nets up: 1169 (26,87% of the total networks scanned)
  • Total nets with our hosts up: 929 (79% of the total nets up, and 21.35% of the total networks scanned)


The top 5 hosts assigned are:
  1. Host x.x.x.1: is up in 610 networks
  2. Host x.x.x.129: is up in 488 networks
  3. Host x.x.x.65: is up in 438 networks
  4. Host x.x.x.17: is up in 413 networks
  5. Host x.x.x.41 and x.x.x.254: are up in 425 networks


In conclusion, if you search only for the hosts 1,129,65,17,41 and 254 being active in a network, you will find out at least the 80% of the networks. 
This is a huge improvement for the 80% of the networks.

Step 3 and 4, find hosts up and port scan

These steps are a common nmap command. Nmap first find out if the host is up and later it scan it.
We have to use some more options in the nmap command. They were:
--open, to only store the open ports.
--reason, to help the verification process.
--max-rtt-timetout 800ms, this was because some networks with a large amount of hosts down, or hosts with a large amount of ports filtered make nmap spend more time scanning. We do not want to spend a lot of time on this, but we want to have statistically meaningful results.

dnmap

It quickly become clear that one machine alone with not suffice to finish the project in time. And trying to use several nmaps at the same time would be very difficult to coordinate. That is why we created a tool called dnmap, it is a distributed nmap framework that really helped us with this.
Check out its own web page: dnmap

Experiments

We conducted experiments during almost 2 weeks to finish our job. We had to learn a lot, fix bugs and have fun, but finally we manage to finish one country.
We still have to add more features to the dnmap, and create a automated analysis tool, but that can wait now.

Results

Results will be published here near the 18 of March. Keep tuned!

Powered by
✕