Tag: icmp

A 2D Heat Map for an ISP Network Operator’s IP address usage, and host ping latency

I wish to graphically visualize how large blocks of IP addresses are utilized by their network operator, and gain some insight into network “areas” with latency or possibly network health issues. This 2D ping latency heat map, relies upon a way to quickly ping a large range of IPs. In another POST, I provided a fast, highly concurrent icmp ping python application, for quickly pinging large IP blocks. I use an AWS US-WEST-2 EC2 server to ping the IP blocks below, and create the 2D ping latency heat maps.

The python visualization code is here , in GITHUB.

If one thinks about consecutive IP addresses, as being “close” to each other; I’d like to map a block of sequential IPs, such that consecutive IPs would be contiguously grouped “closely” into a new x-y cartesian space. In 2006 the XKCD webcomic first mapped the IPv4 internet to a 2D space, using hilbert curve fractal mapping.

The following IP heat map is a ping latency visualization, are for the 188.114.96.0/20 network. All 4096 IPs in this network, are either unpingable and marked as white space, or their ping latency times are in a range of colors. The faster ping times are dark blue in color, and ping times greater than 350 milliseconds are brighter red. The tick marks are in increments of 16 and a 16×16 area is a /24 subnet.

A single .png file, 2D ping latency heat map

This view might let you know; not much of the /20 subnet is pingable, or in-use. This snapshot of ping latency times, taken at 05-26-2020 05:04:01 UTC, was likely during a period of high network usage.

An animated GIF view, of multiple 2D ping latency heat map snapshots

The following GIF is an animation of 82 hourly ping latency png file snapshots. It appears there are 1-3 hourly periods during the day, where groupings of ping latency times jump from ~ 170 msec to ~400 msec . Ping times can be a proxy for network capacity. This view could suggest; network links / routers supporting groups of servers, are heavily loaded during peak usage times. Ping times are often a reflection of how-busy each router’s control plane is, or if the router control plane provides resources to icmp functionality. This post references a BFD protocol parser, that may provide better data insights into network link capacity via BFD timing.

The following is a gallery of interesting CDN server subnets:

I would also like to reference, a few other IP mapping efforts …

  • Open source Python mapping code I borrowed. Thank you Professor John Burkardt
  • Antonios Chariton @DaknObCS and his mapping of 36C3
  • Ben Cox @Benjojo12 tweets and blog posts, initially got me jazzed about BGP things & IP mapping references.
  • The ANT team and their internet mapping work

Raspberry Pi python test script, for ping and DNS reverse lookup

I have an intermittent DSL internet connectivity issue and wish to document the brief outages. I wrote a python 3 network connectivity test script for a Raspberry Pi (RPi model B Plus Rev 1.3), which is connected directly to my home router via ethernet. The script tested DSL connectivity by pinging another customer’s host IP. While at it, the script tested the ISP’s DNS‘ ability to reverse lookup the IPs FQDN (fully qualified domain name). My ISP has DNS issues quite often, and the DNS check has been a better “internet” functional test. During the DNS issues, I have IP network connectivity, but DNS availability breaks all applications.

The script pings an IP, and also asks the DNS to reverse lookup that IP for its FQDN. Since the RPi cron job is only granular to 1 minute, the script repeats pings and DNS lookups every second, for a minute. The githhub document notes how to cron the job. For RPi specific cron details: https://www.raspberrypi.org/documentation/linux/usage/cron.md

For access to the Python 3 script / code: https://github.com/jearlcalkins/ping-dns-logger-rpi

This is a nice tutorial, on setting-up an RPi for headless and ethernet configuration: https://learn.sparkfun.com/tutorials/headless-raspberry-pi-setup/ethernet-with-static-ip-address

fast python concurrent ping scan

I wish to ping a large /16 (65536 IPs) network space for pingable hosts, in a short period of time. I’ve provided a fast python concurrent ping application, as a tool for finding large network IP blocks with:

  • servers having patterns of slow ping latency times and possibly peak usage
  • network outages
  • networks handing-out IPs with DHCP, and disconnecting clients
  • networks and ISPs, well-utilizing their IP allocations

The ping wiki is a nice explanation of how ping works, but pinging every IP in a /16 (65536) is prohibitively slow. The GIT python code “ping” code avoids serially pinging each device, but sends a burst of ping echo commands to a block of 128 IPs. The application then recovers the ping echo packets, from its network packet capture. This concurrent ping mechanism, can ping an entire /20 IP block (4096 IPs) in 41 seconds. This ping application timing came from an AWS-EC2 t2-tiny AMI linux client.

The “ping an entire /16 (65536 IP) network quickly” use-case, is a great concurrency GOLANG problem. On the other hand, I need the python practice.

Fast Python Concurrent Ping – Theory of Operations

The python application, starts by turning-on a tcpdump capture, then sends a single-threaded burst of X sequential ICMP echo request packets to a small block of 128 IPs. After the last ICMP in the burst is sent, the app lets tcpdump run for 1 more second. The tcpdump pcap file is then parsed for ICMP responses, and the “ping time” is recovered and logged. The application iterates and repeats the burst-send and pcap listen process, until the entire network space has been pinged.

This blast mechanism of pinging a small block of 128 IPs, and recovering ICMP echo responses in a parallel network capture, decouples the application from maintaining 128 ping threads. The end-point IPs in the block, simply respond with an ICMP echo response concurrently.