I ran into a few problems recently trying to make a tool for the Recurse center, and in doing so, I learned a bit about networks – stuff I might have heard before but I’ve never had to wade through in a significant way. Until now, it basically boils down to probing the local network, and starts with the Net Mask

Finding all devices on the network

My first task is to find all the tasks on the network. This began with the local network; at home I had something like 192.168.1.0/24 and here I’ve got a “ten dot”, 10.0.0.0/16. So, turns out the number following the slash is the net mask which helps us figure out how many possible addresses could be on a network. But first we have to look at the IP address as something well, parsable:

Four Bytes

An IP address is four bytes, four times eight bits, seperated by periods. So an eight bit number can only get to 256, or 2 to the power of eight. These four bytes, or 32 bits, can be separated into two uses, kind of like a mac address. The first part identifies the local network and is common to all connections, and the second part is a unique address - the range of possible unique addresses (and it is here that was interesting in my use case), is found by subtracting the total bits by the net mask. That is to say a ten dot network, with net mask 16, has 16 bits reserved for unique addresses – the 24 net mask only has eight bits reserved for addresses. So the /16 net mask is a bigger one.

Broadcast

My first reaction to find all connected devices was to ping every possible address and wait for a response without error. This has a couple problems. First of all, on a 16 bit net mask, like the Recurse Center, this can take a while to ping all possible address (that is 255 * 255). Second, and more importantly, not all devices are going to respond to a ping, and even if they do, it is not garanteed that the response will make it back to my device. Turns out the network isn’t so infallible.

So the solution is to instead fire off one ping to the broadcast address; this is the top most address on a network, say 192.168.1.255 or at RC 10.0.255.255. These addresses are reserved for no one machine, but for broadcasting to everyone. Ping this (including a -b flag, and atleast two times -c2) and when on a network like here, you’ll get back dozens of pings and not so few duplicates too (luckily ping handles these nicely. Then the question is simply of taking the ip address after ...from: and before the first :, no regex required.

But the same problem remains: ping is not a sure way of getting all the devices on a network (althought it’s a nice approximation). The best and next step is arp.

ARP Protocol

The arp protocol is a low level packet used in networks to find a device’s mac address. There on your computer, if it’s linux, ought to a file called /proc/net/arp, or something rather, which has what we call an arp table. After broadcast ping it’ll be filled up with the devices which got back to you, and their mac addres. So using Go, and a snazzy library at github.com/mostlygeek/arp, one can make a map from the table and identify any given mac address with it’s ip as a key. (This takes us one step closer to quantifying what kind of devices are in the space, and how many have glowing apples on them.

The real trick, however, to finding all the devices connected is by using an arp broadcast. Unlike the ping, this packet is sent through the router to all connected devices, and it asks one question:

Who Has 192.168.1.133?

Or some other address.

All devices then reply kindly with a media access control address, or mac address:

I do! And my mac address is: 8c:70:5a:27:af:44

That is, their unique device identifier, starting some Vendor info and ending with their personal id, all in six groups of two hexadecimal digits, octets. These numbers are in base16, ending in F (16). The broadcast type mac will be the max: FF:FF:FF:FF:FF:FF.

But more importantly, for me, is that the devices reply and confirm their existence, in a way more assured than simply pinging the broadcast.

OUI database

So Mac addresses are registered with the IEEE, and put into a handy database for the querying (either in txt or csv) here. There are a few different libraries for working with this database, I used a Go one, but there are versions for most other languages. Using this, one can look up a Vendor from the Mac address – effectively, however, I only got it to recognize Apple Inc., which although common, it isn’t the only device we’ve got hanging around RC.