DNS status request

OK, so there is a general query you can make of a DNS service. It's called a status request. Type 0 requests are standard queries, requests to ask what is the ip-address of "www.somecompany.com". Type 1 requests are reverse lookups such as what is the domain-name of "131.181.6.130"? Type 2 requests are "What is your current status?" That's just what we want, because it works with any DNS server, and it doesn't require any actual lookups, which might get delegated to other servers.

A DNS status query is easy to construct, but it has to be sent via UDP. So far all the challenges I have been sending are via TCP. So it wasn't responding, even though the DNS server is supposed to be listening on UDP and TCP.

The challenge turns out to be just:

\25\14\20\0\0\0\0\0\0\0\0\0

The first two bytes are insignificant, just a unique ID generated by the client. The next byte holds the status request command, and everything else has to be 0.

Generating reverse dns queries

An obvious way of getting a response time out of any dns server is to ask it to perform a reverse dns query on itself. The format of a dns query is documented elsewhere, and a reverse dns query can be generated, captured using tcpdump and analysed in wireshark easily enough. The trick is to generate the dns query programmatically since a fixed string won't work equally well for all dns servers. What we don't want to do is ask a server for information it must get from a remote service. This will vary too much, whereas the objective is to measure the responsiveness of the service itself.

It might be possible to make an irregular query, one that doesn't require it to look up anything. That way the string could be fixed. I'll investigate on Wednesday.

telnet and similar services on Ubuntu

I wanted to test a telnet service on Ubuntu. Telnet is an insecure terminal service that has largely been superseded by ssh. However, I wanted to add it to my collection of services I could measure responsiveness on. But, boy was that a lot harder than I thought! Ubuntu already installs a telnet client for you. But in order to get the telnet service running you have to install inetd, the super-service, which then spawns telnet when you ask for it, based on the inetd.conf file in /etc/inetd.conf. In fact the inetd 'superserver' spawns other services too, not just telnet. So it will be needed for lots of other services I want to test. Cool, so I entered a line in inetd.conf thus:

telnet  stream  tcp  nowait  root  /usr/sbin/telnetd  telnetd

The fields are supposed to represent service-name, socket-type, protocol name, wait/nowait, username, serverpath server-args. Now if I run the inetd service I can indeed log into localhost using telnet, but not from another machine on another interface. Furthermore, there is no way to specify that the telnetd use another interface. After trying everything I could think of the solution was quite simple: install xinetd instead of old inetd, and hey presto, it all worked.

Stress testing pf versus iptables

We wanted to know whether pf or iptables was faster at implementing a white or blacklist of ip-addresses. How can you generate packets on one computer pretending to be from a wide range of ip-addresses (i.e. spoofed addresses) and then detect and measure the receipt of them on another computer? A third machine running pf or iptables would need to be in the middle to act as a router/firewall. In fact generating spoofed packets can be achieved with a number of tools. We chose hping3, since it does this but also has a listen function that examines packets with a signature and then prints out the packet contents after this signature. So if we send 10,000 spoofed packets as fast as possible from machine A to machine C via machine B (running pf/iptables) then they should all be dropped if the spoofed address is on the black list. Each such lookup in the black list on machine B will take some CPU time, and the effectiveness of that lookup in pf versus iptables is what we were hoping to measure. The only remaining problem was how to generate a mixture of black/white-listed addresses and others that were definitely not on the list. The lists can be generated by a simple perl script, and by adjusting it we managed to also generate a shell script containing a long series of hping3 commands. We then set up hping3 on machine C and pipe the resulting output to a file. Counting the number of words in the file tells us how many packets made it through the firewall. If the flood of packets overwhelmed the router as we hoped then more will be dropped by the more inefficient firewall.

Computing Number of Threads for a Process

Someone suggested that a good way to measure DDoS might be to monitor the number of threads in a process. Under attack, the process of a service might react by creating more threads to handle the new connections. But, how to do it? Someone suggested that a command like:

ps uH p `ps -C <appname> -o pid=` | wc -l

would do it, where '<appname>' is the name of the process being queried. The bit inside the backticks merely retrieves the pid of the named process. It doesn't have to be the precise name – but it does have to be a unique substring of that process's name. The 'wc -l' bit counts the number of lines in the output of ps, which corresponds to the number of threads. Using the same technique as for the current CPU and memory usage it is easy to retrieve the current number of threads from any process. First set the current app name:

snmpset -v2c -c public localhost DOSTF-MIB::dosTFCurrentAppName.0 = firefox

Then, issue an snmpget on the number of threads for the current application:

snmpget -v2c -c public localhost DOSTF-MIB::dosTFCurrentAppThreadCount.0

And you should get the number of threads printed out:

DOSTF-MIB::dosTFCurrentAppThreadCount.0 = INTEGER: 13

Reading the output of system commands in C

The system command in the standard C library lets you issue a commandline command and receive a return code. But that's not usually what you want. Let's say I wanted to issue a ps command to tell me which processes I was running, then how would I know the result? Actually there is a way, because the output of the command goes to stdout, and it is possible to redirect the output to a real file. For example:

FILE *console = freopen( "console", "w+", stdout );

redirects all standard output to a file called "console". After issuing the command:

ps aux | awk '/firefox/ && !/awk/ {print $4}'

console will contain the percentage of memory used by the 'firefox' process, if it was initiated by me. Then to read the result all you need to do is say:

rewind( console );
res = fscanf( console, "%f", &percent );
fclose( console );

and the float 'percent' will contain the percentage of the memory being used by Firefox. Not very efficient perhaps, but timing it reveals on my system that it only takes 10 milliseconds to execute, which is quick enough. And the information is precious - I can't find it out any other way (that's relatively platform-independent), and it tells me how much memory is being consumed by that particular process whose PID I don't even know.

So what's next?

So now I can get via SNMP the percent CPU usage and percent memory usage by a particular process every second. Next has to be the throughput or goodput of a particular port on a particular interface. This will be much harder and may be impossible.

Implementing the SET operation in a MIB

I wanted to set the name of the current application being monitored and then use snmpget on values that depend on that setting. I realise this is not quite the snmp way but it's just what I need to monitor a particular application's current state. That way I can retrieve for example the current application's memory and CPU usage as a percentage value. The snmp tutorial isn't very forthcoming about the details of how to do this. In particular it doesn't tell you where the commandline oid and value are located. All you get are four parameters:

1. netsnmp_mib_handler *handler
2. netsnmp_handler_registration *reginfo
3. netsnmp_agent_request_info *reqinfo
4. netsnmp_request_info *requests

Try searching the net-snmp site for information about these structures. I couldn't find anything. After some experimentation I managed to work out that the value of an snmpset command is stored in requests->requestvb->val.string and its length in requests->requestvb->val_len. The OID is stored in requests->requestvb->name (an array of ints) and its length in requests->requestvb->name_length (consistent huh?). I ignored most of the SET states (MODE_SET_RESERVE1, MODE_SET_RESERVE2, MODE_SET_FREE, MODE_SET_ACTION and MODE_SET_UNDO), and just provided code for MODE_SET_COMMIT. (The other states are for allocating and deallocating temporary memory). The storage for the string is just a global variable in the module. I just declared it as

static char currentAppName[APP_NAME_LEN];

where APP_NAME_LEN is 128. On receipt of a MODE_SET_COMMIT command I just copy the value at requests->requestvb->val.string into currentAppName. Of course I put a check in to stop more than 128 bytes being copied. I'm not that dumb.

What's next?

The next task is to compute currentAppCPUUsage and currentAppMemoryUsage based on the currentAppName value.