Simple CVE stats from 2001-2011

Year 2001 has 1538 vulnerabilities
2001 had 8 vulns of type:  Buffer Errors
2001 had 4 vulns of type:  Cryptographic Issues
2001 had 4 vulns of type:  Path Traversal
2001 had 2 vulns of type:  Authentication Issues
2001 had 2 vulns of type:  Permissions, Privileges, and Access Control
2001 had 2 vulns of type:  Code Injection
2001 had 2 vulns of type:  Input Validation
2001 had 1 vulns of type:  Resource Management Errors
2001 had 1 vulns of type:  Link Following
2001 had 0 vulns of type:  Credentials Management
2001 had 0 vulns of type:  Cross-Site Request Forgery (CSRF)
2001 had 0 vulns of type:  Cross-Site Scripting
2001 had 0 vulns of type:  Format String Vulnerability
2001 had 0 vulns of type:  Configuration
2001 had 0 vulns of type:  Information Leak / Disclosure
2001 had 0 vulns of type:  Numeric Errors
2001 had 0 vulns of type:  OS Command Injections
2001 had 0 vulns of type:  Race Conditions
2001 had 0 vulns of type:  SQL Injection
Total vulns in 2001 with CWE: 26
Percentage of vulns with CWE: 1.69050715214564%


Year 2002 has 2368 vulnerabilities
2002 had 41 vulns of type:  Buffer Errors
2002 had 32 vulns of type:  Permissions, Privileges, and Access Control
2002 had 32 vulns of type:  Cross-Site Scripting
2002 had 29 vulns of type:  Input Validation
2002 had 17 vulns of type:  Information Leak / Disclosure
2002 had 13 vulns of type:  Path Traversal
2002 had 9 vulns of type:  Configuration
2002 had 8 vulns of type:  Credentials Management
2002 had 8 vulns of type:  Code Injection
2002 had 7 vulns of type:  SQL Injection
2002 had 6 vulns of type:  Numeric Errors
2002 had 6 vulns of type:  Resource Management Errors
2002 had 5 vulns of type:  Authentication Issues
2002 had 3 vulns of type:  Cryptographic Issues
2002 had 2 vulns of type:  Race Conditions
2002 had 2 vulns of type:  Link Following
2002 had 1 vulns of type:  Cross-Site Request Forgery (CSRF)
2002 had 1 vulns of type:  Format String Vulnerability
2002 had 1 vulns of type:  OS Command Injections
Total vulns in 2002 with CWE: 223
Percentage of vulns with CWE: 9.41722972972973%


Year 2003 has 1515 vulnerabilities
2003 had 59 vulns of type:  Buffer Errors
2003 had 40 vulns of type:  Cross-Site Scripting
2003 had 30 vulns of type:  Input Validation
2003 had 25 vulns of type:  Information Leak / Disclosure
2003 had 24 vulns of type:  Permissions, Privileges, and Access Control
2003 had 17 vulns of type:  Path Traversal
2003 had 13 vulns of type:  Code Injection
2003 had 12 vulns of type:  Configuration
2003 had 12 vulns of type:  SQL Injection
2003 had 9 vulns of type:  Authentication Issues
2003 had 9 vulns of type:  Credentials Management
2003 had 8 vulns of type:  Cryptographic Issues
2003 had 6 vulns of type:  Resource Management Errors
2003 had 4 vulns of type:  Numeric Errors
2003 had 2 vulns of type:  Format String Vulnerability
2003 had 2 vulns of type:  Race Conditions
2003 had 2 vulns of type:  Link Following
2003 had 0 vulns of type:  Cross-Site Request Forgery (CSRF)
2003 had 0 vulns of type:  OS Command Injections
Total vulns in 2003 with CWE: 274
Percentage of vulns with CWE: 18.0858085808581%


Year 2004 has 2669 vulnerabilities
2004 had 30 vulns of type:  Buffer Errors
2004 had 22 vulns of type:  Permissions, Privileges, and Access Control
2004 had 20 vulns of type:  Cross-Site Scripting
2004 had 9 vulns of type:  Path Traversal
2004 had 9 vulns of type:  Input Validation
2004 had 8 vulns of type:  SQL Injection
2004 had 6 vulns of type:  Authentication Issues
2004 had 6 vulns of type:  Credentials Management
2004 had 6 vulns of type:  Code Injection
2004 had 5 vulns of type:  Configuration
2004 had 4 vulns of type:  Information Leak / Disclosure
2004 had 4 vulns of type:  Resource Management Errors
2004 had 3 vulns of type:  Cryptographic Issues
2004 had 3 vulns of type:  Format String Vulnerability
2004 had 2 vulns of type:  Race Conditions
2004 had 2 vulns of type:  Link Following
2004 had 1 vulns of type:  Numeric Errors
2004 had 1 vulns of type:  OS Command Injections
2004 had 0 vulns of type:  Cross-Site Request Forgery (CSRF)
Total vulns in 2004 with CWE: 141
Percentage of vulns with CWE: 5.28287748220307%


Year 2005 has 4684 vulnerabilities
2005 had 64 vulns of type:  Buffer Errors
2005 had 48 vulns of type:  SQL Injection
2005 had 32 vulns of type:  Permissions, Privileges, and Access Control
2005 had 31 vulns of type:  Resource Management Errors
2005 had 28 vulns of type:  Cross-Site Scripting
2005 had 21 vulns of type:  Input Validation
2005 had 20 vulns of type:  Code Injection
2005 had 18 vulns of type:  Information Leak / Disclosure
2005 had 15 vulns of type:  Numeric Errors
2005 had 10 vulns of type:  Path Traversal
2005 had 5 vulns of type:  Link Following
2005 had 4 vulns of type:  Authentication Issues
2005 had 3 vulns of type:  Cryptographic Issues
2005 had 3 vulns of type:  Configuration
2005 had 2 vulns of type:  Credentials Management
2005 had 2 vulns of type:  Race Conditions
2005 had 1 vulns of type:  Cross-Site Request Forgery (CSRF)
2005 had 1 vulns of type:  Format String Vulnerability
2005 had 1 vulns of type:  OS Command Injections
Total vulns in 2005 with CWE: 309
Percentage of vulns with CWE: 6.59692570452605%


Year 2006 has 7043 vulnerabilities
2006 had 199 vulns of type:  Code Injection
2006 had 145 vulns of type:  Buffer Errors
2006 had 87 vulns of type:  Cross-Site Scripting
2006 had 84 vulns of type:  SQL Injection
2006 had 74 vulns of type:  Resource Management Errors
2006 had 63 vulns of type:  Input Validation
2006 had 50 vulns of type:  Permissions, Privileges, and Access Control
2006 had 37 vulns of type:  Numeric Errors
2006 had 29 vulns of type:  Information Leak / Disclosure
2006 had 21 vulns of type:  Path Traversal
2006 had 17 vulns of type:  Format String Vulnerability
2006 had 14 vulns of type:  Authentication Issues
2006 had 8 vulns of type:  Cryptographic Issues
2006 had 7 vulns of type:  Race Conditions
2006 had 6 vulns of type:  Configuration
2006 had 5 vulns of type:  Credentials Management
2006 had 3 vulns of type:  Cross-Site Request Forgery (CSRF)
2006 had 2 vulns of type:  OS Command Injections
2006 had 1 vulns of type:  Link Following
Total vulns in 2006 with CWE: 852
Percentage of vulns with CWE: 12.0971177055232%


Year 2007 has 6505 vulnerabilities
2007 had 451 vulns of type:  Buffer Errors
2007 had 366 vulns of type:  Cross-Site Scripting
2007 had 296 vulns of type:  Code Injection
2007 had 263 vulns of type:  SQL Injection
2007 had 229 vulns of type:  Permissions, Privileges, and Access Control
2007 had 228 vulns of type:  Input Validation
2007 had 164 vulns of type:  Path Traversal
2007 had 107 vulns of type:  Numeric Errors
2007 had 104 vulns of type:  Resource Management Errors
2007 had 96 vulns of type:  Information Leak / Disclosure
2007 had 69 vulns of type:  Authentication Issues
2007 had 41 vulns of type:  Cross-Site Request Forgery (CSRF)
2007 had 36 vulns of type:  Configuration
2007 had 31 vulns of type:  Format String Vulnerability
2007 had 25 vulns of type:  Link Following
2007 had 24 vulns of type:  Credentials Management
2007 had 19 vulns of type:  Cryptographic Issues
2007 had 18 vulns of type:  Race Conditions
2007 had 6 vulns of type:  OS Command Injections
Total vulns in 2007 with CWE: 2573
Percentage of vulns with CWE: 39.554189085319%


Year 2008 has 7031 vulnerabilities
2008 had 1480 vulns of type:  SQL Injection
2008 had 981 vulns of type:  Cross-Site Scripting
2008 had 582 vulns of type:  Buffer Errors
2008 had 574 vulns of type:  Permissions, Privileges, and Access Control
2008 had 467 vulns of type:  Input Validation
2008 had 447 vulns of type:  Path Traversal
2008 had 385 vulns of type:  Code Injection
2008 had 322 vulns of type:  Resource Management Errors
2008 had 222 vulns of type:  Authentication Issues
2008 had 221 vulns of type:  Information Leak / Disclosure
2008 had 177 vulns of type:  Link Following
2008 had 166 vulns of type:  Numeric Errors
2008 had 119 vulns of type:  Cross-Site Request Forgery (CSRF)
2008 had 69 vulns of type:  Credentials Management
2008 had 61 vulns of type:  Cryptographic Issues
2008 had 41 vulns of type:  Configuration
2008 had 33 vulns of type:  Format String Vulnerability
2008 had 25 vulns of type:  Race Conditions
2008 had 12 vulns of type:  OS Command Injections
Total vulns in 2008 with CWE: 6384
Percentage of vulns with CWE: 90.797895036268%


Year 2009 has 4848 vulnerabilities
2009 had 734 vulns of type:  Cross-Site Scripting
2009 had 673 vulns of type:  SQL Injection
2009 had 558 vulns of type:  Buffer Errors
2009 had 329 vulns of type:  Permissions, Privileges, and Access Control
2009 had 266 vulns of type:  Code Injection
2009 had 247 vulns of type:  Input Validation
2009 had 245 vulns of type:  Path Traversal
2009 had 237 vulns of type:  Resource Management Errors
2009 had 164 vulns of type:  Numeric Errors
2009 had 148 vulns of type:  Authentication Issues
2009 had 141 vulns of type:  Information Leak / Disclosure
2009 had 86 vulns of type:  Cryptographic Issues
2009 had 84 vulns of type:  Cross-Site Request Forgery (CSRF)
2009 had 56 vulns of type:  Credentials Management
2009 had 47 vulns of type:  Configuration
2009 had 32 vulns of type:  Race Conditions
2009 had 29 vulns of type:  Link Following
2009 had 23 vulns of type:  Format String Vulnerability
2009 had 11 vulns of type:  OS Command Injections
Total vulns in 2009 with CWE: 4110
Percentage of vulns with CWE: 84.7772277227723%


Year 2010 has 4696 vulnerabilities
2010 had 578 vulns of type:  SQL Injection
2010 had 566 vulns of type:  Cross-Site Scripting
2010 had 536 vulns of type:  Buffer Errors
2010 had 319 vulns of type:  Permissions, Privileges, and Access Control
2010 had 299 vulns of type:  Input Validation
2010 had 270 vulns of type:  Resource Management Errors
2010 had 256 vulns of type:  Path Traversal
2010 had 248 vulns of type:  Code Injection
2010 had 162 vulns of type:  Information Leak / Disclosure
2010 had 154 vulns of type:  Numeric Errors
2010 had 66 vulns of type:  Cross-Site Request Forgery (CSRF)
2010 had 62 vulns of type:  Cryptographic Issues
2010 had 56 vulns of type:  Authentication Issues
2010 had 51 vulns of type:  Credentials Management
2010 had 33 vulns of type:  Race Conditions
2010 had 26 vulns of type:  Link Following
2010 had 21 vulns of type:  Configuration
2010 had 12 vulns of type:  Format String Vulnerability
2010 had 12 vulns of type:  OS Command Injections
Total vulns in 2010 with CWE: 3727
Percentage of vulns with CWE: 79.3654173764906%


Year 2011 has 3733 vulnerabilities
2011 had 648 vulns of type:  Buffer Errors
2011 had 372 vulns of type:  Input Validation
2011 had 367 vulns of type:  Cross-Site Scripting
2011 had 366 vulns of type:  Resource Management Errors
2011 had 295 vulns of type:  Information Leak / Disclosure
2011 had 285 vulns of type:  Permissions, Privileges, and Access Control
2011 had 120 vulns of type:  Numeric Errors
2011 had 107 vulns of type:  SQL Injection
2011 had 92 vulns of type:  Code Injection
2011 had 91 vulns of type:  Path Traversal
2011 had 60 vulns of type:  Authentication Issues
2011 had 57 vulns of type:  Cross-Site Request Forgery (CSRF)
2011 had 57 vulns of type:  Cryptographic Issues
2011 had 34 vulns of type:  Configuration
2011 had 32 vulns of type:  Credentials Management
2011 had 26 vulns of type:  Link Following
2011 had 14 vulns of type:  Race Conditions
2011 had 13 vulns of type:  OS Command Injections
2011 had 8 vulns of type:  Format String Vulnerability
Total vulns in 2011 with CWE: 3044
Percentage of vulns with CWE: 81.5429949102599%
Total: 49439

To be honest, I am a bit dismayed at the quality of the data. 2001 only categorized 1.7% of the vulns recorded (I am sure most, if not all, were added retroactively). The highest percentage of vulns that had been categorized was 90% in 2008. I find it interesting that the first few years are dominated by buffer overflows (perhaps because of poor data), and then around 2008, web vulns become the top recorded and categorized. Perhaps this is because of the vast amount new web technologies emerging. That is, until 2011 where buffer overflows are once again the most. I used the xml files from the NIST and my source code that I used to generate the stats is on github. Using LINQ, so it isn't super speedy. Takes a few minutes. Works with Mono or .NET.

Communicating with your NeXpose server via Mono/.NET

I have a public repo on github that houses my nexpose-sharp library. It is written in C# and consumes the NeXpose XML API (both 1.1 and 1.2). Here is an example of how easy it is to get all the vuln checks NeXpose has:

using System;
using System.Xml;
using nexposesharp;

namespace nexposeclient
{
class MainClass
{
public static void Main (string[] args)
{
using (NexposeSession session = new NexposeSession("192.168.56.101"))
{
session.Authenticate("nexpose"/*user*/, "nexpose"/*password*/);

using (NexposeManager11 manager = new NexposeManager11(session))
{
XmlDocument vulns = manager.GetVulnerabilityListing();

int i = 0;
foreach (XmlNode vuln in vulns.FirstChild.ChildNodes)
{
string vulnID = vuln.Attributes["id"].Value;

XmlDocument deets = manager.GetVulnerabilityDetails(vulnID);

string title = deets.FirstChild.FirstChild.Attributes["title"].Value;
string severity = deets.FirstChild.FirstChild.Attributes["severity"].Value;

Console.WriteLine(String.Format("{0} has a severity of {1} and an id of {2}", title, severity, vulnID));

i++;
}

Console.WriteLine("\n\nTotal vulnerabilities in database: " + i);
}
}
}
}
}


The library has 2 manager implementations. The above example use the 1.1 API. A NexposeManager12 class exists that inherits from NexposeManager11 (available from NeXpose 4.0) and implements the extended 1.2 API (available for NeXpose installations of 4.8+). I am currently in the process of writing some unit tests, which will be committed as soon as possible.

You can grab a copy of NeXpose Community Edition today and try it out!

My NHibernate Configuration for Mono and PostgreSQL

I love C#. It is by far my favorite language. I have been using mono to write C# application on linux for a few years now.

Ruby is growing on me, but only because of Metasploit. I wouldn't even bother with Ruby if it weren't for MSF. Generally, I switch back and forth between C# and Ruby (and, coincidentally, Monodevelop and vim respectively) a few times a day, or even going at both at the same time.

One of my personal projects requires a lot of DB stuff. At a previous job, I was introduced to NHibernate, this is by far the easiest way to manage your DB objects within your code. While this job was Windows centric (MSSQL, visual studio, etc...), I have adapted what I learned to Linux as well. I like PostgreSQL more than I like MySQL, and am very happy that NHibernate supports this dialect. It wasn't straight forward, though, figuring out the exact details. Maybe this will help someone in the same boat I am in.

The most important thing is you hibernate.hbm.xml. This is where you SQL connection string is, and where you tell NHibernate what dialect of SQL you are using:

<?xml version="1.0" encoding="utf-8"?>
<hibernate-configuration  xmlns="urn:nhibernate-configuration-2.2" >
    <session-factory>
        <property name="connection.provider">NHibernate.Connection.DriverConnectionProvider</property>
        <property name="connection.driver_class">NHibernate.Driver.NpgsqlDriver</property>
        <property name="connection.connection_string">
            Server=192.168.1.156;Port=5432;Database=pgdb;User Id=postgres;Password=postgres;SSL=true;
        </property>
        <property name="dialect">NHibernate.Dialect.PostgreSQL82Dialect</property>
  <property name='proxyfactory.factory_class'>NHibernate.ByteCode.LinFu.ProxyFactoryFactory, NHibernate.ByteCode.LinFu</property>
    </session-factory>
</hibernate-configuration>

This requires Npgsql.dll, which runs with Mono just fine. The small detail that caused me much stress early on was the dialect property name.

<property name="dialect">NHibernate.Dialect.PostgreSQL82Dialect</property>

At first, I was missing the "82" in the middle of the dialect name. This defaults to PostgreSQL 7.4. I am running 8.4, and there were incompatibilities that caused issues. After a day or two, I finally figured out what was causing my problems.

I hope this helps others.

Non-trivial key names

It seems that you run into instances where someone working with the registry doesn't know quite how to use it. Well, let's start with some code first.

Say we have a regex to carve out the data we want:

Regex nk = new Regex(@"nk[\x2c|\x20]\x00.{7}\x01.{117}");

Chances are this data chunk will have a lot of junk at the end. Most key names as far as I can tell under < 65 characters long, but there are instances where a name legitimately runs longer than that. Here is one example:

It's not a root key!
Offset to Parent: 4145008
Number of Subkeys: 0
Offset to Subkey LF Blocks: 1061109567
Number of values: 1
Offset to value list: 4144959
Offset to security key: 7421704
Offset to classname: 1061109567
Offset to ?trash?: 1634757999
Name Length: 58 bytes
Classname Length: 0
Partial Name: C:|WINDOWS|Microsoft.NET|Framework|v3.5|AddInProcess3
Name: C:|WINDOWS|Microsoft.NET|Framework|v3.5|AddInProcess3

It's not a root key!
Offset to Parent: 4144959
Number of Subkeys: 0
Offset to Subkey LF Blocks: 1061109567
Number of values: 1
Offset to value list: 4144992
Offset to security key: 1061109592
Offset to classname: 4156278
Offset to ?trash?: 1231316033
Name Length: 20590 bytes
Classname Length: 28530
Partial Name: cess32,version="3.5.0.0",publicKeyToken="b77a5c561934

The first node key name has a length of 58 bytes. Pretty normal. But the second node key has a name 20590 bytes long. It also has to do with the .NET Framework. (*sigh* Microsoft...)

I can't carve out 20000 byte long chunks for each key node to satisfy the needs of names that shouldn't really be names, that would be crazy. It just so happened that this name was throwing an IndexOutOfrangeException. I decided I could use this to my advantage.

I could pick a sane number for the size of the regex that would get 90% of my key names and simply work around the longer names (in the short term at any rate).

My code ended up looking like this:


else if (i == (int)0x0048) //name length
{
byte[] lengthBytes = new byte[dword]; //should only be a word length. not sure why I need to make this dword

for (int k = 0;k<word;k++)
{
lengthBytes[k] = bs[i+k];
}

nameLength = BitConverter.ToInt32(lengthBytes, 0);

Console.WriteLine(String.Format("Name Length: {0} bytes", nameLength.ToString()));

i += word;
}

... //other else if's here

else if (i == (int)0x004C) //key name
{
int length = nameLength;

char[] blah = new char[length];

for (int k = 0; k < length;k++)
{
try
{
blah[k] = (char)bs[i+k];
}
catch(Exception ex)
{
//sometimes you get stupid long names (someone not know inghow the registry works)
//when this happens, we will just read what we can and return what we get. Doesn't have to
//be perfect since we can load the full name at a later time when the user clicks the list item
//for most names, this won't be needed.
if (ex.GetType() == typeof(IndexOutOfRangeException))
{
Console.WriteLine("Partial Name: " + new string(blah));

i += bs.Length - i;
k = length;

continue;
}
else
throw ex;

}
}

Console.WriteLine("Name: " + new string(blah));

i += bs.Length - i; //we are done.
}


I figure for a list of values, you won't be showing more than 100 or so characters until you pick the specific key out of the list and it loads the full details. At that point you can read the entire name and show it to the full user.

Windows Registry with Mono, pt2 - Node Keys

I have had a bit more time on my hands to work on being able to read the registry without using advapi32.dll. Today I was able to hack up a small (incomplete) class for node keys that builds the framework for breaking apart and manipulating the data.

To start off, from this file, we can get the offsets we need to read to get the right data.

the nk-Record
=============

Offset Size Contents
0x0000 Word ID: ASCII-"nk" = 0x6B6E
0x0002 Word for the root-key: 0x2C, otherwise 0x20
0x0004 Q-Word write-date/time in windows nt notation
0x0010 D-Word Offset of Owner/Parent key
0x0014 D-Word number of sub-Keys
0x001C D-Word Offset of the sub-key lf-Records
0x0024 D-Word number of values
0x0028 D-Word Offset of the Value-List
0x002C D-Word Offset of the sk-Record
0x0030 D-Word Offset of the Class-Name
0x0044 D-Word Unused (data-trash)
0x0048 Word name-length
0x004A Word class-name length
0x004C ???? key-name

It's pretty straight forward. In every fragment, we can go to specific offsets and get the data we want. This ends up looking like this:


public NodeKey (string data)
{
ASCIIEncoding enc = new ASCIIEncoding();

byte[] bs = enc.GetBytes(data);

//the lengths we will be working with.
int word = 2;
int dword = word+word; //double word
int qword = dword+dword; //quad word

for (int i = 0; i < bs.Length;)
{
//making sure it is nk
if (i == (int)0x0000) //header
{
if ((int)bs[0] == 110)
{
if ((int)bs[1] == 107)
{
i += word;
continue;
}
else
{
throw new Exception("This may be a damaged nk block. If so, fix the header and try again.");
}
}
else
{

throw new Exception("Not a nk");
}
}

else if (i == (int)0x0002) //is it a root key?
{
if (bs[i] == (byte)0x2C)
{
//It's a root key!
Console.WriteLine("It's a root key!");

}
else
{
//it's not a root key!
Console.WriteLine("It's not a root key!");
}

i += word; //move up 2 elements
continue;
}

else if (i == (int)0x0004) //timestamp in long smb form blegh
{
byte[] blah = new byte[qword];
for (int k = 0;k<qword;k++)
{
blah[k] = bs[i+k];
}

i+= qword;
}
else if (i == (int)0x0010) //offset to parent
{
i += dword;
}
else if (i == (int)0x0014) //number of subkeys
{
i += dword;
}
else if (i == (int)0x001C) //offset to subkey lf blocks
{
i += dword;
}
else if (i == (int)0x0024) //number of values
{
i += dword;
}
else if (i == (int)0x0028) //offset of value list
{
i += dword;
}
else if (i == (int)0x002C) //offset to the sk block
{
i += dword;
}
else if (i == (int)0x0030) //offset to classname
{
i += dword;
}
else if (i == (int)0x0044) //this is trash supposedly
{
i += dword;
}
else if (i == (int)0x0048) //name length
{
i += word;
}
else if (i == (int)0x004A) //class name length
{
i += word;
}
else if (i == (int)0x004C) //key name
{
int length = bs.Length - i;

char[] blah = new char[length];

for (int k = 0; k < length;k++)
{
blah[k] = (char)bs[i+k];
}

Console.WriteLine(blah);

i += length; //we are done.
}
else i+= word; //debugging purposes
}
}



If you notice, however, my code is not complete. I am starting with the most useful stuff first and moving on that way. A more complete class will keep the key name length in a local variable and use that instead of bs.Length when reading the key name later. With the current implementation, I read in too many bytes and grab some extra key headers :-/. You could create properties that are privately set and publicly get'able and set the properties to their respective values, to make it truly object oriented.

Another thing to point out is i is being incremented by the length read each time. It isn't arbitrary. This way next go around we are at the offset we need to be at.

One thing I look forward to implementing is lazy loading of parents and children. If you would like to test this, class, you can see my previous post on initially reading and deciphering the windows registry in C#. Just use this in your for loop instead:


foreach (Match mx in nk.Matches (d)) {

all++;
NodeKey key = new NodeKey(mx.Value);
}


Analyzing the Windows NT registry without advapi32.dll using Mono (PoC)

I have been doing some challenges for a contest and one requires analyzing a set of Windows NT registry hives. Regedit really sucks (though it does run in wine). I decided it would be more fun to write a small library that can read the registry hives without relying on p/invoke and advapi32.dll on Windows. I have some small code that carves out the data I need, though I am running into a problem on the software hive supplied. Maybe someone can point me in the right direction.

A lot of my information came from this text file which I found, and have updated some with information that I found missing.

As far as I can tell, there are 6 data types to be carved out of the hives. regf file headers, hbin blocks, node keys, value keys, and lf/h (lh on XP) blocks. There are also security keys (with a sk header) within node keys. The following regex's should carve out the data from the registry files so you may parse out the information you need.

Regex regf = new Regex (@"^regf.{508}");
Regex nk = new Regex (@"nk[\x2c|\x20]\x00.{7}\x01.{64}");
Regex vk = new Regex (@"vk.{3}\x00\x00[\x00|\x80].{64}");
Regex hbin = new Regex (@"hbin.{4}\x00\x10\x00\x00.{8}");
Regex lf = new Regex (@".{4}l[f|h][0-65535].{8}"); //lf or lh on winxp

But in order to search the hive, we need to read it in. This isn't very efficient, and I am aware of this. It works.

using (FileStream fs = File.OpenRead (path)) {
 var data = new byte[checked((int)fs.Length)];
 int i = 0;
 int read;
     
 using (var ms = new MemoryStream (checked((int)fs.Length))) {
      
  while ((read = fs.Read (data, 0, data.Length)) > 0) {
   ms.Write (data, 0, read);
   i += read;
  }
      
  byte[] hive = ms.ToArray ();
  char[] cList = new char[fs.Length];
      
  i = 0;
  foreach (byte b in hive)
   cList[i++] = (char)b;
      
         string d = new string (cList);
 
      
  int all = 0;
      
  foreach (Match mx in lf.Matches (d)) { //you can change out the regex you want here.
   byte[] bb = new byte[mx.Value.Length];
   char[] cb = new char[mx.Value.Length];
       
   for (int k = 0; k < mx.Value.Length; k++) {
    bb[k] = (byte)mx.Value[k];
    cb[k] = (char)bb[k];
    
   }
   
   all++;
   
   //Console.WriteLine (new string (cb));
  }
  
  Console.WriteLine (all.ToString ());
  all = 0;
 }
}

Basically, we read in the hive into a MemoryStream, convert the stream into a byte array, move that into a char array from which we create a string to search for the regexs in. Yes, we store 4 copies of the registry in memory. I am sure there are better ways to do this.

Then we loop through each match and count them. Of course we are working with binary streams, so if you choose to write the data carved out to the console, it will look like random data (to the untrained eye at least).

Running through all the hives supplied, I get this output:

/home/bperry/SAM
nk[\x2c|\x20]\x00.{7}\x01.{64}
47
.{4}l[f|h][0-65535].{8}
0
vk.{3}\x00\x00[\x00|\x80].{64}
36
hbin.{4}\x00\x10\x00\x00.{8}
6
^regf.{508}
1

/home/bperry/software
nk[\x2c|\x20]\x00.{7}\x01.{64}
43147
.{4}l[f|h][0-65535].{8}
6
vk.{3}\x00\x00[\x00|\x80].{64}
54708
hbin.{4}\x00\x10\x00\x00.{8}
2917
^regf.{508}
0

/home/bperry/system
nk[\x2c|\x20]\x00.{7}\x01.{64}
11189
.{4}l[f|h][0-65535].{8}
4
vk.{3}\x00\x00[\x00|\x80].{64}
21926
hbin.{4}\x00\x10\x00\x00.{8}
1121
^regf.{508}
1

/home/bperry/default
nk[\x2c|\x20]\x00.{7}\x01.{64}
554
.{4}l[f|h][0-65535].{8}
0
vk.{3}\x00\x00[\x00|\x80].{64}
1014
hbin.{4}\x00\x10\x00\x00.{8}
58
^regf.{508}
1

/home/bperry/SECURITY
nk[\x2c|\x20]\x00.{7}\x01.{64}
220
.{4}l[f|h][0-65535].{8}
0
vk.{3}\x00\x00[\x00|\x80].{64}
147
hbin.{4}\x00\x10\x00\x00.{8}
10
^regf.{508}
1

The number printed after the regex is the number of matches found. The data is fully carved out, so the only thing left is to break it apart to get the relevant data. If you will notice however, software reports 0 regf file headers, and I cannot figure out why. Any thoughts?

Odd Math.Round() behaviour in Mono (same as .NET?)

I had read on a forum somewhere that Math.Round() rounded to the closest even number. I mentioned this to the guys in #mosa and one mentioned that seems like it would be a bug, as it should go from 4.5 -> 5, rather than the (supposedly) expected 4.5 -> 4, with 4 being the closest even number. Another example is 9.5 -> 10 because 10 is the closest even number (9 is odd).

I wrote a simple method to test this.

using System;

namespace round_test
{
 class MainClass
 {
  public static void Main (string[] args)
  {
   double x = 0d;
   
   while (x < 5d)
   {
    Console.WriteLine("actual: " + x.ToString());
    Console.WriteLine("rounded: " + Math.Round(x).ToString());
    
    x += 0.1d;
   }
  }
 }
}

The output using mono wasn't very consistent. For instance, 2.5 -> 3 while the rest of the n.5 -> closest even number to n (4.5 -> 4, 3.5 -> 4). (full output here). Odd behaviour, bug??


EDIT: It seems there are two types of rounding, explained on MSDN. The default is banker's rounding.

Maths, pt1 and other news

I recently found a really awesome project, MOSA (Managed Operating System Alliance). I haven't had more fun hacking and breaking code in a long time. It's just really neat being able to write your operating system in C#.

One of my projects is building an operating system that performs floating-point arithmetic and fast fourier transforms as kind of a benchmark for the operating system/Ahead-Of-Time compiler. A few years ago, I ported John Walker's FBENCH to C# and I thought this would be an excellent candidate for the task. He also has a benchmark, FFBENCH (Fast-Fourier Transforms) which I plan on porting in the next few days as well.

The MOSA project, however, is quite young compared to other projects like it. It isn't very complete at all and isn't really useful yet. I plan on helping out with this a bit. For instance, I took the trig functions John Walker uses in FBENCH (he defined all the trig functions in case you didn't want to use math.h) and moved them over to C#. It wasn't terribly difficult, just a bit tedious. If you would like a copy of these methods, you may get them here. If all goes well and according to plan, these methods will go into Korlib, the core library MOSA uses for the OS.

I will be making a post in the next few days regarding the ports of the two math benchmarks to C#, and maybe even an image of my OS that can run in QEMU! A new release had been made of the benchmarks since I ported last, so this morning was spent porting the new FBENCH to C# and it is working quite dandily (is that a word?). FFBENCH should be even easier. There are a few tests I want to run regarding these benchmarks. Speed of Mono vs .NET arithmetically and the speed of using System.Math vs my methods. Hopefully I have some good results to show soon.


Also, in other news, the OpenVAS Build Repo has added a 10.10 repository for Ubuntu. I have been testing it on my network here with virtual machines strewn about the house and everything seems dandy! I highly recommend you check it out if you are interested. As soon as I get a bit more free time, I will be releasing two virtual appliances updated to run the od-autoassess script (x86 and x86_64) on Maverick with all the new features I have implemented in the script.

Adding the repository is easy, and you can follow my guide here (updated yesterday) to get a new VM up and running. If you find any problems, let me know!

Counting words while watching a video

So, a friend of mine asked for some help on a statistics assignment in which he counted the amount of times any given word was said during a movie. He asked if I could write up an app that would help him do this fairly easily, so I said sure.

It took me about an hour and a half to get it written and the bugs worked out and then he decided it was taking too long and just started doing it by hand, and didn't tell me about it while I still worked on it.

Whatever. Maybe someone else can find it useful.

The video won't show up in the picture, but it plays anything windows media player can play.

While watching the video, pressing 'r' will record a time for the word being recorded, rather than having to sit with the mouse or keep the focus on the record button. In the end, it spits out a result as such (per friends specifications):

The compiled executable is
here.

The source code is here.


Have fun counting words now! If anyone actually wants this for linux, I could look into porting it to GTK and gstreamer.

FtpWebRequest uploading

This may be obvious to some people but WebRequestMethods.File.UploadFile is not the same as WebRequestMethods.Ftp.UploadFile.

If you keep getting an error trying to upload the file along the lines of 'This method is not supported', make sure you are using the correct request method.

On Optimization

A few days ago, I became curious about a problem. It wasn't necessarily my problem, but it had been something I'd encountered throughout the past few years I guess. This time around, it was dealing with file path manipulation, so I decided to think of every way I'd seen file path manipulation done and check the speed, GC, and RAM usage throughout each test and see which method was fastest and less RAM dependent. I am not claiming these tests are thorough or even accurate, but the results are interesting. One thing I noticed was that it didn't matter what test was ran first, it always had a large gap in the RAM and GC usage, so I assume it has to do with loading dependent libraries and them being cached in later methods. This recreated every file path on my windows partition (/windows) since I knew I wouldn't get any permission errors.

The main lines in the code that I was testing are the following:

...
string fileName = d + dirSeparator + info.Name; //dirSeparator is defined once at the beginning of the method.
...
string fileName = d + "/" + info.Name; 
...
string fileName = d + Path.DirectorySeparatorChar + info.Name;
...
string fileName = Path.Combine(d, info.Name);
...

And the results:

bperry@bperry-desktop:~/Projects/PathingSpeedTest/PathingSpeedTest/bin/Release$ mono PathingSpeedTest.exe 
Building cache...

Starting escape from outside iterations...
Took 23 seconds
Most RAM: 78932kb (Started with 78800kb)
Most GC: 620kb (Started with 604kb)

Starting escape from inside iterations...
Took 29 seconds
Most RAM: 78828kb (Started with 78828kb)
Most GC: 616kb (Started with 612kb)

Starting environment escape...
Took 29 seconds
Most RAM: 78828kb (Started with 78828kb)
Most GC: 632kb (Started with 600kb)

Starting Path.Combine() test...
Took 29 seconds
Most RAM: 78828kb (Started with 78828kb)
Most GC: 616kb (Started with 612kb)

bperry@bperry-desktop:~/Projects/PathingSpeedTest/PathingSpeedTest/bin/Release$ 

May not be the best way to test this, so I am open to suggestions. Source code is here (is monodevelop, so not sure if it will open in VS).

StackOverflowException overriding Page in custom class

I was overriding the Page in a custom class that defined user control specific things that my app would do while being run. Each user control inherited from this UserControl class that inherited from System.Web.UI.UserControl. The problem was every time I tried to access the property, I would get a StackOverflowException. My code:


public new VolatileMinds.Web.Common.Page Page { get { return this.Page as VolatileMinds.Web.Common.Page; } }


ended up looking like this;


public new VolatileMinds.Web.Common.Page { get { return System.Web.HttpContext.Current.Handler as VolatileMinds.Web.Common.Page; } }


The problem was that it was trying to recursively cast the Page over and over again, causing the StackOverflowException. Using the Current.Handler sovles this issue.

NOTE: I don't really use absolute namespaces when doing this stuff, I thought it would be easier, however, to understand what was happening if I did.

Credit Card Validator in C#

I needed a credit card validator for a few of my projects. I found a few snippets of code throughout google, but nothing really just giving me what I needed, so I wanted to post my class. Maybe it will help others doing the same thing I did.

    public class CardValidator
    {
        public string CardType { get; private set; }
        public bool IsValid { get; private set; }
        public string ResultingError { get; private set; }
       
        public string CardNumber { get; set; }
        public DateTime CardExpiration { get; set; }

        public void Validate()
        {
            IsValid = false;

            if (string.IsNullOrEmpty(CardNumber))
            {
                ResultingError = "Card number empty....";
                return;
            }

            if (CardNumber.Length > 16)
            {
                ResultingError = "Card number too long";
                return;
            }

            foreach (char digit in CardNumber)
            {
                if (!char.IsDigit(digit))
                {
                    ResultingError = "Card number contains invalid characters";
                    return;
                }
            }

            if (CardExpiration < DateTime.Today)
            {
                ResultingError = "Card has expired.";
                return;
            }

            int sum = 0;
            for (int i = CardNumber.Length - 1; i >= 0; i--)
            {
                if (i % 2 == CardNumber.Length % 2)
                {
                    int n = int.Parse(CardNumber.Substring(i, 1)) * 2;
                    sum += (n / 10) + (n % 10);
                }
                else
                {
                    sum += int.Parse(CardNumber.Substring(i, 1));
                }
            }

            IsValid = (sum % 10 == 0);

            if (IsValid == true)
            {
                switch (CardNumber.Substring(0, 1))
                {
                    case "3":
                        CardType = "AMEX/Diners Club/JCB";
                        break;
                    case "4":
                        CardType = "VISA";
                        break;
                    case "5":
                        CardType = "MasterCard";
                        break;
                    case "6":
                        CardType = "Discover";
                        break;
                    default:
                        CardType = "Unknown";
                        break;
                }
            }
            else
            {
                CardType = "Invalid";
            }
        }
    }

Getting an XmlElement from string in C#

I had a bunch of Xml coming at me in a POST response and needed it to be an XmlElement. I ended up doing writing a quick method that does the trick, but I am not sure if it is the best way (making it an extension method made sense in context).

public XmlElement ToXmlElement (this string xml)
{
XmlDocumentFragment frag = new XmlDocument().CreateDocumentFragment();

frag.InnerXml = xml;

return frag.FirstChild as XmlElement;
}

The problem with this is you have to assume the string you are passing in is well-f0rmed XML.

Running SQL scripts in order from C# code

I have a folder of SQL scripts being compiled as embedded resources. They are named as such:

01 FirstTable.sql
02 SecondTable.sql
etc...

so that way I can run them in the order they need to be run in when say, resetting a database. The problem I ran into was getting the resources through reflection gave them to me in the wrong order... the second script was trying to be run first and it relies on the first script, so that obviously didn't work. Running Array.Sort() on the script list fixed this problem. The code ended up looking like:


public void Reset()
{
if (string.IsNullOrEmpty(ConnectionString) && Connection == null)
throw new Exception("Connection string and connection are null.");
else
{
if (Connection == null)
Connection = new MySqlConnection(ConnectionString);

Connection.Open();

MySqlCommand cmd = new MySqlCommand();
cmd.CommandText = "DROP DATABASE SystemsLogica; CREATE DATABASE SystemsLogica; USE SystemsLogica;";
cmd.CommandType = System.Data.CommandType.Text;
cmd.Connection = Connection;

cmd.ExecuteNonQuery();

Assembly asm = Assembly.GetExecutingAssembly();

string[] scripts = asm.GetManifestResourceNames();
Array.Sort(scripts);

foreach (string file in scripts)
{
Stream res = asm.GetManifestResourceStream(file);
byte[] resbytes = new byte[res.Length];

res.Read(resbytes, 0, (int)res.Length);

Console.WriteLine(file);
Console.WriteLine("-----------------");
Console.WriteLine(Encoding.ASCII.GetString(resbytes));
Console.Write("\n\n\n");


using (cmd = new MySqlCommand())
{
cmd.CommandText = Encoding.ASCII.GetString(resbytes);
cmd.CommandType = System.Data.CommandType.Text;
cmd.Connection = Connection;

cmd.ExecuteNonQuery();
}

}

Connection.Close();
}
}


This works great in mono. This was just a test method, so there is no real error checking, so be careful.