Hey,
I've been developing my project since accepted students were announced and many things have been created, changed, recreated, fixed, re-fixed, tested, retested, and still being tested of course =)
So, I will be writing about what is already done and some other things, so when I read this I will know what I need to do :)
Scan Scheduler
Scan Scheduler has been developed to schedule scans (but it may schedule anything).
- Key features:
- Multi-platform. Actually, this is the main reason to do this Scheduler;
- Uses cron format for defining scheduling time[1];
- PyGTK GUI;
- Save scans output to a directory;
- Send scans output through email;
- Add scans to the Network Inventory. This is another main reason for developing this Scheduler;
- Based on scheduler profiles[2].
[1] Almost the same cron format is being used (';' instead of ','), but this will be changed.
[2] Each "Scheduled Scan" has an unique profile name that identifies it, and one or more "Scheduled Scan" may use a "Scheduling Profile" (this defines when scans should run) and also has an unique name that identifies it.
- Some "extra" info and development time of Scan Scheduler:
- Cron parser was done already, Adriano did it.
- Scheduler Profiles was Adriano's idea and I though and still think it is good enough to keep it;
- Took around 1 week (a bit less) to develop Scheduler core and Scheduler GUI;
- Created a "SMTP Account Editor" GUI for creating and managing SMTP schemas to be used inside "Scan Scheduler Editor" for sending emails.
- After this one week, I've improved the Scheduler controller several times but still has one main "bug" left [3] and:
- This is almost ready for use except for:
- [3] As you may know, some nmap scan options requires root to execute. So, I'm still looking for a way on how to start it as root inside UMIT. I've tried using gksu, but then Scheduler will use config files inside root home. Running with sudo at console causes the Scheduler to not create a control file at user home dir, but it runs fine and everything that it needs to do is done, except that it can't be stopped by the controller since it didn't create the control file.
Strangely enough this isn't an issue for win32 :p
Network InventoryWhy write yet another Network Inventory ?Before the official start (28th May), I've been thinking about how to make a nice Network Inventory. I did some research and for my surprise, most of "Network Inventories" were more like a Software and Hardware Inventory that gathered information from local network using some win32 protocol that I forgot the name now (sorry).
After some search I found PBNJ, nice! At least someone thinks the same way I do about what a Network Inventory should be. But (maybe I could be wrong and I'm sorry for that) it is just impossible to visualize and organize a historic using PBNJ alone, since it doesn't offer a GUI or anything else. Also, I think it would be way too hard to handle many changes in a small to medium network with it. PBNJ author seems to be a good person (I just talked a bit with him on IRC asking for permission to talk about PBNJ for a lecture that I talked a bit about my project, and he was very nice), so I hope he doesn't get mad if he reads this.
Other project that I know that does a lot more than I pretend to do in this summer (winter for me) is Splunk, it handles everything you throw at it. Their team have very nice people, talked with them for the same reason mentioned before.
So, UMIT Network Inventory comes to help you understand your network(s) situation, with a nice GUI that includes unlimited Historic per device, Search, Scheduler and an eye-candy and useful Timeline, and is Open Source of course.
What was done to make Network Inventory exist ?Before the official start I saw that current UMIT database schema would be no help for doing an Inventory. So a new schema needed to be done, flexible enough to handle every piece of nmap xml output [4] and some other things related to the Inventory.
And there was a new schema! Nice! It has evolved and it is on a very good stage now. So, I created a python package for handling this database. This database consists of 30+ tables, and a lot of triggers for forcing foreign key integrity (since sqlite doesn't enforce it). So, what this packages does up to the moment:
- Inserts XML nmap output into this database;
- Store/Retrieve any piece of data;
- Store/Retrieve things related to Inventories;
- Retrieve combined data;
- Performs search;
- Store/Retrieve Inventory Changes;
- Update Inventory Changes;
- Grabs Inventory Changes by timerange (just started but works already).
With this new database, it was possible create and maintain several Inventories.
[4] It is still missing some features from latest nmap versions, but this is easily extendable.
The never-ending journey: The Network Inventory GUIMan.. let me tell you a thing, since I was announced as an accepted student, I've written around 35.000+ lines of code, and "throw away" more than half (right now on my branch there is 15k+ of my code).
The Timeline widget have been written and rewritten several times, always getting better (at least I think) till actual stage. All previous versions never worked with real data, this is, they were never really integrated within Network Inventory. Only this latest version is finally working with real data, and finally it seems I found a good Timeline solution, that accept data in a very flexible format and it is very easy to add new graph "types" to it (right now it supports line graph and area graph).
Other key-piece of Network Inventory GUI was the Changes List and Changes Diff (Changes Diff is an improved version of current DiffCompare found in UMIT). Before current Interface, I've tried several things to create a nice historic visualization (I ended up deprecating some of them even before making them to the repository).
After all, I think it was really important to try several things to see what goes better. Also, I can't count this as a time waste since it helped me a lot to improve the interface, to improve my thinking on how to handle Inventory changes and some other things.
It has been a really good journey, and is still being. More features to come, especially for Timeline, very soon.
Ok, so what Network Inventory does right now ?This text has gotten long enough, so, here comes the features list:
- Inventory creation/editing;
- Network and Host Discovery (using nmap);
- Timeline for navigating through changes in a better way (right now it supports just yearly view);
- Archives any xml nmap output (if you know some that breaks it, send me a copy or at least report the error please);
- Handles an unlimited historic for each device found in scans;
- Perform "scan diff" for ports, extraports, fingerprint, osclasses, os match;
- Displays a list with short changes description per device;
- Shows a more complete set of changes when something is selected at "list with short changes description";
- Searches for ip, hostname, ports, services, MAC, fingerprint, os match and os classes data;
Expect a much better integration with Timeline and the rest of Network Inventory for the following day/week(s).
If someone wants to follow current development, you will need to checkout my repository at sourceforge using svn. https://umit.svn.sourceforge.net/svnroot/umit/branch/ggpolo
That was it, thanks for reading ;)
