Saturday, December 15, 2007
Umit has featured "The Bourne Ultimatum" movie, helping some CIA agents to hack into a mail server. We've put some screenshots of the movie, made by Robert Chui, who first noticed the appearance of Umit in the movie, in our screenshots session. In the screenshot, you can see Umit running in the background using a modified theme, and in the bash window, you can see UMIT written somewhere in the top of the terminal. Hey movie directors! Sit and wait for UmitMapper and NetworkInventory, which are about to be integrated into Umit!
Monday, October 08, 2007
I'm pleased to announce the Umit 0.9.5 Release Candidate 1, UmitWeb 0.1 Beta 1 and the new Umit Web Site which were designed by João Paulo Pacheco and developed by e effort of volunteers leaded by Luís Bastião which were a GSoC 2007 student of Umit this year. Yet, this Umit release won't feature most of the GSoC exciting projects that we had this year, such as the Umit Mapper, the Interface Editor or the Network Inventory. Those are going to be integrated to Umit and released in a latter version after the stable release of version 0.9.5. This release is the result of several bug fixes, and the integration of all the work done by Adriano and Frederico on the Independent Features project during this Google Summer of Code. The Umit 0.9.5-RC1 and UmitWeb 0.1-B1 are currently available for download at the source forge project page in the following formats:
- Source packages compressed in the formats: tar.gz, tar.bz2 and zip
- Windows installer with Umit and every related dependencies
Linux users should take a look at README file inside source packages for installation instructions.
- Fixed some installation issues
- Fixed some issues related to configuration files and files permissions
- Now you can run umit giving it some command line arguments
- There are some usability improvements made as well, and the crash report tool were improved also
- Some core improvements were made also, making Umit more stable and reliable
- Fixed the loads of bugs reported on our bug tracker
 Umit project download page at source forge: https://sourceforge.net/project/showfiles.php?group_id=142490
 Umit project bug report page at source forge: https://sourceforge.net/tracker/?func=add&group_id=142490&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;atid=752647
 Umit repository (anonymous read access allowed): https://svn.sourceforge.net/svnroot/umit
 Umit website: http://umit.sourceforge.net
 Umit Blog: http://umitproject.blogspot.com
Friday, August 24, 2007
I'm jumping on the bandwagon and presenting you with a demo video for the Network Inventory.
It is like a tutorial video, you may view it at http://ggpolo.googlepages.com/umitni_demo.htm
The video quality is not so good cause I needed to resize to fit on most people screens.
So this completes my shortest post ever, so you don't fall asleep ;)
Wednesday, August 22, 2007
The UmitMapper is a tool to visualize networks topology, and security informations about it. The propose sent to Nmap and Umit called RadialNet, and can be found here. I'm presenting you with a demonstration of UmitMapper tool. At this time you can get it in the SVN. I'm working in a tarball release at now. Bellow you can view the video demonstration.
I hope that this tool is useful for all. Suggestions and comments are wellcome. To test the UmitMapper download it from SVN.
svn co https://umit.svn.sourceforge.net/svnroot/umit/branch/joao umit
Run the Umit with the command: sudo python umit. The use of privileged user is a requirement of Nmap traceroute. Instructions to use the tool are in the README_GSOC file in the SVN. Bellow some screenshots:
To understand how to work the visualization you can read the GSoC proposal here. The UmitMapper is tested under Linux and Mac OS, but probably work in windows too.
Att, João Paulo de Souza Medeiros.
Monday, August 20, 2007
I'm here to present you with my work developed during this Google Summer of Code.
My project was renamed of 'Profile and Wizard Editor' to 'Umit Interface Editor' sometime ago. But I want tell you that my work is most over the first name. But Umit Interface Editor sounds better and it's a very good approach of what is my project.
I would like announce a little modification about a implementation. Now the options at options.xml have arg_type. This is a detail important refer because some people use profile, options and wizard in your GSoC projects. Feel free to criticize and comment.
A new way of Edit Profiles was introduce at UMIT. I call to it: Profile Manager.
Focus on the Umit Interface Editor basically it's a editor of xml file: profile_editor.xml, wizard.xml and options.xml. It use the same way that manually you should be. There are three mode of editing: Profile, Wizard And Options. In the Options Edit mode you can add, remove, update the options, changing name, args etc. The Profile and Wizard was edited by the same way. In a notebook you can add/remove/rename/move sections (Tabs). And the same for items, you can add, remove, move, rename new options using the Option List created using Option Edit Mode.
Use at my brach: python umit -e or python umit and go menubar: Profile > Interface Editor and Profile Manager.
About this Google Summer of Code I should say that I like this experience very much . I never development anything like this. I learn a lot in different areas. I learn things that I never will know in the University, it's a great great Summer for me.
Soon as possible I will do another post to talk about the new improvements of my project. But my mainly concern is change in the kind of write xml files and change edit mode of Wizard.
Below it's a video of a demo 'Umit Interface Editor' that I recorded. I'm apologize of my bad bad English. It's only to you see. Sorry next try should be better, I promise.
Saturday, August 11, 2007
I'm posting more news about Network Inventory, Scheduler and "standard" UMIT.
Many things have changed, again, to better. I will be listing major changes since 30th July, 2007 here:
- Changes in UMIT:
- Configuration files are updated when a new version is installed, this fixes:
- Problems most users had, something like UMIT raising Errors because it couldn't find config files and some other things.
- Directories structure is cleaner now
- Changes in Scheduler GUI:
- It will show some warnings when you try to run it as a normal user inside UMIT or Network Inventory;
- I've made its interface a bit more generic, so both at umit interface as well NI interface, you will see the same Schedulter interface.
- Scheduler option 'Save outputs to' now expects only a directory.
- Changes in Scheduler Core:
- It cleans up temporary files correctly when it stops;
- The Cron Parser used on it now uses ',' as separator instead of ';' ;
- Writes log messages to $HOME/.umit/scheduler.log;
- You can start Scheduler specifying some umit config dir, so you can start it at system startup, or by some other user and use other user config dir (if you can write to it of course);
- Changes in Network Inventory GUI:
- You can view Timeline for single hosts;
- Graph Preferences starts with current Timeline graph settings now;
- Added auto-refresher for Timeline, default is 5 minutes;
- Added refresh option to Options in Changes list display (Historic visualization);
- It is possible to run scans whenever you want now, right clicking on some Inventory at Historic tab;
- You can view hosts by ipv4 address (like before), ipv6, mac or hostname (default now) at Historic now. In case a host doesn't have the correspondent data for current view mode, it fallbacks to ipv4 address.
- Added a dialog for controlling data removal;
- Changes in Network Inventory Core:
- Added support for removing data that is N days old;
Also, a quick installation guide follows:
- Remove your old umit installation:
- Run uninstall_umit and then remove your $HOME/.umit;
- cd ggpolo (path to where you checked out my repo)
- sudo python install_scripts/linux/setup.py install
Now, a screenshot. The GUI is basically the same as previous version, and in this one I don't have much data as you can see by my Timeline graph:
Latest Network Inventory screenshot
Monday, July 30, 2007
Like I said before, Timeline was being integrated with NI and now it is ready to use.
What things have changed since two weeks ago:
- Select between "Yearly View", "Monthly View", "Daily View" and "Hourly View";
- Filter settings are kept, before, changing graph mode or something else would reset it;
- TLBarDisplay (an "extra" widget for Timeline) now has a nice color transition effect;
- Changes listing have been reworked, so it was possible to use it inside Timeline;
- More ways for grabbing data;
- A lot of several minor (and some not so minor) things have been done, to make it possible to Timeline work with all new changes.
- Configure something to remove data older than N days;
- A more flexible data range setting;
- Search for changes over time*;
There are probably many other things, I just pointed something that I considered most needed.
You can view Latest UMIT Network Inventory screenshot
Tuesday, July 17, 2007
Now, to the point. On my last post I said it was all sudo's fault for not creating Scheduler control file correctly, and I was wrong.
I wasn't even looking at that problem now, I was continuing with Timeline integration with rest of umitInventory and umitDB but for some unknown reason I just wanted to fix that strange bug.
From the previous post: "Running with sudo at console causes the Scheduler to not create a control file at user home dir, but it runs fine and everything that it needs to do is done, except that it can't be stopped by the controller since it didn't create the control file."
But what was happening was: I've done a function that checks if a process is running at all, on UNIX, it uses kill for that. So, it was working like this: send kill to the pid with signal 0, if it throws an OSError exception the process is not running... WRONG!
Throwing an OSError doesn't mean the process is not running, I needed to check errno also. An OSError with errno equals 3 ('ESRCH') means the process isn't running (errno 3 is 'No such process'), but since I ran the process with sudo it was returning errno equals 1 that means 'Operation not permitted' and then it was returning False and the control file would get deleted and I would think that the control file wasn't created at all.
Now this bug is fixed at least =)
And again, forgive me SUDO.
Monday, July 16, 2007
I've been developing my project since accepted students were announced and many things have been created, changed, recreated, fixed, re-fixed, tested, retested, and still being tested of course =)
So, I will be writing about what is already done and some other things, so when I read this I will know what I need to do :)
Scan Scheduler has been developed to schedule scans (but it may schedule anything).
- Key features:
- Multi-platform. Actually, this is the main reason to do this Scheduler;
- Uses cron format for defining scheduling time;
- PyGTK GUI;
- Save scans output to a directory;
- Send scans output through email;
- Add scans to the Network Inventory. This is another main reason for developing this Scheduler;
- Based on scheduler profiles.
 Each "Scheduled Scan" has an unique profile name that identifies it, and one or more "Scheduled Scan" may use a "Scheduling Profile" (this defines when scans should run) and also has an unique name that identifies it.
- Some "extra" info and development time of Scan Scheduler:
- Cron parser was done already, Adriano did it.
- Scheduler Profiles was Adriano's idea and I though and still think it is good enough to keep it;
- Took around 1 week (a bit less) to develop Scheduler core and Scheduler GUI;
- Created a "SMTP Account Editor" GUI for creating and managing SMTP schemas to be used inside "Scan Scheduler Editor" for sending emails.
- After this one week, I've improved the Scheduler controller several times but still has one main "bug" left  and:
- This is almost ready for use except for:
-  As you may know, some nmap scan options requires root to execute. So, I'm still looking for a way on how to start it as root inside UMIT. I've tried using gksu, but then Scheduler will use config files inside root home. Running with sudo at console causes the Scheduler to not create a control file at user home dir, but it runs fine and everything that it needs to do is done, except that it can't be stopped by the controller since it didn't create the control file.
Strangely enough this isn't an issue for win32 :p
Why write yet another Network Inventory ?
Before the official start (28th May), I've been thinking about how to make a nice Network Inventory. I did some research and for my surprise, most of "Network Inventories" were more like a Software and Hardware Inventory that gathered information from local network using some win32 protocol that I forgot the name now (sorry).
After some search I found PBNJ, nice! At least someone thinks the same way I do about what a Network Inventory should be. But (maybe I could be wrong and I'm sorry for that) it is just impossible to visualize and organize a historic using PBNJ alone, since it doesn't offer a GUI or anything else. Also, I think it would be way too hard to handle many changes in a small to medium network with it. PBNJ author seems to be a good person (I just talked a bit with him on IRC asking for permission to talk about PBNJ for a lecture that I talked a bit about my project, and he was very nice), so I hope he doesn't get mad if he reads this.
Other project that I know that does a lot more than I pretend to do in this summer (winter for me) is Splunk, it handles everything you throw at it. Their team have very nice people, talked with them for the same reason mentioned before.
So, UMIT Network Inventory comes to help you understand your network(s) situation, with a nice GUI that includes unlimited Historic per device, Search, Scheduler and an eye-candy and useful Timeline, and is Open Source of course.
What was done to make Network Inventory exist ?
Before the official start I saw that current UMIT database schema would be no help for doing an Inventory. So a new schema needed to be done, flexible enough to handle every piece of nmap xml output  and some other things related to the Inventory.
And there was a new schema! Nice! It has evolved and it is on a very good stage now. So, I created a python package for handling this database. This database consists of 30+ tables, and a lot of triggers for forcing foreign key integrity (since sqlite doesn't enforce it). So, what this packages does up to the moment:
- Inserts XML nmap output into this database;
- Store/Retrieve any piece of data;
- Store/Retrieve things related to Inventories;
- Retrieve combined data;
- Performs search;
- Store/Retrieve Inventory Changes;
- Update Inventory Changes;
- Grabs Inventory Changes by timerange (just started but works already).
 It is still missing some features from latest nmap versions, but this is easily extendable.
The never-ending journey: The Network Inventory GUI
Man.. let me tell you a thing, since I was announced as an accepted student, I've written around 35.000+ lines of code, and "throw away" more than half (right now on my branch there is 15k+ of my code).
The Timeline widget have been written and rewritten several times, always getting better (at least I think) till actual stage. All previous versions never worked with real data, this is, they were never really integrated within Network Inventory. Only this latest version is finally working with real data, and finally it seems I found a good Timeline solution, that accept data in a very flexible format and it is very easy to add new graph "types" to it (right now it supports line graph and area graph).
Other key-piece of Network Inventory GUI was the Changes List and Changes Diff (Changes Diff is an improved version of current DiffCompare found in UMIT). Before current Interface, I've tried several things to create a nice historic visualization (I ended up deprecating some of them even before making them to the repository).
After all, I think it was really important to try several things to see what goes better. Also, I can't count this as a time waste since it helped me a lot to improve the interface, to improve my thinking on how to handle Inventory changes and some other things.
It has been a really good journey, and is still being. More features to come, especially for Timeline, very soon.
Ok, so what Network Inventory does right now ?
This text has gotten long enough, so, here comes the features list:
- Inventory creation/editing;
- Network and Host Discovery (using nmap);
- Timeline for navigating through changes in a better way (right now it supports just yearly view);
- Archives any xml nmap output (if you know some that breaks it, send me a copy or at least report the error please);
- Handles an unlimited historic for each device found in scans;
- Perform "scan diff" for ports, extraports, fingerprint, osclasses, os match;
- Displays a list with short changes description per device;
- Shows a more complete set of changes when something is selected at "list with short changes description";
- Searches for ip, hostname, ports, services, MAC, fingerprint, os match and os classes data;
If someone wants to follow current development, you will need to checkout my repository at sourceforge using svn. https://umit.svn.sourceforge.net/svnroot/umit/branch/ggpolo
That was it, thanks for reading ;)
Monday, July 09, 2007
If you're curious to know what Umit's students are working on, join us on trying our software and checkout our repository. The student's works are separated by diretories inside the branch dir.
- bass_boy (Rodolfo) - The UmitWeb! Run umitweb.py and point your browser to localhost:8059. Login: user1, Password: 123
- ggpolo (Guilherme) - Network Inventory and Scan Scheduler. Execute python umit -i
- hildon (Adriano) - Porting of Umit interface to Maemo
- indep_features (Adriano and Frederico) - Independent features and bug hunting.
- joao (João) - UmitMapper. Run a scan, and take a look on the shinny new tab that shows João's radial map of your network.
- k0p (Luís Bastião) - Umit Interface Editor. A better way to customize your Umit's profile and wizard interfaces. Run umitInterfaceEditor/uie.py
- max (Max) - NSE Facilitator. An easier way to play and create your NSE Scripts.
- pavel (Pavel) - Nmap Python Wrapper. A module intended to let you do things like: import nmap; nmap.run_scan("localhost")
- umitdb (Guilherme and João) - A new Database schema and module for Umit. Better arrangement, and other stuffs improved.
The only thing we're missing is you trying it out, and having fun with us!
 - Nmap 4.22SOC1 Release announcement - http://seclists.org/nmap-dev/2007/q3/0030.html
 - Umit's Bug Tracker - http://sourceforge.net/tracker/?group_id=142490&atid=752647
Monday, May 28, 2007
After a few days, I made a draft that describes how it will work. The mechanism is quite simple: each User will be assigned to a Role, and that role can have several Access Permissions. This appear to be an obvious way to do authorization. But how could access divided in permissions?
I found a way that I guess it's interesting: each Role will have definitions about details of command composition. For example, a Role to an user that can execute OS detection will have a permission to execute nmap with the '-O' parameter. In other hand, if a role doesn't have a permission that define the '-sV' command, users assigned to that role cannot do detection of service's versions.
The better way to find this option inside the nmap command line is comparing it with regular expressions. There are advantages to use regular expressions to indentify parts of a command. The permission definition can me written in an XML file (like options and profiles), and it can be extended by the user. For example, if a user want to create a new permission definition, He/she can easily do it by editing the security.xml and put inside it the properly regular expression associated with that permission.
Other important thing that is important to think is about how umit will handle the permissions order in each role. I think that the best way to do it is 'chaining' all permissions and categorize it by defining a priority order. This way, umit will work like a 'command firewall', filtering the options to allow or deny the command execution.
I'm still thinking if the best way to store user roles is in XML files or a database. I have this issue to store users too. I'll realize tests and researches about security issues in each case.
Next week will be time to implement my thoughts and test it. Let's roll in the first official GSoC week. Here we go!
From: UmitWeb POST
Wednesday, May 23, 2007
These modifications will feature the next Umit version, which is been integrated to Nmap this month, and soon will be launched for testing. This new is worth a post here because Umit has reached again one more milestone, and the quantity of users which will use Umit will increase A LOT after that, leaving us with a good user base for testing and giving us feedback with sugestions, and inspiring us on making an even better tool to make your work faster every day.
Soon, Umit will feature brand new features like UmitMapper, NetworkInventory, Profile/Wizard Editor, NSE Facilitator, UmitWeb, and whole load of features intended to make you waste less time working and more time with your familly. That's our goal!
So, stay tunned for the next Nmap release, give it a try and let us know your thoughts about it, sending us a bug report if you find a bug or an email if you just want to give us some feedback or sugestion about the tool.
 - http://sourceforge.net/tracker/?group_id=142490&atid=752647
 - My e-mail is py.adriano at that google mail that we all love ;-)
Thursday, April 12, 2007
So, it came the time to announce the students that are going to participate at this GSoC. Sorry for taking too long to release the results, but I was trying to get some more slots to have as more of you guys working this year in this open source dream. Unfortunately, I can't accept everyone. I must confess that it was really hard to come out with the list of the selected students, because you guys are really talented.
First, I would like to thank everyone for the interest on Umit and for having applied a proposal to the Umit Project this year. Just the fact of knowing that such talented folks like you are interested on Umit is very gratifying, and stimulates me to keep working to provide a better tool for the whole open source community.
I would like to thank Google for this great opportunity which they're giving to the open source community, and recognize that Umit came to life because of the opportunity I had with GSoC 2005. Also, I have a special thank to Fyodor, who believed in my capabilities and mentored me during GSoC 2005 and 2006.
Everyone who applied to GSoC this year is already a winner. Having a good idea and making a good proposal to sell it is hard and takes time. Just the fact of giving this first step is more meaningful than been successful in your idea, because your success relies on your initiative (that first step you gave sending your proposal...).
Without initiative, ideas are nothing but lost musings. So, if were not selected this year, keep your chin up, and go ahead my friends, this is not the end of the road yet. Those who are still interested on working with us at Umit regardless of been selected this year are *VERY* welcome, and I'll be glad to register your name and e-mail at the Umit's hall of fame (The project credits! :-D). Also, working with us will give you a huge advantage over oter students next year, you're going to learn a lot and still boost your resume.
Now, I'm honored to present you the selected students for this GSoC:
1 - Rodolfo da Silva Carvalho (UmitWeb) - UmitWeb is going to be web interface for Umit, with which everyone will be able to use Umit remotely. Have you ever thought about schedulling a scan remotely, and having the result at your inbox later? Rodolfo will make it real for you. Rodolfo is studying english at Wizard School, and he owns the degree of Bachelor on Information Systems. He knows Umit since the very beginning, and also made some sketches of the UmitWeb last year.
2 - Frederico Silva Ribeiro (Independent features) - There are loads of bug hunting, new features and usability improvements to be made this Summer, and Frederico is the one responsible for that. If you have any bug to report, do it now, and he will get rid of them. I'll be helping him on this during this Summer. Frederico is a student from UNIVERSO (Salgado de Oliveira University) in the major Internet and Computer Networks.
3 - João Paulo de Souza Medeiros (UmitMapper) - Have you ever thought about trying to grab your whole network with your hands? Well, maybe you'll be able to do that with your mouse with the project that João is going to implement for us this year. His idea is based on scientific propositions about graphs dispositions, and the goal is to make it as easy, smooth and usable as possible, so you can have a handy graph of your network where you can browse freely to investigate whatever you need with just a couple of clicks inside the Umit
interface. João is a student from Federal University of Rio Grande do Norte, in the major Computer Engineering.
4 - Guilherme Henrique Polo Gonçalves (Network Inventory and Scan Scheduler) - Watching what happens to your whole network everyday, and notice what has changed since the last look you gave is a hard task. Guilherme is going to make a tool to ease your task on that. A tool that is going to be your eyes in the network, and will point out to you what has changed in the network since the last verification in a easy and usable fashion. Hope this could help users to get home earlier! Guilherme is a student from the State University
of Maringá, in the major of Computer Science.
5 - Luis Antônio Bastião Silva (Profile/Wizard interface editor) - Did you once felt like editing the Profile or Wizard interface to put that brand new option that the brand new alpha version of Nmap is making available, but were confused with the XML files you would have to edit? Luis is going to make a easy to use interface editor to ease this task, so you'll be able to quickly add, remove or edit any option displayed on the Profile or Wizard windows. Luis is a student from Universidade de Aveiro in Engenharia Computadores e Telemática
(Computer Engineering and Telematics), in the major of Computer Science.
6 - Pavel Klemenkov (Nmap Wrapper) - Umit currently lacks on integration with the runtime interaction options that Nmap provide. Pavel is going to make a wrapper of the Nmap lib, so Umit will be able to run Nmap as if it was a python lib! And other persons will also be able to take advantage on that, creating python scripts to deal with nmap, and create their own solutions to problems that Umit don't solve yet. Pavel is a sudent from Lomonosov Moscow State University, in the major of Computational Mathematics and Cybernetics.
7 - Maxim I. Gavrilov (NSE Facilitator and Independent Features) - Umit must follow this NSE era that is arriving. In order to keep pace with Nmap, Max is going to implement some tools to ease the task of using NSE from Umit interface. Also, he is going to help us on independent features. Max is a student from SPbSU ITMO (www.ifmo.ru), in the major of Mathematical Modeling, pursuing a master degree.
The result will be officially published by Google soon, and the result can change a bit. But I don't believe it though.
I would also like to announce that I'll be creating three mailing lists for the Umit Project, as follows:
* Developers - Intended for developers and users interested on Umit development subjects, help, bugs discussion, suggest improvements, etc.
* Announce - Low traffic mailing list for all of you who wants to receive really hot and filtered updates from Umit.
* SoC - Private list for selected students communication. It will be used for sending status reports, sugestions, private subjects discussion and students interaction.
After this announcement, I'll be adding the selected students to the SoC list, and I'll send further instructions through it. I'll be also creating an IRC channel, so the whole community can participate on the project definition phase which will be the period from April 12th to May 27th.
Good luck to all of you!
Friday, March 16, 2007
Thursday, March 15, 2007
Why is this post so important? Just in case you want a GSoC T-shirt and a stippend of U$4.500,00 to help us improve this open source project!
If you want to participate as a student, take a look on our ideas and start writing your proposal, and feel free to send us proposals with your own ideas also.
Hurry! The students application deadline is March 24th.