Tuesday, April 28, 2009

Umit Summer of Code 2009

This year we had so many great proposals and highly skilled students that we couldn't accept with our limited slots on Google Summer of Code that we decided to create our own parallel program called Umit Summer of Code.

The goal is to have those great projects and students working with us during this summer to create some high quality open source software.


Unfortunatelly, we don't have any budget to afford the same stippend the Google Summer of Code students will receive from Google. But, we can offer a lot of benefits for all of those who wants to volunteers for our Umit Summer of Code.

  • A mentor assigned to conduct and help you throughtout your project. That's the best way to get into Open Source for real, and receive tips and guidelines from folks who already develop successful Open Source software. You're not going to find that experience in any how-to or tutorial in the internet.
  • Your project and your name spreaded along with Umit. Whenever Umit goes, your name and project will go also.
  • Boost your resume. That's something that you can definetly mention in your resume, and will certainly boost it.
  • Experience. You'll get development experience that you'll hardly have somewhere else. You'll face issues on several topics, and will have to provide a high quality software to wide range of users world wide.
  • Google T-Shirt. One of the most important parts ;-). You'll receive a Google T-Shirt after USoC for your participation and success.
  • Certificate. We'll provide you with an official Umit Project participation certificate, which may even include hours if you think that may help you with your university.
  • Credits. We'll mention your work and name along with all announcements we'll do for our sponsored students.
  • Recommendation letter. Through your work, we'll get to know you and we'll certainly be pleased to write you a recomendation letter for whatever purpose you may need. Some students have already benefited from our recomendation letters.
More detail on our program...

Everyone is welcome to participate! We'll be receiving proposals until May 1st.

Monday, April 27, 2009

What's new in upcoming weeks for UMPA?

20th of April Google has announced the accepted students for Google Summer of Code 2009 and I'm proud to say I was accepted again by Umit Project to develop UMPA library.
So be prepare that upcoming weeks will bring some noise and massive commits for UMPA ;-)

Anyway, I would like to share ideas which I'm going to develop during the summer.
There are 4 main features which I would like to hack.


libpcap wrapping

Currently, UMPA doesn't receive packet. It works only in one way (sending). The ideal option to support receiving packets is to use the common library called libpcap. By this, we can use well known format to save and load files with network data. The common and compatible format, allows to reuse files with other applications. To use this library with UMPA, python wrapper is needed. There are several wrappers, like impacket[1], pylibpcap[2] and pypcap[3]. Due to licence incompatiblity or unsupported Windows platform, I would like to use pypcap as a python wrapper for libpcap library. There are still some problems with the wrapper (e.g. function pcap_dispatch() is not supported) but by summing all prons and cons up, this choice fits better our needs than others.

So first, I'm planning to extend pypcap for our needs. I will add pcap_dispatch() functionality and any others if needed. I will test the wrapper against the most common operating systems like GNU/Linux, MS Windows or MacOSX. I have already talked with the main author of pypcap and he agrees this idea and is glad that I would join to pypcap's community.

receiving packets

After preparing and testing pypcap, I'm going to write receiving support for UMPA. It will be done by providing API to functions like available_devices(), receive_one(), receive_loop() and others. I'm going to talk with the author of PacketManipulator about his needs in this scope and to set common API together. Filtering packets will be available too.

Also, I will provide a function for simple action-reaction system (by using pcap_dispatch()). In this system, user could to register callback functions for expected network's frames. Simple scenerio: user send packet A and expect the respond B or C. He registers 2 callbacks for receiving system. And depending which respond will be received (B or C), appropriate function will be called.

Asynchronous Scheduler

UMPA provides an extension called scheduler. It provides functionality to set initial delay for sending packets and interval between sending next frames. This extension in current state is blocking. It means that during a delay or interval time, library freezes process (UMPA uses time.sleep() function). My goal is to write new scheduler which will be non-blocking. Because UMPA is a library, using multiprocessing is improper. Threads are not the best option neither. They would solve some part of problems, like not freezing process, so GUI like PacketManipulator wouldn't be freezed too. On the other hand, new problems will occur with synchronization of threads during receiving packets. To avoid this, I'm going to use asynchronous technique. There are several already done frameworks for this (Twisted[4], asyncore[5] and others). I'm going to use asyncore for several reasons like:

  • it's distributed with Python Standard Library (no additional dependencies)
  • it's very lightweight (minimal additional CPU load)
  • it avoids to write asynchronous scheduler from a scratch (and speeds up developing process)

The only problem is that currently asyncore doesn't support future callbacks. There is an opened ticket for this issue[6] with attached patches and it's going to be commited soon. I'm going to rewrite this patch, to get back-compatibility with older Python's versions than trunk and distribute own version of asyncore with UMPA (with all compatible manner).

After all, UMPA will provide both schedulders, blocking and non-blocking. The reason to keep both is to give users a chance to pick appropriate for own needs (blocking scheduling will provide more friendly environment and its easier to use it for short applications).

Template System

The third feature which I'm going to implement is the template system.

Longterm simulation usecase describes goals of this idea:

User prepares a scenerio of the network simulation. He set rules and behaviours for received packets. UMPA will create new packets on demand and will be able to use information received from the packets to construct suitable packets to send them back. By creating complex scenerio, it's possible to keep UMPA alive for weeks and simulate some hardwares or softwares behaviours.

UMPA will become really powerful tool with this feature. It can be use in many simulation processes by hackers or even scientists (for analyse or provide scientist theories). Template system will be useful also in daily short-term cases. Like for checking stability of the network hardware by administrators. This feature is providing some intelligent for created packets!

Template system will be based on Python files for 2 reasons.

  • it will get some features by low cost (like loops, conditions)
  • user who is using UMPA library knows Python and he doesn't have to learn another (template) language

Additional functions will be provided to get easy way to write templates, especially for long-term cases where using too many conditions may fail (e.g. to generate sequence of packets). To clarify, async scheduler will be strongly integrated part of template system.

Protocol Implementation

I will implement missing protocols like ICMP or ARP. There is few limitation in core of UMPA to implement e.g. ICMP. I will restructure or rewrite part of core code if I would come across any problems during implementation phase.

So, this is how I see it now. Hope to see this (and more) done in August. UMPA would be incredible powerful then.
Check UMPA website for current status!

[1] http://oss.coresecurity.com/projects/impacket.html
[2] http://pylibpcap.sourceforge.net
[3] http://code.google.com/p/pypcap/
[4] http://twistedmatrix.com
[5] http://docs.python.org/library/asyncore.html
[6] http://bugs.python.org/issue1641

Monday, April 20, 2009

Summer of Code 2009

I'm pleased to announce the students that are going to work on Umit Project during this Google Summer of Code. This year we received 26 great proposals, but we got only 5 sponsored slots from Google. It was very tough to decide on which projects to sponsor, and we tried to base our decision on what would be the best for the Organization future.

This year, like the past one, we used a selection criteria in which we asked students to accomplish some tasks and interact with community to measure students' responsiveness, dedication, interest, personality and experience.

Although these tasks were important to rank students, we actually use other subjective meanings to decide on who is getting the slot. One of the parameters we use to decide is the proposal's importance for the organization and how we see it fitting on our long term goals, how well structured is the proposal, time the student will have available to dedicate on the project, etc.

Unfortunatelly, we had to let some really nice folks outside this year. We fought really hard to get more slots so we could fit everybody in, but we couldn't get more from Google. We understand that they're doing their best to embrace all the open source community, and we're grateful for whatever we can take.

For all the students we couldn't accept, we sincerely wish that they could stay around and get involved on the project. Although they couldn't get into Summer of Code this year, they can always get into Umit Project, and they will always be welcome to participate and help. By doing this, you'll get a boost in your resume and you'll greatly increase your chances for Summer of Code next year.

Another important detail from this year selection, is that this we had a brave young willing to participate as a volunteer, because he isn't 18 yet and he can't participate officially according to Google's rules. He desires to be subject to the same rules, deadlines and requirements that our sponsored students and he proposed it to be like that since the beginning. That's a wonderful example of dedication and desire to participate. We hope to see more examples like him!

Here follows the list of our selected students:
And our hero:
Please, join us welcoming them! And follow us throughout our Summer of Code on our Blog or Twitter.

Tuesday, April 14, 2009

Umit @ Twitter

We have created a twitter account for The Umit Project. We'll be publishing the lastest minor updates there, while we'll be also posting updates here on the blog as always, although they'll be often more elaborated. Everyone is welcome to follow us there!

Saturday, April 11, 2009

Umit 1.0-BETA2 Release

I'm pleased to announce the release of Umit 1.0-BETA2, featuring a lot of fixes and improvements. Last release was a success, and we integrated a lot of new features that our students have developed during last Summer of Codes. This 1.0 serie is very special, and we recomend everyone to try. If possible, please, report any bug that you may find.

Main changes:
  • We changed documentation [0]. Now, we're using sphinx, and it looks great.
  • Changed our module structure. Now, instead of umitCore and umitGUI we have umit.core and umit.gui . If you've being using our modules, ask for help in our devel mailing list [1] to see the best way to update your code.
  • Fixed an old issue related to .umit configuration files while updating to a new version of Umit. You don't need to care about these files anymore, while updating Umit.
  • Improved our installers.
  • Conducted other fixes reported.
If you're interested in the full change log, please take a look at our new changelog page [2]. Thanks to Guilherme Polo, who developed this changelog solution for us.

If you find any bug, please report it using our Trac[3], or using the bug report tool that comes with Umit, and that you can access in the main interface.

Download the installers, or source packages, here[4].

Special thanks to our dedicated community, which have being working hard on delivering a better Umit (alphabetical order): Bartosz Skowron, Daniel Cassiano, Devtar Singh, Francesco Piccino, Guilherme Polo, João Medeiros, Luís Silva, Rodolfo Carvalho.

Kind Regards,

[0] - Umit's new documentation
[1] - Umit's Development Mailing List
[2] - Changelog Page
[3] - Trac
[4] - Download Page