Umit project

Friday, March 29, 2013

Alpha Release of Open Monitor

We're proud to announce an Alpha release of Open Monitor!

It features a desktop (Windows, Linux, Mac OS X) and a mobile agent (Android only at the moment, iOS coming soon), and with this release, we intend to put these agents to work around the world and begin to collect precious data about internet connectivity all around.

During this alpha, we hope to be able to detect and polish all the remaining rough edges for a following beta release. We count on the community to help us test and collect this data that will be already made available in real time through Open Monitor's website for the world to use.

If you'd like to try the desktop agent, follow the install instructions here.
For the android mobile agent, .

Thursday, March 21, 2013

Honoring Certificates for all Past Successful Participantes of GSoC through Umit

At Umit we always wanted to say an extra thank you for all of the students and mentors that helped us through Google Summer of Code since 2007. The certificate idea came up, and we decided to make it special and deliver to all of those heroes that made the good code now our community can enjoy for free.

Design Details

We've participated on Google Summer of Code all years since 2007 (in 2005 and 2006 Adriano Marques participated as student), thus we'd have to make 6 different certificates featuring GSoC's logo of each year.

To make it special, we added some design perks to it: the Color Scheme and an Easter Egg.

Color Scheme

Umit's anniversary is considered to be August 29 2005, which is around the time the first GSoC was over, and the first release of Umit was made public.

For each certificate, we got the epoch number for the anniversary in that year (08/29/2007, 08/29/2008, ...), extracted the 6 digits in the middle of the epoch and splited it in 3 pairs of hex numbers. Then, we turned those numbers into colors (RGB) and generated an inverse color for it using a XOR of the extracted number against white in RGB (the number FFFFFF).

The result was a set of colors for the top and bottom stripes and the Easter Egg in the background.

We even wrote a script to generate the HTML below showing those colors!

2007 - #883708 - #77c8f7

2008 - #199932 - #e666cd

2009 - #515292 - #aead6d

2010 - #830652 - #7cf9ad

2011 - #146012 - #eb9fed

2012 - #462236 - #b9ddc9

Easter Egg

The other special thing about these certificates is that they feature real code that is present in our repositories since 2005 and that all of those being honored have gone through at least once in their life with Umit Project. It is also part of the About of the Umit Network Scanner and one of the places where we keep our permanent honors to all of those who helped our community.

The Certificates

Ok, enough details! Here are the certificates!

Monday, September 03, 2012

Google Summer of Code 2012 - Results

This summer was INTENSE!

Furious coding all around with these warriors! Lots of code delivered, and some neat new softwares all around to ease your life of network monitoring and internet connectivity monitoring.

Ladies and Gentlemen, join us congratulating the heroes!

Open Monitor All Star Team

Tianwei Liu

Desktop agent for Open Monitor: We call an agent the piece of software that goes in our user's devices and that keeps scanning for sites to confirm their status and sending reports to the aggregator so people can follow internet availability in real time from out main site. Read more about the desktop agent.

Dai Zhiwei

iOS agent for Open Monitor: Dai built from scratch an agent for iOS! Now, apple lovers can also be helpers in our pursuit to monitor the internet connectivity world wide. Read more about the ios agent.

Narendran Tangarajan

Distributed Hash Tables for inter peer communication between Open Monitor agents: This is a kademlia like approach at retrieving location of peers in a scalable way without relying on a central server for such. Narendran implemented this solution for our android and desktop agents this summer. Read more about his work.

Gautam

Mobile Sniffer and Android agent for Open Monitor: Finished up our mobile sniffer and improved our android agent. Both will be available from Google Play soon!

Orçun Avsar

Aggregator for Open Monitor: This is how we call the main server that receives, processes and displays the reports generated by the agents while testing the internet connectivity around the world. Read more about the aggregator.

Site Status

Alin Predoi

Not yet another site availability status, no. Our focus is on being a fail safe net that can receive traffic from your site in case your site fails and your users are about to see an ugly 500 server error message instead, and show them a friendly message from you and allowing them to subscribe to be notified as soon as your systems are back. This way, you won't lose that traffic that went on just when that leap second got updated and your servers went away.

Network Administrator

Amit Pal

A non cumbersome approach at monitoring your servers in the cloud (or somewhere else), without requiring complex setup and configurations. For those who want something easy to customize, this is definitely a killer for service monitoring and alarms. Read more about the Network Administrator.

Umit Network Scanner for Android

Gorjan Petrovski

Completely rethought interface, with concurrent scans support and natively built nmap scanner. Best of all: in your android device. This is what Gorjan accomplished this Summer, and that will soon be available in your Google Play store!

We're impressed with the amazing work these students managed to put together this Summer! Special thanks to Google, and specially to those who run this awesome program! Without it, we wouldn't have been able to go so far!

Friday, August 31, 2012

Inter-peer P2P routing in OpenMonitor - Overview

Hi all. This is Narendran Thangarajan from India.
In GSoC 2012, OpenMonitor system has been revamped and empowered in various ways. I am happy to have made a contribution through the inter-peer routing implementation. Let me give an introduction into the technology and then explain briefly how it fits into the OpenMonitor system.

Why did we need P2P routing?

Before GSoC 2012, OpenMonitor was more of a centralized system. All the peers and super peers were connected to the central aggregator and they relied heavily on this central aggregator for authentication, receiving tests, sending reports etc. But what if the central aggregator itself was blocked due to censorship? Or if the aggregator is facing a minor outage? Thats when the need to modularise the OpenMonitor system arises. If the responsibilities of the aggregator could be delegated to super peers seamlessly, then the tight dependency of the aggregator's availability could be eliminated.

Thus we looked into different alternatives on how to make our OpenMonitor system more scalable and available. Listed below are the most adopted DHT based P2P routing algorithms currently.

Content Addressable Network
Chord
Pastry
Tapestry
Kademlia

We decided that Kademlia Routing algorithm is the best suited P2P logic for our scenario. Lets make a quick visit to Kademlia algorithm now.

Kademlia :

Paper URL : http://www.cs.rice.edu/Conferences/IPTPS02/109.pdf

To understand Kademlia, first we must get to know about Distributed Hash Tables. DHT is the technique powering BitTorrent tracker concept. As it sounds, a DHT network consists of a hash table, distributed across multiple nodes in such a manner that the impact of a node suddenly leaving the network is minimal.

Distributed Hash Tables

When you search for any content (or 'value') on this hash table, a hash function maps it to a unique key, which is mapped to a node ID. Kademlia takes utmost O(log(n)) searches, in a network containing n nodes, until it finds the node where the required data is present.

Each node in Kademlia is assigned a Node ID which is unique in a P2P network at any instant of time. Besides being unique, the node ID serves another purpose of routing. Two nodes are considered to be nearby if the XOR value between two node IDs is small. Difference between two node ID values is considered to be the distance between those two nodes. Each node has a 128 bit routing table which can store a list of node information. Each bit stores information in the form of using which node lookup becomes fast and easy.
Kademlia or 'Kad' is a concept. There are various implementations of the Kad concept in various languages. To list a few

TomP2P
Libcage
Maidsafe
Entangled

We first started off with Maidsafe-DHT. I was able to cross compile the library to accommodate with Python based desktop agent and the Android (Java) based mobile agent. But one of my cool fellow Umit developers Dai found that Maidsafe was not completely compatible with iOS. So we had to move to an alternative. During the initial stages, we also found that Maidsafe was huge. Too huge to be running as a binary on a thin client like the mobile phone. So we chose "libcage" implementation created by Yuuki Takano. Its a C++ based P2P routing framework with NAT traversal. Libcage was originally done as an implementation of DTUN concept proposed by Takano. You can find the source code of this project here. Considering Maidsafe, libcage was much simpler and lightweight, enabling it to be run even under stern resource constraints.

Cross Compiling Libcage for Python :

As I said, libcage is written in C++. In order for a program written in C++ to be used by a python program, the C++ library should be wrapped to run smoothly on the destination platform. Wrapping C++ modules to work with python could be enabled in the following ways - SWIG (Simplified Wrapper and Interface Generator) and Boost Python. Finally we chose Boost Python to enable cross compiling libcage library to be used by C++ programs. Here are the steps :

Clone libcage from https://github.com/ytakano/libcage.git
Create your own mylogic.cpp file using the libcage libraries. Use the examples present in /examples folder.
Add the following module to your mylogic.cpp file to make the methods visible to Boost Python.

BOOST_PYTHON_MODULE(libcagepeers){

def("python_method1",actual_method1);
def("python_method2",actual_method2);
}

Next, create a shared library using "g++ -shared -Wl,-soname,libcagepeers.so.1 -o libcagepeers.so getpeers.cpp" command. Make sure you link the necessary libraries, say boost, libcage and crypto. Executing this command creates the necessary libcagepeers.so shared library, which will act as a python module as well.

Thus, we can just insert a "import libcagepeers" line into our python code and start using the shared library.

Cross Compiling Libcage for Android :

Android NDK is little complex when compared to Android SDK. But learning it is proportionately more rewarding. Here are the steps to make libcage usable with your Android app. Be warned, the official version of NDK doesn't support certain features like wchar, wstring etc. which are often used in the libraries involved. So I made use of Crystax NDK.

Create a "jni" folder under the root directory of your project.
Cross compile all the required libraries like boost, event and crypto using android toolchain.
1. Create an independent toolchain using make-standalone-toolchain.sh command.
2. Set the environment variables

export SYSROOT=/home/sunshadow/Documents/android-ndk-r7-crystax-5.beta2/platforms/android-4/arch-arm

export CC=arm-linux-androideabi-gccexport CXX=arm-linux-androideabi-g++

// For instance to cross compile libevent library.

sudo ./configure --build=arm --target=arm-linux-androideabi --enable-shared
sudo make
sudo make install

Now you must find the required static libraries in the .libs folder. Move the libevent*.a and libevent*.la libraries to our jni folder in android project's root directory. Repeat the same steps for boost, crypto and libcage. After this step, its exactly what we need to do for Android NDK projects ie. Create an Android.mk file that specifies the C and C++ flags, include folders, libraries to be linked and the source files to be compiled. When implementing our mylogic.cpp file, we must wrap the function to be exposed to Android using JNI_EXPORT JNICALL .

With respect to OpenMonitor, we also made a few changes in the libcage library, to make certain underlying data structures like the routing table and the listening port visible to the python and android layer. This is the overview on the background processes involved in making a P2P routing library named libcage to work with the desktop agent and the mobile agent of OpenMonitor.

Cheers :)

Sunday, August 26, 2012

Open Monitor Desktop Agent 2012

Hi everybody,

I am Tianwei, from China. I am a student of Google Summer of Code 2012.The GSoC 2012 has just reached the end! I have a wonderful summer with UMIT. Let’s have a look what we have done this summer.

1. Login Form

When the users open desktop agent, a login form will pop up for the uses to input username and password.

About the settings button, the user can set the url of the aggregator cloud.

When the user input wrong username/password, the alter window will pop up.

2. Preference Window

a) Tests Page

In the Test Page, the user can select and delete the Test Sets, but the website test is necessary, when the user try to remove all tests, the website is still there.

At the same time, the update setting and HTTP Throttled Test Switch is here.

b) Advanced Page

Task assign, task scheduler, report uploader and test fetcher can be configured. The current desktop agent support English and Chinese, later more language will be added.

c) Peer Info Page

Some information about peer id, test set version, cloud URL are in this page. The users can add super peer information(ip and port) by manual.

d) General Page

In this page, the users can configure the startup, auto login, desktop notification. Version details can show us.

e) Update Page

Automatically update or manually update can be selected. The desktop will check software update information in startup stage.

f) Feedback Page

Service suggestion or website suggestion can be sent to aggregator.

3. Logs Window

4. Event Window

5. Dashboard Window

a) Capacity Item

This page contains communication grade, service tests result and other peer information.

b) Sent Details

c) Task Details

d) Connection Items

e) Reports

6. About Window

7. Bug Report Window

When the desktop agent meet a crash, it will produce a crash report window which contains bug information and it will be sent to our website.

8. Software update Window

In the startup process, when the desktop agent check new version , it will show the icon in right-top position.

9. Super Peer Behavior

a) Without GUI

b) How to set super peer in desktop agent

Of course, the aggregator side should mark this peer is super peer.

c) When the super peer starts up, it will connect the peers in his peer database.

d) How the desktop agent add super peer by manual

In Preference Page -> Add Super Peer -> restart the desktop agent. The desktop agent will try to connect super peer and get necessary token information without aggregator.