Hi all. This is Narendran Thangarajan from India.
In GSoC 2012, OpenMonitor system has been revamped and empowered in various ways. I am happy to have made a contribution through the inter-peer routing implementation. Let me give an introduction into the technology and then explain briefly how it fits into the OpenMonitor system.
When you search for any content (or 'value') on this hash table, a hash function maps it to a unique key, which is mapped to a node ID. Kademlia takes utmost O(log(n)) searches, in a network containing n nodes, until it finds the node where the required data is present.
Now you must find the required static libraries in the .libs folder. Move the libevent*.a and libevent*.la libraries to our jni folder in android project's root directory. Repeat the same steps for boost, crypto and libcage. After this step, its exactly what we need to do for Android NDK projects ie. Create an Android.mk file that specifies the C and C++ flags, include folders, libraries to be linked and the source files to be compiled. When implementing our mylogic.cpp file, we must wrap the function to be exposed to Android using JNI_EXPORT JNICALL .
With respect to OpenMonitor, we also made a few changes in the libcage library, to make certain underlying data structures like the routing table and the listening port visible to the python and android layer. This is the overview on the background processes involved in making a P2P routing library named libcage to work with the desktop agent and the mobile agent of OpenMonitor.
Cheers :)
In GSoC 2012, OpenMonitor system has been revamped and empowered in various ways. I am happy to have made a contribution through the inter-peer routing implementation. Let me give an introduction into the technology and then explain briefly how it fits into the OpenMonitor system.
Why did we need P2P routing?
Before GSoC 2012, OpenMonitor was more of a centralized system. All the peers and super peers were connected to the central aggregator and they relied heavily on this central aggregator for authentication, receiving tests, sending reports etc. But what if the central aggregator itself was blocked due to censorship? Or if the aggregator is facing a minor outage? Thats when the need to modularise the OpenMonitor system arises. If the responsibilities of the aggregator could be delegated to super peers seamlessly, then the tight dependency of the aggregator's availability could be eliminated.
Thus we looked into different alternatives on how to make our OpenMonitor system more scalable and available. Listed below are the most adopted DHT based P2P routing algorithms currently.
- Content Addressable Network
- Chord
- Pastry
- Tapestry
- Kademlia
We decided that Kademlia Routing algorithm is the best suited P2P logic for our scenario. Lets make a quick visit to Kademlia algorithm now.
Kademlia :
To understand Kademlia, first we must get to know about Distributed Hash Tables. DHT is the technique powering BitTorrent tracker concept. As it sounds, a DHT network consists of a hash table, distributed across multiple nodes in such a manner that the impact of a node suddenly leaving the network is minimal.
 |
| Distributed Hash Tables |
Each node in Kademlia is assigned a Node ID which is unique in a P2P network at any instant of time. Besides being unique, the node ID serves another purpose of routing. Two nodes are considered to be nearby if the XOR value between two node IDs is small. Difference between two node ID values is considered to be the distance between those two nodes. Each node has a 128 bit routing table which can store a list of node information. Each bit stores information in the form of using which node lookup becomes fast and easy.
Kademlia or 'Kad' is a concept. There are various implementations of the Kad concept in various languages. To list a few
Kademlia or 'Kad' is a concept. There are various implementations of the Kad concept in various languages. To list a few
- TomP2P
- Libcage
- Maidsafe
- Entangled
We first started off with Maidsafe-DHT. I was able to cross compile the library to accommodate with Python based desktop agent and the Android (Java) based mobile agent. But one of my cool fellow Umit developers Dai found that Maidsafe was not completely compatible with iOS. So we had to move to an alternative. During the initial stages, we also found that Maidsafe was huge. Too huge to be running as a binary on a thin client like the mobile phone. So we chose "libcage" implementation created by Yuuki Takano. Its a C++ based P2P routing framework with NAT traversal. Libcage was originally done as an implementation of DTUN concept proposed by Takano. You can find the source code of this project here. Considering Maidsafe, libcage was much simpler and lightweight, enabling it to be run even under stern resource constraints.
Cross Compiling Libcage for Python :
As I said, libcage is written in C++. In order for a program written in C++ to be used by a python program, the C++ library should be wrapped to run smoothly on the destination platform. Wrapping C++ modules to work with python could be enabled in the following ways - SWIG (Simplified Wrapper and Interface Generator) and Boost Python. Finally we chose Boost Python to enable cross compiling libcage library to be used by C++ programs. Here are the steps :
- Clone libcage from https://github.com/ytakano/libcage.git
- Create your own mylogic.cpp file using the libcage libraries. Use the examples present in /examples folder.
- Add the following module to your mylogic.cpp file to make the methods visible to Boost Python.
BOOST_PYTHON_MODULE(libcagepeers){
def("python_method1",actual_method1);
def("python_method2",actual_method2);
}
- Next, create a shared library using "g++ -shared -Wl,-soname,libcagepeers.so.1 -o libcagepeers.so getpeers.cpp" command. Make sure you link the necessary libraries, say boost, libcage and crypto. Executing this command creates the necessary libcagepeers.so shared library, which will act as a python module as well.
Thus, we can just insert a "import libcagepeers" line into our python code and start using the shared library.
Cross Compiling Libcage for Android :
Android NDK is little complex when compared to Android SDK. But learning it is proportionately more rewarding. Here are the steps to make libcage usable with your Android app. Be warned, the official version of NDK doesn't support certain features like wchar, wstring etc. which are often used in the libraries involved. So I made use of Crystax NDK.
- Create a "jni" folder under the root directory of your project.
- Cross compile all the required libraries like boost, event and crypto using android toolchain.
- Create an independent toolchain using make-standalone-toolchain.sh command.
- Set the environment variables
export SYSROOT=/home/sunshadow/Documents/android-ndk-r7-crystax-5.beta2/platforms/android-4/arch-armexport CC=arm-linux-androideabi-gccexport CXX=arm-linux-androideabi-g++
// For instance to cross compile libevent library.
sudo ./configure --build=arm --target=arm-linux-androideabi --enable-shared
sudo make
sudo make install
With respect to OpenMonitor, we also made a few changes in the libcage library, to make certain underlying data structures like the routing table and the listening port visible to the python and android layer. This is the overview on the background processes involved in making a P2P routing library named libcage to work with the desktop agent and the mobile agent of OpenMonitor.
Cheers :)
