Wednesday, July 13, 2011

ICM - Mobile Agent

The Umit Internet Connectivity Monitor is a watchdog which will act as a first line of defense against global Internet censorship. Conceptually, it scans for websites and services and provides a real-time stream of censorship events plotted on a web mapping service. Architecturally, it consists of three major entities: 1) The Cloud Aggregator, 2) The Desktop Agent, 3) The Mobile Agent. Both types of agents, perform connectivity tests and then route the results - either directly or through a P2P network - to the aggregator.
In this post, we will dissect the various modules of the mobile agent and discuss its nitty-gritty. Primarily, the mobile agent consists of ten modules, namely:
1) The GUI,
2) Aggregator Communication,
3) P2P Communication,
4) Connectivity Testing,
5) Maps Service,
6) Notifications,
7) Process Management,
8) Unit Tests,
9) Social Network Integration,
10) Search Engine Access.

At this stage, it is useful to mention some of the technologies involved in engineering the mobile agent. The mobile agent uses the Android platform, Apache Maven and the Maven Android Plugin for build management, JUnit Testing Framework for test-driven development, Restlet Android Edition for aggregator communication, Google Maps and OSMDroid for maps, Google Protocol Buffers for serialization, Android JavaMail for mail management, Apache Commons for various components, Twitter4J for the Twitter API, Bing API version 2.0, Google Web Search API, and finally Eclipse as the IDE and the ADT Plugin for Eclipse.

We will now discuss each module one by one:
1) The GUI:
As the name suggests, this module implements the UI of the application using standard Android Views and Widgets. It consists of three main activities/views hosted inside a TabHost. These activities, namely, InformationActivity, MapActivity, and ControlActivity allow the user to interact with the application. The InformationActivity provides a real-time stream of connectivity events - both received from the aggregator and results of tests performed locally. The MapActivity plots connectivity events on top of various map packages. And finally, the ControlActivity allows the user to tweak many of the configuration parameters such as the scanning interval etc.

All communication with the aggregator is handled by this module. The aggregator provides webservices that the mobile agent performs HTTP POST calls on. The webservices have a request/response format, i.e. the client POSTs a Google Protobuf request message (serialized to a Base64 String) and receives a Protobuf response message (Base64 serialized). These messages are encrypted using a 128-bit AES symmetric key cipher. GetEvents, GetTests, GetPeers, and GetSuperPeers webservices are called using an Android Service after configuration intervals. The rest of the webservices are called when required.

Asynchronous communication with other agents (peers) takes place over TCP sockets. The exchanged messages are Protobuf messages serialized to byte arrays encrypted using a 128-bit AES symmetric key cipher. This module also maintains a message queue.

This module performs connectivity tests for both websites and services. In case of websites, the HTTP header is first downloaded and analyzed for the status code. If the status code is normal (200), the website content is downloaded and converted to a Protobuf report message to be sent to the aggregator. Whereas, depending on the service protocol, service tests are performed using the Glasnost model. In the Glasnost model, 2 flows are started from the client to the testing server. The first flow consists of regular service messages while the second flow consists of random bytes sent using the same protocol. Any disparity between the two flows is an indicator of differentiation. To ensure that these tests are performed in the background even when the application is minimized, the Android Service component is used. Each connectivity test is fired off as a TimerTask at a preset regular interval.

The maps service takes connectivity events and plots them on top of mapping packages. At present, the mobile agent supports two packages: 1) Google Maps, and 2) OSMDroid. All events are either marked as normal or differentiated. Normal events use a green marker while differentiated events use a red marker on top of a map overlay.

This module uses a NotificationManager running in a background Service to fire off a notification when events are received from the aggregator or a connectivity test is completed, etc.

The process management module takes care of all process artifacts and parameters. It holds all global objects, cipher keys, actions, runtime parameters and versioning data. Additionally, it also generates report IDs and holds various constants.

This module performs JUnit tests to unit test various components of the other modules. Each test extends the AndroidTestCase and uses standard Assert statements. All tests are fired off using Maven Android Plugin.

The Social Network Integration module uses Twitter4J to connect to the user's Twitter account and send Tweets of important events. The Twitter API uses OAuth for account authentication. To authenticate their account, users are directed to a Twitter page through the ControlActivity. After logging into their Twitter account, users are provided with a Pin number which is entered through the ControlActivity. After this authentication phase, Tweets of important events are automatically sent to the associated account through the Notifications Service.

This module provides access to the search capability of various search engines that the mobile agent requires for its functionality. Currently, it has access to Bing and Google.

Other than these main modules, the mobile agent also contains a Utilities module which holds crypto functions, disk read/write functions, and a profiler which when enabled, logs the time taken by each profiled method. Additionally, a Commons module to hold artifacts common between the aggregator and agents is also provided.


  1. Sure it is! As well as everything we do on Umit Project. Check our dev site: or our main website: