Extending and enabling BigBlueButton Collaboration Platform for External SIP/Mobile Users

BigBlueButton is a known Open Source multi-party Video Conferencing platform. For all good reasons it made it to my software stack for our Smart Glass based Remote Collaboration platform.

First, some background - It all started couple of years back when BigBlueButton (BBB) was at release 0.9 and they had discontinued the native android client that was developed for 0.8 version of BBB. For the android based Smart Glass platform of ours, native android client was crucial, so i decided to use 0.8 version of BBB and built Smart Glass based client on the same lines as 0.8 version of BBB.

The prototype stage of our product was over soon and time was for serious business. Back in the prototype phase, we didn't worry too much about getting all features - including voice conference to work from the android device.  It was now a must-to-have feature and demanded focused effort.

So got an opportunity to take a deep dive into the architecture of the BBB platform - mainly voice conferencing engine. BBB uses "FreeSWITCH" opensource telephony engine for voice communication across the networks. I basically had couple of options -

1. Get BBB 0.8 voice conference to work from outside/external client
2. Upgrade to BBB 0.9
3. Upgrade to BBB 1.1 ( latest version )

Of these #3 was ruled out since it had moved long long away from 0.8 version and does not have android client from BBB and integrating 0.8 android client with this would have been a huge challenge. So considering the timelines, choice was to be made between #1 and #2.  Considering the newer version and few enhanced abilities in 0.9, decided to give it a try. It was a journey in itself ( and some great experience ) , getting the 0.8 android client to work with 0.9 version of BBB.  Many internal things were changed from 0.8 to 0.9 and finally was able to connect to BBB 0.9 server from Android.

The only thing remaining in this BBB 0.9  with Native Android client was getting voice conference to work from outside/external client. By default, since FreeSWITCH is prone to external attacks, it is usually packaged and run as standalone service and accessible only on the loopback/local IP (127.0.0.1). BBB server apps access FreeSWITCH locally.  Getting it exposed to allow external clients ( AKA SIP Clients ) wasn't trivial. The idea was simple, run the FreeSWITCH service on the public IP. But getting it to work by both - local BBB apps and external mobile/SIP client was not straightforward, and possibly getting this to work on  Amazon Web Services (AWS) was additional challenge.

If one googles on this challenge, there are lot of references, all talking variations of configurations, ideas, thoughts for various versions of BBB and FreeSWITCH, needless to say, nothing worked for me.  But one thing was common - all talked about same set of configuration files.  While no combination of configurations worked entirely, all the way along  i was certain that I was on right track since logically it was sounding correct.  Initially the android client voice connection was "Timing Out", at some point of time, I was getting "Connection Refused", later the client was able to connect, but "automatically disconnecting" after 30 secs, later "one way voice call" and finally it all came through fine. It was able to connect Voice Call from android and have a two way communication.

This enabled the android client to connect to BBB 0.8 and 0.81 seamlessly, but voice conference on 0.91 was still an issue. I had to change the default BBB - SIP dial plan to get this to work. By default the dial plan was rejecting the external callers based on IP. I had to modify the condition to accommodate both internal ( BBB Web ) and external (SIP) clients.

Several resources and tools helped all the way along, and many thanks to BBB Developer Google Groups. Last, but not the least, was the X-Lite SIP Soft Phone client. The detailed messages captured and logged by X-Lite SIP Soft Phone were really helpful nailing down the last bits of this challenge.

Long story short, I now have a customized BBB Android client that works seamlessly with BBB 0.8, BBB 0.81 and BBB 0.91. Here are the set of configuration files with settings that worked for me to enable BBB Chat, Audio and Video conferencing (both web and mobile ) on a Ubuntu 10.04 Linux instance inside a Virtual Private Network ( VPC ) on AWS.  I will soon fork the BBB repositories (for Android client) and publish these updates to the community.

The forked and extended version of mconf-mobile repository :

https://www.github.com/abhaychaware/mconf-mobile


-------------------------------------
*******  BBB 0.80  ********
-------------------------------------
Red5 Configuration files
-------------------------------------
/usr/share/red5/webapps/bigbluebutton/WEB-INF/bigbluebutton.properties
> change esl.host = internal ip of the server ( not 127.0.0.1 )

/usr/share/red5/webapps/sip/WEB-INF/bigbluebutton-sip.properties
>  change sip.server.host = internal ip of the server ( not 127.0.0.1 )

FreeSWITCH Configuration files
-----------------------------------------------
/opt/freeswitch/conf/autoload_configs/event_socket.conf.xml
>  change value of "listen-ip" param to internal ip of the server ( not 127.0.0.1 )

/opt/freeswitch/conf/vars.xml
>  delete the line that defines value for variable local_ip_v4   i.e.  <X-PRE-PROCESS cmd="set" data="local_ip_v4=127.0.0.1"/>
>  confirm that "external_rtp_ip" and  "external_sip_ip" are set to "stun:stun.freeswitch.org"

/opt/freeswitch/conf/sip_profiles/external.xml
>      change these params to have external_rtp_ip and external_sip_ip respectively.

<param name="ext-rtp-ip" value="$${external_rtp_ip}"/>
<param name="ext-sip-ip" value="$${external_sip_ip}"/>


/opt/freeswitch/conf/directory/default.xml

>     Add a domain entry for the  public IP of the server

-------------------------------------
*******  BBB 0.81  ********
-------------------------------------
Red5 Configuration files
-------------------------------------
/usr/share/red5/webapps/bigbluebutton/WEB-INF/bigbluebutton.properties
> change freeswitch.esl.host = internal ip of the server ( not 127.0.0.1 )

/usr/share/red5/webapps/sip/WEB-INF/bigbluebutton-sip.properties
>  change bbb.sip.app.ip = internal ip of the server ( not 127.0.0.1 )
>  change freeswitch.ip = internal ip of the server ( not 127.0.0.1 )

FreeSWITCH Configuration files
-----------------------------------------------
/opt/freeswitch/conf/autoload_configs/event_socket.conf.xml
>  change value of "listen-ip" param to internal ip of the server ( not 127.0.0.1 )

/opt/freeswitch/conf/vars.xml
>  delete the line that defines value for variable local_ip_v4   i.e.  <X-PRE-PROCESS cmd="set" data="local_ip_v4=127.0.0.1"/>
>  confirm that "external_rtp_ip" and  "external_sip_ip" are set to the public IP of the server an not "stun:stun.freeswitch.org"

/opt/freeswitch/conf/sip_profiles/external.xml
>      change these params to have external_rtp_ip and external_sip_ip respectively.

<param name="ext-rtp-ip" value="$${external_rtp_ip}"/>
<param name="ext-sip-ip" value="$${external_sip_ip}"/>

/opt/freeswitch/conf/directory/default.xml

>     Add a domain entry for the  public IP of the server

-------------------------------------
*******  BBB 0.91  ********
-------------------------------------
Red5 Configuration files
-------------------------------------
/usr/share/red5/webapps/bigbluebutton/WEB-INF/bigbluebutton.properties
> change esl.host = internal ip of the server ( not 127.0.0.1 )

/usr/share/red5/webapps/sip/WEB-INF/bigbluebutton-sip.properties
>  change sip.server.host = internal ip of the server ( not 127.0.0.1 )
-----------------------------------------------
FreeSWITCH Configuration files
-----------------------------------------------

/opt/freeswitch/conf/autoload_configs/event_socket.conf.xml
>  change value of "listen-ip" param to internal ip of the server ( not 127.0.0.1 )

/opt/freeswitch/conf/vars.xml
>  edit the line that defines value for variable local_ip_v4   i.e.  <X-PRE-PROCESS cmd="set" data="local_ip_v4=127.0.0.1"/>,  set it to server's internal IP.
>  confirm that "external_rtp_ip" and  "external_sip_ip" are set to the public IP of the server and not "stun:stun.freeswitch.org"

/opt/freeswitch/conf/sip_profiles/external.xml
>      change these params to have external_rtp_ip and external_sip_ip respectively.

<param name="ext-rtp-ip" value="$${external_rtp_ip}"/>
<param name="ext-sip-ip" value="$${external_sip_ip}"/>

/opt/freeswitch/conf/dialplan/public/bbb_sip.xml
>      Modify the condition to allow both external and internal callers to connect

/opt/freeswitch/conf/directory/default.xml

>     Add a domain entry for the  public IP of the server

Next steps, wait for BBB 1.1's awesome HTML5 Client to start supporting two way Camera / Video sharing, again exciting times ahead.

Modern Integrations - Need of ESB like Mule

I recently architected a Smart Glass based AR platform. The idea was cool, build a secure, lightweight, reliable, configurable and performant platform for enabling Smart Glasses to connect with enterprise systems. Idea was to allow business users - hands on task force - to work efficiently, execute business processes while keeping their hands free, from various gadgets, scanners, blue-tooth devices etc.

So the middle ware platform was the key component that could help achieve this goal. As a version 1.0 , I architected the entire solution from scratch, building REST based web application platform with JWT tokens securing the services. Few customization hooks allow easy extension of the platform and there we had, a platform that fulfilled the needs and a demoable solution was in place. By design this lightweight/stateless/cache-enabled platform was also scallable - tested upto 300 concurrent users on 4GB/dual core CPU machine with response times as low as 300ms. Perfect.

As the idea/solution saw light of the day, and more and more enquiries started coming in, a natural progression was initialised to move this platform to a more scalable architecture. An ESB based platform was the way to go. Looking at the popularity, availability of the community version and several other factors, MuleSoft was chosen as platform of chioce.  Idea was to leverage the infrastructural capabilities of the ESB - like scaling, HA, performance etc. and focus on enhancing our platform.

Getting started with Mule was pretty straightforward. Downloaded Anypoint Studio, referred to the excellently documented details with examples, some tutorials, some videos online and first application deployed.Tested, works like a charm. Easy.

Challenge now was to make the ESB runtime part of our existing web app based platform and run Mule as a service in the app container. I plan to write a separate technical blog on this one, but in short, the existing documentation helped to good extent. After some r&d and troubleshooting, I was able to embed the Mule Runtime engine inside our web based platform and was able to execute a simple Mule Flow from within this webapp. Woohoo.

It needed some more focused efforts to release first version of our platform on Mule to a respectable stage. Attaching here initial sample flow/s (not disclosing it entirely of course - IP reasons).

Serious work continues to go in enhancing this Mule based platform, several features in roadmap, deployed quite easily on Mule ESB, features like enhanced logging, audit trails, filtering, polling, scheduling and so on.

Awaiting exciting times to come.

Reporting framework with Apache POI, MS Excel and BIRT

Recently I came across a unique requirement, where in an inventory entitlement analysis model was developed in MS Excel and the customer wanted to move away from MS Excel model and turn it into a more scalable, extendable model, which could then further be deployed as SaaS model. Though the requirement was to move away from MS Excel, considering the business domain peculiarities, practicalities and user behavior, using MS Excel in some way was a strong possible solution option. Also the requirements demanded usage of statistical computation for forecasting and estimations where in the calculation power of MS Excel would have come real handy, so that was finally the option of choice. 

Problem Statement

The pressing need of moving away from MS Excel based model for the customer was result of few things. Firstly, the MS Excel based model was “evolved” to what it was and so a lot of static references were introduced all over the sheets and it had reached a state where making a slight change like adding a new product or a warehouse would mean a lot of work and making it very rigid and practically impossible to scale. Another reason to move away from MS Excel based model was the need to have a relational database and a way to maintain full history of actual and forecast data, to be able to do better prediction and enhanced reporting. 

Solution

Basically, I had three options to choose from – 
1. Develop the entire app in excel by re-modeling the data storage and using macros 
2. Use excel as front end and plug-in a database engine like MS Access 
3. Use excel only as calculation engine and develop a web based application that ‘uses’ the excel model using third party libraries for excel api. 

After studying the customer problem statements, requirements and understanding their business, and knowing the typical user behavior, I decided to still make use of MS Excel, in a more optimal fashion, which led me to choose the third option. 

Overall solution then went on evolving as we dug deeper into each component of the high level architecture. JSF was chosen to cater for the pluggable user interface need on account of its recent popularity and active community (struts community didn’t show any activity for last few months). For relational database we chose MySql’s community version – that was an easy choice. Another very crucial requirement was to be able to generate crosstab reports, line and pie charts. Keeping the reporting dynamism in mind, I decided to use a reporting framework – again, an open source one. There were couple of options to choose from where in Jasper and BIRT topped the list. Some of the pros we saw in BIRT compared to Jasper were in terms of ease of development, implementation of crosstabs and drill-down and linked reports. After few POCs, we choose BIRT over Jasper. BIRT also had a third party library (JSF4BIRT) which made it very easy to incorporate BIRT report into JSF. 

The application being a forecasting application, involved some complex evaluation models and formulae which, I thought, would be very easy to implement in MS Excel and fairly complex, otherwise. So I decided to deploy MS Excel to this task. Apache’s POI was then deployed to facilitate integration of the java application with MS Excel documents. 

For other calculations, which were huge in number but not so complex, PLSQL seemed to be right place since if they were to be done in the middle layer, would involve huge number of database calls hampering the performance, so in order to do these calculations closer to where data resides, PLSQL Stored Procedures were deployed. 

There we had, as envisioned, a web based J2EE application where in, the business user could easily maintain the master data and during every forecast run (typically weekly or monthly), download an automatically generated, dynamic MS Excel based input template. He/she would then fill it in at leisure and upload it back into the system. Basic data validations being handled using the MS Excel validation framework itself, System would then do further business validations and dump the data into the staging area. User would then “run” the model to trigger the system to generate forecasts and variety of reports in batch and using BIRT reporting engine, just render the filter based parameterized crosstabs and charts, on the report pages.

Future

This framework could further be enhanced to incorporate various goodies like BI kind of features letting users create their own reports, adapters with ERPs to automate process of data population and report generation, tagging exceptional data points and preserving tags for future reference and so on.

Unstructured Data Mining – A huge challenge or a huge opportunity ?

Gartner estimates that unstructured data constitutes 80% of the whole enterprise data. A huge proportion of this unstructured data comprises chat transcripts, emails and other informal and semi-formal internal and external communications like MS Word and PDF documents. Usually such text is meant for human consumption. However, now with huge amounts of such text being present, both online and within the enterprise, it is important to mine such text using computers.

By Definition, Unstructured Data (or unstructured information) refers to information that either does not have a pre-defined data model and/or does not fit well into relational tables. Unstructured information is typically text-heavy, but may contain data such as dates, numbers, and facts as well.

An insidious dilemma right now is regarding the issue of unstructured information. In businesses and offices, file storage and database is strewn with sensitive data that’s uncategorized, unclassified, and unprotected. With all the data exchanged within and between offices, collecting and organizing these data is proving to be a challenge.

Managing unstructured information is vital for any business, as these uncategorized data may prove to be vital in the decision-making process. Much investment is going into searching and systematizing data in networks. This is because a host of vital information may be found in these free-form texts, both in soft and hard form ) such as the following:

•  Client Responses - This information may just be buried within countless emails and correspondence.
•  Market Rival - A slew of new products and services manufactured by the competition may be analyzed by uncategorized research documents.
•  Market Segments - Feedback from consumers and customers may be derived from call transcripts and user comments.

For a company, the successful classification and management of unstructured information may lead to more profitable decisions and business opportunities.

Dealing with unstructured data
Data mining and text analytics and noisy text analytics techniques are different methods used to find patterns in, or otherwise “interpret”, this information. Common techniques for structuring text usually involve manual tagging with metadata or Part-of-speech tagging for further text mining-based structuring. There are several commercial solutions which help one to analyze and understand unstructured data for business applications. Apache UIMA, an Apache product, on the other hand, is an open source option.

UIMA (Unstructured Information Management Architecture) provides a common framework for processing this information to extract meaning and create structured data about the information.

UIMA analyzes large volumes of unstructured information in order to discover knowledge that is relevant to an end user. An example UIM application might ingest plain text and identify entities, such as persons, places, organizations; or relations, such as works-for or located at.

UIMA enables applications to be decomposed into components, for example "language identification" => "language specific segmentation" => "sentence boundary detection" => "entity detection (person/place names etc.)". To do so, it provides components which implement interfaces defined by the framework and provides self describing metadata via XML descriptor files. It additionally provides capabilities to wrap components as network services, and can scale to very large volumes by replicating processing pipelines over a cluster of networked nodes.

Additional toolset is also provided to further extend UIMA’a capabilities of extracting meaningful information from the unstructured data. One of such tool is called CFE ( Configurable Feature Extractor ) that enables feature extraction in a very generalized way. This is done using rules expressed in FESL (Feature Extraction Specification Language) in XML form. FESL's rule semantics allow the precise identification of the information that is required to be extracted by specifying precise multi-parameter criteria.

In my opinion, Unstructured Data Management will provide huge help to government and semi-government agencies, more than IT industries, where millions of unstructured/semi-structured documents are lying on shelves in physical form or in computers in soft form, waiting to be explored, read again and referred to.

There are some open source, powerful search platforms like Apache Solr, which claim to integrate with UIMA seamlessly which further strengthens the case for use of these open source technologies put together to solve the problem of this enterprise world. Having said that, is that really a huge problem or a challenge to face or really an opportunity that an IT service company should grab?

References
http://en.wikipedia.org/wiki/
http://www.unstructuredinformation.com/
http://uima.apache.org
http://lucene.apache.org/solr/
http://domino.research.ibm.com/library/Cyberdig.nsf/papers/BA59E9190C9534B4852574F000482E86/$File/rc24673.pdf

Exploring ANTLR

Getting ANTLR to work, atleast as a proof of concept, wasn't trivial as I was very new to the whole concept of defining language constructs, writing compilers and so on. But once you get accustomed to the terms and this technology, you feel really empowered. You can theoretically write your own language or convert anything, into anything else :)

I am going to cover basics about ANTLR here and the output of the POC.

The Big Picture

*   The lexical analyzer, or lexer, breaks up the input stream into tokens.
*   Tokens are vocabulary symbols emanating from the lexical analyzer.
*   The parser feeds off this token stream and tries to recognize the sentence structure.
*   Abstract syntax tree (AST) is a TREE representation of the language syntax.
*   Output can be AST or text ( using StringTemplates ).

Definitions, Tools and usage

*   Grammar describes the syntax of a language.
*   ANTLTWorks – IDE to write, verify, test, debug the grammar
*   Commandline - java org.antlr.Tool grammar.g
*   Generates Tokens, Lexer, Parser java classes

Imagine a maze with a single entrance and single exit that has words written on the floor. Every path from entrance to exit generates a sentence by “saying” the words in sequence. In a sense, the maze is analogous to a grammar that defines a language.


Grammar Example

prog : stat+ ;
stat : expr NEWLINE
       | ID '=' expr NEWLINE
       | NEWLINE;
expr : multExpr (('+' |'-' ) multExpr)* ;
multExpr : atom ('*' atom)* ;
atom : INT 
       | ID
       | '(' expr ')’ ;
ID  : ('a'..'z' |'A'..'Z' )+ ;
INT : '0'..'9' + ;
WS  : (' ' |'\t' |'\n' |'\r' )+ {skip();} ;

Test Class

public class Test {
   public static void main(String[] args) throws Exception {
   // Create an input character stream from standard in
   ANTLRInputStream input = new ANTLRInputStream(System.in);

   // Create an ExprLexer that feeds from that stream
   ExprLexer lexer = new ExprLexer(input);

   // Create a stream of tokens fed by the lexer
   CommonTokenStream tokens = new CommonTokenStream(lexer);

   // Create a parser that feeds off the token stream
   ExprParser parser = new ExprParser(tokens);

   // Begin parsing at rule prog
   parser.prog();
   }
}

AST vs Parse Tree for 3+4*5

An AST is to be distinguished from a parse tree, which represents the sequence of rule invocations used to match an input stream.

Grammar File Structure

Filename : grammarName.g

/** This is a document comment */
grammarType grammarName;
«optionsSpec»
«tokensSpec»
«attributeScopes»
«actions»
/** doc comment */
rule1 : ... | ... | ... ;
rule2 : ... | ... | ... ;
...

The order of the grammar sections must be as shown, with the rules appearing after all the other sections.


Sample Grammar

grammar T;
options {
 language=Java;
}
@members {
 String s;
 public String changeCase(String in) 
 {
  return in.toUpperCase();
 }
}
r  : ID '#' {s = $ID.text; System.out.println("found "+s);} -> declTemplate(id={$ID.text}) ;
ID : 'a'..'z' + ;
WS : (' ' |'\n' |'\r' )+ {skip();} ; // ignore whitespace

Template file : mytemplate.stg

declTemplate (id) ::=  <<
   var  ;
>>

Actions embedded within Rules

grammar T;
options {language=Java;}
@header {
      package org.antlr.test;
} 
r[int a, String b] returns [int n]
@init {
 $n=$a; // init return value
}
@after {
 System.out.println("returning value n="+$n);
}
 : ... {$n=23;}
 | ... {$n=9;}
 | ... {$n=1;}
 | // use initialized value of n
;

Rules Error Handling

void rule() throws RecognitionException {
  try {
    «rule-body»
  }
  catch (RecognitionException re) {
  reportError(re);
  recover(input,re);
 }
}

Overriding default error handling

To get these rich error messages, override two methods from BaseRecognizer,
displayRecognitionError( )
getTokenErrorDisplay( )

Example of a rich message

$ java Test
<< 1+;
<< EOF
>> line 1:2 [prog, stat, expr, multExpr, atom] no viable alternative,
token=[@2,2:2=';',<7>,1:2] (decision=5 state 0)
decision=<<35:1: atom : ( INT | '(' expr ')' );>>
found expr: 1+;

Grammar Level Options

options {
language = Java / C#;
output = template / AST;
backtrack = true / false;
memorize = true / false;
rewrite = true / false;
}

Rule Attributes

Attribute  Type
---------  ------------------------------------------------------------
text    String   // text matched by the rule
start    Token   // first token matched by the rule
stop    Token   // last token matched by the rule
tree    Object  // AST computed by the rule
st         StringTemplate  // Template computed for this rule

Example

r  : ID '#' {s = $ID.text; System.out.println("found "+s);} -> declTemplate(id={$ID.st}) ;

POC : PLSQL to JAVA Converter



References

Websites
http://www.antlr.org/
http://www.stringtemplate.org/
http://www.antlr.org/grammar/list
http://www.antlr.org/grammar/1279318813752/PLSQL.g

Books
 The.Definitive.ANTLR.Reference.pdf

Rules Engine on mobile platform : Getting JRuleEngine to do what it promised ..

During my work on rule engine on mobile platform, I came across this JRuleEngine which I briefly touched upon, in my last blog here. In there, I mentioned that I extended JRuleEngine to support multiple "facts" of same type. I thought it would be worth-while to publish what and how I did it, so this blog.


The last released ( 2008 ? ) version of JRuleEngine said in the documentation that it supports multiple facts of same type, but it actually didn't. All the facts which are loaded in the working memory are stored in a map with the getClass().getName() as key, which obviously means that only one fact of a "type" can be stored since the key in the map must be unique. So the last fact that was fed into working memory always wins, and is considered for rules execution. This is not what ( and most of us ) really wanted. 


To overcome this issue, there were a few considerations that were needed. 
(a)  Working memory should be capable of loading multiple facts of same type
(b)  Rules engine should be able to query for and receive an array of matching facts
(c)  The action/s should be able to act on all the matching facts
(d)  The result should be able to return all the facts
(e)  This behavior should be followed by all rules in the rule file


To take care of this issue, first thing required was to store the facts with a unique identifier, and so the engine would need a method to be called upon the facts which returned a unique identifier.  To achieve this, I created an abstract class named "ParentObject" which defined a "getId()" method and inherited all the facts from this ParentObject. So all the facts had to provide implementation for the getId() method. In my case, this method just returns the unique id field ( primary key in database ). But this id alone may not be unique across different types of facts, so while adding these facts into the working memory map, I appended getClass().getName() with the returned value from getId() to make this instance entry really unique.


Fact object : 
-----------------
public class FactObject extends ParentObject {
...
...
    public int getId() 
    {
         return this.id;
    }
...
...
}


Rule Engine :
 -----------------

public class StatefulRuleSessionImpl implements StatefulRuleSession {

...

...

if (object instanceof ParentObject)

{

    workingMemory.put(object.getClass().getName()+"_"+((ParentObject)object).getId(),object);

}

...

...

}


I then wrote a method to retrieve all the matching facts from the working memory. So whenever executeRules() is called, the engine now calls this new method that returns all the facts matching the "if" condition/s in the rule/s.


To see if the fact passes the constraint in the rule, the rule engine first gets the matching facts from working memory, executes the LHS in the constraint, gets the return value and then parses the RHS of the constraint in the rule. It then compares the return value from the method call on fact with the result of parsing RHS, and decides if the facts pass the constraint. So on the same lines, my code gets all the matching facts from the working memory, stores the fact id and the result for each fact from LHS expression in to a map, and then later loops through the map to filter out the facts which do not pass the first constraint. 


This check is then recursively done for each constraint, one by one, leaving behind only the facts which match all the constraints in the rule.


Then comes the "actions" part.  The action needs to be executed on all the facts which matched all the constraints in last step, so my extended code now evaluates the LHS part of the action once and then in loop, executes the RHS part of the action for all the facts that passed last step.


The rule engine then repeats this process for all actions within the rule, for all rules in the rules file. 


The result, by default, gathers all facts anyway, so didn't have to do anything special there.

Rules Engine on mobile platform : Getting Rule engine to work on Mobile Platform

One fine morning I started googling about rules engines, mostly open source. I came across this Drools rule engine. Sounded exciting. Getting rules engine to work on windows wasn’t a huge challenge, but since I was totally new to Drools concept, RETE algorithm, it was a bit challenging. Within a matter of couple of days, I was able to do a proof of concept with Drools. The next challenge was to do something that was never done before or very few folks out there have tried, was to get Drools to work on a mobile platform. Since Android’s getting very popular these days and since it has good java support ( and of-course I had recently bought a new Android phone ), Android was the mobile platform of choice !

Getting Drools to work on Android, wasn’t as easy as it appeared initially. My thought ( which was later found to be too optimistic to be true ) was that Drools is java based and android supports java, so just port drools library and application to Android and run the application. The assumption was fare and simple, I needed to run a java app on a platform that supports java. The ride was fun but wasn’t as easy as it sounds.

I had two strategies to begin with. “A” – to compile and load rules at run time on android and “B” – to pre-compile the rules and package the precompiled/serialized rules and de-serialize and load on android. Option “B” appeared to be more suitable for a “mobile” type environment, at least at this initial stage, but option “A” was better from application/rule upgrade standpoint, so I started with option “A”.

After getting a basic Hello World android app built using Eclipse and Android SDK, I pushed it out to my android device and boom! It just worked !! During this small ( ? ) exercise I learnt a lot of good android stuff which proved to be very useful in later part of this journey– how to root an android device, adb tools, pushing and pulling files to and from the device, packaging application and so on.  Later, upgrading the app to use SQLlite database wasn’t a huge deal.

Main challenge (and head-ache) started when I starting writing the Drools part of my proof-of-concept app. I started off with a minimal set of drools runtime libraries ( to keep the app size to minimum), viz. drools-core, drools-compiler, drools-api , antlr and the mvel jar. Using compatible versions of these components is very important. I chose the drools 5.1.0 version and obtained compatible versions of the dependencies as and when required through out this exercise. In fact, I had switched to drools 5.2 but had to switch back to 5.1 since the packaging tool did not like drools-compiler jar from 5.2 and failed with stack-overflow error and I didn’t have time to figure out why. So stepped back to 5.1 since my aim was to get any version of drools to work on android anyway. When I compiled the first drools-android app, it started complaining about eclipse jdt jar not being in classpath which I added thereafter. This first compilation gave me little more insight of what happens when you compile/package an android app. 

My initial idea of android’s support for java had a new dimension now, when I found out that the packing tool re-compiles the sun java compiled classes into another format called DEX ( Dalvik Executable ) format, which could be interpreted by the DVM ( Dalvik virtual machine ) that runs on Android. This led to another discovery that Android’s java is not sun’s java, but it is the Google implementation of java. Obviously to keep the vm lightweight, they have only partially implemented the java spec ( This was going to add another obstacle in my journey down the stream, which I had no knowledge about, at this point of time ).

Ok. So when I packaged and build my first small drools android app and ported on to android phone, it installed successfully ( woo hoo ! ) , but crashed when I opened up the installed app. I enabled the debugging and “adb logcat” showed me why it crashed. The DVM crashed with an error saying “parent class loader may not be null”. When I opened up the drools source, org.drools.util.CompositeClassLoader, I found the constructor that was the culprit.

    public CompositeClassLoader(final ClassLoader parentClassLoader) {

        super(null);

        loader.set( new DefaultLoader() );

    }

Upon further investigation and googling, I found a blog post by Kris Verlaenen (http://blog.athico.com/2011/03/jbpm5-lightweight-running-on-android.html) and he had faced the same issue. Thanks to Kris, this was a quick fix for me, to replace super(null) call by super(CompositeClassLoader.class.getClassLoader()). After re-compiling the source and redeploying the app on android, the app again crashed while opening up and this time it failed compiling the rule ( .drl ) file saying that eclise jdt jar is not in the classpath. This is a huge file ( @4mb alone) for a mobile environment. Post some help from the mailing list, I decided to use janino, something like this. To get this to work, I had to add “janino-2.5.15.jar” to the application and classpath.

        …

        Properties properties = new Properties();

        properties.put("drools.dialect.java.compiler", "JANINO")

        KnowledgeBuilderConfiguration kbConfig =

        KnowledgeBuilderFactory.newKnowledgeBuilderConfiguration( properties );

        KnowledgeBuilder kBuilder = KnowledgeBuilderFactory.newKnowledgeBuilder(kbConfig);

        kBuilder.add( ResourceFactory.newFileResource( drlPath ), ResourceType.DRL );

        KnowledgeBase kBase = KnowledgeBaseFactory.newKnowledgeBase();

        kBase.addKnowledgePackages( kBuilder.getKnowledgePackages() );

        …

I then started getting StringIndexOutOfBounds errors and I had to comment out the java version check (dangerous ! ) from AbstractParser.java and ParseTools.java from mvel compiler jar to get past this issue. Moving ahead, I packaged this app ( @ 1.7 mb – packaged ) and ported on the android device. Now it started throwing null pointer exceptions from “build” method of org.drools.rule.builder.RuleBuilder class indicating problem with the compilation. At this time I had spent reasonable amount of time getting this option “A” to work, So I decided to start following my plan “B”.  Here I serialized my knowledge base and wrote it to a .pkg file. ( all referenced classes need to be serializable )

        …

        OutputStream os = new FileOutputStream( rbPath );

        ObjectOutputStream oos = new ObjectOutputStream( os );

        oos.writeObject( kBase );

        …

This way, I got past the compilation problem. Planted ahead of me was the issue that I touched upon briefly earlier – the issue of partial implementation of java by Google. Google has not implemented entire java spec, but left out some packages including java beans package which my app required on android. In its absence, it started throwing “java.lang.NoClassDefFoundError: java.beans.Introspector” errors indicating missing package. On further research and help from mailing list ( thanks to Michael Anstis ) I found out that Apache have developed full java implementation named “harmony” and have a jar named ‘beans.jar’ as a part of their runtime that was what I needed ! I downloaded the jar and re-packaged it using “jarjar” utility and included in the app and it took care of this missing package problem !

The next one standing in the line was issue of de-serializing classes serialized in JVM environment into the DVM environment. KnowledgeBase that I had serialized was in the sun’s java format and what the DVM was expecting when de-serialized was the dalvik executable format. I spent some amount of time on it with almost no success .. so I decided to give up on option B and went back to Option A.

I picked it up where I had left it out. Null pointer exception during rule build. I tried using package builder instead of knowledgebuilder. Mean while I also figured out that the package builder configuration is not able to load the default JavaDialectConfiguration ( didn’t spend enough time to understand why ), so I created one myself and set it to the package builder configuration.

         Properties properties = new Properties();

         properties.setProperty( "drools.dialect.java.compiler","JANINO" );

         PackageBuilderConfiguration cfg = new PackageBuilderConfiguration( properties);

         JavaDialectConfiguration javaConf =  new JavaDialectConfiguration( );

         javaConf.init(cfg);

         cfg.setDialectConfiguration( "java", javaConf );

         PackageBuilder builder = new PackageBuilder( cfg );

         builder.addPackageFromDrl(new InputStreamReader(is));

         RuleBase ruleBase  = RuleBaseFactory.newRuleBase();

         ruleBase.addPackage( builder.getPackage() ); 

It still led to null pointer exceptions, where I was creating the PackageBuilderConfigurator instance. With a few deep dives into drools source, I figured out what the problem was. While creating the package builder configuration, along with properties, I passed “null” as second argument, which did the trick.

         PackageBuilderConfiguration cfg = new PackageBuilderConfiguration( properties, null);

With this done, I was all set to run drools on android , knowing a little about challenges which were going to be set before me along the journey !  All compiled, packaged well, deployed on android, it was now able to go past the rule building ( and the null pointer exception ) and the application crashed with HEAP error .. that was the next challenge – heap / cpu management !

I spent some good amount of time getting over the heap error, but third party libraries being involved, and unavailability of rich profiling tools for android, I decided to give up on Drools and decided to go with something lightweight, like JRuleEngine (http://jruleengine.sourceforge.net/). JRuleEngine being another java implementation and my java - android workbench already set, it was a relatively easy deal to get the vanilla JRuleEngine to work.

JRuleEngine looked like an abandoned work ( no updates post 2008 ), so I had to extend it to get it to do what I needed. The last sourced version of JRules does not support ( eventhough the documentation says, to some extend, it does ! ) inserting multiple facts of same “type” into working memory, since it accesses the facts with getClass().getName(). I extended the JRuleEngine to consider multiple facts of same “type” while evaluating rules ( assumptions ) and executing rules ( actions ), since rules need to be evaluated for every instance of the fact “type” and actions need to be taken on “all” matching facts.

Again, I had to download the Apache Harmony’s beans.jar and awt.jar and re-package it for android’s DVM using jarjar utility. With this jar , extended jrules.jar from JRulesEngine and jsr94.jar in place, following snippet worked very well. ( based on example1 packaged in JRuleEngine. )

            Class.forName( "org.jruleengine.RuleServiceProviderImpl" );

RuleServiceProvider serviceProvider = RuleServiceProviderManager.getRuleServiceProvider( "org.jruleengine" );

RuleExecutionSet res1 = ruleAdministrator.getLocalRuleExecutionSetProvider( null ).createRuleExecutionSet( inStream, null );

// instream is the input stream for the xml based rules file.

            RuleAdministrator ruleAdministrator = serviceProvider.getRuleAdministrator();

            RuleRuntime ruleRuntime = serviceProvider.getRuleRuntime();

String uri = res1.getName();

StatefulRuleSession statefulRuleSession =  (StatefulRuleSession) ruleRuntime.createRuleSession( uri, new HashMap(),RuleRuntime.STATEFUL_SESSION_TYPE );

statefulRuleSession.addObjects( input ); // input is ArrayList of facts

statefulRuleSession.executeRules();

List results = statefulRuleSession.getObjects();

I later modified this basic app and added a clickable list view in recursion which let me implement what I wanted – a rules based application on mobile platform with a guided selling UI J

This marked the final milestone of my journey, a journey that gave me a deep insight of what I would otherwise have always wondered about.