Meta-Programming in Java

Posted by Peter Donald Thu, 09 Mar 2006 05:33:00 GMT

I have been looking over my old code repository and I came across GEL. GEL was a toolkit that allowed me to define entities and events for HLA using an XML data definition language.

The HLA is a family of specifications developed to promote intereoperability and reuse of simulation assets. Unfortunately there is a significant development effort associated with construction of a HLA-compliant federation. Less than 2% of the 2500+ lines of code in the “Hello World” sample HLA application relates to simulation logic. The integration code is a significant cost and there is often a high-level of coupling between the simulation code and the integration code.

GEL was created to simplify HLA developers task by generating all the integration code from a simulation model defined in an XML document. The HLA developer had the sole responsibility of writing the simulation/logic code. Think of GEL as a a poor-mans MDA (I even planned to have a set of separate generators that would allow the simulation objects to be hosted in a Quake3 game). This approach rocks as you only need to fix and update the code in one place (the generator) and you are far more productive at the higher level of abstraction.

Each simulation object in the GEL XML document resulted in the generation of several Java classes by velocity.

  • a Model class for holding data of a specific instance.
  • a ModelListener class that receives events relating to a Model instance.
  • a HLAPeer class that reflects the data in a Model instance into the HLA runtime.
  • a PersistPeer class that persists the data in a Model instance.
  • a number of Value Object classes to represent various events
  • etc…

I had long believed in having a single high-level model of the domain objects and generating. Prior to XML I had used SGML, INI and custom formats to represent the model. On a number of occasions I have attempted to move the model data into the code to make it easier to keep up to date. I have used C/C++’s macro expansions, XDoclet-like processors and Java 1.5 Annotations at various times.

Six months using Ruby On Rails’ has changed my world view. The model classes are very close to the level at which I would create my domain model. The good thing about ruby is that you can always add more language constructs to make sure the code matches your domain model. (I know Lisp-ers have always raved about this but I have never been able to use lisp in a commercial environment). Things like acts_as_list to order your domain objects, acts_as_versioned to version your domain objects or acts_as_threaded to have a threaded tree representation radically simplify the representation of domain objects.

I have been thinking about how I would do this in Java but most of the neat stuff just does not seem possible without the introduction of a Meta-object Protocol or some sort of load-time class file modification.

Consider a simple simulation object. In ruby I would represent it via

class Kettle < Model::Base
  shared_attr :temperature, :float, :resolution => 1.0, :accuracy => 1.0 
end

In java there is no simple equivalent. I could just define the model in a class like below. However this would probably need to be instrumented during loading so that the runtime could monitor changes to the shared attribute.

public class Kettle extends BaseModel
{
  @SharedAttribute(resolution = 1.0, accuracy = 1.0 )
  private float m_temperature;

  public float getTemperature()
  {
    return m_temperature;
  }

  public void setTemperature(final float temperature)
  {
    m_temperature = temperature;
  }
}

Instrumentation in whatever form it takes is fairly complex even if you use something like AspectJ that is perfect for this scenario. Much more complex than writing the equivalent acts_as_* plugin.

The above only works when the instrumentation code does not need to add any methods. However if you want a simulation object to support shared events you need to be able to subscribe, unsubscribe and generate these events which means you need the same boilerplate code repeated over and over.

I can not think of anyway to do this in java so I guess it is back to generating the Java classes from a central model file. Pity I think I liked the ruby way better.

Posted in  | Tags , , , , ,  | 2 comments | no trackbacks

Distributed Case-Based Reasoning using Java

Posted by Peter Donald Wed, 08 Mar 2006 02:05:00 GMT

Over the next few months I will be investing some time into a research project focusing on distributed case-base reasoning. The idea is to establish an ad-hoc network of nodes that each have a separate case-base. When a node attempts to find a solution to a problem it will query both it’s local case-base and case-bases of connected nodes to determine the solution.

The exact algorithms that the solver uses to select nodes to query will have a significant impact on the speed and accuracy with which a solution can be found. The maintainece of the distributed cases-bases via case addition or removal will also need to be addressed in our research.

I will be working on an existing codebase but I have been looking around to see if I can find any free or opensource java case-base reasoning software. Something that I can tack a distribution layer on top of but I have been unable to find any products that are not commercial software and/or under highly restrictive licenses. I wonder if this is an area that just does not appeal to FOSS developers?

Posted in  | Tags , , , , ,  | no comments | no trackbacks

PanMX: Exposing MBeans the easy way

Posted by Peter Donald Mon, 27 Feb 2006 05:15:00 GMT

PanMX removes the pain from developing MBeans. It was a toolkit I developed years ago to make things less painful. I have been cleaning up my subversion repositories and I figured it may be useful for others. So here it is released to the public at large.

The user can use Java 1.5 Annotations to define the MBean interface or allow the toolkit to derive the MBean interface using reflection.

The toolkit allows the user to define ModelMBeans via the ModelMBeanFactory or RMXBeans which are very similar MXBeans via the RMXBeanFactory .

Annotating a Class

An example of how to annotate a Java Class.

@MBean(description = "Component to measure and clean out toxins")
  public class Liver
{
  private float m_toxicity;

  @MxAttribute()
    public float getToxicity()
  {
    return m_toxicity;
  }

  @MxAttribute(description = "Level of toxicity")
    public void setToxicity(final float toxicity)
  {
    m_toxicity = toxicity;
  }

  @MxOperation(description = "Clean out the toxins")
    public void clean()
  {
    m_toxicity = 0;
  }
}

Exposing an MBean on the ‘Server’

To expose an instance of the Liver class as a ModelMBean the user could add the following code;

final Liver liver = new Liver();
final ObjectName liver = new ObjectName("body:part=Liver");
final ModelMBean mBean =
  ModelMBeanFactory.createAnnotatedModelMBean( liver );
final MBeanServer mBeanServer =
  ManagementFactory.getPlatformMBeanServer();
mBeanServer.registerMBean( mBean, objectName );

To expose the object as an RMXBean the user could use the following code;

final Liver liver = new Liver();
final ObjectName liver = new ObjectName("body:part=Liver");
final Object mBean =
  RMXBeanFactory.createAnnotatedRMXBean( liver );
final MBeanServer mBeanServer =
  ManagementFactory.getPlatformMBeanServer();
mBeanServer.registerMBean( mBean, objectName );

Accessing an MBean on the ‘Client’

The user can access the MBeans using the standard mechanisms. Additionally if the user decides to use RMXBeans and has defined the management interface using a java interface then the object can be dynamically proxied. This means that the user can interact with the object as if it was a local object.

The following code example demonstrates accessing a RMXBean as a dynamic proxy.

public interface LiverRMXBean
{
  @MxAttribute()
  float getToxicity();

  @MxAttribute(description = "Level of toxicity")
  void setToxicity(float toxicity);

  @MxOperation(description = "Clean out the toxins")
  void clean();
}

@MBean(description = "Component to measure and clean out toxins", 
       interfaces = {LiverMBean.class})
  public class Liver
  implements LiverMBean
{
  private float m_toxicity;

  public float getToxicity()
  {
    return m_toxicity;
  }

  public void setToxicity(final float toxicity)
  {
    m_toxicity = toxicity;
  }

  public void clean()
  {
    m_toxicity = 0;
  }
}

...
// Registering the Liver MBean
final Liver liver = new Liver();
final ObjectName liver = new ObjectName("body:part=Liver");
final Object mBean =
  RMXBeanFactory.createAnnotatedRMXBean( liver );
final MBeanServer mBeanServer =
  ManagementFactory.getPlatformMBeanServer();
mBeanServer.registerMBean( mBean, objectName );
...

...
// Accessing and using the Liver MBean
final MBeanServerConnection connection = ...;
final ObjectName liver = new ObjectName("body:part=Liver");
final LiverRMXBean liver = (LiverRMXBean)
  RMXBeanFactory.newProxyInstance( liver );

System.out.println( "Current Toxicity Level: " + liver.getToxicity() );
liver.clean();
System.out.println( "Toxicity Level after cleaning: " + liver.getToxicity() )
...

Files

To get your greasy mits on PanMX grab it from one of the following locations

Posted in  | Tags , , ,  | no comments | no trackbacks

Returning to Java

Posted by Peter Donald Tue, 17 Jan 2006 23:07:00 GMT

Over the last year I have wandered away from the Java programming language and experimented with different languages. Over the last six months I have had little contact with java programming except for occasionally helping out a student. However most of my time has been spent reading, teaching and programming in other languages such as Ruby (via Ruby On Rails), Prolog, Erlang and a dash of eLisp and Scheme.

I have just returned to working with Java and I find it lacking. Some of the features that java lacks do not really fit with Javas philosophy such as “open” classes. Why java does not support the richness of rubys collection classes is beyond me.

Open Classes

Ruby has the ability to re-open classes and add, remove and rename methods that have already been defined. At times this has been a lifesaver as I could re-open a class that was defined by a framework and fix bugs without having to keep a separate branch for all my dependencies.

This feature also allowed me to ensure that my code worked across different versions of libraries. For example the Struct class differs betweem Ruby 1.8.2 and Ruby 1.8.3 in the way that it handles symbols with ’?’ but I was able to re-open the class and make the behaviour consistent. I have also been able to fix many rails variations and bugs between versions so that my code would work out of the box.

I don’t think this is a feature that Java or any statically typed language should support but it is useful none the less.

Collection Classes

Ruby also has an amazingly nice set of collection classes. While it is true that the classes may include a lot of cruft, I really like the four search methods defined in Enumerable ; select/find_all, detect/find, reject, and collect/map. (select is aliased to find_all or vice versa as is detect and find, and collect and map). I believe they were originally derived from Smalltalks collection classes and it is a pity they did not find their way into java.

Consider doing something like iterating through a unordered list of 3D vertexes and any in front of the plane are placed in a new list but transformed via a matrix associated with the plane.

results = my_vertexes.select {|v| v.in_front?(my_plane) }.
     collect {|v| v.transform(my_matrix)}

While the equivelent java code is something like

LinkedList<Vertex> tempList = new LinkedList<Vertex>()
for( Vertex v : myVertexes )
{
  if( v.isInFront(myPlane) )
  {
    tempList.add( v );
  }
}
LinkedList<Vertex> results = new LinkedList<Vertex>()
for( Vertex v : tempList )
{
    tempList.add( v.transform(myMatrix) );
}

I suspect that both code snippets could be improved but they are only meant to demonstrate a point. Namely that ruby code tends to support the common case with a far shorter syntax and IMHO smaller cognitive load than equivelent code in java. My guess is that java would make it much easier to optimize the code for speed but that is rarely the most important feature of software.

Conclusion

While I am remembering that there is things I miss about java there is also things that I wish java had. I am finding it difficult to live without rubys “blocks” but still looking forward to hacking on java again.

Posted in  | Tags , ,  | no comments | no trackbacks

Scripting Databases

Posted by Peter Donald Wed, 23 Nov 2005 00:07:00 GMT

I have always considered data modeling as the one of the most critical aspects of software development. A well designed data model can outlive the specific software product it was designed for and provide a valuable asset to the organization. In the ideal scenario the data model will evolve and adapt with the organization as requirements change.

However, as a software developer I have rarely worked in an environment where I needed a deep understanding of any particular vendors database implementation. Recently I have been developing database-centric software on a number of different platforms. It may be that I am missing something but I have yet to find a decent database centric scripting language.

Consider the following problem that I was tackling a month ago. We have a central database server and application server to access the database. Database writes MUST occur through the application server to maintain data integrity due to limitations at our software layer. We also support distributed servers that can periodically synchronize with the central server. The synchronization process requires heavy use of buisness logic to detect and resolve conflicts in synchronized data. Some data may also come from other external systems such as personal or payroll. This data needs to be cleaned and converted into our schema before being synchronized with the central server.

I was tasked with automating the synchronization from an external system to our central server. I also needed to have a test run through the synchronization prior to the real run to stop the process if synchronization would fail. This involved the following steps

  1. Import, convert and clean the data from external database into INCOMING database
  2. Backup INCOMING database
  3. Backup CENTRAL database
  4. Restore CENTRAL database into TEST database
  5. Run synchronization between INCOMING and TEST. This involves;
    1. Startup TEST application
    2. Start INCOMING application and initiate synchronization with TEST
    3. Shutdow TEST application
  6. If synchronization in previous step was successful then synchronize between INCOMING and CENTRAL servers. This involves;
    1. Start INCOMING server and initiate synchronization with CENTRAL
  7. Backup CENTRAL database

At each step along the way we need to log information about progress into another database as the process can take several hours. If an error occurs we need to inform appropriate party.

I ended up implementing this as a stored procedure in Microsoft SQL Server. This is not without it’s problems. For starters it is tied to a specific vendors database server (and possibly a specific database server version). Secondly there is a large number of ugly code hacks. To execute external processes in SQL Server you need to create a job with the command then start the job. Then I poll a system table every 10 seconds until the job has completed using GOTO’s .

If that was not bad enough, I needed to come up with a mechanism to log progress messages to a different database. My problem was that if an error occured during a number of the steps a transaction roll back was issued which reverted all the log messages. The only way I could find to get around this was to open another connection to SQL Server using the SQL-DMO COM object. The COM object only used to write log entries and as it was a different connection it would not be rolled back when the main transaction rolled back. ugly!!

These uglies occur when I was just automating the process. When you get down to the data manipulation and synchronization it gets even less appealing. The code to extract data from the external database and clean it prior to putting it into INCOMING is contained within
  • an xml document defining transformation rules
  • auxilliary SQL scripts to support non-standard rules
  • a look-up-table in another database
The code to synchronize the data between multiple applications is placed within
  • another xml document defining consistancy rules
  • custom java code to support non-standard rules

It is not a pretty sight.

Admittedly if the system was to be rewritten from scratch the whole process would probably be a lot cleaner. But even then, I was skeptical that there was a nice way to implement this. The software would need to be able to define a domain model with rules that contain both imperative/procedural (ie java or some other imperative language) and declarative elements (ie sql and some data constraint language).

Previously I had thought that the best path to tackle this problem was to use some sort of Domain Specific Language to define the declarative aspects of the data model and then define the procedural elements using a language like Java. I have used this approach with success before. I defined the static model characteristerics and data constraints in an XML document and then used Velocity to generate the java code that was enhanced with procedural elements.

Recently I have been playing with Ruby on Rails and I have been re-evaluating my position. Rails has the ActiveRecord library that allows you to define model classes (using the Active Record pattern as described by Martin Fowler). These model classes can define validations that offer a psuedo-constraint language for the data. It also offers support for defining associations, and aggregations between different active record elements and is generally a nice and easy toolkit to use to access relational data. If you need to escape to SQL for performance or conceptual reasons then that is possible with few hassles.

Even more recently I discovered migrations in rails that make it possible to incrementally modify your database schema as your application evolves. You can add or remove columns, tables, indexes etc all the while preserving and migrating data as per application requirements. To upgrade to the latest schema you need only run the “migrate” rake task and be done with it.

This makes rails or more specifically ActiveRecord a very strong contender for my toolkit of choice to script database It would make it possible to avoid vendor specific stored procedures or SQL, to a certain degree and make it much easier to develop software to manipulate schemas and data.

The only negative is that it is in ruby and I have a greater understanding of the java language. Then again maybe ruby does not require the breadth of understanding java does – ir is much simpler to just get stuff done.

Maybe ruby is the next java.

Posted in ,  | no comments | no trackbacks

Older posts: 1 2