Philipp H. Oser's Blog

EL4J Release 1.6 is out!

2009-06-07T11:22:00.000-07:00

I am glad to announce the new EL4J release 1.6!

There are quite a large number of new features:

DataExtent for controlling eager loading of Hibernate Entities
JAX-WS upgrade
First release of the Hibernate offliner
Cleaned up env support (to share config between Maven and Spring, now each module can incrementally define its configuration in an XML file)
Database scripts can now be versioned and only the "right" upgrade scripts are then applied
ID fixer improvements (to get hibernate session semantics even outside of a Hibernate session, e.g. with remoting)
Support for the Maven Sonar plugin
Better handling of the PK with Hibernate (our convenience parent class for entity objects)
Full JPA support in Hibernate (e.g. @PrePersist can be enabled).
Upgrades for many libraries C3P0 instead of DBCP, Spring, Maven, …
Bugfixes and clean-ups

More info can be found in the release notes:
http://el4j.svn.sourceforge.net/viewvc/*checkout*/el4j/trunk/el4j/etc/ReleaseNotes.txt

Convenient Disconnected Mode in your Rich Java Application with Hibernate Offliner

2008-12-23T02:13:00.000-08:00

In EL4J, we have recently published the open-source Hibernate Offliner, a lightweight layer above Hibernate that supports 2 modes: online and offline (=disconnected) mode. In the online mode the application works on objects that lie in a remote database and that are served via an application server. In the offline mode, the application works on objects that lie in a local database that runs on the client node. The transitions between online and offline mode are handled almost transparently for the application.
A typical use-case of the Hibernate Offliner is a salesforce-support application that runs both when connected to the application server AND when the sales representative is disconnected from the network.

Under the hood
As in traditional database applications, we use the DAO pattern to abstract details of the database access. When switching from online to offline mode, we use a different DAO instance that uses the local database instead and records deleted objects (currently with an interceptor). A DAO registry (a factory for DAO instances) makes the right DAO selection transparent for the application. In order to make an application offline-capable as transparently as possible, a service-layer can be duplicated in the local application. This avoids that the service-layer needs to be fully aware of the offline-capability. All object-operations are done on the level of Hibernate. This means the Hibernate Offliner adds no dependency to the underlying database brand.

Going offline
When switching between the online and the offline mode, you use a set of hibernate queries to indicate what parts of your data you want to replicate on your local database. From this moment on, all future database accesses go to the local database.

Synchronizing data with the master database & going online again
Once you are reconnected to the application server, the local changes (detected via a newer object id/version pair and the recorded deletes (either delete locally only or delete in master db)) get committed in chunks (for scalability) to the master database. What if someone else has changed a same object instance in parallel (e.g. optimistic locking failures)? When going online again, the Offliner indicates conflicts in a data structure that can then be handled by the application (there is no generic conflict handling approach that works in all cases).

Transparency of the offline-capability for your application
For your application it can be to a large extent transparent that it supports online and offline mode: Data accesses work in exactly the same way, whether your application is offline or not. When you adapt an application to support offlining, it just needs to treat the 2 transitions: “go offline” and “go online” (this includes handling of potential conflicts). These two transitions typically require a few calls to the Offliner that can be nicely separated from the rest of the application. In other words, the concern “offlining” is nicely separated from other concerns.
The 2 pictures "Before" and "After" indicate a possible scenario:

Architecture constraints & open issues
What do you need to take into consideration when using this infrastructure? The following limitations exist:

Currently only support for primary keys (PKs) of type int, long and String are implemented. The PKs on entity objects are also used to identify whether they are local or remote: so the PKs of the local and the remote database must be distinct.
Between online and offline mode, the application mustn't keep objects (as their PKs change).
At the moment, the Hibernate Offliner intercepts DAO-access (for deleted objects) on the EL4J generic DAO infrastructure (this is a convenience support that provides DAO with basic CRUD operations without coding). The core of the Hibernate Offliner is independent of this choice.
The Offliner cannot guess missing „incoming“ links (so chose your offlining-queries judiciously). E.g., if you have a person class with a parent pointer and children set (1:n, one-way) and offline only the parent, fetching it from the local database and calling getChildren() will return a (wrong) empty set.
Hibernate cascading save and the Offliner may not run efficiently together

Acknowledgments
The bean browser was designed and implemented by David Bernhard, thanks for the good work!

More info
Have a look at the svn head of EL4J under framework/modules/offliner or contact the EL4J team.

DataExtends to control lazy loading with Hibernate

2008-11-27T08:46:00.000-08:00

Who has not suffered from LazyLoadingExceptions with JPA/ Hibernate? Often one uses the open session in view pattern to improve the situation. When sending hibernate-loaded classes over the network, not even this pattern solves the issue.
Thanks to an idea of Michael Vorburger, we have implemented a small abstraction called DataExtent. When loading an object, a DataExtent defines what extent of its "surrounding" object graph we want to load as well.
For this, you 1) define first a data extent, and 2) use it in your findById or findByCriteria methods of the DAO. You would put the often required DataExtents as constants in your DAO, other DataExtents can be defined on the fly.

The following picture illustrates a possible class graph, with different extents we may be interested in:

Sample code:

fileWithoutContentExtent =
new DataExtent(File.class)  
  .with("name", "lastModified", "fileSize", "mimeType");

fullFileExtent =
new DataExtent(File.class).  
with("name", "content", "lastModified", "fileSize", "mimeType");

// The Extent Object of type 'Person'
myExtent = new DataExtent(Person.class);
// Construct a complex graph:
// Person has a List of Teeth, a Tooth has a 'Person' as owner,
// the owner has a list of 'Person' as friends, the friends are again
// the same 'Person'-entity as defined in the beginning.
myExtent.withSubentities(
collection("teeth",
entity(Tooth.class)
 .with("owner")
),
collection("friends", myExtent.getRootEntity())
);

Then you can easily control what should be loaded in the DAO (our DAOs are typically a subclass of our generic DAO):
myPerson = myDao.findById(myPk, myExtent); or
myPersonList = myDao.findByCriteria(myCriteria, myExtent);

References:

2008-09-01T00:33:00.000-07:00

I just came back from the SOA Security event of ipt and Credit Suisse - unfortunately quite disappointed:

At the door of the event, I was refused entry because I work at a concurrent of ipt (the event runs under "SOA Interest Group"). I have no problem understanding limited access to events, but it should be stated when the event is published - or one should be notified before going there. The event was publicly announced here.
The reservation system (it runs on the web site of ipt) is broken: I reserved myself twice (separated by 3 days), and I did neither get the promised answering email nor an error response (I did not hear from them at all). Later they removed the remark that they promise an answering email - so I figured I could go without confirmation.
It is sad that the bank Credit Suisse, which used to have a good reputation, works with such a dicey partner.

Remark: I just checked their website again: in the mean time, they added some more concrete specification of who can go - they seem to like publishing unfinished stuff...

Jazoon 08 & EL4J 1.4

2008-07-01T05:15:00.000-07:00

After a long silence due to intensive times, I am glad to announce 2 things:

Jazoon 08, the international Java conference is over. Here is the presentation I held together with Martin Zeltner: Jazoon 08 presentation on Module-based development with Spring and Maven. The presentation of Jean-François Poilpret on the new Sun App Framework for Swing (we use this framework extensively in EL4J) can be found here .

We have released the version 1.4 of EL4J. Improvements include: more convenience methods for database access, new support to layout Swing GUIs in XML (similar to XAML), many maven improvements, jconsole can now connect by default to EL4J applications, clean-ups, ... . For a complete list of features, please consult the release notes.

EL4J release 1.3 is out!

2008-03-06T03:17:00.000-08:00

We are glad to announce the new el4j release!

Main features: New JSF application template based on Seam and Facelets (no need for EJB 3), simplified database access (less config duplication, more auto detection, more query possibilities), streamlined multi-environment support (make the same properties easily available in both maven (build and launch time) and Spring (run-time)), demo for single-jar deployment, new trouble-shooting guide (it helps with frequent issues that happen when you work with the technologies included in EL4J), improved doc, recursive maven builds (maven rec: as maven knows your dependencies it can now build your project and the projects it depends on automatically), clean-ups, ....

Full release notes: http://el4j.svn.sourceforge.net/viewvc/*checkout*/el4j/trunk/el4j/etc/ReleaseNotes.txt
SF.net site: http://el4j.sourceforge.net/index.html

EL4J release 1.2 is out!

2007-12-18T00:17:00.000-08:00

Main features: New Swing GUI framework (compilation of existing frameworks & demo), JAXWS usable via POJO programming model, new library versions, webstart demo, improved doc, seam demo, clean-ups, ....

Full release notes: http://el4j.svn.sourceforge.net/viewvc/*checkout*/el4j/trunk/el4j/etc/ReleaseNotes.txt
SF.net site: http://el4j.sourceforge.net/index.html

Presentation @ Jazoon 07 (European JavaOne Conference)

2007-07-06T17:43:00.001-07:00

I held 2 presentations @ Jazoon this year, a demo about EL4J and a keynote on the heterogeneity of Java. Here are the slides: EL4J demo keynote . Recently we have made quite some clean ups in EL4J. I hope to publish a next release soon.

Jazoon scriptlet challenge for european students

2007-03-21T01:59:00.000-07:00

The company I work for sponsors a scriptlet programming challenge to mark the 2007 Jazoon conference (JavaOne of Europe) in Zurich.

The goal is not to spend a lot of time coding, but to realize an interesting idea. Particularly encouraged are scriptlets using new features of Java 6, Groovy 1.0 and Mashups.
The number of raw lines of code is limited to 100 lines.

You can win 3 free student entries for Jazoon and Amazon book vouchers.

More details

Dependency graph for maven 2

2007-01-25T13:44:00.000-08:00

When more and more maven modules are added to a project, it's sometimes useful to see a graph showing the dependencies between all modules. This can be done with the EL4J maven depgraph plugin. I think the quickest way to show the benefits is by showing its output when applied on itself:

There are 2 modes of the plugin: one shows the dependencies of all the modules in a project. The other mode just shows the dependencies of the currently active directory. You can filter the modules to include. The output is then processed with the graphviz tool.

For more info on the plugin I refer to the code (the global website has not yet been re-generated with the new plugins):

http://svn.sourceforge.net/viewvc/el4j/trunk/el4j/maven/plugins/maven-depgraph-plugin/

And while I am at it, we also have a maven database plugin that allows executing SQL scripts and (optionally) launching an embedded database. More details can be found under http://svn.sourceforge.net/viewvc/el4j/trunk/el4j/maven/plugins/maven-database-plugin/

Remoting semantics of Spring slightly changed

2006-10-17T14:35:00.000-07:00

Time passes too quickly, already a month passed since my last blog entry :-(.
We slightly adapted the remoting semantics of Spring. Here's a picture that shows how it works:

In the picture there are 2 JVMs: one has the client and one has the server role. We show the spring application context of each JVM. The rectangles with rounded corners are spring beans (Java objects created by Spring). The normal rectangles are normal Java objects. Arrows are normal Java references. Spring beans with a dashed ring around have an auto proxy (so you could e.g. add interceptors to them).
In the server JVM, to publish POJOs via a remoting protocol, you define an exporter spring bean (this requires no code, only a short Spring configuration). This exporter bean references another spring bean that represents the protocol (this protocol bean fully configures what protocol to use and settings for the protocol). By changing settings on the protocol spring bean, you can fully control how all the linked exporter beans are exported.
In the client JVM, to get access to a remotely published POJO (either via Spring or other remoting), you just define an importer spring bean. This importer spring bean can be accessed like any other spring bean. We tend to share the protocol configuration between the client and the server JVM.

For simpler testing, one could even deploy the spring beans of the client and those of the server in one JVM (omitting the importer and exporter).

Why did we change the semantics of the Spring remoting?

It's easier to switch remoting protocols. The protocol definition and the fact that we remote is completely orthogonal.

You can also remote 2.0/2.1 EJB beans without writing custom delegator classes (as Spring proposes it)

You can optionally enable implicit context passing (this is a topic for another blog entry, you may want to check out my article decribing this if you are impatient).

It allows for easy set up of load-balancing

The following picture illustrates our load-balancing feature:

The load balancing protocol is a so-called composite protocol. It applies the Composite Design Pattern and thus allows the user to compose several of the atomic protocols (i.e., a non-composite protocol such as RMI) into this composite protocol. To the outside, it behaves like an atomic protocol. Note that the load balancing protocol is only used on the client side of a (remote) invocation and requires no modifications to existing remoting protocols.

The load balancing protocol attempts to establish an initial connection to a particular server. If this connection attempt fails, it will ask for the next server from the policy bean and attempt to connect to this server. It repeats this behavior until it succeeds to connect, or no more servers are available. In the latter case, it throws a ch.elca.el4j.services.remoting.protocol.loadbalancing.NoProtocolAvailableException.

Different policies can be plugged to govern the sequence in which protocol instances are invoked.

Pragmatic merging of XML

2006-09-19T14:41:00.000-07:00

Sometimes it is useful to be able to merge different XML document into one. For example when a tool accepts only one XML configuration file, but different subprojects that use the tool together would each be able to add some information to the XML configuration file. A concrete example would be the web.xml file of a J2EE servlet container: there can only be one such file per WAR-file. What if you have 2 subprojects that you would like to deploy into one WAR-file?
You could merge the files by hand or you could set up an merging functionality to do it automatically for you. With the little framework I present here, you can easily set up your own XML file merging. The framework is available under the EL4J project (http://EL4J.sf.net) as part of the xml merger module. It was contributed by Laurent Bovet (from outside of the EL4J core team), thanks.

Let's look at a simple example:

The following simple code implements this:

public String merge(String original, String patch) {
Configurer c = new PropertyXPathConfigurer("xpath.1=/root/d \n matcher.1=ID");
return new ConfigurableXmlMerge(c).merge(new String[]{original, patch} );
}

You may ask: what does it mean to merge XML documents? I think it is difficult to find a general answer to this question:

Sometimes the second XML file should be able override tags of the first XML file
What are the branches that should be merged? Just the same XML tags? Or does some attribute need to be equal?
Sometimes some branches of the XML document may need to be left out

Because of this, the XML merger is set up as a framework. You plug in your own definitions of how matching and mappings are done. There are basic merging semantics implemented in the framework. If you need other rules, you can extend the parts that you are interested in.

Please consult the detailed documentation or the samples of the xml merger in case you would like to know more.

Getting insights in a running Spring application via JMX

2006-09-13T08:57:00.000-07:00

During development it is often practical to be able to get more information about what is going on in a running JVM, e.g.

what Spring configuration was loaded (taking into account all the overridden configuration),
what threads are running and with what properties,
what interceptors are present on what spring beans

The EL4J JMX module allows exactly this. You just add a dependency from your application module to the JMX module and automatically have this and more information available via any web browser.

Here are some screenshots to illustrate this:

First we show the JMX overview page:

Clicking on the fooBean leads to the following view:

Clicking on the Configuration property shows the config values of the bean:

Which corresponds to following spring configuration:

You can also see all the threads that run in the JVM with their attributes:

Additional features:

Change the log4j logging level for a certain package during runtime
Get the current configuration location of log4j
See the measurements from the JaMON performance measuring interceptor
Activate log4j appenders during runtime (e.g. to get more info in case of problems)
See all system properties defined in the JVM (e.g. to get the current CLASSPATH)
Get the order of the configuration files as Spring reads them

All this is available with just the addition of a reference to the JMX module. Remark: to make the spring configuration available to JMX, it is required to use a subclass of the normal XMLApplicationContext.

Modules in the open-source EL4J framework

2006-08-16T12:39:00.000-07:00

EL4J is to a large extent a collection of existing J2EE frameworks. Spring is the most important of the contained frameworks.

One interesting feature of EL4J are modules. They allow to split an application in smaller pieces. EL4J itself also uses modules to split its features in more handy chunks. A module is similar to an eclipse plugin or to a maven project (in fact, an EL4J module is both mapped to an eclipse plugin and a maven module).

A module in EL4J contains (1) code, (2) predefined configuration and (3) dependencies. You can make dependencies on other modules or on jar files. When a module m1 uses another module m2, the module m1 has acces to all the content (code, config and dependencies) of module m2. This works transitively, i.e. if m2 depends on a module m3, m1 has also access to m2.

Sample use: Adding simple performance tracking
Why are modules useful? Let's assume you have an application with the module app. If you want to setup the performance interceptor to automatically time the duration of all calls to certain spring beans, you just make a dependency from your application to the module "light-statistics". From then on, you can access performance information via JMX.
Under the hood, the module light-statistics sets up the JAMON performance interceptor, adds it by default to all spring beans (you can override this) and makes the information available in JMX. No need to configure anything or know more about the working of the module light statistics. By default, all is setup for you.
In case you are not happy with the default, you can change anything. But simple things are simple to set up.

Here's a sample setup of modules:

In your applications, you make a dependency on all EL4J modules you need (above the module YourApplication needs the modules Security and WebTools. In this way, you automatically setup the things that are tedious to set up: jar-dependencies and the default configuration.

Are you afraid that EL4J loads configuration "magically"? You don't have to use this feature, we just recommend it for simplicity. For your application context, you specify the place from where configuration should be loaded from (as always in spring). By convention, we put configuration we recommend to always use for a given module in a directory called mandatory. The location we recommend to add to the list of locations to load configurations from is "classpath*:mandatory/*.xml". Spring then includes all the default configuration automatically. You also have the possibility to exclude a location in case you do not need it.

We need a well-integrated set of best-of-breed tools for J2EE development

2006-08-14T15:03:00.000-07:00

For simple, non-heterogeneous environments, IT-analysts often recommend using the .NET framework as development platform, rather than the J2EE. They say that .NET is simpler and more productive.
This bugs me, because I prefer the Java platform and because I favor the more open community around the J2EE. Great innovation occurs in the J2EE open source world. We also have the better IDEs.

However, when I recommend how to use the J2EE myself, I enumerate many different tools (just to name a few: Spring, Hibernate or Ibatis, Axis, Maven, Ant, JUnit, Log4j, Acegi), from different sources. It is often far from obvious what frameworks one should choose, there are just too many out there (consider e.g. the web framework domain). The integration of these frameworks is also done anew too many times.

EL4J, the open-source project I work on, has the goal to improve this situation. It combines leading J2EE tools, provides sample applications and adds some features of its own. It was already used in 16 projects so far. http://EL4J.sf.net

How do you estimate the situation in the J2EE development world?