Alexander Smirnov’s personal Weblog

May 18, 2011

Neo4j Java EE connector

Filed under: Uncategorized — alexsmirnov @ 12:28 pm

Recently, I needed to add ‘graph relations’ to the data objects in the some JEE application. That application uses JPA and MySQL to store data objects, and first attempt was done by adding “many-to-many” relationships that points back to entity itself, but such schema doesnt work well in SQL. Therefore, we decided to keep graph-like relationships in the Neo4j database.

The next question was: how to add Neo4j to our application and how to coordinate transactions between JPA and Neo4j ? Because we run inside Glassfish 3.1 container, we put graph database outside of application and created Java EE connector neo4j-connector that deployed to server as resource adaptor and manages Neo4j database. Application get instance of GraphDatabaseService from JNDI and doesn’t have to care about database startup/shutdown, configuration, and transactions.

Connector features:

  • Standard JCA 1.6 connector, that can be installed on any Java EE 6 compatible server. For the oldest servers, It’s pretty easy to convert connector to JCA 1.5 API.
  • ResourceAdapter starts Neoj4 database at first request and shutdown it with server.
  • Supports both LocalTransaction and XA transaction. XA support may be not quite correct – instead of using proper XAResource from adapter, it provides access to platform TransactionManager for Neo4j server. Diving into Neo4j internals, I’ve found that it creates couple of XAResource objects and enlists them in the Transaction, while ResourceAdaptor lets to create only one XAResource. Finally, I creater Provider for JEE Server TransactionManager, similar to the Spring Data project.

Usage.

  1. Check out source code from the Github neo4j-connector project and build it with Maven. Project uses some artifacts from the Jboss Maven repository, so you have to configure it before build.
  2. Deploy connector to your application server. For Glassfish 3, there is shell script that deploys resource adapter and configures connector. All what you need is having asadmin application in the path or GLASSFISH_HOME environment variable. The connector supports two configuration options: dir for the Neo4j database location and boolean “xa” property that switches adapter from Local to XA transactions.
  3. Add neo4j-connector-api library to comple your application. JCA classes loaded in the parent Classloader on the server, you don’t need neo4j classes at runtime. For maven, just add dependency to ejb or war project:
</pre>
<dependency>
 <groupId>com.netoprise</groupId>
 <artifactId>neo4j-connector-api</artifactId>
 <version>0.1-SNAPSHOT</version>
 <scope>provided</scope>
 </dependency>
<pre>

There you go – Neo4j GraphDatabaseService now available as JNDI resource!

</pre>
@Resource(mappedName="/eis/Neo4j")
 Neo4JConnectionFactory neo4jConnectionFactory;

@Produces
 GraphDatabaseService createDatabaseService() throws ResourceException{
 return neo4jConnectionFactory.getConnection();
 }
<pre>

In the EJB services, you don’t have to start/stop transactions by hand, it will be done by container if you set apporpriate @TransactionAttribute for EJB/business method.
In the XA mode, you can use JPA/SQL and Neo4j together, and container will take care for data consistency ( of course, database connection also have to use XADataSource ).

Future plans

  1. Add database configuration parameters to adapter.
  2. Support Neo4j High Availability cluster in the adaptor. I plan to start and configure all necessary services directly in the adapter, using Application Server cluster service where possible. Therefore, JEE application can be scaled without any changes in the code, as it supposed for JEE.
  3. Provide JPA style Object to Graph mapping, most likely as the CDI extension.

17 Comments »

  1. Hi Alex,
    Is this API still active? I could not find the corresponding Mavern repo for “neo4j-connector-api” in https://repository.jboss.org/nexus/content/groups/public-jboss

    Comment by Amit — February 19, 2014 @ 11:09 pm

  2. […] dit, c'est qu'en plus de ces bugs, il est impossible de l'utiliser pour déployer le connecteur Neo4J. Et là, il faut en revenir à la base. Enfin, la base, pas tout à fait. Vous savez que maven […]

    Pingback by Déployer un domaine sur Glassfish avec maven | riduidel's wordpress — June 6, 2013 @ 2:56 am

  3. […] connecting Neo4J to JBoss 7 (preferrably JBoss 7.1) you’ll need the Neo4J Connector written by Alexander Smirnov. (Can be found as a maven project here: […]

    Pingback by sBlog – where I blog about Stuff » Blog Archive » Connecting JBoss 7.1 with Neo4J 1.7 — May 18, 2012 @ 4:04 pm

  4. Hey, is there any chance you moved the project to jboss 7.1?
    The Connector gets deployed successfully (just deployed the rar in {serverhome}/standalone/deployments)
    but my application won’t start :
    {“Operation step-2” => {“JBAS014771: Services with missing/unavailable dependencies” => [“jboss.naming.context.java.module.Neo4J_Web.Neo4J_Web.env.\”test.Neo4jClient\”.connectionFactoryjboss.naming.context.java.eis.Neo4jMissing[jboss.naming.context.java.module.Neo4J_Web.Neo4J_Web.env.\”test.Neo4jClient\”.connectionFactoryjboss.naming.context.java.eis.Neo4j]”,”jboss.naming.context.java.module.Neo4J_Web.Neo4J_Web.env.Neo4jjboss.naming.context.java.eis.Neo4jMissing[jboss.naming.context.java.module.Neo4J_Web.Neo4J_Web.env.Neo4jjboss.naming.context.java.eis.Neo4j]”]}}}

    Manifest.mf in {project}/WebRoot/Meta-Inf has line “Dependencies: deployment.neo4j-connector export services” though this is only a workaround as you mentioned above, i’m using the full profile version of jboss so jca 1.6 is included (by ironjacamar1.0.9 in jboss 7.1.1)

    i’m kinda stuck and hoping that somebody reads this (considering that the last post was made almost a year ago.. ^^)

    Comment by salgmachine — May 7, 2012 @ 11:06 am

  5. Hi, I’m trying to use neo4j JCA connector in my JavaEE6 war application and I’m currently facing some bugs.
    I deployed successfully connector into the JBoss AS 7 and I see:

    12:18:34,458 INFO [org.jboss.as.connector.metadata.deployment.ResourceAdapterDeploymentService$AS7RaDeployer] (MSC service thread 1-3) Registered connection factory java:/eis/Neo4j on mdr
    12:18:34,464 INFO [Neo4jResourceAdapter] (MSC service thread 1-3) start()

    Unfortunately when I deployed my application I get the following error:

    12:19:40,730 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-4) MSC00001: Failed to start service jboss.deployment.unit.”grid.war”.component.DBServiceController.START: org.jboss.msc.service.StartException in service jboss.deployment.unit.”grid.war”.component.DBServiceController.START: Failed to start service
    at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1786)
    at org.jboss.msc.service.ServiceControllerImpl$ClearTCCLTask.run(ServiceControllerImpl.java:2291)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [:1.6.0_26]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [:1.6.0_26]
    at java.lang.Thread.run(Thread.java:680) [:1.6.0_26]
    Caused by: java.lang.IllegalStateException: Failed to construct component instance
    at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:153)
    at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:76)
    at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:110)
    at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:123)
    at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44)
    at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1765)
    … 4 more
    Caused by: java.lang.IllegalArgumentException: Can not set com.netoprise.neo4j.connection.Neo4JConnectionFactory field grid.controller.DBServiceController.connectionFactory to com.netoprise.neo4j.connection.Neo4JConnectionFactoryImpl
    at sun.reflect.UnsafeFieldAccessorImpl.throwSetIllegalArgumentException(UnsafeFieldAccessorImpl.java:146) [:1.6.0_26]
    at sun.reflect.UnsafeFieldAccessorImpl.throwSetIllegalArgumentException(UnsafeFieldAccessorImpl.java:150) [:1.6.0_26]
    at sun.reflect.UnsafeObjectFieldAccessorImpl.set(UnsafeObjectFieldAccessorImpl.java:63) [:1.6.0_26]
    at java.lang.reflect.Field.set(Field.java:657) [:1.6.0_26]
    at org.jboss.as.ee.component.ManagedReferenceFieldInjectionInterceptor.processInvocation(ManagedReferenceFieldInjectionInterceptor.java:64)
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:287)
    at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53)
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:287)
    at org.jboss.as.ee.component.ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptor.java:53)
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:287)
    at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53)
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:287)
    at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:44)
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:287)
    at org.jboss.as.ejb3.component.session.SessionInvocationContextInterceptor.processInvocation(SessionInvocationContextInterceptor.java:67)
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:287)
    at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45)
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:287)
    at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
    at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:151)
    … 9 more

    I was trying to retrieve connectionFactory and connection in the following way:

    @Singleton
    @Startup
    public class DBServiceController implements Serializable {

    private static final long serialVersionUID = 1L;
    public static final String NEO4J_NAME = “java:/eis/Neo4j”;

    @Resource(mappedName = “java:/eis/Neo4j”)
    private Neo4JConnectionFactory connectionFactory;

    @Produces
    public GraphDatabaseService createDatabaseService() throws ResourceException {
    return connectionFactory.getConnection();
    }

    public Neo4JConnectionFactory getConnectionFactory() {
    return connectionFactory;
    }

    public void setConnectionFactory(Neo4JConnectionFactory connectionFactory) {
    this.connectionFactory = connectionFactory;
    }

    public DBServiceController() throws ResourceException {
    initialize();
    }

    public void initialize() throws ResourceException {
    Node referenceNode = connectionFactory.getConnection().getReferenceNode();
    System.out.println(“*** Reference Node ***” + referenceNode.getId());
    }

    I would really appreciate any indication or suggestion how to sort it out 😦
    Thanks a milion in advance
    J.

    Comment by JM — August 29, 2011 @ 3:34 am

    • Jboss 7 has no full JSC support yet. The main problem that connector classes are not exposed to application, as it required by JEE6 spec ( Jboss 7 is only Web profile, full profile will come with 7.1 ).
      A workaround exists: add “Dependencies: deployment.neo4j-connector export services” to the META-INF/MANIFEST.MF
      See Jboss 7 documentation for details about classloading.

      Comment by alexsmirnov — August 30, 2011 @ 10:32 pm

      • Thanks for this indication. Unfortunately my manifest.mf content seems to be not working. I played a bit around with different manifest.mf structures but none of them worked for me 😦 I understand that this META-INF/MANIFEST.MF has to be added into the src/main/resources of neo4j-connector. My connector name is simply: neo4j-connector.rar that I deployed into the jboss-as-7.0.1.Final/standalone/deployments. Is that exact syntax of this manifest.mf: Dependencies: deployment.neo4j-connector export services as you suggested ? Or that was just the direction to go ? I would really appreciate if you could attach working manifest.mf structure.

        Comment by JM — August 31, 2011 @ 7:23 am

      • You probably missed my point. That line should be added to the your application, not the neo4j connector. Jboss 7 does not support yet full JEE stack, and JCA is only optional extension to the web profile.
        In the full profile, server should detect references from application to connector resource, and add connector classes to the application classloader.
        Jboss 7 misses that functionality, so you should explicitly define application dependencies, see https://docs.jboss.org/author/display/AS7/Class+Loading+in+AS7 for details.
        Another workaround exists: you can deploy rar as the pert of your EAR application, and use META-INF/ironjakamar.xml to configure connector.
        I run connector test on the Jboss 7, you can use these tests as reference.

        Comment by alexsmirnov — August 31, 2011 @ 12:32 pm

      • Thanks for the great work so far!
        I am to experiencing some problems with JBoss AS 7.

        Here the steps I made:
        I cloned the repo, ran maven and copied the neo4j-connector.rar to $JBOSS_HOME/standalone/deployment/.
        Then I packed my Demo.war (which is essentially your Neo4jClient as a standalone app) and added “Dependencies: deployment.neo4j-connector export services” to $WAR/META-INF/MANIFEST.MF.

        As soon as I start the Server, I get the following errors:

        19:07:03,537 ERROR [org.jboss.as.deployment] (DeploymentScanner-threads – 1) {“Composite operation failed and was rolled back. Steps that failed:” => {“Operation step-2” => {“Services with missing/unavailable dependencies” => [“jboss.deployment.unit.\”neo4jDemo.war\”.component.Neo4jClient.START missing [ jboss.naming.context.java.module.neo4jDemo.neo4jDemo.env/Neo4jClient/connectionFactory ]”,”jboss.naming.context.java.module.neo4jDemo.neo4jDemo.env/Neo4jClient/connectionFactory.jboss.deployment.unit.\”neo4jDemo.war\”.module.neo4jDemo.neo4jDemo.3 missing [ jboss.naming.context.java.eis/Neo4j ]”,”jboss.naming.context.java.module.neo4jDemo.neo4jDemo.env/Neo4j.jboss.deployment.unit.\”neo4jDemo.war\”.module.neo4jDemo.neo4jDemo.2 missing [ jboss.naming.context.java.eis/Neo4j ]”,”jboss.deployment.unit.\”neo4jDemo.war\”.jndiDependencyService missing [ jboss.naming.context.java.module.neo4jDemo.neo4jDemo.env/Neo4jClient/connectionFactory, jboss.naming.context.java.module.neo4jDemo.neo4jDemo.env/Neo4j ]”]}}}

        Did anybody have the same problems or can anyone provide a simple sample webapp?

        Thanks in advance for all your help!

        Wombat

        Comment by Wombat — September 4, 2011 @ 10:08 am

      • Looks like unbounded JNDI resource. How did you define connector resource ? To make connector available in JNDI, you have to configure connection pool and resource.
        In the test, I use ironjakamar.xml for that, in production environment it should be done in server configuration.
        I’m going to move my project to Jboss 7 soon, so I will able to write more concrete description.

        Comment by alexsmirnov — September 8, 2011 @ 9:46 pm

  6. Congrats on the connector! Good idea. Just what i needed.
    Only prob i have is that when pinging the connector from the Glassfish admin interface i get:

    Ping Connection Pool for Neo4jPool is Failed. Ping failed Exception – Could not find any implementation of org.neo4j.kernel.impl.transaction.TransactionManagerProvider with a key=”jee-jta” Please check the server.log for more details. Ping failed Exception – Could not find any implementation of org.neo4j.kernel.impl.transaction.TransactionManagerProvider with a key=”jee-jta” Please check the server.log for more details.

    The “xa” property is set to true (that’s is exactly what i need) but if i set to false it works. Do i need to configure a class implementing org.neo4j.kernel.impl.transaction.TransactionManagerProvider?
    Thx!

    Emilio

    Comment by Emilio Primucci — August 10, 2011 @ 11:36 am

    • I’m aware of this problem. Neo4j uses service loader that looks for service classes in the context ClassLoader. While it goes pretty well in the EJB or web application, GlassFish ping seems to be called in the different context, so service loader cannot find requested service.
      I hoped that proper OSGI bundle would help, but OSGI service loader commented out in Neo4j code.
      In my application, Neo4j connector works well in spite of failed ping request.
      The similar problem comes with Jboss AS7, that doesn’t load connector classes unless you put special property in MANIFEST ( see connector integration test for details ).

      Comment by alexsmirnov — August 10, 2011 @ 12:03 pm

  7. Hi, I’m trying to use neo4j JCA connector in my JavaEE application and I’m currently facing some bugs.
    Namely, the @Produces annotated factory method does nto seems to be called by my EJB.
    I’ve taken a look at your tests in order to see if a complete example existed, but found nothing. Could you provide us an example ?
    Thanks

    Comment by Nicolas Delsaux (@riduidel) — August 8, 2011 @ 5:15 am

    • I saw your question on StackOverflow, but it was removed before I had chance to reply.
      The only difference with my own code is the pool name, I didn’t use / in it.
      Do you see any errors in the server log ?

      Comment by alexsmirnov — August 8, 2011 @ 11:32 am

  8. Great idea!!

    I’ m having an issue in glassfish 3.1: Could not find any implementation of org.neo4j.kernel.impl.transaction.TransactionManagerProvider with a key=”jee-jta”.

    Thanks in advance.
    Regards

    Comment by Juan Pablo Soto — July 21, 2011 @ 7:15 am

  9. Hey this is a fantastic idea. I’ve been looking around today for something like this. I’m also looking at to see how well I can port across my domain to neo4j nodes, as I’m doing a proof of concept for myself.

    Cheers for this

    Comment by William K — July 9, 2011 @ 2:01 am


RSS feed for comments on this post. TrackBack URI

Leave a reply to alexsmirnov Cancel reply

Create a free website or blog at WordPress.com.