Syntax highlighter header

Saturday, 12 June 2021

Running local Docker image in Minikube

I was using Minikube to learn Kubernetes. Kubernetes pulls images from public container registries or your private registry hosted on a public server.  I did not want to upload my confidential container images to any public repository or spend money in creating my private repository hosted on a public server.

In this post I am going to describe a method to run container images from my local machine in single node kubernetes cluster hosted in Minikube. I am doing this experiment on n CentOS 7 Laptop.

Creating a Dockerfile

For creating a container image you first need to create a Dockerfile. In Dockerfile you provide a base image and modification you want on top of the content provided by the base image.

You can refer to https://docs.docker.com/develop/develop-images/dockerfile_best-practices/ for more information on Dockerfile.

I am using a tomcat image and creating my image without doing any modification to it. I just want to test if Minikube will be able to pick my image locally.

My Dockerfile content is:

FROM tomcat:latest

Setting docker environment variables

Minikube runs inside a docker container on linux box and hosts a local docker registry there. You can point to that docker registry by setting some environment variables. These environment variables and their value is listed by running following command.


minikube docker-env

The output on my local machine is:


export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://192.168.49.2:2376"
export DOCKER_CERT_PATH="/home/deployer/.minikube/certs"
export MINIKUBE_ACTIVE_DOCKERD="minikube"

# To point your shell to minikube's docker-daemon, run:
# eval $(minikube -p minikube docker-env)

Creating Docker image

After setting these environment variables you can run following command to build your image:


docker build -t tomcat-jas .

Now my docker image named tomcat-jas is pushed to Minikube's container registry. Now we can use it in docker deployments. 

Creating Kubernetes deployment

We need to create deployment using yaml file instead of providing image name in kubectl command because there we can't provide imagePullPolicy parameter. My tomcat-jas.yaml file has following content:


apiVersion: apps/v1
kind: Deployment
metadata:
  name: tomcat-jas-deployment
  labels:
    app: tomcat-jas
spec:
  replicas: 1
  selector:
    matchLabels:
      app: tomcat-jas
  template:
    metadata:
      labels:
        app: tomcat-jas
    spec:
      containers:
      - name: tomcat-jas
        image: tomcat-jas:latest
        imagePullPolicy: Never
        ports:
        - containerPort: 8080
      - name: httpd
        image: httpd:latest
        ports:
        - containerPort: 80

I am running two containers inside my pod one is httpd running at port 80 and another is tomcat running at port 8080. You can create deployment using following command:

kubectl create -f tomcat-jas.yaml

You can check status of you deployment using following command:


kubectl get deployment

Output on my machine is:


NAME                    READY   UP-TO-DATE   AVAILABLE   AGE
balanced                1/1     1            1           13d
hello-minikube          1/1     1            1           13d
hello-node              1/1     1            1           13d
kubernetes-bootcamp     2/2     2            2           13d
tomcat-jas-deployment   1/1     1            1           2m21s

You can see that deployment is successfully running.

Exposing the deployment as a service

You can expose your newly created deployment to outside world using the following command:


kubectl expose deployment/tomcat-jas-deployment --type LoadBalancer --port 80,8080

You can check status of your service using following command:

kubectl get service

Output on my machine is:


NAME                    TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                       AGE
balanced                LoadBalancer   10.105.156.60   <pending>     8080:31961/TCP                13d
hello-minikube          NodePort       10.106.246.13   <none>        8080:31293/TCP                13d
hello-node              LoadBalancer   10.104.148.22   <pending>     8080:31192/TCP                13d
kubernetes              ClusterIP      10.96.0.1       <none>        443/TCP                       13d
kubernetes-bootcamp     NodePort       10.99.64.116    <none>        8080:30470/TCP                12d
tomcat-jas-deployment   LoadBalancer   10.103.14.144   <pending>     80:30147/TCP,8080:30692/TCP   19s

Accessing tomcat and httpd from your browser

The port 80 of our deployment is mapped to port number 30147 and port 8080 of our deployment is mapped to port 30690 of Minikube host. The IP address of host can be checked with following command:

minikube ip

Output of above command for me is:


192.168.49.2

You can combine port number and host IP address to make URL of exposed service for me URL will be http://192.168.49.2:30147 and http://192.168.49.2:30692

Thursday, 3 June 2021

Fixing JBOSS-LOCAL-USER: javax.security.sasl.SaslException: ELY05128: Failed to read challenge file

 Recently I was trying to access EJBs hosted in wildfly 23 from a remote machine. Earlier I tested the client running on same machine and it was working fine. But when I put the client on a remote machine it started failing with a strange FileNotFoundError.


Caused by: javax.security.sasl.SaslException: Authentication failed: all available authentication mechanisms failed:
   JBOSS-LOCAL-USER: javax.security.sasl.SaslException: ELY05128: Failed to read challenge file [Caused by java.io.FileNotFoundException: /xxx/wildfly/standalone/tmp/auth/local3418030740192890591.challenge (No such file or directory)]
        at org.jboss.remoting3.remote.ClientConnectionOpenListener.allMechanismsFailed(ClientConnectionOpenListener.java:109) ~[jboss-client.jar:20.0.1.Final]
        at org.jboss.remoting3.remote.ClientConnectionOpenListener$Capabilities.handleEvent(ClientConnectionOpenListener.java:445) ~[jboss-client.jar:20.0.1.Final]
        at org.jboss.remoting3.remote.ClientConnectionOpenListener$Capabilities.handleEvent(ClientConnectionOpenListener.java:244) ~[jboss-client.jar:20.0.1.Final]
        at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92) ~[jboss-client.jar:20.0.1.Final]
        at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66) ~[jboss-client.jar:20.0.1.Final]
        at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89) ~[jboss-client.jar:20.0.1.Final]
        at org.xnio.nio.WorkerThread.run(WorkerThread.java:591) ~[jboss-client.jar:20.0.1.Final]
        at ...asynchronous invocation...(Unknown Source) ~[?:?]
        at org.jboss.remoting3.EndpointImpl.connect(EndpointImpl.java:599) ~[jboss-client.jar:20.0.1.Final]
        at org.jboss.remoting3.EndpointImpl.connect(EndpointImpl.java:565) ~[jboss-client.jar:20.0.1.Final]
        at org.jboss.remoting3.ConnectionInfo$None.getConnection(ConnectionInfo.java:82) ~[jboss-client.jar:20.0.1.Final]
        at org.jboss.remoting3.ConnectionInfo.getConnection(ConnectionInfo.java:55) ~[jboss-client.jar:20.0.1.Final]
        at org.jboss.remoting3.EndpointImpl.doGetConnection(EndpointImpl.java:499) ~[jboss-client.jar:20.0.1.Final]
        at org.jboss.remoting3.EndpointImpl.getConnectedIdentity(EndpointImpl.java:445) ~[jboss-client.jar:20.0.1.Final]
        at org.jboss.remoting3.UncloseableEndpoint.getConnectedIdentity(UncloseableEndpoint.java:52) ~[jboss-client.jar:20.0.1.Final]
        at org.wildfly.naming.client.remote.RemoteNamingProvider.getFuturePeerIdentityPrivileged(RemoteNamingProvider.java:151) ~[jboss-client.jar:20.0.1.Final]
        at org.wildfly.naming.client.remote.RemoteNamingProvider.lambda$getFuturePeerIdentity$0(RemoteNamingProvider.java:138) ~[jboss-client.jar:20.0.1.Final]
        at org.wildfly.naming.client.remote.RemoteNamingProvider$$Lambda$80/601221733.run(Unknown Source) ~[?:?]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_31]
        at org.wildfly.naming.client.remote.RemoteNamingProvider.getFuturePeerIdentity(RemoteNamingProvider.java:138) ~[jboss-client.jar:20.0.1.Final]
        at org.wildfly.naming.client.remote.RemoteNamingProvider.getPeerIdentity(RemoteNamingProvider.java:126) ~[jboss-client.jar:20.0.1.Final]
        at org.wildfly.naming.client.remote.RemoteNamingProvider.getPeerIdentityForNaming(RemoteNamingProvider.java:106) ~[jboss-client.jar:20.0.1.Final]
        ... 90 more
        Suppressed: javax.security.sasl.SaslException: ELY05128: Failed to read challenge file
                at org.wildfly.security.sasl.localuser.LocalUserClient.evaluateMessage(LocalUserClient.java:108) ~[jboss-client.jar:20.0.1.Final]
                at org.wildfly.security.sasl.util.AbstractSaslParticipant.evaluateMessage(AbstractSaslParticipant.java:219) ~[jboss-client.jar:20.0.1.Final]
                at org.wildfly.security.sasl.util.AbstractSaslClient.evaluateChallenge(AbstractSaslClient.java:98) ~[jboss-client.jar:20.0.1.Final]
                at org.wildfly.security.sasl.util.AbstractDelegatingSaslClient.evaluateChallenge(AbstractDelegatingSaslClient.java:54) ~[jboss-client.jar:20.0.1.Final]
                at org.wildfly.security.sasl.util.PrivilegedSaslClient.lambda$evaluateChallenge$0(PrivilegedSaslClient.java:55) ~[jboss-client.jar:20.0.1.Final]
                at org.wildfly.security.sasl.util.PrivilegedSaslClient$$Lambda$128/635454149.run(Unknown Source) ~[?:?]
                at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_31]
                at org.wildfly.security.sasl.util.PrivilegedSaslClient.evaluateChallenge(PrivilegedSaslClient.java:55) ~[jboss-client.jar:20.0.1.Final]
                at org.jboss.remoting3.remote.ClientConnectionOpenListener$Authentication.lambda$handleEvent$0(ClientConnectionOpenListener.java:649) ~[jboss-client.jar:20.0.1.Final]
                at org.jboss.remoting3.remote.ClientConnectionOpenListener$Authentication$$Lambda$129/1032360688.run(Unknown Source) ~[?:?]
                at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:991) ~[jboss-client.jar:20.0.1.Final]
                at org.jboss.remoting3.EndpointImpl$TrackingExecutor$$Lambda$127/1508635946.run(Unknown Source) ~[?:?]
                at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) ~[jboss-client.jar:20.0.1.Final]
                at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) ~[jboss-client.jar:20.0.1.Final]
                at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) ~[jboss-client.jar:20.0.1.Final]
                at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) ~[jboss-client.jar:20.0.1.Final]
                at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_31]

Initially the problem looked like to be a bug in LocalUserClient class. But later I found that it is a functionality. For authenticating local users using sasl server creates a challenge file on server and send path of the file to client and client is supposed to return content of that file. If client is also running on same machine then it is able to read content of the file and return and authentication passes. If you are doing it from a remote machine it fails.

So how to authenticate from a remote client?

You need comment out local authentication line from your standalone-full.xml file to force it use remote authentication mechanism.


            <security-realm name="ApplicationRealm">
                <server-identities>
                    <ssl>
                        <keystore path="application.keystore" relative-to="jboss.server.config.dir" keystore-password="xxx" alias="xxx" key-password="xxx" generate-self-signed-certificate-host="localhost"/>
                    </ssl>
                </server-identities>
                <authentication>
                   <!-- <local default-user="$local" allowed-users="*" skip-group-loading="true"/>-->
                    <properties path="application-users.properties" relative-to="jboss.server.config.dir"/>
                </authentication>
                <authorization>
                    <properties path="application-roles.properties" relative-to="jboss.server.config.dir"/>
                </authorization>
            </security-realm>

After that you need to use add-user.sh command to add a user to wildfly. Now create wildfly-config.xml file with your user's credential on client machine.


<configuration>
    <authentication-client xmlns="urn:elytron:1.0">
        <authentication-rules>
            <rule use-configuration="default"/>
        </authentication-rules>
        <authentication-configurations>
            <configuration name="default">
                <sasl-mechanism-selector selector="#ALL"/>
                <set-user-name name="user"/>
                <credentials>
                    <clear-password password="password"/>
                </credentials>
            </configuration>
        </authentication-configurations>
    </authentication-client>
</configuration>

Now you can pass this credentials file to your EJB client using  parameter:


-Dwildfly.config.url=<your dir>/wildfly-config.xml

Now your client should start working without any authentication error. 

Please comment if you need any more information on this.

Saturday, 12 September 2020

Faster way of creating ZIP file

Zip file is normally used for distributing files and directories.  Zip file serves two purposes one is compressing the file and another of creating a achive of multiple files and directories. Now a days with faster internet speed and precompressed files like jpg and mp4 etc. It will be faster to disable compression altogether to speed up the compression. I wrote a simple program in java to create zip file without compression. I was using a mp4 file as content of zip file. With compression enabled it was taking 5 time more time and giving a compression of 5% which is not worth for time spent. Following is the source code:


package zipperformance;

import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.zip.Deflater;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
import java.util.zip.CRC32;

public class ZipCreater {
	public static void main(String[] args) {
		
		long startTime = System.currentTimeMillis();
		zipDirectory("files", "test.zip");
		long endTime = System.currentTimeMillis();
		
		System.out.println(endTime-startTime);
		
		
	}
	
    public static void zipDirectory( String dirPath, String zipFilePath )
    {

        FileOutputStream fos = null;
        try
        {
            // Create the file output streams for both the file and the zip.

            File file = new File( zipFilePath );
            if(file.getParentFile()!=null) {
            	file.getParentFile().mkdirs();
            }
            fos = new FileOutputStream( zipFilePath );
            BufferedOutputStream bos = new BufferedOutputStream(fos);
            ZipOutputStream zos = new ZipOutputStream( fos );
            zos.setLevel(Deflater.BEST_SPEED);
            dirFunc( dirPath, dirPath, zos );

            // Close the file output streams for both the file and the zip.
            zos.flush();
            zos.close();
            fos.close();
        }
        catch ( IOException e )
        {
        	e.printStackTrace();
        }
    }
    
    private static void dirFunc( String dirName, String baseExportPath, ZipOutputStream zos )
    {
        try
        {
            File dirObj = new File( dirName );
            if ( dirObj.exists() == true )
            {
                if ( dirObj.isDirectory() == true )
                {
                    // Create an array of File objects, one for each file or directory in dirObj.
                    File[] fileList = dirObj.listFiles();
                    // Loop through File array and display.
                    for ( int i = 0; i < fileList.length; i++ )
                    {
                        if ( fileList[i].isDirectory() )
                        {
                            dirFunc( fileList[i].getPath(), baseExportPath, zos );
                        }
                        else if ( fileList[i].isFile() )

                        {
                            // Call the zipFunc function
                            zipFunc( fileList[i].getPath(), baseExportPath, zos );
                        }
                    }
                }
            }
        }
        catch ( Exception e )
        {
            e.printStackTrace();
        }
    }

    private static void zipFunc( String filePath, String baseExportPath, ZipOutputStream zos )
    {
        try
        {
            String absolutePath = filePath;
            String prefixFilePath = baseExportPath;

            int index = prefixFilePath.length();

            filePath = filePath.substring( ++index );

            // Create a file input stream and a buffered input stream.
            long size = Files.size(Paths.get(absolutePath));
            FileInputStream fis = new FileInputStream( absolutePath );
            BufferedInputStream bis = new BufferedInputStream( fis );
            CRC32 crc32 = new CRC32();
            byte[] data = new byte[1024*128];
            int byteCount;

            // Create a loop that reads from the buffered input stream and writes
            // to the zip output stream until the bis has been entirely read.
            while ( ( byteCount = bis.read( data, 0, data.length ) ) > -1 )
            {
                crc32.update( data, 0, byteCount );
            }
            long crc=crc32.getValue();

            // Create a Zip Entry and put it into the archive (no data yet).
            ZipEntry fileEntry = new ZipEntry( filePath );
            fileEntry.setMethod(ZipEntry.STORED);
            fileEntry.setSize(size);
            fileEntry.setCrc(crc);
            zos.putNextEntry( fileEntry );

            bis.close();
            fis = new FileInputStream( absolutePath );
            bis = new BufferedInputStream( fis );
            
            // Create a loop that reads from the buffered input stream and writes
            // to the zip output stream until the bis has been entirely read.
            while ( ( byteCount = bis.read( data, 0, data.length ) ) > -1 )
            {
                zos.write( data, 0, byteCount );
            }
        }
        catch ( IOException e )
        {
        	e.printStackTrace();
        }
    }
}


java.lang.ClassNotFoundException: Could not load requested class : org.postgresql.Driver in Wildfly

Recently I was trying migrate our application from wildfly 10 to wildfly 20. We faced the following error:


Unable to create requested service [org.hibernate.engine.jdbc.env.spi.JdbcEnvironment]
        at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1$1.run(PersistenceUnitServiceImpl.java:198)
        at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1$1.run(PersistenceUnitServiceImpl.java:128)
        at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:658)
        at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1.run(PersistenceUnitServiceImpl.java:212)
        at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
        at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
        at java.lang.Thread.run(Thread.java:748)
        at org.jboss.threads.JBossThread.run(JBossThread.java:485)
Caused by: org.hibernate.service.spi.ServiceException: Unable to create requested service [org.hibernate.engine.jdbc.env.spi.JdbcEnvironment]
        at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:275)
        at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:237)
        at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:214)
        at org.hibernate.id.factory.internal.DefaultIdentifierGeneratorFactory.injectServices(DefaultIdentifierGeneratorFactory.java:152)
        at org.hibernate.service.internal.AbstractServiceRegistryImpl.injectDependencies(AbstractServiceRegistryImpl.java:286)
        at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:243)
        at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:214)
        at org.hibernate.boot.internal.InFlightMetadataCollectorImpl.<init>(InFlightMetadataCollectorImpl.java:179)
        at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.complete(MetadataBuildingProcess.java:119)
        at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.metadata(EntityManagerFactoryBuilderImpl.java:1215)
        at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:1246)
        at org.jboss.as.jpa.hibernate5.TwoPhaseBootstrapImpl.build(TwoPhaseBootstrapImpl.java:44)
        at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1$1.run(PersistenceUnitServiceImpl.java:170)
        ... 9 more
Caused by: org.hibernate.boot.registry.classloading.spi.ClassLoadingException: HHH010003: JDBC Driver class not found: org.postgresql.Driver
        at org.hibernate.c3p0.internal.C3P0ConnectionProvider.configure(C3P0ConnectionProvider.java:130)
        at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.configureService(StandardServiceRegistryImpl.java:100)
        at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:246)
        at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:214)
        at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.buildJdbcConnectionAccess(JdbcEnvironmentInitiator.java:145)
        at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:66)
        at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:35)
        at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:94)
        at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:263)
        ... 21 more
Caused by: org.hibernate.boot.registry.classloading.spi.ClassLoadingException: Unable to load class [org.postgresql.Driver]
        at org.hibernate.boot.registry.classloading.internal.ClassLoaderServiceImpl.classForName(ClassLoaderServiceImpl.java:134)
        at org.hibernate.c3p0.internal.C3P0ConnectionProvider.configure(C3P0ConnectionProvider.java:127)
        ... 29 more
Caused by: java.lang.ClassNotFoundException: Could not load requested class : org.postgresql.Driver
        at org.hibernate.boot.registry.classloading.internal.AggregatedClassLoader.findClass(AggregatedClassLoader.java:210)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at org.hibernate.boot.registry.classloading.internal.ClassLoaderServiceImpl.classForName(ClassLoaderServiceImpl.java:131)
        ... 30 more

Here I am going to provide solution to the problem. For loading postgresql classes you need to create a module for postgres and reference that module in your application jar.

Steps for creating modue is:

  1. Copy postgres jar to /home/wildfly/modules/system/layers/base/org/postgres/main/postgresql-42.2.2.jar
  2. Create /home/wildfly/modules/system/layers/base/org/postgres/main/module.xml file with following content.

<module xmlns="urn:jboss:module:1.7" name="org.postgres">
        <resources>
                <resource-root path="postgresql-42.2.2.jar" />
        </resources>
        <dependencies>
                <module name="javax.api"/>
                <module name="javax.transaction.api"/>
        </dependencies>
</module>

Now you need to refer the module from your application JAR. For doing that you need to create jboss-deployment-structure.xml file with following content:


<?xml version="1.0" encoding="UTF-8"?>
  <jboss-deployment-structure>
      <ear-subdeployments-isolated>false</ear-subdeployments-isolated>
          <deployment>
                <resources>
                        <resource-root path="myapplication.jar" />
                </resources>
                <dependencies>
                    <module name="org.hibernate" export="true"/>
                    <module name="javax.api" />
                    <module name="org.postgres" export="true"/>
                    <module name="org.infinispan" />
                    <module name="org.apache.commons.lang" />
                </dependencies>
                <exclusions>
                        <module name="org.jboss.ejb-client"/>
                </exclusions>

          </deployment>
  </jboss-deployment-structure>

Please note that export="true" is required, without this class will not be loaded.



Friday, 11 September 2020

HHH010002: C3P0 using driver: null at URL: null

Recently I was working on porting our application to wildfly 20 from wildfly 10 it involved upgrading hibernate to 5.3.17.  We were using C3P0 connection pool in our persistence.xml. We got the following error in our logs


2020-09-11 19:28:35,785 INFO  [org.hibernate.engine.jdbc.connections.internal.ConnectionProviderInitiator] (ServerService Thread Pool -- 86) HHH000130: Instantiating explicit connection provider: org.hibernate.service.jdbc.connections.internal.C3P0ConnectionProvider
2020-09-11 19:28:35,790 INFO  [org.hibernate.c3p0.internal.C3P0ConnectionProvider] (ServerService Thread Pool -- 86) HHH010002: C3P0 using driver: null at URL: null
2020-09-11 19:28:35,790 INFO  [org.hibernate.c3p0.internal.C3P0ConnectionProvider] (ServerService Thread Pool -- 86) HHH10001001: Connection properties: {user=pgctools, password=****}
2020-09-11 19:28:35,790 INFO  [org.hibernate.c3p0.internal.C3P0ConnectionProvider] (ServerService Thread Pool -- 86) HHH10001003: Autocommit mode: false
2020-09-11 19:28:35,791 WARN  [org.hibernate.c3p0.internal.C3P0ConnectionProvider] (ServerService Thread Pool -- 86) HHH10001006: No JDBC Driver class was specified by property hibernate.connection.driver_class
2020-09-11 19:28:35,824 INFO  [com.mchange.v2.log.MLog] (MLog-Init-Reporter) MLog clients using slf4j logging.
2020-09-11 19:28:35,965 INFO  [com.mchange.v2.c3p0.C3P0Registry] (ServerService Thread Pool -- 86) Initializing c3p0-0.9.5.2 [built 08-December-2015 22:06:04 -0800; debug? true; trace: 10]
2020-09-11 19:28:36,021 INFO  [org.hibernate.c3p0.internal.C3P0ConnectionProvider] (ServerService Thread Pool -- 86) HHH10001007: JDBC isolation level: <unknown>
2020-09-11 19:28:36,050 INFO  [com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource] (ServerService Thread Pool -- 86) Initializing c3p0 pool... com.mchange.v2.c3p0.PoolBackedDataSource@8c1576a0 [ connectionPoolDataSource -> com.mchange.v2.c3p0.WrapperConnectionPoolDataSource@9df832bb [ acquireIncrement -> 3, acquireRetryAttempts -> 30, acquireRetryDelay -> 1000, autoCommitOnClose -> true, automaticTestTable -> null, breakAfterAcquireFailure -> false, checkoutTimeout -> 0, connectionCustomizerClassName -> null, connectionTesterClassName -> com.mchange.v2.c3p0.impl.DefaultConnectionTester, contextClassLoaderSource -> caller, debugUnreturnedConnectionStackTraces -> false, factoryClassLocation -> null, forceIgnoreUnresolvedTransactions -> false, forceSynchronousCheckins -> false, identityToken -> 1hge13bacyt6x751iz2mtg|5014cd2c, idleConnectionTestPeriod -> 0, initialPoolSize -> 20, maxAdministrativeTaskTime -> 0, maxConnectionAge -> 0, maxIdleTime -> 300, maxIdleTimeExcessConnections -> 0, maxPoolSize -> 200, maxStatements -> 50, maxStatementsPerConnection -> 0, minPoolSize -> 20, nestedDataSource -> com.mchange.v2.c3p0.DriverManagerDataSource@974ea785 [ description -> null, driverClass -> null, factoryClassLocation -> null, forceUseNamedDriverClass -> false, identityToken -> 1hge13bacyt6x751iz2mtg|5051daf8, jdbcUrl -> null, properties -> {user=******, password=******} ], preferredTestQuery -> SELECT 1;, privilegeSpawnedThreads -> false, propertyCycle -> 0, statementCacheNumDeferredCloseThreads -> 0, testConnectionOnCheckin -> true, testConnectionOnCheckout -> false, unreturnedConnectionTimeout -> 0, usesTraditionalReflectiveProxies -> false; userOverrides: {} ], dataSourceName -> null, extensions -> {}, factoryClassLocation -> null, identityToken -> 1hge13bacyt6x751iz2mtg|18755649, numHelperThreads -> 3 ]
2020-09-11 19:29:06,113 WARN  [com.mchange.v2.resourcepool.BasicResourcePool] (C3P0PooledConnectionPoolManager[identityToken->1hge13bacyt6x751iz2mtg|18755649]-HelperThread-#2) com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@3c1f0be -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (30). Last acquisition attempt exception: : java.lang.NullPointerException
        at org.postgresql.Driver.parseURL(Driver.java:547)
        at org.postgresql.Driver.acceptsURL(Driver.java:466)
        at java.sql.DriverManager.getDriver(DriverManager.java:299)
        at com.mchange.v2.c3p0.DriverManagerDataSource.driver(DriverManagerDataSource.java:285)
        at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
        at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
        at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
        at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
        at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)
        at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125)
        at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44)
        at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1870)
        at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)

The connection pool was unable to create connection because of missing database URL and driver class. Although the database URL and driver class was configured in persistence.xml still the hibernate was not able to pick it up. I had to download hibernate code and debug it for two days to figure out the problem. Our persistence.xml looked like this:


<persistence xmlns="http://java.sun.com/xml/ns/persistence"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd"
   version="1.0">
   <persistence-unit name="UnitName" transaction-type="JTA">
        <properties>
            <property name="jboss.as.jpa.providerModule" value="org.hibernate"/>
        </properties>
    <provider>org.hibernate.ejb.HibernatePersistence</provider>
      <jta-data-source>java:/PostgresDS</jta-data-source>

      <properties>
        <property name="hibernate.dialect" value="org.hibernate.dialect.PostgreSQLDialect"/>
        <property name="hibernate.show_sql" value="false"/>
        <property name="hibernate.format_sql" value="false" />
        <property name="hibernate.transaction.jta.platform" value="org.hibernate.service.jta.platform.internal.JBossAppServerJtaPlatform"/>
        <property name="hibernate.transaction.manager_lookup_class" value="org.hibernate.transaction.JBossTransactionManagerLookup"/>
        <property name="hibernate.session_factory_name" value="java:/jboss/SessionFactory"/>
        <property name="hibernate.cache.use_query_cache" value="true"/>
        <property name="hibernate.cache.use_second_level_cache" value="true"/>
        <property name="hibernate.cache.infinispan.cachemanager" value="java:jboss/infinispan/container/hibernate"/>
        <property name="hibernate.current_session_context_class" value="jta" />

       <property name="hibernate.connection.provider_class" value="org.hibernate.service.jdbc.connections.internal.C3P0ConnectionProvider"/>
       <property name="hibernate.connection.driver_class" value="org.postgresql.Driver" />
       <property name="hibernate.connection.url" value="jdbc:postgresql://localhost:5432/database" />
       <property name="hibernate.connection.username" value="user" />
       <property name="hibernate.connection.password" value="password" />
       <property name="hibernate.c3p0.min_size" value="20"/>
       <property name="hibernate.c3p0.max_size" value="200"/>
       <property name="hibernate.c3p0.timeout" value="300"/>
       <property name="hibernate.c3p0.max_statements" value="50"/>
       <property name="hibernate.c3p0.idle_test_periods" value="3000"/>
       <property name="hibernate.c3p0.testConnectionOnCheckin" value="true"/>
       <property name="hibernate.c3p0.preferredTestQuery" value="SELECT 1;"/>
       <property name="hibernate.c3p0.autoCommitOnClose" value="true"/>
      </properties>
   </persistence-unit>
</persistence>

The root cause of the problem was the datasource mentioned in persistence.xml file. Hibernate removes URL and database driver settings from configuration if datasource is provided in the configuration. It is because if datasource is provided then datasource will be used for getting connections so there is no use of database URL and driver class. But C3P0 connection pool required that. I removed the jta-data-source setting and it still did not work because wildfly inserted a default datasource with name ExampleDS into our persistence unit. We need to disable insertion of default data source using a wildfly specific property in persistence.xml file.

The following line needs to be inserted:


<property name="wildfly.jpa.allowdefaultdatasourceuse" value="false" />

After adding the above property C3P0 connection pool started working.