Encountering "ConcurrentAccessTimeoutException" when making high-volume calls to @Stateless classes

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Encountering "ConcurrentAccessTimeoutException" when making high-volume calls to @Stateless classes

Kean Erickson
 I'm using TomEE 7.0.4, EJB version 3.1
I often see this when making a high volume of calls to methods in
@Stateless classes:

javax.ejb.ConcurrentAccessTimeoutException: No instances available in
Stateless Session Bean pool.  Waited 30 SECONDS

I've set maxSize to 1000 in my tomee.xml and I still receive the error just
as soon as I would without maxSize specified at all, which leads me to
believe it's not doing anything. I see recommendations on this list to use
@Singleton with a read lockmode, but that doesn't seem to make a lot of
sense with our current arrangement.



Thanks!
-Kean
Reply | Threaded
Open this post in threaded view
|

Re: Encountering "ConcurrentAccessTimeoutException" when making high-volume calls to @Stateless classes

jgallimore
Hi

Can you get a thread dump when you're seeing this issue?

Thanks

Jon

On Wed, Oct 10, 2018 at 1:50 AM Kean Erickson <[hidden email]>
wrote:

>  I'm using TomEE 7.0.4, EJB version 3.1
> I often see this when making a high volume of calls to methods in
> @Stateless classes:
>
> javax.ejb.ConcurrentAccessTimeoutException: No instances available in
> Stateless Session Bean pool.  Waited 30 SECONDS
>
> I've set maxSize to 1000 in my tomee.xml and I still receive the error just
> as soon as I would without maxSize specified at all, which leads me to
> believe it's not doing anything. I see recommendations on this list to use
> @Singleton with a read lockmode, but that doesn't seem to make a lot of
> sense with our current arrangement.
>
>
>
> Thanks!
> -Kean
>
Reply | Threaded
Open this post in threaded view
|

Re: Encountering "ConcurrentAccessTimeoutException" when making high-volume calls to @Stateless classes

Kean Erickson
Sure, thread dump is attached. I had to censor out references to my code, but calls in the Stateless class in question are marked with three exclamations, "!!!"

After a few minutes, this happens for every call  to a method in that @Stateless class, which is acquired and used in non-EJB classes via BeanManager.getReference


10-Oct-2018 10:35:50.749 SEVERE [pool-17-thread-9]  javax.ejb.ConcurrentAccessTimeoutException: No instances available in Stateless Session Bean pool.  Waited 30 SECONDS
    at org.apache.openejb.core.stateless.StatelessInstanceManager.getInstance(StatelessInstanceManager.java:226)
    at org.apache.openejb.core.stateless.StatelessContainer.invoke(StatelessContainer.java:204)
    at org.apache.openejb.core.ivm.EjbObjectProxyHandler.synchronizedBusinessMethod(EjbObjectProxyHandler.java:265)
    at org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(EjbObjectProxyHandler.java:260)
    at org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(EjbObjectProxyHandler.java:89)
    at org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(BaseEjbProxyHandler.java:347)
    at (method in stateless class)
    at (method in non-EJB class)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)




On Wed, Oct 10, 2018 at 1:45 AM Jonathan Gallimore <[hidden email]> wrote:
Hi

Can you get a thread dump when you're seeing this issue?

Thanks

Jon

tomee-dump.txt (307K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Encountering "ConcurrentAccessTimeoutException" when making high-volume calls to @Stateless classes

jgallimore
At first glance, it looks like your threads are actually stuck trying to
borrow a database connection from the pool. What does your stateless pool
config and your datasource config look like (remove anything sensitive,
like passwords etc)?

Jon

On Wed, Oct 10, 2018 at 7:13 PM Kean Erickson <[hidden email]>
wrote:

> Sure, thread dump is attached. I had to censor out references to my code,
> but calls in the Stateless class in question are marked with three
> exclamations, "!!!"
>
> After a few minutes, this happens for every call  to a method in that
> @Stateless class, which is acquired and used in non-EJB classes via
> BeanManager.getReference
>
>
> 10-Oct-2018 10:35:50.749 SEVERE [pool-17-thread-9]
> javax.ejb.ConcurrentAccessTimeoutException: No instances available in
> Stateless Session Bean pool.  Waited 30 SECONDS
>     at
> org.apache.openejb.core.stateless.StatelessInstanceManager.getInstance(StatelessInstanceManager.java:226)
>     at
> org.apache.openejb.core.stateless.StatelessContainer.invoke(StatelessContainer.java:204)
>     at
> org.apache.openejb.core.ivm.EjbObjectProxyHandler.synchronizedBusinessMethod(EjbObjectProxyHandler.java:265)
>     at
> org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(EjbObjectProxyHandler.java:260)
>     at
> org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(EjbObjectProxyHandler.java:89)
>     at
> org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(BaseEjbProxyHandler.java:347)
>     at (method in stateless class)
>     at (method in non-EJB class)
>     at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>     at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>     at java.lang.Thread.run(Thread.java:748)
>
>
>
>
> On Wed, Oct 10, 2018 at 1:45 AM Jonathan Gallimore <
> [hidden email]> wrote:
>
>> Hi
>>
>> Can you get a thread dump when you're seeing this issue?
>>
>> Thanks
>>
>> Jon
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Encountering "ConcurrentAccessTimeoutException" when making high-volume calls to @Stateless classes

Kean Erickson
That actually answered my question, I didn't have a proper stateless pool
config in my tomee.xml... I assumed everything would be configured under
<Resource> so I was putting the maxSize property there, hence the property
not working. I added a <Container> and put the prop there instead. Things
are working as expected now. Thanks!

On Wed, Oct 10, 2018 at 12:02 PM Jonathan Gallimore <
[hidden email]> wrote:

> At first glance, it looks like your threads are actually stuck trying to
> borrow a database connection from the pool. What does your stateless pool
> config and your datasource config look like (remove anything sensitive,
> like passwords etc)?
>
> Jon
>
> On Wed, Oct 10, 2018 at 7:13 PM Kean Erickson <[hidden email]>
> wrote:
>
> > Sure, thread dump is attached. I had to censor out references to my code,
> > but calls in the Stateless class in question are marked with three
> > exclamations, "!!!"
> >
> > After a few minutes, this happens for every call  to a method in that
> > @Stateless class, which is acquired and used in non-EJB classes via
> > BeanManager.getReference
> >
> >
> > 10-Oct-2018 10:35:50.749 SEVERE [pool-17-thread-9]
> > javax.ejb.ConcurrentAccessTimeoutException: No instances available in
> > Stateless Session Bean pool.  Waited 30 SECONDS
> >     at
> >
> org.apache.openejb.core.stateless.StatelessInstanceManager.getInstance(StatelessInstanceManager.java:226)
> >     at
> >
> org.apache.openejb.core.stateless.StatelessContainer.invoke(StatelessContainer.java:204)
> >     at
> >
> org.apache.openejb.core.ivm.EjbObjectProxyHandler.synchronizedBusinessMethod(EjbObjectProxyHandler.java:265)
> >     at
> >
> org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(EjbObjectProxyHandler.java:260)
> >     at
> >
> org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(EjbObjectProxyHandler.java:89)
> >     at
> >
> org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(BaseEjbProxyHandler.java:347)
> >     at (method in stateless class)
> >     at (method in non-EJB class)
> >     at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> >     at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> >     at java.lang.Thread.run(Thread.java:748)
> >
> >
> >
> >
> > On Wed, Oct 10, 2018 at 1:45 AM Jonathan Gallimore <
> > [hidden email]> wrote:
> >
> >> Hi
> >>
> >> Can you get a thread dump when you're seeing this issue?
> >>
> >> Thanks
> >>
> >> Jon
> >>
> >
>