RE: [Newsletter] Re: [Newsletter] Re: [Newsletter] Re: TomEE Performance

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

RE: [Newsletter] Re: [Newsletter] Re: [Newsletter] Re: TomEE Performance

fabian-a.richter
Thx Jonathan, but we are using haveged since 2 months into development :)

-----Original Message-----
From: Jonathan S. Fisher <[hidden email]>
Sent: Saturday, September 08, 2018 6:16 PM
To: [hidden email]
Subject: *EXT* [Newsletter] Re: [Newsletter] Re: [Newsletter] Re: TomEE Performance

Hey one more note, if you're on a virtualized Linux system and having the random pool as a bottleneck, take a look at http://www.issihosts.com/haveged.
This is a pretty simple way to increase the entropy pool system-wide and has solved several issues for me in the past. Be sure to read the fine print of course for your particular business case.

Cheers!

On Tue, Sep 4, 2018 at 7:26 AM Jean-Louis Monteiro <[hidden email]>
wrote:

> You are welcome.
>
> --
> Jean-Louis Monteiro
> http://twitter.com/jlouismonteiro
> http://www.tomitribe.com
>
> On Tue, Sep 4, 2018 at 2:17 PM, <[hidden email]>
> wrote:
>
> > Hi Romain,
> >
> > thank you so much for your detailed feedback. After I created the
> > second threaddump I had a look into it myself and came to the same conclusions.
> We
> > will investigate further, and if there are any more TomEE related
> > performance issues, I will get back in contact.
> >
> > Thanks all for the replies!
> >
> > Best
> > Fabian
> >
> > -----Original Message-----
> > From: Romain Manni-Bucau <[hidden email]>
> > Sent: Tuesday, September 04, 2018 2:08 PM
> > To: [hidden email]
> > Subject: *EXT* [Newsletter] Re: [Newsletter] Re: TomEE Performance
> >
> > Hi Fabian,
> >
> > a few pointers and directions to check:
> >
> > 1. Ensure to test with securerandom.source=/dev/./urandom since you
> > use a lot of crypto 2. You use hsqldb which is known to not scale
> > very well
> with
> > the concurrency, maybe switch to another database with a correctly
> > configured connection pool 3. You use DataBaseRealm which does a
> > lookup
> of
> > the connection for each authentication which is synchronized and has
> > no cache on the password hashes so it can be slow at runtime
> >
> > Personally i would start by using a fast realm (even if it always
> > says
> > "ok") to validate this hypothesis before investigating other cases.
> >
> > Romain Manni-Bucau
> > @rmannibucau <https://twitter.com/rmannibucau> |  Blog <
> > https://rmannibucau.metawerx.net/> | Old Blog <
> > http://rmannibucau.wordpress.com> | Github <
> https://github.com/rmannibucau>
> > | LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book <
> > https://www.packtpub.com/application-development/java-
> > ee-8-high-performance>
> >
> >
> > Le mar. 4 sept. 2018 à 12:15, <[hidden email]> a
> > écrit :
> >
> > > You were right, my student did something wrong with the first
> > > thread dump, I uploaded a correct one:
> > > https://gist.github.com/TrustedGate/83781f05950f245a3bfd2388c5bfd7
> > > ab
> > >
> > > Also at the bottom there is a vmstat result, its definitely
> > > multithreaded
> > > :)
> > >
> > > -----Original Message-----
> > > From: Jean-Louis Monteiro <[hidden email]>
> > > Sent: Tuesday, September 04, 2018 11:59 AM
> > > To: [hidden email]
> > > Subject: *EXT* [Newsletter] Re: TomEE Performance
> > >
> > > I would need to check on this one and probably investigate a bit
> > > more what you are doing.
> > >
> > > That being said, I can confirm only one thread is currently working.
> > > So either jmeter is only sending monothreaded requests or there is
> > > something else.
> > > But you aren't doing multiple requests in parallel
> > >
> > > --
> > > Jean-Louis Monteiro
> > > http://twitter.com/jlouismonteiro
> > > http://www.tomitribe.com
> > >
> > > On Tue, Sep 4, 2018 at 11:53 AM,
> > > <[hidden email]>
> > > wrote:
> > >
> > > > We have
> > > >
> > > > @Resource
> > > > WebServiceContext webserviceContext;
> > > >
> > > > in our SOAP api class, that should not work with @Singleton, or
> > > > am I mistaken?
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Jean-Louis Monteiro <[hidden email]>
> > > > Sent: Tuesday, September 04, 2018 11:41 AM
> > > > To: [hidden email]
> > > > Subject: *EXT* [Newsletter] Re: [Newsletter] Re: [Newsletter] Re:
> > > > TomEE Performance
> > > >
> > > > Ah ok. Well I was asking if you were injecting when you took the
> > > > thread dump because from a server point of view I saw only one
> > > > thread working.
> > > > If you were using jmeter with multiple virtual users, I was
> > > > expecting to see more than one thread working.
> > > >
> > > > I'll double check.
> > > >
> > > > @Singleton is by default using Lock WRITE which prevents
> > > > multiple threads to access the singleton.
> > > > If you don't need synchronization or if your code is thread safe
> > > > (no instance variables, etc). You can safely use @Singleton with
> > > > Lock READ which will be faster than @Stateless because there is
> > > > no pool involved. And again, there is no tuning to do
> > > >
> > > >
> > > >
> > > > --
> > > > Jean-Louis Monteiro
> > > > http://twitter.com/jlouismonteiro http://www.tomitribe.com
> > > >
> > > > On Tue, Sep 4, 2018 at 11:37 AM,
> > > > <[hidden email]>
> > > > wrote:
> > > >
> > > > > Injecting? The jmeter was running during the threaddump. What
> > > > > do you mean with injecting?
> > > > >
> > > > > Is Singleton (Lock.READ) not even more of a bottleneck when it
> > > > > comes to multiple concurrent requests? IIRC we tried Singleton
> > > > > before, but not sure what the reason was why we went with
> > @Stateless...
> > > > >
> > > > > -----Original Message-----
> > > > > From: Jean-Louis Monteiro <[hidden email]>
> > > > > Sent: Tuesday, September 04, 2018 11:30 AM
> > > > > To: [hidden email]
> > > > > Subject: *EXT* [Newsletter] Re: [Newsletter] Re: TomEE
> > > > > Performance
> > > > >
> > > > > Yes exactly.
> > > > >
> > > > > Were you injecting anything when you took the jstack?
> > > > > It seems that only one thread is working.
> > > > >
> > > > > We would need you to do it when you are injecting.
> > > > > If you could also give us the CPU usage when you take the
> > > > > jstack that'd be great.
> > > > > You can run `vmstat 5` on another terminal so you can see
> > > > > what's going
> > > > on.
> > > > >
> > > > > You are using Stateless Session beans. Any reason you aren't
> > > > > using a plain singleton?
> > > > > With Singleton (Lock.READ) there is no pooling involved, so
> > > > > you don't have anything to configure or tune.
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Jean-Louis Monteiro
> > > > > http://twitter.com/jlouismonteiro http://www.tomitribe.com
> > > > >
> > > > > On Tue, Sep 4, 2018 at 10:53 AM,
> > > > > <[hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Like this:
> > > > > > https://gist.github.com/TrustedGate/f670c079088404f42d69aabd
> > > > > > 409d
> > > > > > e7c4
> > > ?
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Jean-Louis Monteiro <[hidden email]>
> > > > > > Sent: Tuesday, September 04, 2018 8:38 AM
> > > > > > To: [hidden email]
> > > > > > Subject: *EXT* [Newsletter] Re: TomEE Performance
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Around SOAP, there are a couple of possible optimizations.
> > > > > > What would be helpful is to get into the docker container
> > > > > > when you are over the linear zone and get a jstack of the
> > > > > > tomee
> process.
> > > > > > Post it here or in gist and put the link here.
> > > > > >
> > > > > > Jean-Louis
> > > > > >
> > > > > > --
> > > > > > Jean-Louis Monteiro
> > > > > > http://twitter.com/jlouismonteiro http://www.tomitribe.com
> > > > > >
> > > > > > On Tue, Sep 4, 2018 at 8:27 AM,
> > > > > > <[hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > Hey,
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > we have been running some performance tests with our
> > > > > > > application (TomEE
> > > > > > > 7.0.5 based) and are stuck:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Until 4 core VMs (or docker containers) we see a linear
> > > > > > > increase in performance, which is great and was anticipated.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > But after 4 cores, we barely get 10% (with 8 cores) more
> > > > > > > performance of the system.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > We used jmeter based load tests (SOAP calls) with
> > > > > > > 10/20/30/40 and
> > > > > > > 100 threads, VM to VM via 1gbit, and 4 GB of RAM.
> > > > > > >
> > > > > > >
> > > > > > > We played around with session bean pool sizes (min set to
> > > > > > > thread count, max to 1000) and stateful bean pool settings
> > > > > > > and also with jvm heap size and GC parameters, to no avail.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Are there any more performance parameters we can toy
> > > > > > > around with in TomEE or Tomcat that you can recommend?
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Thank you and best
> > > > > > >
> > > > > > > Fabian
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

--
Jonathan | [hidden email]
Pessimists, see a jar as half empty. Optimists, in contrast, see it as half full.
Engineers, of course, understand the glass is twice as big as it needs to be.

smime.p7s (7K) Download Attachment