Rework TomEE 9 to use Tomcat 10 and build webapps & tar.gz

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Rework TomEE 9 to use Tomcat 10 and build webapps & tar.gz

David Blevins-2
Hi All,

I've done some work to redo how we build our TomEE 9 binaries.  The primary goals I was after in the rework:

 - Give us the ability to upgrade dependency versions; Tomcat 10, Eclipselink, etc
 - Ensure we're publishing wars and tar.gz; Rodney's been asking for this.

Some minor things I did while I was in there:

 - Ensure the server identifies as "TomEE 9.0.0" rather than "TomEE 8.0.7"
 - Publish wars, zips, tars to `org.apache.tomee` groupId rather than `org.apache.tomee.jakarta`


I did this by basically copying the `tomee` directory from the main `tomee` repo and deleting everything that wasn't a webapp or the final `apache-tomee` module. After that there was lots of pom.xml tweaking to get a smaller version of the parent pom.xml created.

Future plans:

 - Copy and update CDI, Bean Validation TCK modules from TomEE 8

Future possibilities:

 - Generate "boms" for each dist like we do in TomEE 8 to get usable Maven deps
 - Add Arquillian adapter, perhaps via bytecode transformation of existing TomEE 8 adpater
 - Copy and update MicroProfile TCK modules from TomEE 8 - bytecode transforming them would be necessary

One aspect I'm interested to try tomorrow is potentially have the repo's parent pom.xml inherit from the TomEE 8.0.7-SNAPSHOT parent pom.  The theory there is that we'd inherit the versions from TomEE 8 and could/would only specify the versions we want to be different.  This would keep maintenance/duplication down and allow us to easily see what is different between TomEE 8 and 9 in terms of library versions.

Since the bytecode processing has been changed, here's a brief overview of how it works in this PR:

 - in each tomee-foo-webapp, wars are built as usual
 - in each tomee-foo-webapp the Eclipse Transformer and TomEE Patch Plugin is setup and produces a tomee-foo-webapp-transformed-9.0.0-M7-SNAPSHOT.war file
 - in the `apache-tomee` module we pull in each "transformed" webapp and build the server dists as usual, but now on a Tomcat 10.0.4 zip

I've booted the server and seen things work, but I haven't updated our TCK setup to test there.

Since it is a big change I've not pushed it and thought it would be better to create a PR so we could discuss.

 - https://github.com/apache/tomee-jakarta/pull/1

I'll wait to merge till I hear some feedback, comments, etc.  It's a lot to digest, so even "sounds promising" is good :)  Not expecting a line-by-line code review, but that's certainly welcome if you have that kind of energy.


Thoughts?



--
David Blevins
http://twitter.com/dblevins
http://www.tomitribe.com


smime.p7s (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Rework TomEE 9 to use Tomcat 10 and build webapps & tar.gz

Jean-Louis MONTEIRO
Hi David,

I'm glad you moved on this. As I mentioned a little while ago, the current
approach was not working.

I had a quick look into the PR even though I did not get a chance to check
it out and try locally. Ideally we should publish some snapshots, it does
not hurt. And then fire up a TCK build.

I'm overall fine with the approach. Sounds like a good compromise to avoid
big maintenance costs without blocking us too much on the Jakarta EE 9 and
forward.
We'll see over time if we can improve, but it looks like a good starting
point.

--
Jean-Louis Monteiro
http://twitter.com/jlouismonteiro
http://www.tomitribe.com


On Tue, Mar 23, 2021 at 8:07 AM David Blevins <[hidden email]>
wrote:

> Hi All,
>
> I've done some work to redo how we build our TomEE 9 binaries.  The
> primary goals I was after in the rework:
>
>  - Give us the ability to upgrade dependency versions; Tomcat 10,
> Eclipselink, etc
>  - Ensure we're publishing wars and tar.gz; Rodney's been asking for this.
>
> Some minor things I did while I was in there:
>
>  - Ensure the server identifies as "TomEE 9.0.0" rather than "TomEE 8.0.7"
>  - Publish wars, zips, tars to `org.apache.tomee` groupId rather than
> `org.apache.tomee.jakarta`
>
>
> I did this by basically copying the `tomee` directory from the main
> `tomee` repo and deleting everything that wasn't a webapp or the final
> `apache-tomee` module. After that there was lots of pom.xml tweaking to get
> a smaller version of the parent pom.xml created.
>
> Future plans:
>
>  - Copy and update CDI, Bean Validation TCK modules from TomEE 8
>
> Future possibilities:
>
>  - Generate "boms" for each dist like we do in TomEE 8 to get usable Maven
> deps
>  - Add Arquillian adapter, perhaps via bytecode transformation of existing
> TomEE 8 adpater
>  - Copy and update MicroProfile TCK modules from TomEE 8 - bytecode
> transforming them would be necessary
>
> One aspect I'm interested to try tomorrow is potentially have the repo's
> parent pom.xml inherit from the TomEE 8.0.7-SNAPSHOT parent pom.  The
> theory there is that we'd inherit the versions from TomEE 8 and could/would
> only specify the versions we want to be different.  This would keep
> maintenance/duplication down and allow us to easily see what is different
> between TomEE 8 and 9 in terms of library versions.
>
> Since the bytecode processing has been changed, here's a brief overview of
> how it works in this PR:
>
>  - in each tomee-foo-webapp, wars are built as usual
>  - in each tomee-foo-webapp the Eclipse Transformer and TomEE Patch Plugin
> is setup and produces a tomee-foo-webapp-transformed-9.0.0-M7-SNAPSHOT.war
> file
>  - in the `apache-tomee` module we pull in each "transformed" webapp and
> build the server dists as usual, but now on a Tomcat 10.0.4 zip
>
> I've booted the server and seen things work, but I haven't updated our TCK
> setup to test there.
>
> Since it is a big change I've not pushed it and thought it would be better
> to create a PR so we could discuss.
>
>  - https://github.com/apache/tomee-jakarta/pull/1
>
> I'll wait to merge till I hear some feedback, comments, etc.  It's a lot
> to digest, so even "sounds promising" is good :)  Not expecting a
> line-by-line code review, but that's certainly welcome if you have that
> kind of energy.
>
>
> Thoughts?
>
>
>
> --
> David Blevins
> http://twitter.com/dblevins
> http://www.tomitribe.com
>
>
   --
    Jean-Louis Monteiro
    http://twitter.com/jlouismonteiro
    http://www.tomitribe.com
Reply | Threaded
Open this post in threaded view
|

Re: Rework TomEE 9 to use Tomcat 10 and build webapps & tar.gz

David Blevins-2

> On Mar 23, 2021, at 1:43 AM, Jean-Louis Monteiro <[hidden email]> wrote:
>
> Hi David,
>
> I'm glad you moved on this. As I mentioned a little while ago, the current
> approach was not working.
>
> I had a quick look into the PR even though I did not get a chance to check
> it out and try locally. Ideally we should publish some snapshots, it does
> not hurt. And then fire up a TCK build.
I updated our tomee-tck setup so it uses these snapshots, then went ahead and pushed some snapshots.

Two TCK runs are going now for the EE 9.1 TCK on JDK 8 and 11:

 - EE 9.1 on JDK8 https://tck.work/tomee/build?id=1616541975465
 - EE 9.1 on JDK8 https://tck.work/tomee/build?id=1616542933031

Results are still rolling in, but it looks good so far.

> I'm overall fine with the approach. Sounds like a good compromise to avoid
> big maintenance costs without blocking us too much on the Jakarta EE 9 and
> forward.
> We'll see over time if we can improve, but it looks like a good starting
> point.

Thanks for the feedback!

Question for you.  I'm now remembering I need to wire back up the `jakarta-direct.properties` file you added.  I temporarily removed it from the plugin configuration as we now have the plugin used 3 times.  Should be easy to make it a dependency so it's visible.  If we're luck we can even mark it provided and have it still work -- then it won't be in the actual wars, zips, tars.

Do you vaguely recall what tests/areas this fixed?


-David


smime.p7s (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Rework TomEE 9 to use Tomcat 10 and build webapps & tar.gz

David Blevins-2
Hi All,

Some updates.  I went ahead and merged that change so we're now building wars, zips, tars and officially using Tomcat 10 in TomEE 9!

We're using Mojarra 3.0.0 and Eclipselink 3.0.0 and I've made improvements to the TomEE Patch Plugin to ensure that the versions we have in the dists is completely unmodified by the Eclipse Transformer or TomEE Patch Plugin.  Here's how that works (I'll add this to the README).

In the configuration of the plugin you can have a directive like the following:

        <configuration>
          <select>tomee-plume-webapp-transformed-.*\.war</select>
          <patchSources>
            <source>${project.basedir}/../../transform/src/patch/java/</source>
            <source>${project.basedir}/src/patch/java/</source>
          </patchSources>
          <replace>
            <jars>
              <jakarta.faces-3.0.0.jar>org.glassfish:jakarta.faces:jar:3.0.0</jakarta.faces-3.0.0.jar>
              <eclipselink-3.0.0.jar>org.eclipse.persistence:eclipselink:jar:3.0.0</eclipselink-3.0.0.jar>
            </jars>
            <resources>
              <openejb-version.properties>${project.build.outputDirectory}/openejb-version.properties</openejb-version.properties>
            </resources>
          </replace>
        </configuration>

## select

The `<select>` setting is what tells the plugin which binaries to modify.  In the above setting we're saying we want to patch the output of the Eclipse Transformer.  The input to the transformer is the regular `tomee-plume-webapp-9.0.0-M123-SNAPSHOT.war` file and the output is a new `tomee-plume-webapp-transformed-9.0.0-M123-SNAPSHOT.war` file.  It's the "transformed" file we want to further patch.

## patchSources

The `<patchSources>` list allows us to specify locations where *.java files live.  We will copy these into the `target/` directory and compile them using any *.jar files we find in the war file to create the classpath.  These compiled classes are then used to overwrite any classes by that name in any part of the war or its libraries.  This is how we handle corner cases that are too complicated to deal with using bytecode tools -- we can just specify an alternate source file.

Being able to have multiple locations for patch files keeps us from having to duplicate patch files as we build multiple war files in multiple modules.  The above configuration says grab the *.java files from the `transform` module and the *.java files from this module.

## replace/jars

The `<replace><jars>` list allows us to tell the plugin, "when you see a `jakarta.faces-3.0.0.jar` in the war file, replace it with the `org.glassfish:jakarta.faces:jar:3.0.0` artifact from our local maven repo."  We can use this to effectively restore any jars we do not want the Eclipse Transformer to modify.  This will be any jar file that already fully supports the Jakarta namespace, like the latest Eclipselink, Mojarra or MyFaces.  Of course in theory the Transformer shouldn't have a negative impact on jars that already support the new namespace, but why risk it when it's easy to gain 100% confidence the jar we ship is byte for byte the same one produced by the respective project.

This setting could potentially also be used to do library upgrades via the patch plugin.

## replace/resources

The `<replace><jars>` list allows us to tell the plugin, "when you see an `openejb-version.properties` file anywhere in the war file or its libraries, replace it with the specially modified version from `target/classes/`."  In the module we're generating a new `openejb-version.properties` so we can change the version TomEE reports from "8.0.7-SNAPSHOT" to "9.0.0-M7-SNAPSHOT"

# Current Jakarta EE 9.1 Results

With all these changes to library upgrades and how we're doing the bytecode modification, we're now down to 218 test failures against the March 8th snapshot of the Jakarta EE 9.1 TCK.

 - https://tck.work/tomee/build?id=1616775424416

The Jakarta EE 9.1 release is scheduled for mid-may, so that'd give us till end of April to get on the official ballot and surprise the world :)  It would take some intense hours over the next month and a bunch of work both here and in Jakarta, but I think we have a real shot.  Not an easy shot, but one worth going for.

I'll try to post some details tomorrow on running the TCK locally.  The current instructions don't cover 9.1.


-David


smime.p7s (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Rework TomEE 9 to use Tomcat 10 and build webapps & tar.gz

Jean-Louis MONTEIRO
Hey David and all,

Thanks for the detailed write up.
I made some changes to being able to patch Tomcat both in tomee-jakarta and
in the Maven tomee-patch-plugin.
Made some small changes to bring back Jenkins to green because it was
failing.

We should be good to move on.

While doing this, I realized an issue with the packaging that makes the
execution fail.
Maven seems to pull both the -SNAPSHOT jar and the timestamped version like

$ ls apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee*

> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-catalina-8.0.7-20210327.040242-103.jar
>    apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-jdbc-8.0.7-SNAPSHOT.jar
> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-catalina-8.0.7-SNAPSHOT.jar
>
> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-juli-8.0.7-20210327.040051-103.jar
> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-common-8.0.7-SNAPSHOT.jar
>
> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-loader-8.0.7-20210327.040109-103.jar
> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-config-8.0.7-20210327.035954-103.jar
>
>  apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-loader-8.0.7-SNAPSHOT.jar
> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-config-8.0.7-SNAPSHOT.jar
>
> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-mojarra-8.0.7-SNAPSHOT.jar
> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-jaxrs-8.0.7-20210327.040512-103.jar
>
> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-security-8.0.7-SNAPSHOT.jar
> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-jaxrs-8.0.7-SNAPSHOT.jar
>
>  apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-webapp-8.0.7-SNAPSHOT.jar
> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-jdbc-8.0.7-20210327.040122-103.jar
>
>  apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-webservices-8.0.7-20210327.040612-103.jar
>

If someone can look at it, or I'll have a look today
--
Jean-Louis Monteiro
http://twitter.com/jlouismonteiro
http://www.tomitribe.com


On Sat, Mar 27, 2021 at 5:05 AM David Blevins <[hidden email]>
wrote:

> Hi All,
>
> Some updates.  I went ahead and merged that change so we're now building
> wars, zips, tars and officially using Tomcat 10 in TomEE 9!
>
> We're using Mojarra 3.0.0 and Eclipselink 3.0.0 and I've made improvements
> to the TomEE Patch Plugin to ensure that the versions we have in the dists
> is completely unmodified by the Eclipse Transformer or TomEE Patch Plugin.
> Here's how that works (I'll add this to the README).
>
> In the configuration of the plugin you can have a directive like the
> following:
>
>         <configuration>
>           <select>tomee-plume-webapp-transformed-.*\.war</select>
>           <patchSources>
>
> <source>${project.basedir}/../../transform/src/patch/java/</source>
>             <source>${project.basedir}/src/patch/java/</source>
>           </patchSources>
>           <replace>
>             <jars>
>
> <jakarta.faces-3.0.0.jar>org.glassfish:jakarta.faces:jar:3.0.0</jakarta.faces-3.0.0.jar>
>
> <eclipselink-3.0.0.jar>org.eclipse.persistence:eclipselink:jar:3.0.0</eclipselink-3.0.0.jar>
>             </jars>
>             <resources>
>
> <openejb-version.properties>${project.build.outputDirectory}/openejb-version.properties</openejb-version.properties>
>             </resources>
>           </replace>
>         </configuration>
>
> ## select
>
> The `<select>` setting is what tells the plugin which binaries to modify.
> In the above setting we're saying we want to patch the output of the
> Eclipse Transformer.  The input to the transformer is the regular
> `tomee-plume-webapp-9.0.0-M123-SNAPSHOT.war` file and the output is a new
> `tomee-plume-webapp-transformed-9.0.0-M123-SNAPSHOT.war` file.  It's the
> "transformed" file we want to further patch.
>
> ## patchSources
>
> The `<patchSources>` list allows us to specify locations where *.java
> files live.  We will copy these into the `target/` directory and compile
> them using any *.jar files we find in the war file to create the
> classpath.  These compiled classes are then used to overwrite any classes
> by that name in any part of the war or its libraries.  This is how we
> handle corner cases that are too complicated to deal with using bytecode
> tools -- we can just specify an alternate source file.
>
> Being able to have multiple locations for patch files keeps us from having
> to duplicate patch files as we build multiple war files in multiple
> modules.  The above configuration says grab the *.java files from the
> `transform` module and the *.java files from this module.
>
> ## replace/jars
>
> The `<replace><jars>` list allows us to tell the plugin, "when you see a
> `jakarta.faces-3.0.0.jar` in the war file, replace it with the
> `org.glassfish:jakarta.faces:jar:3.0.0` artifact from our local maven
> repo."  We can use this to effectively restore any jars we do not want the
> Eclipse Transformer to modify.  This will be any jar file that already
> fully supports the Jakarta namespace, like the latest Eclipselink, Mojarra
> or MyFaces.  Of course in theory the Transformer shouldn't have a negative
> impact on jars that already support the new namespace, but why risk it when
> it's easy to gain 100% confidence the jar we ship is byte for byte the same
> one produced by the respective project.
>
> This setting could potentially also be used to do library upgrades via the
> patch plugin.
>
> ## replace/resources
>
> The `<replace><jars>` list allows us to tell the plugin, "when you see an
> `openejb-version.properties` file anywhere in the war file or its
> libraries, replace it with the specially modified version from
> `target/classes/`."  In the module we're generating a new
> `openejb-version.properties` so we can change the version TomEE reports
> from "8.0.7-SNAPSHOT" to "9.0.0-M7-SNAPSHOT"
>
> # Current Jakarta EE 9.1 Results
>
> With all these changes to library upgrades and how we're doing the
> bytecode modification, we're now down to 218 test failures against the
> March 8th snapshot of the Jakarta EE 9.1 TCK.
>
>  - https://tck.work/tomee/build?id=1616775424416
>
> The Jakarta EE 9.1 release is scheduled for mid-may, so that'd give us
> till end of April to get on the official ballot and surprise the world :)
> It would take some intense hours over the next month and a bunch of work
> both here and in Jakarta, but I think we have a real shot.  Not an easy
> shot, but one worth going for.
>
> I'll try to post some details tomorrow on running the TCK locally.  The
> current instructions don't cover 9.1.
>
>
> -David
>
>
   --
    Jean-Louis Monteiro
    http://twitter.com/jlouismonteiro
    http://www.tomitribe.com
Reply | Threaded
Open this post in threaded view
|

Re: Rework TomEE 9 to use Tomcat 10 and build webapps & tar.gz

Jean-Louis MONTEIRO
Ok looks like we are back to normal. Everything is working so far. Bad news
is that I have no idea what really fixed the problem.
But we are having a lot of NoSuchMethodError with OpenWebBeans

--
Jean-Louis Monteiro
http://twitter.com/jlouismonteiro
http://www.tomitribe.com


On Mon, Mar 29, 2021 at 9:21 AM Jean-Louis Monteiro <
[hidden email]> wrote:

> Hey David and all,
>
> Thanks for the detailed write up.
> I made some changes to being able to patch Tomcat both in tomee-jakarta
> and in the Maven tomee-patch-plugin.
> Made some small changes to bring back Jenkins to green because it was
> failing.
>
> We should be good to move on.
>
> While doing this, I realized an issue with the packaging that makes the
> execution fail.
> Maven seems to pull both the -SNAPSHOT jar and the timestamped version like
>
> $ ls apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee*
>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-catalina-8.0.7-20210327.040242-103.jar
>>    apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-jdbc-8.0.7-SNAPSHOT.jar
>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-catalina-8.0.7-SNAPSHOT.jar
>>
>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-juli-8.0.7-20210327.040051-103.jar
>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-common-8.0.7-SNAPSHOT.jar
>>
>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-loader-8.0.7-20210327.040109-103.jar
>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-config-8.0.7-20210327.035954-103.jar
>>
>>  apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-loader-8.0.7-SNAPSHOT.jar
>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-config-8.0.7-SNAPSHOT.jar
>>
>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-mojarra-8.0.7-SNAPSHOT.jar
>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-jaxrs-8.0.7-20210327.040512-103.jar
>>
>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-security-8.0.7-SNAPSHOT.jar
>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-jaxrs-8.0.7-SNAPSHOT.jar
>>
>>  apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-webapp-8.0.7-SNAPSHOT.jar
>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-jdbc-8.0.7-20210327.040122-103.jar
>>
>>  apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-webservices-8.0.7-20210327.040612-103.jar
>>
>
> If someone can look at it, or I'll have a look today
> --
> Jean-Louis Monteiro
> http://twitter.com/jlouismonteiro
> http://www.tomitribe.com
>
>
> On Sat, Mar 27, 2021 at 5:05 AM David Blevins <[hidden email]>
> wrote:
>
>> Hi All,
>>
>> Some updates.  I went ahead and merged that change so we're now building
>> wars, zips, tars and officially using Tomcat 10 in TomEE 9!
>>
>> We're using Mojarra 3.0.0 and Eclipselink 3.0.0 and I've made
>> improvements to the TomEE Patch Plugin to ensure that the versions we have
>> in the dists is completely unmodified by the Eclipse Transformer or TomEE
>> Patch Plugin.  Here's how that works (I'll add this to the README).
>>
>> In the configuration of the plugin you can have a directive like the
>> following:
>>
>>         <configuration>
>>           <select>tomee-plume-webapp-transformed-.*\.war</select>
>>           <patchSources>
>>
>> <source>${project.basedir}/../../transform/src/patch/java/</source>
>>             <source>${project.basedir}/src/patch/java/</source>
>>           </patchSources>
>>           <replace>
>>             <jars>
>>
>> <jakarta.faces-3.0.0.jar>org.glassfish:jakarta.faces:jar:3.0.0</jakarta.faces-3.0.0.jar>
>>
>> <eclipselink-3.0.0.jar>org.eclipse.persistence:eclipselink:jar:3.0.0</eclipselink-3.0.0.jar>
>>             </jars>
>>             <resources>
>>
>> <openejb-version.properties>${project.build.outputDirectory}/openejb-version.properties</openejb-version.properties>
>>             </resources>
>>           </replace>
>>         </configuration>
>>
>> ## select
>>
>> The `<select>` setting is what tells the plugin which binaries to
>> modify.  In the above setting we're saying we want to patch the output of
>> the Eclipse Transformer.  The input to the transformer is the regular
>> `tomee-plume-webapp-9.0.0-M123-SNAPSHOT.war` file and the output is a new
>> `tomee-plume-webapp-transformed-9.0.0-M123-SNAPSHOT.war` file.  It's the
>> "transformed" file we want to further patch.
>>
>> ## patchSources
>>
>> The `<patchSources>` list allows us to specify locations where *.java
>> files live.  We will copy these into the `target/` directory and compile
>> them using any *.jar files we find in the war file to create the
>> classpath.  These compiled classes are then used to overwrite any classes
>> by that name in any part of the war or its libraries.  This is how we
>> handle corner cases that are too complicated to deal with using bytecode
>> tools -- we can just specify an alternate source file.
>>
>> Being able to have multiple locations for patch files keeps us from
>> having to duplicate patch files as we build multiple war files in multiple
>> modules.  The above configuration says grab the *.java files from the
>> `transform` module and the *.java files from this module.
>>
>> ## replace/jars
>>
>> The `<replace><jars>` list allows us to tell the plugin, "when you see a
>> `jakarta.faces-3.0.0.jar` in the war file, replace it with the
>> `org.glassfish:jakarta.faces:jar:3.0.0` artifact from our local maven
>> repo."  We can use this to effectively restore any jars we do not want the
>> Eclipse Transformer to modify.  This will be any jar file that already
>> fully supports the Jakarta namespace, like the latest Eclipselink, Mojarra
>> or MyFaces.  Of course in theory the Transformer shouldn't have a negative
>> impact on jars that already support the new namespace, but why risk it when
>> it's easy to gain 100% confidence the jar we ship is byte for byte the same
>> one produced by the respective project.
>>
>> This setting could potentially also be used to do library upgrades via
>> the patch plugin.
>>
>> ## replace/resources
>>
>> The `<replace><jars>` list allows us to tell the plugin, "when you see an
>> `openejb-version.properties` file anywhere in the war file or its
>> libraries, replace it with the specially modified version from
>> `target/classes/`."  In the module we're generating a new
>> `openejb-version.properties` so we can change the version TomEE reports
>> from "8.0.7-SNAPSHOT" to "9.0.0-M7-SNAPSHOT"
>>
>> # Current Jakarta EE 9.1 Results
>>
>> With all these changes to library upgrades and how we're doing the
>> bytecode modification, we're now down to 218 test failures against the
>> March 8th snapshot of the Jakarta EE 9.1 TCK.
>>
>>  - https://tck.work/tomee/build?id=1616775424416
>>
>> The Jakarta EE 9.1 release is scheduled for mid-may, so that'd give us
>> till end of April to get on the official ballot and surprise the world :)
>> It would take some intense hours over the next month and a bunch of work
>> both here and in Jakarta, but I think we have a real shot.  Not an easy
>> shot, but one worth going for.
>>
>> I'll try to post some details tomorrow on running the TCK locally.  The
>> current instructions don't cover 9.1.
>>
>>
>> -David
>>
>>
   --
    Jean-Louis Monteiro
    http://twitter.com/jlouismonteiro
    http://www.tomitribe.com
Reply | Threaded
Open this post in threaded view
|

Re: Rework TomEE 9 to use Tomcat 10 and build webapps & tar.gz

Jean-Louis MONTEIRO
Might be an outdated snapshot because I don't seem to reproduce locally and
I'm up to date
--
Jean-Louis Monteiro
http://twitter.com/jlouismonteiro
http://www.tomitribe.com


On Tue, Mar 30, 2021 at 8:43 PM Jean-Louis Monteiro <
[hidden email]> wrote:

> Ok looks like we are back to normal. Everything is working so far. Bad
> news is that I have no idea what really fixed the problem.
> But we are having a lot of NoSuchMethodError with OpenWebBeans
>
> --
> Jean-Louis Monteiro
> http://twitter.com/jlouismonteiro
> http://www.tomitribe.com
>
>
> On Mon, Mar 29, 2021 at 9:21 AM Jean-Louis Monteiro <
> [hidden email]> wrote:
>
>> Hey David and all,
>>
>> Thanks for the detailed write up.
>> I made some changes to being able to patch Tomcat both in tomee-jakarta
>> and in the Maven tomee-patch-plugin.
>> Made some small changes to bring back Jenkins to green because it was
>> failing.
>>
>> We should be good to move on.
>>
>> While doing this, I realized an issue with the packaging that makes the
>> execution fail.
>> Maven seems to pull both the -SNAPSHOT jar and the timestamped version
>> like
>>
>> $ ls apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee*
>>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-catalina-8.0.7-20210327.040242-103.jar
>>>    apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-jdbc-8.0.7-SNAPSHOT.jar
>>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-catalina-8.0.7-SNAPSHOT.jar
>>>
>>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-juli-8.0.7-20210327.040051-103.jar
>>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-common-8.0.7-SNAPSHOT.jar
>>>
>>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-loader-8.0.7-20210327.040109-103.jar
>>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-config-8.0.7-20210327.035954-103.jar
>>>
>>>  apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-loader-8.0.7-SNAPSHOT.jar
>>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-config-8.0.7-SNAPSHOT.jar
>>>
>>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-mojarra-8.0.7-SNAPSHOT.jar
>>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-jaxrs-8.0.7-20210327.040512-103.jar
>>>
>>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-security-8.0.7-SNAPSHOT.jar
>>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-jaxrs-8.0.7-SNAPSHOT.jar
>>>
>>>  apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-webapp-8.0.7-SNAPSHOT.jar
>>> apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-jdbc-8.0.7-20210327.040122-103.jar
>>>
>>>  apache-tomee-plume-9.0.0-M7-SNAPSHOT/lib/tomee-webservices-8.0.7-20210327.040612-103.jar
>>>
>>
>> If someone can look at it, or I'll have a look today
>> --
>> Jean-Louis Monteiro
>> http://twitter.com/jlouismonteiro
>> http://www.tomitribe.com
>>
>>
>> On Sat, Mar 27, 2021 at 5:05 AM David Blevins <[hidden email]>
>> wrote:
>>
>>> Hi All,
>>>
>>> Some updates.  I went ahead and merged that change so we're now building
>>> wars, zips, tars and officially using Tomcat 10 in TomEE 9!
>>>
>>> We're using Mojarra 3.0.0 and Eclipselink 3.0.0 and I've made
>>> improvements to the TomEE Patch Plugin to ensure that the versions we have
>>> in the dists is completely unmodified by the Eclipse Transformer or TomEE
>>> Patch Plugin.  Here's how that works (I'll add this to the README).
>>>
>>> In the configuration of the plugin you can have a directive like the
>>> following:
>>>
>>>         <configuration>
>>>           <select>tomee-plume-webapp-transformed-.*\.war</select>
>>>           <patchSources>
>>>
>>> <source>${project.basedir}/../../transform/src/patch/java/</source>
>>>             <source>${project.basedir}/src/patch/java/</source>
>>>           </patchSources>
>>>           <replace>
>>>             <jars>
>>>
>>> <jakarta.faces-3.0.0.jar>org.glassfish:jakarta.faces:jar:3.0.0</jakarta.faces-3.0.0.jar>
>>>
>>> <eclipselink-3.0.0.jar>org.eclipse.persistence:eclipselink:jar:3.0.0</eclipselink-3.0.0.jar>
>>>             </jars>
>>>             <resources>
>>>
>>> <openejb-version.properties>${project.build.outputDirectory}/openejb-version.properties</openejb-version.properties>
>>>             </resources>
>>>           </replace>
>>>         </configuration>
>>>
>>> ## select
>>>
>>> The `<select>` setting is what tells the plugin which binaries to
>>> modify.  In the above setting we're saying we want to patch the output of
>>> the Eclipse Transformer.  The input to the transformer is the regular
>>> `tomee-plume-webapp-9.0.0-M123-SNAPSHOT.war` file and the output is a new
>>> `tomee-plume-webapp-transformed-9.0.0-M123-SNAPSHOT.war` file.  It's the
>>> "transformed" file we want to further patch.
>>>
>>> ## patchSources
>>>
>>> The `<patchSources>` list allows us to specify locations where *.java
>>> files live.  We will copy these into the `target/` directory and compile
>>> them using any *.jar files we find in the war file to create the
>>> classpath.  These compiled classes are then used to overwrite any classes
>>> by that name in any part of the war or its libraries.  This is how we
>>> handle corner cases that are too complicated to deal with using bytecode
>>> tools -- we can just specify an alternate source file.
>>>
>>> Being able to have multiple locations for patch files keeps us from
>>> having to duplicate patch files as we build multiple war files in multiple
>>> modules.  The above configuration says grab the *.java files from the
>>> `transform` module and the *.java files from this module.
>>>
>>> ## replace/jars
>>>
>>> The `<replace><jars>` list allows us to tell the plugin, "when you see a
>>> `jakarta.faces-3.0.0.jar` in the war file, replace it with the
>>> `org.glassfish:jakarta.faces:jar:3.0.0` artifact from our local maven
>>> repo."  We can use this to effectively restore any jars we do not want the
>>> Eclipse Transformer to modify.  This will be any jar file that already
>>> fully supports the Jakarta namespace, like the latest Eclipselink, Mojarra
>>> or MyFaces.  Of course in theory the Transformer shouldn't have a negative
>>> impact on jars that already support the new namespace, but why risk it when
>>> it's easy to gain 100% confidence the jar we ship is byte for byte the same
>>> one produced by the respective project.
>>>
>>> This setting could potentially also be used to do library upgrades via
>>> the patch plugin.
>>>
>>> ## replace/resources
>>>
>>> The `<replace><jars>` list allows us to tell the plugin, "when you see
>>> an `openejb-version.properties` file anywhere in the war file or its
>>> libraries, replace it with the specially modified version from
>>> `target/classes/`."  In the module we're generating a new
>>> `openejb-version.properties` so we can change the version TomEE reports
>>> from "8.0.7-SNAPSHOT" to "9.0.0-M7-SNAPSHOT"
>>>
>>> # Current Jakarta EE 9.1 Results
>>>
>>> With all these changes to library upgrades and how we're doing the
>>> bytecode modification, we're now down to 218 test failures against the
>>> March 8th snapshot of the Jakarta EE 9.1 TCK.
>>>
>>>  - https://tck.work/tomee/build?id=1616775424416
>>>
>>> The Jakarta EE 9.1 release is scheduled for mid-may, so that'd give us
>>> till end of April to get on the official ballot and surprise the world :)
>>> It would take some intense hours over the next month and a bunch of work
>>> both here and in Jakarta, but I think we have a real shot.  Not an easy
>>> shot, but one worth going for.
>>>
>>> I'll try to post some details tomorrow on running the TCK locally.  The
>>> current instructions don't cover 9.1.
>>>
>>>
>>> -David
>>>
>>>
   --
    Jean-Louis Monteiro
    http://twitter.com/jlouismonteiro
    http://www.tomitribe.com
Reply | Threaded
Open this post in threaded view
|

Re: Rework TomEE 9 to use Tomcat 10 and build webapps & tar.gz

David Blevins-2
In reply to this post by Jean-Louis MONTEIRO
> On Mar 30, 2021, at 11:43 AM, Jean-Louis Monteiro <[hidden email]> wrote:
>
> Ok looks like we are back to normal. Everything is working so far. Bad news
> is that I have no idea what really fixed the problem.

I had the same thing happen to me last week.  I did a clean build and it magically went away.

I'm in the same boat as you in not feeling very relieved.  It'll most likely come back as an issue.  When it happened to me I noticed it was the original war that had the issue first.  Of course the translated war had it also as did the server zip.

> But we are having a lot of NoSuchMethodError with OpenWebBeans

I definitely see the latest zips/tars are not working.  Haven't had a chance to dig in yet.


-David


smime.p7s (1K) Download Attachment