[TCK] Status and plans

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[TCK] Status and plans

Jean-Louis MONTEIRO
Hi community,

Trying to give visibility as much as possible.
Here is a more high level status on Jakarta EE 8 TCK (EE 9 is close so I'll
focus on EE 8)

I would say that most of the integration work has been done.
We are mostly in a final stage where we need to check what a test is doing,
and either find a better configuration for TomEE or fix it so it passes the
TCK

We are on the 98/99% with about 400 failures of the 32+k tests.
See build https://tck.work/tomee/build?id=1607604502810 for example.

We've already got some challenges or issues opened against Jakarta EE TCK
or sub projects.

   -

   JPA
   -

      https://github.com/eclipse-ee4j/jpa-api/issues/299
      -

      https://github.com/eclipse-ee4j/jpa-api/issues/300
      -

   JTA
   -

      https://github.com/eclipse-ee4j/jta-api/issues/185
      -

   Authentication (JASPIC)
   -

      https://github.com/eclipse-ee4j/authentication/issues/115
      -

      https://github.com/eclipse-ee4j/authentication/issues/116
      -

      https://github.com/eclipse-ee4j/authentication/issues/117
      -

   EJB
   -

      https://github.com/eclipse-ee4j/ejb-api/issues/137


They do explain some failures in their areas.
JTA I added a workaround for the moment that's why we are at 100%.

Tomcat has some known failures on EL, JSP, Servet, WebSockets
We need to find them and fix the remaining stuff.

If you guys want to help, you can pick up one of the failures.
Mind to check if they are either related to the tickets above or if they
are known Tomcat issues.

https://cwiki.apache.org/confluence/display/TOMCAT/Jakarta+EE+TCKs

Of course, if you have any question or if you need assistance, let us know.
--
Jean-Louis Monteiro
http://twitter.com/jlouismonteiro
http://www.tomitribe.com
   --
    Jean-Louis Monteiro
    http://twitter.com/jlouismonteiro
    http://www.tomitribe.com
Reply | Threaded
Open this post in threaded view
|

Re: [TCK] Status and plans

Jean-Louis MONTEIRO
Just a small heads up

Here is the very last result of this week
https://tck.work/tomee/build?id=1607726026267

We managed to fix about 100 tests, so we are at about 250 of the finish
line.
Small notes

1/ I have added some excludes for tests which were bad and that got fixed
in Jakarta EE 9 - See
https://github.com/apache/tomee-tck/blob/master/src/test/resources/ts.jtx#L19

I have probably more to add, but we'll do on the flow.

2/ Jaspic went down because I removed the work around I did back during
summer. Now that I opened a ticket to the JASPIC team, we need to wait for
their feedback, but I don't want to mess with the results artificially.

Have a good weekend and thanks
Great community effort.

--
Jean-Louis Monteiro
http://twitter.com/jlouismonteiro
http://www.tomitribe.com


On Thu, Dec 10, 2020 at 3:12 PM Jean-Louis Monteiro <
[hidden email]> wrote:

> Hi community,
>
> Trying to give visibility as much as possible.
> Here is a more high level status on Jakarta EE 8 TCK (EE 9 is close so
> I'll focus on EE 8)
>
> I would say that most of the integration work has been done.
> We are mostly in a final stage where we need to check what a test is
> doing, and either find a better configuration for TomEE or fix it so it
> passes the TCK
>
> We are on the 98/99% with about 400 failures of the 32+k tests.
> See build https://tck.work/tomee/build?id=1607604502810 for example.
>
> We've already got some challenges or issues opened against Jakarta EE TCK
> or sub projects.
>
>    -
>
>    JPA
>    -
>
>       https://github.com/eclipse-ee4j/jpa-api/issues/299
>       -
>
>       https://github.com/eclipse-ee4j/jpa-api/issues/300
>       -
>
>    JTA
>    -
>
>       https://github.com/eclipse-ee4j/jta-api/issues/185
>       -
>
>    Authentication (JASPIC)
>    -
>
>       https://github.com/eclipse-ee4j/authentication/issues/115
>       -
>
>       https://github.com/eclipse-ee4j/authentication/issues/116
>       -
>
>       https://github.com/eclipse-ee4j/authentication/issues/117
>       -
>
>    EJB
>    -
>
>       https://github.com/eclipse-ee4j/ejb-api/issues/137
>
>
> They do explain some failures in their areas.
> JTA I added a workaround for the moment that's why we are at 100%.
>
> Tomcat has some known failures on EL, JSP, Servet, WebSockets
> We need to find them and fix the remaining stuff.
>
> If you guys want to help, you can pick up one of the failures.
> Mind to check if they are either related to the tickets above or if they
> are known Tomcat issues.
>
> https://cwiki.apache.org/confluence/display/TOMCAT/Jakarta+EE+TCKs
>
> Of course, if you have any question or if you need assistance, let us know.
> --
> Jean-Louis Monteiro
> http://twitter.com/jlouismonteiro
> http://www.tomitribe.com
>
   --
    Jean-Louis Monteiro
    http://twitter.com/jlouismonteiro
    http://www.tomitribe.com
Reply | Threaded
Open this post in threaded view
|

Re: [TCK] Status and plans

Jean-Louis MONTEIRO
Hi all,

Here is the last run
https://tck.work/tomee/build?id=1608554936631

I just closed the vote on Apache Johnzon and released it. TomEE is up to
date now and we should be 100% on JSONP.
On JSTL,
https://tck.work/tomee/tests?path=com.sun.ts.tests.jstl&build=1608554936631&status=FAILED

Locally only the import fails but I have a challenge opened to understand.
https://github.com/eclipse-ee4j/jstl-api/issues/140

JAX RS and WebSockets are still areas where I haven't spent time, so if
some want to dig, that'd be great.

Thiago is working on the Servlet part.
Richard did the Quartz upgrade, I'll see if I can address the EJB32 issue
with the endDate prior to startDate (see related thread).

I'll fire up a new build when I have figured out the JSTL issue so we also
have a JSONP green bar ;-)

--
Jean-Louis Monteiro
http://twitter.com/jlouismonteiro
http://www.tomitribe.com


On Sat, Dec 12, 2020 at 9:50 AM Jean-Louis Monteiro <
[hidden email]> wrote:

> Just a small heads up
>
> Here is the very last result of this week
> https://tck.work/tomee/build?id=1607726026267
>
> We managed to fix about 100 tests, so we are at about 250 of the finish
> line.
> Small notes
>
> 1/ I have added some excludes for tests which were bad and that got fixed
> in Jakarta EE 9 - See
> https://github.com/apache/tomee-tck/blob/master/src/test/resources/ts.jtx#L19
>
> I have probably more to add, but we'll do on the flow.
>
> 2/ Jaspic went down because I removed the work around I did back during
> summer. Now that I opened a ticket to the JASPIC team, we need to wait for
> their feedback, but I don't want to mess with the results artificially.
>
> Have a good weekend and thanks
> Great community effort.
>
> --
> Jean-Louis Monteiro
> http://twitter.com/jlouismonteiro
> http://www.tomitribe.com
>
>
> On Thu, Dec 10, 2020 at 3:12 PM Jean-Louis Monteiro <
> [hidden email]> wrote:
>
>> Hi community,
>>
>> Trying to give visibility as much as possible.
>> Here is a more high level status on Jakarta EE 8 TCK (EE 9 is close so
>> I'll focus on EE 8)
>>
>> I would say that most of the integration work has been done.
>> We are mostly in a final stage where we need to check what a test is
>> doing, and either find a better configuration for TomEE or fix it so it
>> passes the TCK
>>
>> We are on the 98/99% with about 400 failures of the 32+k tests.
>> See build https://tck.work/tomee/build?id=1607604502810 for example.
>>
>> We've already got some challenges or issues opened against Jakarta EE TCK
>> or sub projects.
>>
>>    -
>>
>>    JPA
>>    -
>>
>>       https://github.com/eclipse-ee4j/jpa-api/issues/299
>>       -
>>
>>       https://github.com/eclipse-ee4j/jpa-api/issues/300
>>       -
>>
>>    JTA
>>    -
>>
>>       https://github.com/eclipse-ee4j/jta-api/issues/185
>>       -
>>
>>    Authentication (JASPIC)
>>    -
>>
>>       https://github.com/eclipse-ee4j/authentication/issues/115
>>       -
>>
>>       https://github.com/eclipse-ee4j/authentication/issues/116
>>       -
>>
>>       https://github.com/eclipse-ee4j/authentication/issues/117
>>       -
>>
>>    EJB
>>    -
>>
>>       https://github.com/eclipse-ee4j/ejb-api/issues/137
>>
>>
>> They do explain some failures in their areas.
>> JTA I added a workaround for the moment that's why we are at 100%.
>>
>> Tomcat has some known failures on EL, JSP, Servet, WebSockets
>> We need to find them and fix the remaining stuff.
>>
>> If you guys want to help, you can pick up one of the failures.
>> Mind to check if they are either related to the tickets above or if they
>> are known Tomcat issues.
>>
>> https://cwiki.apache.org/confluence/display/TOMCAT/Jakarta+EE+TCKs
>>
>> Of course, if you have any question or if you need assistance, let us
>> know.
>> --
>> Jean-Louis Monteiro
>> http://twitter.com/jlouismonteiro
>> http://www.tomitribe.com
>>
>
   --
    Jean-Louis Monteiro
    http://twitter.com/jlouismonteiro
    http://www.tomitribe.com
Reply | Threaded
Open this post in threaded view
|

Re: [TCK] Status and plans

Jean-Louis MONTEIRO
https://tck.work/tomee/build?id=1608726402626

Boom, JSTL 1 failure and JSONP 0 failure.
We are down to 209 failures
--
Jean-Louis Monteiro
http://twitter.com/jlouismonteiro
http://www.tomitribe.com


On Wed, Dec 23, 2020 at 10:34 AM Jean-Louis Monteiro <
[hidden email]> wrote:

> Hi all,
>
> Here is the last run
> https://tck.work/tomee/build?id=1608554936631
>
> I just closed the vote on Apache Johnzon and released it. TomEE is up to
> date now and we should be 100% on JSONP.
> On JSTL,
> https://tck.work/tomee/tests?path=com.sun.ts.tests.jstl&build=1608554936631&status=FAILED
>
> Locally only the import fails but I have a challenge opened to understand.
> https://github.com/eclipse-ee4j/jstl-api/issues/140
>
> JAX RS and WebSockets are still areas where I haven't spent time, so if
> some want to dig, that'd be great.
>
> Thiago is working on the Servlet part.
> Richard did the Quartz upgrade, I'll see if I can address the EJB32 issue
> with the endDate prior to startDate (see related thread).
>
> I'll fire up a new build when I have figured out the JSTL issue so we also
> have a JSONP green bar ;-)
>
> --
> Jean-Louis Monteiro
> http://twitter.com/jlouismonteiro
> http://www.tomitribe.com
>
>
> On Sat, Dec 12, 2020 at 9:50 AM Jean-Louis Monteiro <
> [hidden email]> wrote:
>
>> Just a small heads up
>>
>> Here is the very last result of this week
>> https://tck.work/tomee/build?id=1607726026267
>>
>> We managed to fix about 100 tests, so we are at about 250 of the finish
>> line.
>> Small notes
>>
>> 1/ I have added some excludes for tests which were bad and that got fixed
>> in Jakarta EE 9 - See
>> https://github.com/apache/tomee-tck/blob/master/src/test/resources/ts.jtx#L19
>>
>> I have probably more to add, but we'll do on the flow.
>>
>> 2/ Jaspic went down because I removed the work around I did back during
>> summer. Now that I opened a ticket to the JASPIC team, we need to wait for
>> their feedback, but I don't want to mess with the results artificially.
>>
>> Have a good weekend and thanks
>> Great community effort.
>>
>> --
>> Jean-Louis Monteiro
>> http://twitter.com/jlouismonteiro
>> http://www.tomitribe.com
>>
>>
>> On Thu, Dec 10, 2020 at 3:12 PM Jean-Louis Monteiro <
>> [hidden email]> wrote:
>>
>>> Hi community,
>>>
>>> Trying to give visibility as much as possible.
>>> Here is a more high level status on Jakarta EE 8 TCK (EE 9 is close so
>>> I'll focus on EE 8)
>>>
>>> I would say that most of the integration work has been done.
>>> We are mostly in a final stage where we need to check what a test is
>>> doing, and either find a better configuration for TomEE or fix it so it
>>> passes the TCK
>>>
>>> We are on the 98/99% with about 400 failures of the 32+k tests.
>>> See build https://tck.work/tomee/build?id=1607604502810 for example.
>>>
>>> We've already got some challenges or issues opened against Jakarta EE
>>> TCK or sub projects.
>>>
>>>    -
>>>
>>>    JPA
>>>    -
>>>
>>>       https://github.com/eclipse-ee4j/jpa-api/issues/299
>>>       -
>>>
>>>       https://github.com/eclipse-ee4j/jpa-api/issues/300
>>>       -
>>>
>>>    JTA
>>>    -
>>>
>>>       https://github.com/eclipse-ee4j/jta-api/issues/185
>>>       -
>>>
>>>    Authentication (JASPIC)
>>>    -
>>>
>>>       https://github.com/eclipse-ee4j/authentication/issues/115
>>>       -
>>>
>>>       https://github.com/eclipse-ee4j/authentication/issues/116
>>>       -
>>>
>>>       https://github.com/eclipse-ee4j/authentication/issues/117
>>>       -
>>>
>>>    EJB
>>>    -
>>>
>>>       https://github.com/eclipse-ee4j/ejb-api/issues/137
>>>
>>>
>>> They do explain some failures in their areas.
>>> JTA I added a workaround for the moment that's why we are at 100%.
>>>
>>> Tomcat has some known failures on EL, JSP, Servet, WebSockets
>>> We need to find them and fix the remaining stuff.
>>>
>>> If you guys want to help, you can pick up one of the failures.
>>> Mind to check if they are either related to the tickets above or if they
>>> are known Tomcat issues.
>>>
>>> https://cwiki.apache.org/confluence/display/TOMCAT/Jakarta+EE+TCKs
>>>
>>> Of course, if you have any question or if you need assistance, let us
>>> know.
>>> --
>>> Jean-Louis Monteiro
>>> http://twitter.com/jlouismonteiro
>>> http://www.tomitribe.com
>>>
>>
   --
    Jean-Louis Monteiro
    http://twitter.com/jlouismonteiro
    http://www.tomitribe.com
Reply | Threaded
Open this post in threaded view
|

Re: [TCK] Status and plans

Jean-Louis MONTEIRO
Hi all,

With the end of the year and some work constraints, I haven't been able to
really work too much on the TCK.
But I was back on it a bit last week and this week.

Did some work on Jakarta EE 9 to see where we were at and I've been able to
get down to 400+ failures which is not bad at all.
https://tck.work/tomee/project?id=8189

I realized that some tests are still using old namespaces. Aside from that
being a TCK bug that I'm going to submit and fix, it strikes me that TomEE
should be able to support both new and old namespaces in XML descriptors.
At least, our code is built to support it, so if it does not. Then we need
to fix.

So I decided to revert the transformer rules to rename the namespaces and
did a new run which adds about 1000 failures mainly on JPA.
I need to do a further analysis to see how to fix it. I'll also open a new
thread on our TomEE 9 and the approach taken. We might need to use a
slightly different path.

On Jakarta EE 8, we are down to 174 tests to fix, yeah !!!!
https://tck.work/tomee/project?id=818

On EJB 30, aside from the AroundConstruct failure, the datasource tests are
passing locally, so there is probably an influence from other tests.
On jaspic, the callback handler from Tomcat is still an issue, but no
answer from the spec, so everyone pushed the ball on the other side.
On jaxrs, good progress but still work to do.
On jpa, failures are mainly related to open issues on TCK.
On jstl, the single failure is the same
On securityapi, our implementation is in draft mode and not enough tested
and probably lack in terms of implementation
On servlet and websockets, we are better, but still need to see where
failures are expected (tomcat challenges opened) or not.

If some of you guys want to help, please shoot so we don't duplicate the
effort.





--
Jean-Louis Monteiro
http://twitter.com/jlouismonteiro
http://www.tomitribe.com


On Wed, Dec 23, 2020 at 4:35 PM Jean-Louis Monteiro <
[hidden email]> wrote:

> https://tck.work/tomee/build?id=1608726402626
>
> Boom, JSTL 1 failure and JSONP 0 failure.
> We are down to 209 failures
> --
> Jean-Louis Monteiro
> http://twitter.com/jlouismonteiro
> http://www.tomitribe.com
>
>
> On Wed, Dec 23, 2020 at 10:34 AM Jean-Louis Monteiro <
> [hidden email]> wrote:
>
>> Hi all,
>>
>> Here is the last run
>> https://tck.work/tomee/build?id=1608554936631
>>
>> I just closed the vote on Apache Johnzon and released it. TomEE is up to
>> date now and we should be 100% on JSONP.
>> On JSTL,
>> https://tck.work/tomee/tests?path=com.sun.ts.tests.jstl&build=1608554936631&status=FAILED
>>
>> Locally only the import fails but I have a challenge opened to understand.
>> https://github.com/eclipse-ee4j/jstl-api/issues/140
>>
>> JAX RS and WebSockets are still areas where I haven't spent time, so if
>> some want to dig, that'd be great.
>>
>> Thiago is working on the Servlet part.
>> Richard did the Quartz upgrade, I'll see if I can address the EJB32 issue
>> with the endDate prior to startDate (see related thread).
>>
>> I'll fire up a new build when I have figured out the JSTL issue so we
>> also have a JSONP green bar ;-)
>>
>> --
>> Jean-Louis Monteiro
>> http://twitter.com/jlouismonteiro
>> http://www.tomitribe.com
>>
>>
>> On Sat, Dec 12, 2020 at 9:50 AM Jean-Louis Monteiro <
>> [hidden email]> wrote:
>>
>>> Just a small heads up
>>>
>>> Here is the very last result of this week
>>> https://tck.work/tomee/build?id=1607726026267
>>>
>>> We managed to fix about 100 tests, so we are at about 250 of the finish
>>> line.
>>> Small notes
>>>
>>> 1/ I have added some excludes for tests which were bad and that got
>>> fixed in Jakarta EE 9 - See
>>> https://github.com/apache/tomee-tck/blob/master/src/test/resources/ts.jtx#L19
>>>
>>> I have probably more to add, but we'll do on the flow.
>>>
>>> 2/ Jaspic went down because I removed the work around I did back during
>>> summer. Now that I opened a ticket to the JASPIC team, we need to wait for
>>> their feedback, but I don't want to mess with the results artificially.
>>>
>>> Have a good weekend and thanks
>>> Great community effort.
>>>
>>> --
>>> Jean-Louis Monteiro
>>> http://twitter.com/jlouismonteiro
>>> http://www.tomitribe.com
>>>
>>>
>>> On Thu, Dec 10, 2020 at 3:12 PM Jean-Louis Monteiro <
>>> [hidden email]> wrote:
>>>
>>>> Hi community,
>>>>
>>>> Trying to give visibility as much as possible.
>>>> Here is a more high level status on Jakarta EE 8 TCK (EE 9 is close so
>>>> I'll focus on EE 8)
>>>>
>>>> I would say that most of the integration work has been done.
>>>> We are mostly in a final stage where we need to check what a test is
>>>> doing, and either find a better configuration for TomEE or fix it so it
>>>> passes the TCK
>>>>
>>>> We are on the 98/99% with about 400 failures of the 32+k tests.
>>>> See build https://tck.work/tomee/build?id=1607604502810 for example.
>>>>
>>>> We've already got some challenges or issues opened against Jakarta EE
>>>> TCK or sub projects.
>>>>
>>>>    -
>>>>
>>>>    JPA
>>>>    -
>>>>
>>>>       https://github.com/eclipse-ee4j/jpa-api/issues/299
>>>>       -
>>>>
>>>>       https://github.com/eclipse-ee4j/jpa-api/issues/300
>>>>       -
>>>>
>>>>    JTA
>>>>    -
>>>>
>>>>       https://github.com/eclipse-ee4j/jta-api/issues/185
>>>>       -
>>>>
>>>>    Authentication (JASPIC)
>>>>    -
>>>>
>>>>       https://github.com/eclipse-ee4j/authentication/issues/115
>>>>       -
>>>>
>>>>       https://github.com/eclipse-ee4j/authentication/issues/116
>>>>       -
>>>>
>>>>       https://github.com/eclipse-ee4j/authentication/issues/117
>>>>       -
>>>>
>>>>    EJB
>>>>    -
>>>>
>>>>       https://github.com/eclipse-ee4j/ejb-api/issues/137
>>>>
>>>>
>>>> They do explain some failures in their areas.
>>>> JTA I added a workaround for the moment that's why we are at 100%.
>>>>
>>>> Tomcat has some known failures on EL, JSP, Servet, WebSockets
>>>> We need to find them and fix the remaining stuff.
>>>>
>>>> If you guys want to help, you can pick up one of the failures.
>>>> Mind to check if they are either related to the tickets above or if
>>>> they are known Tomcat issues.
>>>>
>>>> https://cwiki.apache.org/confluence/display/TOMCAT/Jakarta+EE+TCKs
>>>>
>>>> Of course, if you have any question or if you need assistance, let us
>>>> know.
>>>> --
>>>> Jean-Louis Monteiro
>>>> http://twitter.com/jlouismonteiro
>>>> http://www.tomitribe.com
>>>>
>>>
   --
    Jean-Louis Monteiro
    http://twitter.com/jlouismonteiro
    http://www.tomitribe.com