Nuking tables on tearDown() for CRUD tests

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Nuking tables on tearDown() for CRUD tests

Alexander Saint Croix-2
Hello,

I'm doing CRUD tests on a collection of integrated entities in a shared
persistence unit, and am encountering some errors because of crufty tables
from previous tests.  I'm looking for a means of guaranteeing that the
tables for each of my entities is devoid of entries after each test runs.
The appropriate place to do this would be in the tearDown() method of the
unit test class.

My environment is OpenEJB (3.0.0 core, latest snapshot), OpenJPA and an
in-memory HSQLDB.  OS X on Java 5.

At the moment, I've implemented a stateless session bean to manage
persistence for me.  It's using an injected entity manager to perform a
series of DELETE statements.  Here's how the EM is set up:

    @PersistenceContext(unitName = "party-test-unit", type =
PersistenceContextType.TRANSACTION)
    private EntityManager entityManager;

I want to stress that I was previously removing entries from the database by
getting a list of components of a given class, then iteratively deleting
them.  This worked just fine, but was very inefficient and slowed down the
builds more than was acceptable.

My clear() method defines a String[] of table names, then recursively
creates a Query("DELETE FROM " + table) for each of the bean class names in
the package I'm testing. Here is the code:

    public void clear() {
        Set<Query> queries = new HashSet<Query>();
        String[] types = {
                "Address",
                "AssociatedAddress",
                "EmailAddress",
                "GeographicAddress",
                "ISOCountryCode",
                "Locale",
                "TelecomAddress",
                "WebPageAddress",
                "AssignedResponsibility",
                "Capability",
                "PartyRelationship",
                "PartyRelationshipType",
                "PartyRole",
                "PartyRoleType",
                "Responsibility",
                "Organization",
                "OrganizationName",
                "Party",
                "PartySignature",
                "Person",
                "PersonName",
                "Preference",
                "PreferenceOption",
                "PreferenceType",
                "Property",
                "RegisteredIdentifier"};
        for(String table : types) {
            queries.add(entityManager.createQuery("DELETE FROM " + table));
        }
        for(Query query : queries) query.executeUpdate();
    }

This method *usually* works.  However, sometimes I get *very* strange
nonapplication exceptions that are causing what appear to be intermittent
test failures.  By intermittent I mean a given unit test method might pass
during one "mvn clean build" process, then fail immediately afterward during
a second "mvn clean build" process, without any alterations to the source or
testing code.  This happens on some tests, sometimes, and not on others,
other times, and I can discern no clear pattern.  One example of the error
output is listed at the bottom of this message.

So, what I'm really looking for is a performant way to guarantee that my
tables are empty between unit tests, so that I'm essentially starting from a
completely clean environment.  I welcome any suggestions, and will even
entertain non-specification compliant mechanisms provided by OpenEJB or
OpenJPA to accomplish this.

Dain mentioned that I might completely drop and restart OpenEJB between
tests.  He also mentioned that OpenJPA might have a way to wipe the database
tables clean for a given persistence unit.

I'm interested in any ideas or feedback about this.

Regards,
--
Alexander R. Saint Croix





As promised, here is the error:
-------------------------------------------------------------------------------
Test set: org.eremite.corm.party.PersistenceTest
-------------------------------------------------------------------------------
Tests run: 8, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.749 sec
<<< FAILURE!
testPartySignatureAnnotations(org.eremite.corm.party.PersistenceTest)  Time
elapsed: 0.466 sec  <<< ERROR!
javax.ejb.EJBException: The bean encountered a non-application exception.;
nested exception is:
    <openjpa-1.0.1-r420667:592145 fatal general error>
org.apache.openjpa.persistence.PersistenceException: no-saved-fields
    at org.apache.openejb.core.ivm.BaseEjbProxyHandler.convertException(
BaseEjbProxyHandler.java:366)
    at org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(
BaseEjbProxyHandler.java:323)
    at org.apache.openejb.util.proxy.Jdk13InvocationHandler.invoke(
Jdk13InvocationHandler.java:49)
    at $Proxy19.clear(Unknown Source)
    at org.eremite.corm.party.PersistenceTest.testPartySignatureAnnotations(
PersistenceTest.java:189)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:585)
    at junit.framework.TestCase.runTest(TestCase.java:164)
    at junit.framework.TestCase.runBare(TestCase.java:130)
    at junit.framework.TestResult$1.protect(TestResult.java:110)
    at junit.framework.TestResult.runProtected(TestResult.java:128)
    at junit.framework.TestResult.run(TestResult.java:113)
    at junit.framework.TestCase.run(TestCase.java:120)
    at junit.framework.TestSuite.runTest(TestSuite.java:228)
    at junit.framework.TestSuite.run(TestSuite.java:223)
    at org.junit.internal.runners.OldTestClassRunner.run(
OldTestClassRunner.java:35)
    at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(
JUnit4TestSet.java:62)
    at
org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(
AbstractDirectoryTestSuite.java:138)
    at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(
AbstractDirectoryTestSuite.java:125)
    at org.apache.maven.surefire.Surefire.run(Surefire.java:132)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:585)
    at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(
SurefireBooter.java:290)
    at org.apache.maven.surefire.booter.SurefireBooter.main(
SurefireBooter.java:818)
Caused by: <openjpa-1.0.1-r420667:592145 fatal general error>
org.apache.openjpa.persistence.PersistenceException: no-saved-fields
    at org.apache.openjpa.kernel.StateManagerImpl.dirtyCheck(
StateManagerImpl.java:799)
    at org.apache.openjpa.kernel.BrokerImpl$ManagedCache.dirtyCheck(
BrokerImpl.java:4620)
    at org.apache.openjpa.kernel.BrokerImpl$ManagedCache.access$000(
BrokerImpl.java:4360)
    at org.apache.openjpa.kernel.BrokerImpl.hasTransactionalObjects(
BrokerImpl.java:3739)
    at org.apache.openjpa.kernel.BrokerImpl.setDirty(BrokerImpl.java:3856)
    at org.apache.openjpa.kernel.StateManagerImpl.setPCState(
StateManagerImpl.java:207)
    at org.apache.openjpa.kernel.StateManagerImpl.dirty(
StateManagerImpl.java:1532)
    at org.apache.openjpa.kernel.StateManagerImpl.dirty(
StateManagerImpl.java:1471)
    at org.apache.openjpa.kernel.StateManagerImpl.dirtyCheck(
StateManagerImpl.java:808)
    at org.apache.openjpa.kernel.BrokerImpl$ManagedCache.dirtyCheck(
BrokerImpl.java:4620)
    at org.apache.openjpa.kernel.BrokerImpl$ManagedCache.access$000(
BrokerImpl.java:4360)
    at org.apache.openjpa.kernel.BrokerImpl.hasTransactionalObjects(
BrokerImpl.java:3739)
    at org.apache.openjpa.kernel.BrokerImpl.setDirty(BrokerImpl.java:3856)
    at org.apache.openjpa.kernel.StateManagerImpl.setPCState(
StateManagerImpl.java:205)
    at org.apache.openjpa.kernel.StateManagerImpl.delete(
StateManagerImpl.java:1070)
    at org.apache.openjpa.kernel.BrokerImpl.delete(BrokerImpl.java:2518)
    at org.apache.openjpa.kernel.SingleFieldManager.delete(
SingleFieldManager.java:387)
    at org.apache.openjpa.kernel.SingleFieldManager.delete(
SingleFieldManager.java:373)
    at org.apache.openjpa.kernel.SingleFieldManager.delete(
SingleFieldManager.java:330)
    at org.apache.openjpa.kernel.SingleFieldManager.delete(
SingleFieldManager.java:284)
    at org.apache.openjpa.kernel.StateManagerImpl.cascadeDelete(
StateManagerImpl.java:2817)
    at org.apache.openjpa.kernel.BrokerImpl.delete(BrokerImpl.java:2517)
    at org.apache.openjpa.kernel.BrokerImpl.delete(BrokerImpl.java:2482)
    at org.apache.openjpa.kernel.QueryImpl.deleteInMemory(QueryImpl.java
:1020)
    at
org.apache.openjpa.kernel.ExpressionStoreQuery$DataStoreExecutor.executeDelete
(ExpressionStoreQuery.java:684)
    at org.apache.openjpa.kernel.QueryImpl.delete(QueryImpl.java:1008)
    at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:801)
    at org.apache.openjpa.kernel.QueryImpl.deleteAll(QueryImpl.java:866)
    at org.apache.openjpa.kernel.QueryImpl.deleteAll(QueryImpl.java:862)
    at org.apache.openjpa.kernel.DelegatingQuery.deleteAll(
DelegatingQuery.java:541)
    at org.apache.openjpa.persistence.QueryImpl.executeUpdate(QueryImpl.java
:314)
    at org.eremite.corm.party.BeanManagerImpl.clear(BeanManagerImpl.java:66)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:585)
    at
org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke
(ReflectionInvocationContext.java:146)
    at
org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(
ReflectionInvocationContext.java:129)
    at org.apache.openejb.core.interceptor.InterceptorStack.invoke(
InterceptorStack.java:67)
    at org.apache.openejb.core.stateless.StatelessContainer._invoke(
StatelessContainer.java:210)
    at org.apache.openejb.core.stateless.StatelessContainer._invoke(
StatelessContainer.java:188)
    at org.apache.openejb.core.stateless.StatelessContainer.invoke(
StatelessContainer.java:165)
    at org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(
EjbObjectProxyHandler.java:217)
    at org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(
EjbObjectProxyHandler.java:77)
    at org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(
BaseEjbProxyHandler.java:321)
    ... 26 more
Reply | Threaded
Open this post in threaded view
|

Re: Nuking tables on tearDown() for CRUD tests

Jacek Laskowski
On Jan 6, 2008 11:42 PM, Alexander Saint Croix
<[hidden email]> wrote:

> Dain mentioned that I might completely drop and restart OpenEJB between
> tests.  He also mentioned that OpenJPA might have a way to wipe the database
> tables clean for a given persistence unit.

Here's one possible solution for you - the usage of in-memory db and
<property name="openjpa.jdbc.SynchronizeMappings"
value="buildSchema(SchemaAction='add,deleteTableContents')" />. Until
an em is in use the tables are there in db. Once it's closed, at
openejb shutdown, the tables will get dropped automatically. You may
also want to remove the target directory where the db sits in.

See http://www.jaceklaskowski.pl/wiki/Zasady_zapisu_zmian_do_bazy_danych_w_JPA#Konfiguracja_JPA_-_persistence.xml
for some configuration settings. The article is about how entity
changes are persisted in db using different jpa providers and is
written in Polish, but the configuration files should be
well-understandable by non-Polish speakers too.

Jacek

--
Jacek Laskowski
http://www.JacekLaskowski.pl
Reply | Threaded
Open this post in threaded view
|

Re: Nuking tables on tearDown() for CRUD tests

Alexander Saint Croix-2
Jacek,

Thank you very much for the reply. I am using an in-memory db, and I tried
the property below with no success.  Because the db is in memory, there's
nothing to delete in the file system.

Cheers,
--
Alex



On Jan 6, 2008 5:44 PM, Jacek Laskowski <[hidden email]> wrote:

> On Jan 6, 2008 11:42 PM, Alexander Saint Croix
> <[hidden email]> wrote:
>
> > Dain mentioned that I might completely drop and restart OpenEJB between
> > tests.  He also mentioned that OpenJPA might have a way to wipe the
> database
> > tables clean for a given persistence unit.
>
> Here's one possible solution for you - the usage of in-memory db and
> <property name="openjpa.jdbc.SynchronizeMappings"
> value="buildSchema(SchemaAction='add,deleteTableContents')" />. Until
> an em is in use the tables are there in db. Once it's closed, at
> openejb shutdown, the tables will get dropped automatically. You may
> also want to remove the target directory where the db sits in.
>
> See
> http://www.jaceklaskowski.pl/wiki/Zasady_zapisu_zmian_do_bazy_danych_w_JPA#Konfiguracja_JPA_-_persistence.xml
> for some configuration settings. The article is about how entity
> changes are persisted in db using different jpa providers and is
> written in Polish, but the configuration files should be
> well-understandable by non-Polish speakers too.
>
> Jacek
>
> --
> Jacek Laskowski
> http://www.JacekLaskowski.pl
>
Reply | Threaded
Open this post in threaded view
|

Re: Nuking tables on tearDown() for CRUD tests

Alexander Saint Croix-2
In reply to this post by Alexander Saint Croix-2
I'll ask the OpenEJB guys if this is possible.

Cheers,
--
Alex



On Jan 6, 2008 7:46 PM, Pinaki Poddar <[hidden email]> wrote:

>
> This error is related to dynamic runtime enhancement (my guess). If you
> can,
> switch to build-time enhancement, to verify this guess.
> --
> View this message in context:
> http://www.nabble.com/Nuking-tables-on-tearDown%28%29-for-CRUD-tests-tp14655065p14657182.html
> Sent from the OpenJPA Users mailing list archive at Nabble.com.
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Nuking tables on tearDown() for CRUD tests

Dain Sundstrom
In reply to this post by Jacek Laskowski
This only works if he shuts down OpenEJB between each test method,  
which would be slow.

-dain

On Jan 6, 2008, at 3:44 PM, Jacek Laskowski wrote:

> On Jan 6, 2008 11:42 PM, Alexander Saint Croix
> <[hidden email]> wrote:
>
>> Dain mentioned that I might completely drop and restart OpenEJB  
>> between
>> tests.  He also mentioned that OpenJPA might have a way to wipe  
>> the database
>> tables clean for a given persistence unit.
>
> Here's one possible solution for you - the usage of in-memory db and
> <property name="openjpa.jdbc.SynchronizeMappings"
> value="buildSchema(SchemaAction='add,deleteTableContents')" />. Until
> an em is in use the tables are there in db. Once it's closed, at
> openejb shutdown, the tables will get dropped automatically. You may
> also want to remove the target directory where the db sits in.
>
> See http://www.jaceklaskowski.pl/wiki/ 
> Zasady_zapisu_zmian_do_bazy_danych_w_JPA#Konfiguracja_JPA_-
> _persistence.xml
> for some configuration settings. The article is about how entity
> changes are persisted in db using different jpa providers and is
> written in Polish, but the configuration files should be
> well-understandable by non-Polish speakers too.
>
> Jacek
>
> --
> Jacek Laskowski
> http://www.JacekLaskowski.pl

Reply | Threaded
Open this post in threaded view
|

Re: Nuking tables on tearDown() for CRUD tests

Dain Sundstrom
In reply to this post by Alexander Saint Croix-2
Can you try running each delete query in a separate transaction?  
Something like this in your session bean:

public void clear(String type) {
     Query query = entityManager.createQuery("DELETE FROM " + type)
     query.executeUpdate();
}


and something like this in the test case:

protected void tearDown() {
     String[] types = {...};
     for(String type : types) {
         bean.clear(type);
     }
}

One final thing, if you have foreign key constrained relationships,  
you will need to make sure the cascade delete settings are correct.  
For example, if you have an order with many line items and foreign  
key constraints, you must delete the line items before the order can  
be deleted.  This is normally accomplished with a cascade delete  
setting on the relationship.

-dain

On Jan 6, 2008, at 2:42 PM, Alexander Saint Croix wrote:

> Hello,
>
> I'm doing CRUD tests on a collection of integrated entities in a  
> shared
> persistence unit, and am encountering some errors because of crufty  
> tables
> from previous tests.  I'm looking for a means of guaranteeing that the
> tables for each of my entities is devoid of entries after each test  
> runs.
> The appropriate place to do this would be in the tearDown() method  
> of the
> unit test class.
>
> My environment is OpenEJB (3.0.0 core, latest snapshot), OpenJPA  
> and an
> in-memory HSQLDB.  OS X on Java 5.
>
> At the moment, I've implemented a stateless session bean to manage
> persistence for me.  It's using an injected entity manager to  
> perform a
> series of DELETE statements.  Here's how the EM is set up:
>
>     @PersistenceContext(unitName = "party-test-unit", type =
> PersistenceContextType.TRANSACTION)
>     private EntityManager entityManager;
>
> I want to stress that I was previously removing entries from the  
> database by
> getting a list of components of a given class, then iteratively  
> deleting
> them.  This worked just fine, but was very inefficient and slowed  
> down the
> builds more than was acceptable.
>
> My clear() method defines a String[] of table names, then recursively
> creates a Query("DELETE FROM " + table) for each of the bean class  
> names in
> the package I'm testing. Here is the code:
>
>     public void clear() {
>         Set<Query> queries = new HashSet<Query>();
>         String[] types = {
>                 "Address",
>                 "AssociatedAddress",
>                 "EmailAddress",
>                 "GeographicAddress",
>                 "ISOCountryCode",
>                 "Locale",
>                 "TelecomAddress",
>                 "WebPageAddress",
>                 "AssignedResponsibility",
>                 "Capability",
>                 "PartyRelationship",
>                 "PartyRelationshipType",
>                 "PartyRole",
>                 "PartyRoleType",
>                 "Responsibility",
>                 "Organization",
>                 "OrganizationName",
>                 "Party",
>                 "PartySignature",
>                 "Person",
>                 "PersonName",
>                 "Preference",
>                 "PreferenceOption",
>                 "PreferenceType",
>                 "Property",
>                 "RegisteredIdentifier"};
>         for(String table : types) {
>             queries.add(entityManager.createQuery("DELETE FROM " +  
> table));
>         }
>         for(Query query : queries) query.executeUpdate();
>     }
>
> This method *usually* works.  However, sometimes I get *very* strange
> nonapplication exceptions that are causing what appear to be  
> intermittent
> test failures.  By intermittent I mean a given unit test method  
> might pass
> during one "mvn clean build" process, then fail immediately  
> afterward during
> a second "mvn clean build" process, without any alterations to the  
> source or
> testing code.  This happens on some tests, sometimes, and not on  
> others,
> other times, and I can discern no clear pattern.  One example of  
> the error
> output is listed at the bottom of this message.
>
> So, what I'm really looking for is a performant way to guarantee  
> that my
> tables are empty between unit tests, so that I'm essentially  
> starting from a
> completely clean environment.  I welcome any suggestions, and will  
> even
> entertain non-specification compliant mechanisms provided by  
> OpenEJB or
> OpenJPA to accomplish this.
>
> Dain mentioned that I might completely drop and restart OpenEJB  
> between
> tests.  He also mentioned that OpenJPA might have a way to wipe the  
> database
> tables clean for a given persistence unit.
>
> I'm interested in any ideas or feedback about this.
>
> Regards,
> --
> Alexander R. Saint Croix
>
>
>
>
>
> As promised, here is the error:
> ----------------------------------------------------------------------
> ---------
> Test set: org.eremite.corm.party.PersistenceTest
> ----------------------------------------------------------------------
> ---------
> Tests run: 8, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:  
> 5.749 sec
> <<< FAILURE!
> testPartySignatureAnnotations
> (org.eremite.corm.party.PersistenceTest)  Time
> elapsed: 0.466 sec  <<< ERROR!
> javax.ejb.EJBException: The bean encountered a non-application  
> exception.;
> nested exception is:
>     <openjpa-1.0.1-r420667:592145 fatal general error>
> org.apache.openjpa.persistence.PersistenceException: no-saved-fields
>     at  
> org.apache.openejb.core.ivm.BaseEjbProxyHandler.convertException(
> BaseEjbProxyHandler.java:366)
>     at org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(
> BaseEjbProxyHandler.java:323)
>     at org.apache.openejb.util.proxy.Jdk13InvocationHandler.invoke(
> Jdk13InvocationHandler.java:49)
>     at $Proxy19.clear(Unknown Source)
>     at  
> org.eremite.corm.party.PersistenceTest.testPartySignatureAnnotations(
> PersistenceTest.java:189)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:39)
>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:25)
>     at java.lang.reflect.Method.invoke(Method.java:585)
>     at junit.framework.TestCase.runTest(TestCase.java:164)
>     at junit.framework.TestCase.runBare(TestCase.java:130)
>     at junit.framework.TestResult$1.protect(TestResult.java:110)
>     at junit.framework.TestResult.runProtected(TestResult.java:128)
>     at junit.framework.TestResult.run(TestResult.java:113)
>     at junit.framework.TestCase.run(TestCase.java:120)
>     at junit.framework.TestSuite.runTest(TestSuite.java:228)
>     at junit.framework.TestSuite.run(TestSuite.java:223)
>     at org.junit.internal.runners.OldTestClassRunner.run(
> OldTestClassRunner.java:35)
>     at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(
> JUnit4TestSet.java:62)
>     at
> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTest
> Set(
> AbstractDirectoryTestSuite.java:138)
>     at  
> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(
> AbstractDirectoryTestSuite.java:125)
>     at org.apache.maven.surefire.Surefire.run(Surefire.java:132)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:39)
>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:25)
>     at java.lang.reflect.Method.invoke(Method.java:585)
>     at  
> org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(
> SurefireBooter.java:290)
>     at org.apache.maven.surefire.booter.SurefireBooter.main(
> SurefireBooter.java:818)
> Caused by: <openjpa-1.0.1-r420667:592145 fatal general error>
> org.apache.openjpa.persistence.PersistenceException: no-saved-fields
>     at org.apache.openjpa.kernel.StateManagerImpl.dirtyCheck(
> StateManagerImpl.java:799)
>     at org.apache.openjpa.kernel.BrokerImpl$ManagedCache.dirtyCheck(
> BrokerImpl.java:4620)
>     at org.apache.openjpa.kernel.BrokerImpl$ManagedCache.access$000(
> BrokerImpl.java:4360)
>     at org.apache.openjpa.kernel.BrokerImpl.hasTransactionalObjects(
> BrokerImpl.java:3739)
>     at org.apache.openjpa.kernel.BrokerImpl.setDirty
> (BrokerImpl.java:3856)
>     at org.apache.openjpa.kernel.StateManagerImpl.setPCState(
> StateManagerImpl.java:207)
>     at org.apache.openjpa.kernel.StateManagerImpl.dirty(
> StateManagerImpl.java:1532)
>     at org.apache.openjpa.kernel.StateManagerImpl.dirty(
> StateManagerImpl.java:1471)
>     at org.apache.openjpa.kernel.StateManagerImpl.dirtyCheck(
> StateManagerImpl.java:808)
>     at org.apache.openjpa.kernel.BrokerImpl$ManagedCache.dirtyCheck(
> BrokerImpl.java:4620)
>     at org.apache.openjpa.kernel.BrokerImpl$ManagedCache.access$000(
> BrokerImpl.java:4360)
>     at org.apache.openjpa.kernel.BrokerImpl.hasTransactionalObjects(
> BrokerImpl.java:3739)
>     at org.apache.openjpa.kernel.BrokerImpl.setDirty
> (BrokerImpl.java:3856)
>     at org.apache.openjpa.kernel.StateManagerImpl.setPCState(
> StateManagerImpl.java:205)
>     at org.apache.openjpa.kernel.StateManagerImpl.delete(
> StateManagerImpl.java:1070)
>     at org.apache.openjpa.kernel.BrokerImpl.delete(BrokerImpl.java:
> 2518)
>     at org.apache.openjpa.kernel.SingleFieldManager.delete(
> SingleFieldManager.java:387)
>     at org.apache.openjpa.kernel.SingleFieldManager.delete(
> SingleFieldManager.java:373)
>     at org.apache.openjpa.kernel.SingleFieldManager.delete(
> SingleFieldManager.java:330)
>     at org.apache.openjpa.kernel.SingleFieldManager.delete(
> SingleFieldManager.java:284)
>     at org.apache.openjpa.kernel.StateManagerImpl.cascadeDelete(
> StateManagerImpl.java:2817)
>     at org.apache.openjpa.kernel.BrokerImpl.delete(BrokerImpl.java:
> 2517)
>     at org.apache.openjpa.kernel.BrokerImpl.delete(BrokerImpl.java:
> 2482)
>     at org.apache.openjpa.kernel.QueryImpl.deleteInMemory
> (QueryImpl.java
> :1020)
>     at
> org.apache.openjpa.kernel.ExpressionStoreQuery
> $DataStoreExecutor.executeDelete
> (ExpressionStoreQuery.java:684)
>     at org.apache.openjpa.kernel.QueryImpl.delete(QueryImpl.java:1008)
>     at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:801)
>     at org.apache.openjpa.kernel.QueryImpl.deleteAll(QueryImpl.java:
> 866)
>     at org.apache.openjpa.kernel.QueryImpl.deleteAll(QueryImpl.java:
> 862)
>     at org.apache.openjpa.kernel.DelegatingQuery.deleteAll(
> DelegatingQuery.java:541)
>     at org.apache.openjpa.persistence.QueryImpl.executeUpdate
> (QueryImpl.java
> :314)
>     at org.eremite.corm.party.BeanManagerImpl.clear
> (BeanManagerImpl.java:66)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:39)
>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:25)
>     at java.lang.reflect.Method.invoke(Method.java:585)
>     at
> org.apache.openejb.core.interceptor.ReflectionInvocationContext
> $Invocation.invoke
> (ReflectionInvocationContext.java:146)
>     at
> org.apache.openejb.core.interceptor.ReflectionInvocationContext.procee
> d(
> ReflectionInvocationContext.java:129)
>     at org.apache.openejb.core.interceptor.InterceptorStack.invoke(
> InterceptorStack.java:67)
>     at org.apache.openejb.core.stateless.StatelessContainer._invoke(
> StatelessContainer.java:210)
>     at org.apache.openejb.core.stateless.StatelessContainer._invoke(
> StatelessContainer.java:188)
>     at org.apache.openejb.core.stateless.StatelessContainer.invoke(
> StatelessContainer.java:165)
>     at  
> org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(
> EjbObjectProxyHandler.java:217)
>     at org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(
> EjbObjectProxyHandler.java:77)
>     at org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(
> BaseEjbProxyHandler.java:321)
>     ... 26 more

Reply | Threaded
Open this post in threaded view
|

Re: Nuking tables on tearDown() for CRUD tests

Alexander Saint Croix-2
I'll try moving the string array into the client and running each delete in
a separate transaction.  I'll also see if altering the order of deletes
helps the process along.  Some of the relationships entail cascading
deletes, others do not, and not all are bidirectional, so depending on how
sensitive the data source or persistence provider are, I might have to go
carefully through the entities and figure out the proper order for them to
be deleted.  If some rules could be derived about that, we might look into
building tools to assist with this sort of process later on.

I'll let you know how it goes.  I have to admit, working on this is
addictive.

Cheers, and thanks!
--
Alex



On Jan 8, 2008 3:36 PM, Dain Sundstrom <[hidden email]> wrote:

> Can you try running each delete query in a separate transaction?
> Something like this in your session bean:
>
> public void clear(String type) {
>     Query query = entityManager.createQuery("DELETE FROM " + type)
>     query.executeUpdate();
> }
>
>
> and something like this in the test case:
>
> protected void tearDown() {
>     String[] types = {...};
>     for(String type : types) {
>         bean.clear(type);
>     }
> }
>
> One final thing, if you have foreign key constrained relationships,
> you will need to make sure the cascade delete settings are correct.
> For example, if you have an order with many line items and foreign
> key constraints, you must delete the line items before the order can
> be deleted.  This is normally accomplished with a cascade delete
> setting on the relationship.
>
> -dain
>
> On Jan 6, 2008, at 2:42 PM, Alexander Saint Croix wrote:
>
> > Hello,
> >
> > I'm doing CRUD tests on a collection of integrated entities in a
> > shared
> > persistence unit, and am encountering some errors because of crufty
> > tables
> > from previous tests.  I'm looking for a means of guaranteeing that the
> > tables for each of my entities is devoid of entries after each test
> > runs.
> > The appropriate place to do this would be in the tearDown() method
> > of the
> > unit test class.
> >
> > My environment is OpenEJB (3.0.0 core, latest snapshot), OpenJPA
> > and an
> > in-memory HSQLDB.  OS X on Java 5.
> >
> > At the moment, I've implemented a stateless session bean to manage
> > persistence for me.  It's using an injected entity manager to
> > perform a
> > series of DELETE statements.  Here's how the EM is set up:
> >
> >     @PersistenceContext(unitName = "party-test-unit", type =
> > PersistenceContextType.TRANSACTION)
> >     private EntityManager entityManager;
> >
> > I want to stress that I was previously removing entries from the
> > database by
> > getting a list of components of a given class, then iteratively
> > deleting
> > them.  This worked just fine, but was very inefficient and slowed
> > down the
> > builds more than was acceptable.
> >
> > My clear() method defines a String[] of table names, then recursively
> > creates a Query("DELETE FROM " + table) for each of the bean class
> > names in
> > the package I'm testing. Here is the code:
> >
> >     public void clear() {
> >         Set<Query> queries = new HashSet<Query>();
> >         String[] types = {
> >                 "Address",
> >                 "AssociatedAddress",
> >                 "EmailAddress",
> >                 "GeographicAddress",
> >                 "ISOCountryCode",
> >                 "Locale",
> >                 "TelecomAddress",
> >                 "WebPageAddress",
> >                 "AssignedResponsibility",
> >                 "Capability",
> >                 "PartyRelationship",
> >                 "PartyRelationshipType",
> >                 "PartyRole",
> >                 "PartyRoleType",
> >                 "Responsibility",
> >                 "Organization",
> >                 "OrganizationName",
> >                 "Party",
> >                 "PartySignature",
> >                 "Person",
> >                 "PersonName",
> >                 "Preference",
> >                 "PreferenceOption",
> >                 "PreferenceType",
> >                 "Property",
> >                 "RegisteredIdentifier"};
> >         for(String table : types) {
> >             queries.add(entityManager.createQuery("DELETE FROM " +
> > table));
> >         }
> >         for(Query query : queries) query.executeUpdate();
> >     }
> >
> > This method *usually* works.  However, sometimes I get *very* strange
> > nonapplication exceptions that are causing what appear to be
> > intermittent
> > test failures.  By intermittent I mean a given unit test method
> > might pass
> > during one "mvn clean build" process, then fail immediately
> > afterward during
> > a second "mvn clean build" process, without any alterations to the
> > source or
> > testing code.  This happens on some tests, sometimes, and not on
> > others,
> > other times, and I can discern no clear pattern.  One example of
> > the error
> > output is listed at the bottom of this message.
> >
> > So, what I'm really looking for is a performant way to guarantee
> > that my
> > tables are empty between unit tests, so that I'm essentially
> > starting from a
> > completely clean environment.  I welcome any suggestions, and will
> > even
> > entertain non-specification compliant mechanisms provided by
> > OpenEJB or
> > OpenJPA to accomplish this.
> >
> > Dain mentioned that I might completely drop and restart OpenEJB
> > between
> > tests.  He also mentioned that OpenJPA might have a way to wipe the
> > database
> > tables clean for a given persistence unit.
> >
> > I'm interested in any ideas or feedback about this.
> >
> > Regards,
> > --
> > Alexander R. Saint Croix
> >
> >
> >
> >
> >
> > As promised, here is the error:
> > ----------------------------------------------------------------------
> > ---------
> > Test set: org.eremite.corm.party.PersistenceTest
> > ----------------------------------------------------------------------
> > ---------
> > Tests run: 8, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> > 5.749 sec
> > <<< FAILURE!
> > testPartySignatureAnnotations
> > (org.eremite.corm.party.PersistenceTest)  Time
> > elapsed: 0.466 sec  <<< ERROR!
> > javax.ejb.EJBException: The bean encountered a non-application
> > exception.;
> > nested exception is:
> >     <openjpa-1.0.1-r420667:592145 fatal general error>
> > org.apache.openjpa.persistence.PersistenceException: no-saved-fields
> >     at
> > org.apache.openejb.core.ivm.BaseEjbProxyHandler.convertException(
> > BaseEjbProxyHandler.java:366)
> >     at org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(
> > BaseEjbProxyHandler.java:323)
> >     at org.apache.openejb.util.proxy.Jdk13InvocationHandler.invoke(
> > Jdk13InvocationHandler.java:49)
> >     at $Proxy19.clear(Unknown Source)
> >     at
> > org.eremite.corm.party.PersistenceTest.testPartySignatureAnnotations(
> > PersistenceTest.java:189)
> >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >     at sun.reflect.NativeMethodAccessorImpl.invoke(
> > NativeMethodAccessorImpl.java:39)
> >     at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> > DelegatingMethodAccessorImpl.java:25)
> >     at java.lang.reflect.Method.invoke(Method.java:585)
> >     at junit.framework.TestCase.runTest(TestCase.java:164)
> >     at junit.framework.TestCase.runBare(TestCase.java:130)
> >     at junit.framework.TestResult$1.protect(TestResult.java:110)
> >     at junit.framework.TestResult.runProtected(TestResult.java:128)
> >     at junit.framework.TestResult.run(TestResult.java:113)
> >     at junit.framework.TestCase.run(TestCase.java:120)
> >     at junit.framework.TestSuite.runTest(TestSuite.java:228)
> >     at junit.framework.TestSuite.run(TestSuite.java:223)
> >     at org.junit.internal.runners.OldTestClassRunner.run(
> > OldTestClassRunner.java:35)
> >     at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(
> > JUnit4TestSet.java:62)
> >     at
> > org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTest
> > Set(
> > AbstractDirectoryTestSuite.java:138)
> >     at
> > org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(
> > AbstractDirectoryTestSuite.java:125)
> >     at org.apache.maven.surefire.Surefire.run(Surefire.java:132)
> >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >     at sun.reflect.NativeMethodAccessorImpl.invoke(
> > NativeMethodAccessorImpl.java:39)
> >     at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> > DelegatingMethodAccessorImpl.java:25)
> >     at java.lang.reflect.Method.invoke(Method.java:585)
> >     at
> > org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(
> > SurefireBooter.java:290)
> >     at org.apache.maven.surefire.booter.SurefireBooter.main(
> > SurefireBooter.java:818)
> > Caused by: <openjpa-1.0.1-r420667:592145 fatal general error>
> > org.apache.openjpa.persistence.PersistenceException: no-saved-fields
> >     at org.apache.openjpa.kernel.StateManagerImpl.dirtyCheck(
> > StateManagerImpl.java:799)
> >     at org.apache.openjpa.kernel.BrokerImpl$ManagedCache.dirtyCheck(
> > BrokerImpl.java:4620)
> >     at org.apache.openjpa.kernel.BrokerImpl$ManagedCache.access$000(
> > BrokerImpl.java:4360)
> >     at org.apache.openjpa.kernel.BrokerImpl.hasTransactionalObjects(
> > BrokerImpl.java:3739)
> >     at org.apache.openjpa.kernel.BrokerImpl.setDirty
> > (BrokerImpl.java:3856)
> >     at org.apache.openjpa.kernel.StateManagerImpl.setPCState(
> > StateManagerImpl.java:207)
> >     at org.apache.openjpa.kernel.StateManagerImpl.dirty(
> > StateManagerImpl.java:1532)
> >     at org.apache.openjpa.kernel.StateManagerImpl.dirty(
> > StateManagerImpl.java:1471)
> >     at org.apache.openjpa.kernel.StateManagerImpl.dirtyCheck(
> > StateManagerImpl.java:808)
> >     at org.apache.openjpa.kernel.BrokerImpl$ManagedCache.dirtyCheck(
> > BrokerImpl.java:4620)
> >     at org.apache.openjpa.kernel.BrokerImpl$ManagedCache.access$000(
> > BrokerImpl.java:4360)
> >     at org.apache.openjpa.kernel.BrokerImpl.hasTransactionalObjects(
> > BrokerImpl.java:3739)
> >     at org.apache.openjpa.kernel.BrokerImpl.setDirty
> > (BrokerImpl.java:3856)
> >     at org.apache.openjpa.kernel.StateManagerImpl.setPCState(
> > StateManagerImpl.java:205)
> >     at org.apache.openjpa.kernel.StateManagerImpl.delete(
> > StateManagerImpl.java:1070)
> >     at org.apache.openjpa.kernel.BrokerImpl.delete(BrokerImpl.java:
> > 2518)
> >     at org.apache.openjpa.kernel.SingleFieldManager.delete(
> > SingleFieldManager.java:387)
> >     at org.apache.openjpa.kernel.SingleFieldManager.delete(
> > SingleFieldManager.java:373)
> >     at org.apache.openjpa.kernel.SingleFieldManager.delete(
> > SingleFieldManager.java:330)
> >     at org.apache.openjpa.kernel.SingleFieldManager.delete(
> > SingleFieldManager.java:284)
> >     at org.apache.openjpa.kernel.StateManagerImpl.cascadeDelete(
> > StateManagerImpl.java:2817)
> >     at org.apache.openjpa.kernel.BrokerImpl.delete(BrokerImpl.java:
> > 2517)
> >     at org.apache.openjpa.kernel.BrokerImpl.delete(BrokerImpl.java:
> > 2482)
> >     at org.apache.openjpa.kernel.QueryImpl.deleteInMemory
> > (QueryImpl.java
> > :1020)
> >     at
> > org.apache.openjpa.kernel.ExpressionStoreQuery
> > $DataStoreExecutor.executeDelete
> > (ExpressionStoreQuery.java:684)
> >     at org.apache.openjpa.kernel.QueryImpl.delete(QueryImpl.java:1008)
> >     at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:801)
> >     at org.apache.openjpa.kernel.QueryImpl.deleteAll(QueryImpl.java:
> > 866)
> >     at org.apache.openjpa.kernel.QueryImpl.deleteAll(QueryImpl.java:
> > 862)
> >     at org.apache.openjpa.kernel.DelegatingQuery.deleteAll(
> > DelegatingQuery.java:541)
> >     at org.apache.openjpa.persistence.QueryImpl.executeUpdate
> > (QueryImpl.java
> > :314)
> >     at org.eremite.corm.party.BeanManagerImpl.clear
> > (BeanManagerImpl.java:66)
> >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >     at sun.reflect.NativeMethodAccessorImpl.invoke(
> > NativeMethodAccessorImpl.java:39)
> >     at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> > DelegatingMethodAccessorImpl.java:25)
> >     at java.lang.reflect.Method.invoke(Method.java:585)
> >     at
> > org.apache.openejb.core.interceptor.ReflectionInvocationContext
> > $Invocation.invoke
> > (ReflectionInvocationContext.java:146)
> >     at
> > org.apache.openejb.core.interceptor.ReflectionInvocationContext.procee
> > d(
> > ReflectionInvocationContext.java:129)
> >     at org.apache.openejb.core.interceptor.InterceptorStack.invoke(
> > InterceptorStack.java:67)
> >     at org.apache.openejb.core.stateless.StatelessContainer._invoke(
> > StatelessContainer.java:210)
> >     at org.apache.openejb.core.stateless.StatelessContainer._invoke(
> > StatelessContainer.java:188)
> >     at org.apache.openejb.core.stateless.StatelessContainer.invoke(
> > StatelessContainer.java:165)
> >     at
> > org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(
> > EjbObjectProxyHandler.java:217)
> >     at org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(
> > EjbObjectProxyHandler.java:77)
> >     at org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(
> > BaseEjbProxyHandler.java:321)
> >     ... 26 more
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Nuking tables on tearDown() for CRUD tests

Alexander Saint Croix-2
After working with devs on both the OpenEJB and the OpenJPA teams, I think
that for anything other than trivial persistence units and entity relations,
it is probably necessary to manually find a list of all of the entities of a
given type, iterate over that list, and use the entitymanager.remove()
method to clean out the entities in the datastore between unit tests.
Although this method is extremely slow and will be a big hit to
productivity, I cannot get a succession of "DELETE FROM table" queries to
reliably remove objects without crashing because of entity relations and
foreign keys, and have not yet had the former method crash on me.

The "DELETE FROM table" mechanism seems to wreak havoc on the cascading
rules and cause problems for itself when done in succession to multiple
tables--I don't know enough about the guts of the query execution mechanism
to say why, and that's alright.  If we need to build an example application
for either project to test this behavior at a later time, I'll be able to
provide a sufficiently complex persistence unit to really stress test
things.

In the meantime, I really am itching to get back to actual development, so
I'm going to revert and move forward using the other method for the time
being. Thanks to Dain, Panaki, Jacek, Adam, and Patrick.

Cheers,
--
Alex
Reply | Threaded
Open this post in threaded view
|

Fwd: Nuking tables on tearDown() for CRUD tests

Alexander Saint Croix-2
This was one suggestion by Adam Hardy on the users@openjpa list.  I might
look into dbUnit, but wonder whether it is ideally suited for the container
injection mechanisms we're using.

Cheers,
--
Alex


---------- Forwarded message ----------
From: Adam Hardy <[hidden email]>
Date: Jan 8, 2008 5:52 PM
Subject: Re: Nuking tables on tearDown() for CRUD tests
To: [hidden email]


Alexander Saint Croix on 08/01/08 23:08, wrote:
> After working with devs on both the OpenEJB and the OpenJPA teams, I think
> that for anything other than trivial persistence units and entity
relations,
> it is probably necessary to manually find a list of all of the entities of
a

> given type, iterate over that list, and use the entitymanager.remove()
> method to clean out the entities in the datastore between unit tests.
> Although this method is extremely slow and will be a big hit to
> productivity, I cannot get a succession of "DELETE FROM table" queries to
> reliably remove objects without crashing because of entity relations and
> foreign keys, and have not yet had the former method crash on me.
>
> The "DELETE FROM table" mechanism seems to wreak havoc on the cascading
> rules and cause problems for itself when done in succession to multiple
> tables--I don't know enough about the guts of the query execution
mechanism
> to say why, and that's alright.  If we need to build an example
application
> for either project to test this behavior at a later time, I'll be able to
> provide a sufficiently complex persistence unit to really stress test
> things.
>
> In the meantime, I really am itching to get back to actual development, so
> I'm going to revert and move forward using the other method for the time
> being. Thanks to Dain, Panaki, Jacek, Adam, and Patrick.

I recommended DbUnit for handling test data before, and in terms of the
performance hit, it won't be much problem at all.

However it would require a list of all tables from which to delete, and it
must
be ordered to take account of referential integrity constraints.

DbUnit also allows you to store all data in XML files which can be loaded
fresh
for each test. Obviously this uses the same list of tables as above, just in
reverse. And you need to ditch and recreate the entity manager between each
test. It wasn't quick or easy to set up a system to handle the test data
like
this, but it's paid dividends.
Reply | Threaded
Open this post in threaded view
|

Re: Nuking tables on tearDown() for CRUD tests

dblevins
Administrator
I swore I thought I mentioned using fork mode, but just in case.   If  
you use fork mode with the in-memory db, there's nothing to clean.  
Have you experimented with that route.

We're still using jaxb to unmarshall the persistence.xml, but I've  
been thinking of cutting that out which might save you 1 second each  
test.  JAXB takes a while to initialize the first time it's used in a  
vm.

-David

On Jan 8, 2008, at 5:59 PM, Alexander Saint Croix wrote:

> This was one suggestion by Adam Hardy on the users@openjpa list.  I  
> might
> look into dbUnit, but wonder whether it is ideally suited for the  
> container
> injection mechanisms we're using.
>
> Cheers,
> --
> Alex
>
>
> ---------- Forwarded message ----------
> From: Adam Hardy <[hidden email]>
> Date: Jan 8, 2008 5:52 PM
> Subject: Re: Nuking tables on tearDown() for CRUD tests
> To: [hidden email]
>
>
> Alexander Saint Croix on 08/01/08 23:08, wrote:
>> After working with devs on both the OpenEJB and the OpenJPA teams,  
>> I think
>> that for anything other than trivial persistence units and entity
> relations,
>> it is probably necessary to manually find a list of all of the  
>> entities of
> a
>> given type, iterate over that list, and use the  
>> entitymanager.remove()
>> method to clean out the entities in the datastore between unit tests.
>> Although this method is extremely slow and will be a big hit to
>> productivity, I cannot get a succession of "DELETE FROM table"  
>> queries to
>> reliably remove objects without crashing because of entity  
>> relations and
>> foreign keys, and have not yet had the former method crash on me.
>>
>> The "DELETE FROM table" mechanism seems to wreak havoc on the  
>> cascading
>> rules and cause problems for itself when done in succession to  
>> multiple
>> tables--I don't know enough about the guts of the query execution
> mechanism
>> to say why, and that's alright.  If we need to build an example
> application
>> for either project to test this behavior at a later time, I'll be  
>> able to
>> provide a sufficiently complex persistence unit to really stress test
>> things.
>>
>> In the meantime, I really am itching to get back to actual  
>> development, so
>> I'm going to revert and move forward using the other method for the  
>> time
>> being. Thanks to Dain, Panaki, Jacek, Adam, and Patrick.
>
> I recommended DbUnit for handling test data before, and in terms of  
> the
> performance hit, it won't be much problem at all.
>
> However it would require a list of all tables from which to delete,  
> and it
> must
> be ordered to take account of referential integrity constraints.
>
> DbUnit also allows you to store all data in XML files which can be  
> loaded
> fresh
> for each test. Obviously this uses the same list of tables as above,  
> just in
> reverse. And you need to ditch and recreate the entity manager  
> between each
> test. It wasn't quick or easy to set up a system to handle the test  
> data
> like
> this, but it's paid dividends.

Reply | Threaded
Open this post in threaded view
|

Re: Nuking tables on tearDown() for CRUD tests

Alexander Saint Croix-2
David, Dain and I tested this last night, and here's the final conclusion we
came to.

The initial problem was that retrieving and iterating over a list of the
entities from the entity manager and deleting them one at a time is
incredibly slow (one of my test cases took 4 minutes and 23 seconds to run
through ten test methods for less than 100 entity instances) and the tests
slowed down to a crawl.  The persistence provider (in this case OpenJPA)
isn't built for this sort of usage, so we decided to try a second technique
involving executing native SQL queries directly against the database by way
of the EntityManager.

The queries we determined to run were a sequence of "DELETE FROM foo"
queries, where "foo" is replaced with the name of the actual table / class.
Thus, for Person objects, we'd execute a "DELETE FROM Person" query.

However, this is a potentially dangerous practice as well, for reasons which
are now clear.  I was deleting tables in an order that did not take into
consideration maintaining referential integrity of foreign keys.  Because
many of my objects are tightly integrated, it is important to delete the
tables in a particular order, such that reference objects are wiped clean
first.

I was able to isolate this behavior by experimentally altering the order of
table deletions.  Certain cases would always throw errors, others would
throw none.  The errors each had a characteristic error stack trace with
many repetitions of the following line:

WARN - Unexpected exception from afterCompletion; continuing
> <openjpa-1.0.1-r420667:592145 fatal general error>
> org.apache.openjpa.persistence.PersistenceException: no-saved-fields
>         at org.apache.openjpa.kernel.StateManagerImpl.dirtyCheck(
> StateManagerImpl.java:799)
>

So, anywhere an entity table has a foreign key column you need to either
perform a cascading delete with the entityManager, nullify the reference
(assuming the foreign key columns are optional), or make certain to delete
the referencing bean first.

The recommended method to clear the foreign key columns is to run update
queries.  This is a good mechanism for breaking circular references on
teardown. An example looks like this:

"UPDATE Customer c set.myfk = null"

So, to sum things up, direct deletion against the database is much faster
than working through the EntityManager interface in some cases, but you need
to determine the precise order you are tearing items down, or design your
tests so that they are resilient to cruft in the database, if you intend on
running tests with an embedded OpenEJB container.

For the record, we also tried setting up the POM for the project to fork the
testing JVM, but this mechanism does not create a new JVM for each test
METHOD.  It only creates a new JVM for each test CASE.

I'm now running direct deletes, and being sensitive about foreign key
references, deletion order, and circular loops.  My final solution will
initially include a combination of direct deletes and EntityManager remove
calls via OpenJPA, and I will iteratively tune it until I've got a reliable
scraping method for my library.

One suggestion / idea for the future would be to provide a tool to analyze a
persistence unit and generate this sort of fast delete mechanism as an array
of query strings that could be serialized for any given persistence unit and
marshalled when needed for rapid clearing of the database during entity
tests.

A great big thanks to everyone on both teams who helped me isolate and
resolve this issue.
--
Alexander R. Saint Croix.



On Jan 8, 2008 8:21 PM, David Blevins <[hidden email]> wrote:

> I swore I thought I mentioned using fork mode, but just in case.   If
> you use fork mode with the in-memory db, there's nothing to clean.
> Have you experimented with that route.
>
> We're still using jaxb to unmarshall the persistence.xml, but I've
> been thinking of cutting that out which might save you 1 second each
> test.  JAXB takes a while to initialize the first time it's used in a
> vm.
>
> -David