Threads when launching multiple tests

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Threads when launching multiple tests

banbury_ian
Hi,

I'm executing a set of tests on a third-party product. The Continuous
Integration launch seems to hold each test active. As each test starts a
Spring app, and the Spring app sets up a DataSourceBean, the result is that
the db runs out of conections before the tests are complete, causing
subsequent tests to fail with an SQL exception. For some reason the CT
launch thread then fails to terminate and has to be stopped manually.
Is it possible to make CT run each test in an individual thread that is
terminated at the end of the test?

Cheers,
Ian.



_______________________________________________
Continuous-testing-plugin-discuss mailing list
[hidden email]
https://lists.csail.mit.edu/mailman/listinfo/continuous-testing-plugin-discuss
Reply | Threaded
Open this post in threaded view
|

RE: Threads when launching multipletests

Ilja Preuss
> I'm executing a set of tests on a third-party product. The Continuous
> Integration launch seems to hold each test active. As each test
> starts a Spring app, and the Spring app sets up a DataSourceBean, the
> result is that the db runs out of conections before the tests are
> complete, causing subsequent tests to fail with an SQL exception. For
> some reason the CT launch thread then fails to terminate and has to
> be stopped manually. Is it possible to make CT run each test in an
> individual thread that is terminated at the end of the test?

I'm not sure I understand what you mean by "holding each test active", but
as far as I know, the Junit architecture allows test runners to reference
test instances as long as they like.

Are you using tearDown to free your resources?

Cheers, Ilja


_______________________________________________
Continuous-testing-plugin-discuss mailing list
[hidden email]
https://lists.csail.mit.edu/mailman/listinfo/continuous-testing-plugin-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Threads when launching multiple tests

dsaff
In reply to this post by banbury_ian
(Sorry to everyone for previously accidentally taking the discussion
off-list)

Ian,

Thanks for the additional information.  I'm currently trying a few
things out on CT to see if I can reproduce the behavior.  Do you have
the source code for these tests?  Is there an explicit call to release
db connections?  From where is it called?  Thanks,

    David Saff

Ian Priest wrote:

> Hi David,
>
> when I run the tests outside CT they release all resource once each
> test class is complete. I've had a look at the test code and it is
> using some abstract test classes to enable testing of various
> implementation types of an interface. It looks to me like it's using
> the abstract test pattern mentioned in an earlier post to the
> mail-list. (Sorry to be a bit vague about what the tests are doing.
> It's a 3rd party product that I'm trying to integrate so I'm having to
> dig into the code to try and find answers).
>
> The behaviour would seem to indicate that the CT runner is holding
> onto an instance of the test class after it's run and that is causing
> the test class (or more specifically the spring app that the test
> class starts) to hang onto it's db connections rather than releasing
> them.
> When I run the tests through Ant the db connection count goes up to 10
> as each test class is started, then back down to 0 when each test
> class finishes it's tests and terminates. When I run them through CT
> the number of db conections goes up by 10 as each test class is
> created but doesn't go down again when the class finishes it's tests.
>
> Does the CT runner hang onto an instance until all the tests are done
> - for example to get the test stats at the end of the run?
>
> It's arguable that the fault lies with the test classes failing to do
> a tear-down (and as they're not my code I'm happy to accept that
> they're flawed ;-) ), but I'm still curious about how CT behaves.
>
> Cheers,
> Ian.
>
>> From: David Saff <[hidden email]>
>> To: Ian Priest <[hidden email]>
>> Subject: Re: [Continuous-testing-plugin-discuss] Threads when
>> launching multiple tests
>> Date: Tue, 29 Nov 2005 14:10:09 -0500
>>
>> Ian,
>>
>> The behavior of threads within your test run should not be affected
>> by continuous testing.  Do you see behavior like this at all when
>> running your tests outside of CT?
>>
>> In the past, we have had problems running test suites that involve
>> TestDecorators.  Is that the case here?
>>
>> Thanks,
>>
>>    David Saff
>>
>> Ian Priest wrote:
>>
>>> Hi,
>>>
>>> I'm executing a set of tests on a third-party product. The
>>> Continuous Integration launch seems to hold each test active. As
>>> each test starts a Spring app, and the Spring app sets up a
>>> DataSourceBean, the result is that the db runs out of conections
>>> before the tests are complete, causing subsequent tests to fail with
>>> an SQL exception. For some reason the CT launch thread then fails to
>>> terminate and has to be stopped manually.
>>> Is it possible to make CT run each test in an individual thread that
>>> is terminated at the end of the test?
>>>
>>> Cheers,
>>> Ian.
>>>
>>>
>>>
>>> _______________________________________________
>>> Continuous-testing-plugin-discuss mailing list
>>> [hidden email]
>>> https://lists.csail.mit.edu/mailman/listinfo/continuous-testing-plugin-discuss 
>>>
>>>
>>
>>
>
>


_______________________________________________
Continuous-testing-plugin-discuss mailing list
[hidden email]
https://lists.csail.mit.edu/mailman/listinfo/continuous-testing-plugin-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Threads when launching multiple tests

banbury_ian
Hi David,

please note that it looks like this problem is to do with the way Eclipse
runs JUnit tests rather than CT. However, if you're still interested...

I can recreate it quite easily. You'll need Jars for Spring; Jakata Commons
dbcp, logging, pool, connector and collections; log4j and junit to copy what
I did. You'll also need mysql and it's java connectors if you use a mysql db
like I did, but any db should do.

Set up a (for example) MySql database, then create an Eclipse project. Add
all the above Jars to the project's classpath. Create a Spring
application-context.xml file and add a DataSourceBean to it as follows,
adjusting the db details to suit the database you've set up...

<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE beans PUBLIC '-//SPRING//DTD BEAN//EN'
'http://www.springframework.org/dtd/spring-beans.dtd'>

<beans>

    <!-- Datasource bean -->
    <bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close">
        <property name="driverClassName">
            <value>org.gjt.mm.mysql.Driver</value>
        </property>
        <property name="url">
            <value>jdbc:mysql:///CTThreadRecreate</value>
        </property>
        <property name="username">
            <value>ctthreadrecreate</value>
        </property>
        <property name="password">
            <value>ctthreadrecreate</value>
        </property>
    </bean>

</beans>

I then created a set of test classes, named Test1 to Test10 as follows...

import junit.framework.TestCase;

import org.apache.commons.dbcp.BasicDataSource;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

public class Test1 extends TestCase {

        private static ApplicationContext ctx =
            new ClassPathXmlApplicationContext("application-context.xml");

        public void test1() throws Exception {
                // Get the bean to connect to the db

                BasicDataSource ds = (BasicDataSource)ctx.getBean("dataSource");
                java.sql.Connection c = ds.getConnection();
                c.close();

                // Fail the test
                //throw new Exception("Test failed");

        }

}

I then set up a JUnit launch config to "run all the tests in the the
project" and ran the tests. Edit the  test that gets run last so that it
sleeps for a reasonable length of time before exiting. This is to give
yourself time to examine the dbase connection stats.
In MySql you can just start a MqSQL command-line client and execute "show
processlist" to view the number of connections. Do this just before running
the tests again and it'll probably show 1 or 2 processes. Now run the tests,
and while the last one is sleeping run "show processlist" again in MySQL.
It'll now show somewhere around 11 connections. When the sleeping test
finishes and the JUnit run config drops out all the connections will be
dropped.

Important note - I haven't enabled CT on the project, so I'm guessing that
actually this behaviour is coming from Eclipse's standard JUnit run config,
unless your CT stuff hijacks it in some way.

It looks as though something in the Eclipse JUnit runner is hanging on to a
reference to each test class instance until all the tests have run. It might
even be possible to recreate without using Spring at all. Just create a
static reference to a BasicDataSource in each test and use that to get the
connection. I suspect there are two fixes for this: Get the third party to
change their unit tests so that they explicitly stop the Spring app when
each test class is finshed, or increase the number of permitted connections
to my database(!). (Unfortunately using set-up and tearDown to make the
tests behave will make them run very slowly indeed due to the overhead of
starting the Spring app for each test.)

Thanks for the time you've spent so far. If you don't want to take this any
further that's fine by me. I'll see if I can get more info from the Eclipse
forums.

Cheers,
Ian.


>From: David Saff <[hidden email]>
>To: Ian Priest <[hidden email]>
>CC: [hidden email]
>Subject: Re: [Continuous-testing-plugin-discuss] Threads when launching
>multiple tests
>Date: Wed, 30 Nov 2005 06:49:36 -0500
>
>(Sorry to everyone for previously accidentally taking the discussion
>off-list)
>
>Ian,
>
>Thanks for the additional information.  I'm currently trying a few things
>out on CT to see if I can reproduce the behavior.  Do you have the source
>code for these tests?  Is there an explicit call to release db connections?
>  From where is it called?  Thanks,
>
>    David Saff
>
>Ian Priest wrote:
>
>>Hi David,
>>
>>when I run the tests outside CT they release all resource once each test
>>class is complete. I've had a look at the test code and it is using some
>>abstract test classes to enable testing of various implementation types of
>>an interface. It looks to me like it's using the abstract test pattern
>>mentioned in an earlier post to the mail-list. (Sorry to be a bit vague
>>about what the tests are doing. It's a 3rd party product that I'm trying
>>to integrate so I'm having to dig into the code to try and find answers).
>>
>>The behaviour would seem to indicate that the CT runner is holding onto an
>>instance of the test class after it's run and that is causing the test
>>class (or more specifically the spring app that the test class starts) to
>>hang onto it's db connections rather than releasing them.
>>When I run the tests through Ant the db connection count goes up to 10 as
>>each test class is started, then back down to 0 when each test class
>>finishes it's tests and terminates. When I run them through CT the number
>>of db conections goes up by 10 as each test class is created but doesn't
>>go down again when the class finishes it's tests.
>>
>>Does the CT runner hang onto an instance until all the tests are done -
>>for example to get the test stats at the end of the run?
>>
>>It's arguable that the fault lies with the test classes failing to do a
>>tear-down (and as they're not my code I'm happy to accept that they're
>>flawed ;-) ), but I'm still curious about how CT behaves.
>>
>>Cheers,
>>Ian.
>>
>>>From: David Saff <[hidden email]>
>>>To: Ian Priest <[hidden email]>
>>>Subject: Re: [Continuous-testing-plugin-discuss] Threads when launching
>>>multiple tests
>>>Date: Tue, 29 Nov 2005 14:10:09 -0500
>>>
>>>Ian,
>>>
>>>The behavior of threads within your test run should not be affected by
>>>continuous testing.  Do you see behavior like this at all when running
>>>your tests outside of CT?
>>>
>>>In the past, we have had problems running test suites that involve
>>>TestDecorators.  Is that the case here?
>>>
>>>Thanks,
>>>
>>>    David Saff
>>>
>>>Ian Priest wrote:
>>>
>>>>Hi,
>>>>
>>>>I'm executing a set of tests on a third-party product. The Continuous
>>>>Integration launch seems to hold each test active. As each test starts a
>>>>Spring app, and the Spring app sets up a DataSourceBean, the result is
>>>>that the db runs out of conections before the tests are complete,
>>>>causing subsequent tests to fail with an SQL exception. For some reason
>>>>the CT launch thread then fails to terminate and has to be stopped
>>>>manually.
>>>>Is it possible to make CT run each test in an individual thread that is
>>>>terminated at the end of the test?
>>>>
>>>>Cheers,
>>>>Ian.
>>>>
>>>>
>>>>
>>>>_______________________________________________
>>>>Continuous-testing-plugin-discuss mailing list
>>>>[hidden email]
>>>>https://lists.csail.mit.edu/mailman/listinfo/continuous-testing-plugin-discuss
>>>>
>>>>
>>>
>>>
>>
>>
>



_______________________________________________
Continuous-testing-plugin-discuss mailing list
[hidden email]
https://lists.csail.mit.edu/mailman/listinfo/continuous-testing-plugin-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Threads when launching multiple tests

dsaff
Ian Priest wrote:

> Important note - I haven't enabled CT on the project, so I'm guessing
> that actually this behaviour is coming from Eclipse's standard JUnit
> run config, unless your CT stuff hijacks it in some way.

Actually, it does hijack JUnit runs even when tests aren't being run
continuously, so I'd be interested if you see the same behavior when you
disable the CT feature entirely.  Thanks,

    David Saff

_______________________________________________
Continuous-testing-plugin-discuss mailing list
[hidden email]
https://lists.csail.mit.edu/mailman/listinfo/continuous-testing-plugin-discuss