Disabling test cases

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Disabling test cases

Rolf Theunissen
Hi,

It is common practice to disable test because they are not stable. I don't want to discuss if that is good or bad practice. However, I think we can improve a bit on how we disable test cases.

It is common practice to comment test-cases out to disable them. Or, to add if statements such that test cases are skipped. As a result it is not obvious how many tests are currently disabled. In the first case (commenting out) the test is just not counted, in the latter case (if statement) the test-case always passes.

Many of the tests are now migrated to JUnit4. JUnit4 has support to ignore test-cases. When a test case or suite should be ignored it can be annotated with @Ignore. When a test-case should be ignored on one OS only, assume statements can be used.

The benefit of using @Ignore and assume is that test results get labeled as 'ignored'. This makes it obvious that many test are currently not run for a piece of code.

Best Regards,
Rolf

_______________________________________________
platform-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/platform-dev
Reply | Threaded
Open this post in threaded view
|

Re: Disabling test cases

Lars Vogel-2
+1

Rolf Theunissen <[hidden email]> schrieb am Mo., 27. Jan. 2020, 14:25:
Hi,

It is common practice to disable test because they are not stable. I don't want to discuss if that is good or bad practice. However, I think we can improve a bit on how we disable test cases.

It is common practice to comment test-cases out to disable them. Or, to add if statements such that test cases are skipped. As a result it is not obvious how many tests are currently disabled. In the first case (commenting out) the test is just not counted, in the latter case (if statement) the test-case always passes.

Many of the tests are now migrated to JUnit4. JUnit4 has support to ignore test-cases. When a test case or suite should be ignored it can be annotated with @Ignore. When a test-case should be ignored on one OS only, assume statements can be used.

The benefit of using @Ignore and assume is that test results get labeled as 'ignored'. This makes it obvious that many test are currently not run for a piece of code.

Best Regards,
Rolf
_______________________________________________
platform-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/platform-dev

_______________________________________________
platform-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/platform-dev