Speed-up of gerrit builds

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Speed-up of gerrit builds

Jonah Graham
Hi cdt-devers,

I have made an update to how gerrit patches are verified with Hudson.
Rather than one large job (cdt-verify) that does all the work, 4
smaller jobs are run in parallel that each test a different part of
CDT. This allows for the following highlights:

1) Time from git push to build result reduced from 52 minutes to 25 minutes

       2x improvement

2) Time from git push to compile-only build completing so you know if
there is a silly compile only error (like forgetting to commit a file
or forgetting to update pom.xml when version changes) reduced from 52
minutes to 7 minutes -

       7x improvement

3) Time to rebuild failing test because Bug 499777 isn't resolved yet
reduced from 52 minutes to as little as 17 (depending on what failed).
This takes advantage of being able to retrigger only the job that
failed instead of the whole build -

       3x improvement


In gerrit the Hudson job votes the same, but instead of reporting one
job result, it reports 4, for example:

    Hudson CI
    Patch Set 3: Verified-1
    Build Unstable

    https://hudson.eclipse.org/cdt/job/cdt-verify-test-dsf-gdb-only/2/ : SUCCESS
    https://hudson.eclipse.org/cdt/job/cdt-verify-test-cdt-other/2/ : SUCCESS
    https://hudson.eclipse.org/cdt/job/cdt-verify-compile-only/3/ : SUCCESS
    https://hudson.eclipse.org/cdt/job/cdt-verify-test-cdt-ui-only/3/ : UNSTABLE


You can see all the jobs here: https://hudson.eclipse.org/cdt/view/Gerrit/
You can see my notes in the bug: https://bugs.eclipse.org/515304

Please let me know on cdt-dev or in the bug above if you have any
comments/concerns on my approach.

Thanks,
Jonah

~~~
Jonah Graham
Kichwa Coders Ltd.
www.kichwacoders.com
_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cdt-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Speed-up of gerrit builds

Marc-André Laperle
How did you do the multiple jobs? I tried setting multiple with the Gerrit Trigger before and only one of the jobs every triggered.

From: [hidden email] <[hidden email]> on behalf of Jonah Graham <[hidden email]>
Sent: Friday, April 14, 2017 8:13:44 PM
To: CDT General developers list.
Subject: [cdt-dev] Speed-up of gerrit builds
 
Hi cdt-devers,

I have made an update to how gerrit patches are verified with Hudson.
Rather than one large job (cdt-verify) that does all the work, 4
smaller jobs are run in parallel that each test a different part of
CDT. This allows for the following highlights:

1) Time from git push to build result reduced from 52 minutes to 25 minutes

       2x improvement

2) Time from git push to compile-only build completing so you know if
there is a silly compile only error (like forgetting to commit a file
or forgetting to update pom.xml when version changes) reduced from 52
minutes to 7 minutes -

       7x improvement

3) Time to rebuild failing test because Bug 499777 isn't resolved yet
reduced from 52 minutes to as little as 17 (depending on what failed).
This takes advantage of being able to retrigger only the job that
failed instead of the whole build -

       3x improvement


In gerrit the Hudson job votes the same, but instead of reporting one
job result, it reports 4, for example:

    Hudson CI
    Patch Set 3: Verified-1
    Build Unstable

    https://hudson.eclipse.org/cdt/job/cdt-verify-test-dsf-gdb-only/2/ : SUCCESS
    https://hudson.eclipse.org/cdt/job/cdt-verify-test-cdt-other/2/ : SUCCESS
    https://hudson.eclipse.org/cdt/job/cdt-verify-compile-only/3/ : SUCCESS
    https://hudson.eclipse.org/cdt/job/cdt-verify-test-cdt-ui-only/3/ : UNSTABLE


You can see all the jobs here: https://hudson.eclipse.org/cdt/view/Gerrit/
You can see my notes in the bug: https://bugs.eclipse.org/515304

Please let me know on cdt-dev or in the bug above if you have any
comments/concerns on my approach.

Thanks,
Jonah

~~~
Jonah Graham
Kichwa Coders Ltd.
www.kichwacoders.com
_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cdt-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Speed-up of gerrit builds

Jonah Graham
Hi Marc-Andre,

I am not sure what I did differently, I just duplicated the cdt-verify
job and then tuned it. I saw that gerrit trigger supported multiple
jobs when I contributed something to Nebula and got multiple builds.

Perhaps a bug has been fixed since your last attempt?

A while ago CDT had multiple jobs (see for example
https://git.eclipse.org/r/#/c/86315/) running too, but they were both
doing the same thing.

HTH
Jonah
~~~
Jonah Graham
Kichwa Coders Ltd.
www.kichwacoders.com


On 16 April 2017 at 17:57, Marc-André Laperle
<[hidden email]> wrote:

> How did you do the multiple jobs? I tried setting multiple with the Gerrit
> Trigger before and only one of the jobs every triggered.
> ________________________________
> From: [hidden email] <[hidden email]> on behalf of
> Jonah Graham <[hidden email]>
> Sent: Friday, April 14, 2017 8:13:44 PM
> To: CDT General developers list.
> Subject: [cdt-dev] Speed-up of gerrit builds
>
> Hi cdt-devers,
>
> I have made an update to how gerrit patches are verified with Hudson.
> Rather than one large job (cdt-verify) that does all the work, 4
> smaller jobs are run in parallel that each test a different part of
> CDT. This allows for the following highlights:
>
> 1) Time from git push to build result reduced from 52 minutes to 25 minutes
>
>        2x improvement
>
> 2) Time from git push to compile-only build completing so you know if
> there is a silly compile only error (like forgetting to commit a file
> or forgetting to update pom.xml when version changes) reduced from 52
> minutes to 7 minutes -
>
>        7x improvement
>
> 3) Time to rebuild failing test because Bug 499777 isn't resolved yet
> reduced from 52 minutes to as little as 17 (depending on what failed).
> This takes advantage of being able to retrigger only the job that
> failed instead of the whole build -
>
>        3x improvement
>
>
> In gerrit the Hudson job votes the same, but instead of reporting one
> job result, it reports 4, for example:
>
>     Hudson CI
>     Patch Set 3: Verified-1
>     Build Unstable
>
>     https://hudson.eclipse.org/cdt/job/cdt-verify-test-dsf-gdb-only/2/ :
> SUCCESS
>     https://hudson.eclipse.org/cdt/job/cdt-verify-test-cdt-other/2/ :
> SUCCESS
>     https://hudson.eclipse.org/cdt/job/cdt-verify-compile-only/3/ : SUCCESS
>     https://hudson.eclipse.org/cdt/job/cdt-verify-test-cdt-ui-only/3/ :
> UNSTABLE
>
>
> You can see all the jobs here: https://hudson.eclipse.org/cdt/view/Gerrit/
> You can see my notes in the bug: https://bugs.eclipse.org/515304
>
> Please let me know on cdt-dev or in the bug above if you have any
> comments/concerns on my approach.
>
> Thanks,
> Jonah
>
> ~~~
> Jonah Graham
> Kichwa Coders Ltd.
> www.kichwacoders.com
> _______________________________________________
> cdt-dev mailing list
> [hidden email]
> To change your delivery options, retrieve your password, or unsubscribe from
> this list, visit
> https://dev.eclipse.org/mailman/listinfo/cdt-dev
>
> _______________________________________________
> cdt-dev mailing list
> [hidden email]
> To change your delivery options, retrieve your password, or unsubscribe from
> this list, visit
> https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cdt-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Speed-up of gerrit builds

Marc-André Laperle

It's very good that it works now. Perhaps I also did a mistake somewhere in the job configs when I tried this. Thanks a lot for looking into this!


Marc-André


From: [hidden email] <[hidden email]> on behalf of Jonah Graham <[hidden email]>
Sent: Sunday, April 16, 2017 1:03:04 PM
To: CDT General developers list.
Subject: Re: [cdt-dev] Speed-up of gerrit builds
 
Hi Marc-Andre,

I am not sure what I did differently, I just duplicated the cdt-verify
job and then tuned it. I saw that gerrit trigger supported multiple
jobs when I contributed something to Nebula and got multiple builds.

Perhaps a bug has been fixed since your last attempt?

A while ago CDT had multiple jobs (see for example
https://git.eclipse.org/r/#/c/86315/) running too, but they were both
doing the same thing.

HTH
Jonah
~~~
Jonah Graham
Kichwa Coders Ltd.
www.kichwacoders.com


On 16 April 2017 at 17:57, Marc-André Laperle
<[hidden email]> wrote:
> How did you do the multiple jobs? I tried setting multiple with the Gerrit
> Trigger before and only one of the jobs every triggered.
> ________________________________
> From: [hidden email] <[hidden email]> on behalf of
> Jonah Graham <[hidden email]>
> Sent: Friday, April 14, 2017 8:13:44 PM
> To: CDT General developers list.
> Subject: [cdt-dev] Speed-up of gerrit builds
>
> Hi cdt-devers,
>
> I have made an update to how gerrit patches are verified with Hudson.
> Rather than one large job (cdt-verify) that does all the work, 4
> smaller jobs are run in parallel that each test a different part of
> CDT. This allows for the following highlights:
>
> 1) Time from git push to build result reduced from 52 minutes to 25 minutes
>
>        2x improvement
>
> 2) Time from git push to compile-only build completing so you know if
> there is a silly compile only error (like forgetting to commit a file
> or forgetting to update pom.xml when version changes) reduced from 52
> minutes to 7 minutes -
>
>        7x improvement
>
> 3) Time to rebuild failing test because Bug 499777 isn't resolved yet
> reduced from 52 minutes to as little as 17 (depending on what failed).
> This takes advantage of being able to retrigger only the job that
> failed instead of the whole build -
>
>        3x improvement
>
>
> In gerrit the Hudson job votes the same, but instead of reporting one
> job result, it reports 4, for example:
>
>     Hudson CI
>     Patch Set 3: Verified-1
>     Build Unstable
>
>     https://hudson.eclipse.org/cdt/job/cdt-verify-test-dsf-gdb-only/2/ :
> SUCCESS
>     https://hudson.eclipse.org/cdt/job/cdt-verify-test-cdt-other/2/ :
> SUCCESS
>     https://hudson.eclipse.org/cdt/job/cdt-verify-compile-only/3/ : SUCCESS
>     https://hudson.eclipse.org/cdt/job/cdt-verify-test-cdt-ui-only/3/ :
> UNSTABLE
>
>
> You can see all the jobs here: https://hudson.eclipse.org/cdt/view/Gerrit/
> You can see my notes in the bug: https://bugs.eclipse.org/515304
>
> Please let me know on cdt-dev or in the bug above if you have any
> comments/concerns on my approach.
>
> Thanks,
> Jonah
>
> ~~~
> Jonah Graham
> Kichwa Coders Ltd.
> www.kichwacoders.com
> _______________________________________________
> cdt-dev mailing list
> [hidden email]
> To change your delivery options, retrieve your password, or unsubscribe from
> this list, visit
> https://dev.eclipse.org/mailman/listinfo/cdt-dev
>
> _______________________________________________
> cdt-dev mailing list
> [hidden email]
> To change your delivery options, retrieve your password, or unsubscribe from
> this list, visit
> https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cdt-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Speed-up of gerrit builds

Nathan Ridge
In reply to this post by Jonah Graham
Hi Jonah,

Just wanted to say, thanks a lot for putting in the effort into making this change! This, together with your fixing or disabling of the most common intermittent failures, has significantly improved the experience of testing one's patch.

Thanks,
Nate
________________________________________
From: [hidden email] <[hidden email]> on behalf of Jonah Graham <[hidden email]>
Sent: April 15, 2017 12:13 AM
To: CDT General developers list.
Subject: [cdt-dev] Speed-up of gerrit builds

Hi cdt-devers,

I have made an update to how gerrit patches are verified with Hudson.
Rather than one large job (cdt-verify) that does all the work, 4
smaller jobs are run in parallel that each test a different part of
CDT. This allows for the following highlights:

1) Time from git push to build result reduced from 52 minutes to 25 minutes

       2x improvement

2) Time from git push to compile-only build completing so you know if
there is a silly compile only error (like forgetting to commit a file
or forgetting to update pom.xml when version changes) reduced from 52
minutes to 7 minutes -

       7x improvement

3) Time to rebuild failing test because Bug 499777 isn't resolved yet
reduced from 52 minutes to as little as 17 (depending on what failed).
This takes advantage of being able to retrigger only the job that
failed instead of the whole build -

       3x improvement


In gerrit the Hudson job votes the same, but instead of reporting one
job result, it reports 4, for example:

    Hudson CI
    Patch Set 3: Verified-1
    Build Unstable

    https://hudson.eclipse.org/cdt/job/cdt-verify-test-dsf-gdb-only/2/ : SUCCESS
    https://hudson.eclipse.org/cdt/job/cdt-verify-test-cdt-other/2/ : SUCCESS
    https://hudson.eclipse.org/cdt/job/cdt-verify-compile-only/3/ : SUCCESS
    https://hudson.eclipse.org/cdt/job/cdt-verify-test-cdt-ui-only/3/ : UNSTABLE


You can see all the jobs here: https://hudson.eclipse.org/cdt/view/Gerrit/
You can see my notes in the bug: https://bugs.eclipse.org/515304

Please let me know on cdt-dev or in the bug above if you have any
comments/concerns on my approach.

Thanks,
Jonah

~~~
Jonah Graham
Kichwa Coders Ltd.
www.kichwacoders.com
_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cdt-dev
Loading...