LLVM/Clang toolchain broken

classic Classic list List threaded Threaded
58 messages Options
123
Reply | Threaded
Open this post in threaded view
|

LLVM/Clang toolchain broken

Joost Kraaijeveld
Hi,

Some time ago I filed
https://bugs.eclipse.org/bugs/show_bug.cgi?id=535565 about the
LLVM/Clang toolchain being broken. At this moment it is not possible to
create a working "C+++ Managed Build" for LLVM/Clang.

I managed to run the CDT in the debuggers as per
https://wiki.eclipse.org/Getting_started_with_CDT_development and I
have an idea about the problem (see my bug-report that contains more
info). I am willing to try to solve the problem. Therefore I have some
questions to get me started.

- Is there any (high level) design documentation available for the LLVM
managedbuilder stuff? Or for GCC?
- Why is there a CDT LLVM preference page? What is/was it's purpose?
There is no such page for GCC?
- I cannot find UI for LLVM/Clang project properties. Can someone
enlighten me?
- The same for the GCC project properties?

TIA

--
Groeten,

Joost Kraaijeveld
Hommelseweg 123
6821 LD Arnhem
06-51855277

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
Reply | Threaded
Open this post in threaded view
|

Re: LLVM/Clang toolchain broken

Joost Kraaijeveld
Hi,

My immediate personal problem is solved by placing a comment on line 62
of LlvmResourceListener.java to prevent
LlvmPreferenceStore.addStdLibUnix() being called.


I suspect that the same should be done on line 59 but I have no Windows
with MinGW to test it with.

--
Groeten,

Joost Kraaijeveld
Hommelseweg 123
6821 LD Arnhem
06-51855277

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
Reply | Threaded
Open this post in threaded view
|

Re: LLVM/Clang toolchain broken

Jonah Graham
Hi Joost,

Thank you for your analysis up until this point.

In my new role as CDT project lead there are a number of areas that I am simply not familiar with (yet!) and this is one of them. 

I can try to provide some background, and I will be very happy to review any patches you provide!

To answer some of your questions:

The LLVM plug-ins were added in Bug 338553 and haven't changed much since then in the particular area of the code you have highlighted. 

From a quick look at this it seems there is a confusion in this code between settings passed to the LLVM compiler and those passed to the CDT indexer and which tool is "in charge". In simple cases that difference probably doesn't matter, but it clearly quickly goes wrong (the case in Bug 527757 is particularly obviously bad).


> - Is there any (high level) design documentation available for the LLVM managedbuilder stuff? Or for GCC?

Have a look at https://wiki.eclipse.org/CDT/designs/MBS it contains design documentation for when managed build was first added and has a number of links that may be useful. 

> - Why is there a CDT LLVM preference page? What is/was it's purpose?

I don't know. There may be answers in the Petri's thesis, but the link to it is dead (see comment).

> There is no such page for GCC?

Because it probably shouldn't exist for LLVM either?

> - I cannot find UI for LLVM/Clang project properties. Can someone
enlighten me?
> - The same for the GCC project properties?

It is in the Project Properties -> C/C++ Build -> Settings. (see attached file to confirm we are talking about the same thing)

Thank you for taking the time to look at this. There is certainly a lot of little bits needed to get a C++ Managed Build project to "just work" and I look forward to supporting you where I can.

Thanks
Jonah




~~~
Jonah Graham
Kichwa Coders
www.kichwacoders.com


On Fri, 17 Jan 2020 at 09:11, Joost Kraaijeveld <[hidden email]> wrote:
Hi,

My immediate personal problem is solved by placing a comment on line 62
of LlvmResourceListener.java to prevent
LlvmPreferenceStore.addStdLibUnix() being called.


I suspect that the same should be done on line 59 but I have no Windows
with MinGW to test it with.

--
Groeten,

Joost Kraaijeveld
Hommelseweg 123
6821 LD Arnhem
06-51855277

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev

Screenshot_2020-01-17_12-16-40.png (128K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: LLVM/Clang toolchain broken

Joost Kraaijeveld
On Fri, 2020-01-17 at 12:20 -0500, Jonah Graham wrote:
> (the case in Bug 527757 is particularly obviously bad).
I am pretty sure that my "solution" described in my previous mail (and
in 535565) also solves 527757 and probably 500186.

> > There is no such page for GCC?
>
> Because it probably shouldn't exist for LLVM either?
I think so.
>
> > - I cannot find UI for LLVM/Clang project properties. Can someone
> enlighten me?
> > - The same for the GCC project properties?
>
> It is in the Project Properties -> C/C++ Build -> Settings. (see
> attached file to confirm we are talking about the same thing)
Ah, no, sorry. I meant where in the source tree of the CDT are those UI
elements. I already discovered the dialogs in application in 2003 or
maybe earlier ;-)

Thanks for assisting!

--
Groeten,

Joost Kraaijeveld
Hommelseweg 123
6821 LD Arnhem
06-51855277

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
Reply | Threaded
Open this post in threaded view
|

Re: LLVM/Clang toolchain broken

Jonah Graham
On Fri, 17 Jan 2020 at 14:38, Joost Kraaijeveld <[hidden email]> wrote:
On Fri, 2020-01-17 at 12:20 -0500, Jonah Graham wrote:
> (the case in Bug 527757 is particularly obviously bad).
I am pretty sure that my "solution" described in my previous mail (and
in 535565) also solves 527757 and probably 500186.

Great!
 

> > There is no such page for GCC?
>
> Because it probably shouldn't exist for LLVM either?
I think so.
>
> > - I cannot find UI for LLVM/Clang project properties. Can someone
> enlighten me?
> > - The same for the GCC project properties?
>
> It is in the Project Properties -> C/C++ Build -> Settings. (see
> attached file to confirm we are talking about the same thing)
Ah, no, sorry. I meant where in the source tree of the CDT are those UI
elements. I already discovered the dialogs in application in 2003 or
maybe earlier ;-)

(embarrassed emoji) Of course - I was a bit on autopilot I suppose. Those UI elements are data driven, the first level of UI is org.eclipse.cdt.managedbuilder.ui.properties.Page_BuildSettings (org.eclipse.cdt.managedbuilder.ui bundle), the data is controlled primarily by XML in the plugin.xmls via the org.eclipse.cdt.managedbuilder.core.buildDefinitions extension point (see /org.eclipse.cdt.managedbuilder.llvm.ui/plugin.xml for LLVM's and /org.eclipse.cdt.managedbuilder.gnu.ui/plugin.xml for GCC's and https://help.eclipse.org/topic/org.eclipse.cdt.doc.isv/reference/extension-points/org_eclipse_cdt_managedbuilder_core_buildDefinitions.html for the extension point documentation).



Thanks for assisting!

I hope that helps.

Jonah

 

--
Groeten,

Joost Kraaijeveld
Hommelseweg 123
6821 LD Arnhem
06-51855277

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
Reply | Threaded
Open this post in threaded view
|

Re: LLVM/Clang toolchain broken

Marc-Andre Laperle-3
In reply to this post by Jonah Graham
Hi,

From what I remember, the intention of the preference page was to be able to add the system headers which a prebuilt clang did not know about at all and you had to specify manually. This preference page would be global to all projects and configurations. But this is not the way normally managed build projects are configured. In fact, it did this global setting by adding the include paths to every projects and every configuration, before every build and regardless of the toolchain! Also, post-build it would always do a full workspace refresh which I had to comment out because people were complaining quickly that Eclipse was unusable while this plugin was installed.

In the midst of the enthusiasm of having LLVM support, I don’t think this plugin was fully tested or thoroughly reviewed because there were/are glaring issues. I personally ended up using the GCC toolchain and just replacing the command with clang and clang++ which worked fine (especially once many OS/distros had a prebuilt clang fully configured with system paths, etc) so the motivation of fixing the LLVM toolchain was/is low. Still, I think it would be a nice project to go back and clean up the LLVM plugin (…or remove it). A first obvious clean-up would be to remove the preference page and any concept of global include path and libs. Another step would be to remove the many tools and toolchain that I don’t believe people use or are still maintained (LLVM-GCC, LLVM + JIT, etc). Then there is general code clean-up (there shouldn’t be obvious comment for every line).

Cheers,
Marc-André

On Jan 17, 2020, at 12:20 PM, Jonah Graham <[hidden email]> wrote:

Hi Joost,

Thank you for your analysis up until this point.

In my new role as CDT project lead there are a number of areas that I am simply not familiar with (yet!) and this is one of them. 

I can try to provide some background, and I will be very happy to review any patches you provide!

To answer some of your questions:

The LLVM plug-ins were added in Bug 338553 and haven't changed much since then in the particular area of the code you have highlighted. 

From a quick look at this it seems there is a confusion in this code between settings passed to the LLVM compiler and those passed to the CDT indexer and which tool is "in charge". In simple cases that difference probably doesn't matter, but it clearly quickly goes wrong (the case in Bug 527757 is particularly obviously bad).


> - Is there any (high level) design documentation available for the LLVM managedbuilder stuff? Or for GCC?

Have a look at https://wiki.eclipse.org/CDT/designs/MBS it contains design documentation for when managed build was first added and has a number of links that may be useful. 

> - Why is there a CDT LLVM preference page? What is/was it's purpose?

I don't know. There may be answers in the Petri's thesis, but the link to it is dead (see comment).

> There is no such page for GCC?

Because it probably shouldn't exist for LLVM either?

> - I cannot find UI for LLVM/Clang project properties. Can someone
enlighten me?
> - The same for the GCC project properties?

It is in the Project Properties -> C/C++ Build -> Settings. (see attached file to confirm we are talking about the same thing)

Thank you for taking the time to look at this. There is certainly a lot of little bits needed to get a C++ Managed Build project to "just work" and I look forward to supporting you where I can.

Thanks
Jonah




~~~
Jonah Graham
Kichwa Coders
www.kichwacoders.com


On Fri, 17 Jan 2020 at 09:11, Joost Kraaijeveld <[hidden email]> wrote:
Hi,

My immediate personal problem is solved by placing a comment on line 62
of LlvmResourceListener.java to prevent
LlvmPreferenceStore.addStdLibUnix() being called.


I suspect that the same should be done on line 59 but I have no Windows
with MinGW to test it with.

--
Groeten,

Joost Kraaijeveld
Hommelseweg 123
6821 LD Arnhem
06-51855277

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
<Screenshot_2020-01-17_12-16-40.png>_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev


_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
Reply | Threaded
Open this post in threaded view
|

Re: LLVM/Clang toolchain broken

Jan Baeyens

Hi

I'm happy to there is lots of attention for the managed build capability of CDT right now :-).
I have been playing with the gnu makefile generator for some years now and I have been impressed with the capabilities provided by the extension points (called build model below).
I realize I only understand a very small portion of the problem domain. For instance I don't know cMake, LLVM, LLVM-GCC, LLVM + JIT... but I also feel I can contribute.
What I do know is this; Whatever tool you use to build, you will always have to combine source code files with commands (executable and parameters).
The way to combine (and the commands) can be very easy or very complex.

When I look at the gnu makefile generator I think the build model is a very good basis to describe "how to combine source code files with commands".
If I look at the implementation of the gnu makefile generator I see 3 problematic design decisions:

1)No instantiation of the build model.
2)Trying to make "good makefiles"
3)Working with strings

I would like to sell the lack of "instantiation of the build model"  as a current weakness in CDT and the addition of a "instantiation of the build model" as the solution.
And please see my questions below.

1)No instantiation of the build model.

Suppose I have the super simple build model
convert each *.c to *.o with the command g++ -o {output} {input}
convert all *.o to {project}.exe with the command link  {inputs} {output}

With a project myExample, containing the files input.c and test.c; 

A instantiation of the build model would look like

Input files (input.c);output files (input.o);command (g++ -o input.o input.c)
Input files (test.c);output files (test.o);command (g++ -o test.o test.c)
Input files (input.o; test.o);output files (myExample.exe);command (link test.o input.o myExample.exe)

My thinking is that with this instantiation of the build model one can easily

1) make a makefile
2) run the commands
3) make a cmake file
....

Even though gnuMakefileGenrator does all this work; it doesn't instantiate the build model in any way, the code goes from build model directly to makefile in pieces.
If we could agree on a data model for the instantiation of the build model we could avoid lots of duplication and have a great addition to CDT.

2)Trying to make "good makefiles"

The second problem I see with gnuMakefileGenerator is that it tries to generate good makefiles. Hints to this are:
1)The possibility to not expand environment variables in the makefile
2)When using nameproviders the makefile becomes completely different

Though it is noble to make readable/maintainable makefiles, it also requires lots of extra resources (like testing) and a high level of knowledge about makefiles for future maintainers.
Using only one "type" of makefiles one can easily reduce:
1)amount of code
2)amount of tests
3)amount of training

3)Working with strings

The code is all based around strings and doesn't use classes for files and so on. I don't think this is due to lack of skill but a simple consequence of the design decisions I already mentioned.
This however adds to the maintenance nightmare.

My questions

My fingers want to tackle this problem but to avoid needless work I feel it is important for me to have a better understanding of how this fits into the cdt community. As such I have following questions.
1)Inside the broad CDT world. Does anyone know of other build models like described in the Managed Build System Extensibility Document in the eclipse help->CDT-plugin developer guide->programmers guide->CDT DOM?
2)Did you ever build a build model based on "Managed Build System Extensibility Document" or others? Can you share your experiences/thoughts?
3)Suppose I build a buildmodel instantiator (and a makefile generator for testing) does anyone think someone will use it?
4)Could this buildmodel instantiator become part of CDT?

Best regards

Jantje

Op 18/01/2020 om 4:50 schreef Marc-Andre Laperle:
Hi,

From what I remember, the intention of the preference page was to be able to add the system headers which a prebuilt clang did not know about at all and you had to specify manually. This preference page would be global to all projects and configurations. But this is not the way normally managed build projects are configured. In fact, it did this global setting by adding the include paths to every projects and every configuration, before every build and regardless of the toolchain! Also, post-build it would always do a full workspace refresh which I had to comment out because people were complaining quickly that Eclipse was unusable while this plugin was installed.

In the midst of the enthusiasm of having LLVM support, I don’t think this plugin was fully tested or thoroughly reviewed because there were/are glaring issues. I personally ended up using the GCC toolchain and just replacing the command with clang and clang++ which worked fine (especially once many OS/distros had a prebuilt clang fully configured with system paths, etc) so the motivation of fixing the LLVM toolchain was/is low. Still, I think it would be a nice project to go back and clean up the LLVM plugin (…or remove it). A first obvious clean-up would be to remove the preference page and any concept of global include path and libs. Another step would be to remove the many tools and toolchain that I don’t believe people use or are still maintained (LLVM-GCC, LLVM + JIT, etc). Then there is general code clean-up (there shouldn’t be obvious comment for every line).

Cheers,
Marc-André

On Jan 17, 2020, at 12:20 PM, Jonah Graham <[hidden email]> wrote:

Hi Joost,

Thank you for your analysis up until this point.

In my new role as CDT project lead there are a number of areas that I am simply not familiar with (yet!) and this is one of them. 

I can try to provide some background, and I will be very happy to review any patches you provide!

To answer some of your questions:

The LLVM plug-ins were added in Bug 338553 and haven't changed much since then in the particular area of the code you have highlighted. 

From a quick look at this it seems there is a confusion in this code between settings passed to the LLVM compiler and those passed to the CDT indexer and which tool is "in charge". In simple cases that difference probably doesn't matter, but it clearly quickly goes wrong (the case in Bug 527757 is particularly obviously bad).


> - Is there any (high level) design documentation available for the LLVM managedbuilder stuff? Or for GCC?

Have a look at https://wiki.eclipse.org/CDT/designs/MBS it contains design documentation for when managed build was first added and has a number of links that may be useful. 

> - Why is there a CDT LLVM preference page? What is/was it's purpose?

I don't know. There may be answers in the Petri's thesis, but the link to it is dead (see comment).

> There is no such page for GCC?

Because it probably shouldn't exist for LLVM either?

> - I cannot find UI for LLVM/Clang project properties. Can someone
enlighten me?
> - The same for the GCC project properties?

It is in the Project Properties -> C/C++ Build -> Settings. (see attached file to confirm we are talking about the same thing)

Thank you for taking the time to look at this. There is certainly a lot of little bits needed to get a C++ Managed Build project to "just work" and I look forward to supporting you where I can.

Thanks
Jonah




~~~
Jonah Graham
Kichwa Coders
www.kichwacoders.com


On Fri, 17 Jan 2020 at 09:11, Joost Kraaijeveld <[hidden email]> wrote:
Hi,

My immediate personal problem is solved by placing a comment on line 62
of LlvmResourceListener.java to prevent
LlvmPreferenceStore.addStdLibUnix() being called.


I suspect that the same should be done on line 59 but I have no Windows
with MinGW to test it with.

--
Groeten,

Joost Kraaijeveld
Hommelseweg 123
6821 LD Arnhem
06-51855277

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
<Screenshot_2020-01-17_12-16-40.png>_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev


_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
Reply | Threaded
Open this post in threaded view
|

Re: LLVM/Clang toolchain broken

Jonah Graham
In reply to this post by Marc-Andre Laperle-3
Thank you for that background information. Your explanation also explains why I had to install llvm package on Ubuntu even though I already had a working clang++. The current code looks for llvm-ar to decide if the LLVM toolchains are available rather than clang or clang++. I can also see things like the linker command in the build definition is llvm-ld which I expect is now out of date.

I would vote to remove the existing LLVM plug-in and for creating a new clang one that does it properly. It may be we can just reuse all the build definitions or extend the existing gcc ones? As for process it may be that simply rebranding the plug-in&feature to clang/clang++ is best. I leave whoever takes up this challenge to decide which is best.

Finally I would like to thank Petri for the original LLVM contribution. While it is now under review, it added value to CDT when it was first added. 

Thank you,
Jonah



~~~
Jonah Graham
Kichwa Coders
www.kichwacoders.com


On Fri, 17 Jan 2020 at 22:51, Marc-Andre Laperle <[hidden email]> wrote:
Hi,

From what I remember, the intention of the preference page was to be able to add the system headers which a prebuilt clang did not know about at all and you had to specify manually. This preference page would be global to all projects and configurations. But this is not the way normally managed build projects are configured. In fact, it did this global setting by adding the include paths to every projects and every configuration, before every build and regardless of the toolchain! Also, post-build it would always do a full workspace refresh which I had to comment out because people were complaining quickly that Eclipse was unusable while this plugin was installed.

In the midst of the enthusiasm of having LLVM support, I don’t think this plugin was fully tested or thoroughly reviewed because there were/are glaring issues. I personally ended up using the GCC toolchain and just replacing the command with clang and clang++ which worked fine (especially once many OS/distros had a prebuilt clang fully configured with system paths, etc) so the motivation of fixing the LLVM toolchain was/is low. Still, I think it would be a nice project to go back and clean up the LLVM plugin (…or remove it). A first obvious clean-up would be to remove the preference page and any concept of global include path and libs. Another step would be to remove the many tools and toolchain that I don’t believe people use or are still maintained (LLVM-GCC, LLVM + JIT, etc). Then there is general code clean-up (there shouldn’t be obvious comment for every line).

Cheers,
Marc-André

On Jan 17, 2020, at 12:20 PM, Jonah Graham <[hidden email]> wrote:

Hi Joost,

Thank you for your analysis up until this point.

In my new role as CDT project lead there are a number of areas that I am simply not familiar with (yet!) and this is one of them. 

I can try to provide some background, and I will be very happy to review any patches you provide!

To answer some of your questions:

The LLVM plug-ins were added in Bug 338553 and haven't changed much since then in the particular area of the code you have highlighted. 

From a quick look at this it seems there is a confusion in this code between settings passed to the LLVM compiler and those passed to the CDT indexer and which tool is "in charge". In simple cases that difference probably doesn't matter, but it clearly quickly goes wrong (the case in Bug 527757 is particularly obviously bad).


> - Is there any (high level) design documentation available for the LLVM managedbuilder stuff? Or for GCC?

Have a look at https://wiki.eclipse.org/CDT/designs/MBS it contains design documentation for when managed build was first added and has a number of links that may be useful. 

> - Why is there a CDT LLVM preference page? What is/was it's purpose?

I don't know. There may be answers in the Petri's thesis, but the link to it is dead (see comment).

> There is no such page for GCC?

Because it probably shouldn't exist for LLVM either?

> - I cannot find UI for LLVM/Clang project properties. Can someone
enlighten me?
> - The same for the GCC project properties?

It is in the Project Properties -> C/C++ Build -> Settings. (see attached file to confirm we are talking about the same thing)

Thank you for taking the time to look at this. There is certainly a lot of little bits needed to get a C++ Managed Build project to "just work" and I look forward to supporting you where I can.

Thanks
Jonah




~~~
Jonah Graham
Kichwa Coders
www.kichwacoders.com


On Fri, 17 Jan 2020 at 09:11, Joost Kraaijeveld <[hidden email]> wrote:
Hi,

My immediate personal problem is solved by placing a comment on line 62
of LlvmResourceListener.java to prevent
LlvmPreferenceStore.addStdLibUnix() being called.


I suspect that the same should be done on line 59 but I have no Windows
with MinGW to test it with.

--
Groeten,

Joost Kraaijeveld
Hommelseweg 123
6821 LD Arnhem
06-51855277

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
<Screenshot_2020-01-17_12-16-40.png>_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
Reply | Threaded
Open this post in threaded view
|

Re: LLVM/Clang toolchain broken

Joost Kraaijeveld
Hi,

At the moment I have:

- created a local bug535565 branch from CDT-9_10_0,
- removed the LLVM-page from Eclipse by removing the LLVM-preference
page in the org.eclipse.cdt.managedbuilder.llvm.ui plugin.xml,
- commented out the code that caused the bugs 535565, 527757 and
probably 500186.

As far as I have tested, it now works for me (Linux, Debian Testing,
i.e. Debian/Bullseye, Clang only). I have attached a patch created by
choosing "Team->Synchronize workspace", select all differences, "create
patch", and then saved it as file.

Some questions...

- Is this enough for getting a minimal real working Clang-support for
the next release of the CDT? If so, what should be my next steps?
If nt: what do you expect?
- Should I remove all the code that is related to the removed LLVM-
preference page?
- What do you hope that I should do more as a minimal effort form
improvement?
- What should be my next steps?

For the future:
 
- I do not quite understand what Jan means. I do think that rethinking
and documenting some parts of the design would be nice though.
- I do not think that *removing* a separate Clang (LLVM???) plug-in is
the way to go. Although Clang is very compatible with GCC, it also has
specific differences.  Copy&paste re-use and manually maintaining
(structural) similarity comes to mind. E.g. GCC and Clang share a lot

of stuff but seem to be surprising unrelated in terms of the CDT-plugin
- I prefer to leave out the non-Clang(++) stuff because I do not know
anything about the LLVM-stuff. I think that the CDT should compile
C/C++ programs.

FWIW: I really *hate* and don't (maybe want to) understand all the
configuration/declarative XML-stuff. I prefer readable real code. Does
anyone knows a readable tutorial for creating Eclipse plug-ins that
explains the whole extension point stuff?

--
Groeten,

Joost Kraaijeveld
Hommelseweg 123
6821 LD Arnhem
06-51855277

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev

org.eclipse.cdt.patch (15K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: LLVM/Clang toolchain broken

Jonah Graham
Hi Joost,

This is probably enough for the next release, but I will have to review. CDT (and most of Eclipse) have strong rules for maintaining API which makes deleting old code non-trivial sometimes. It may be the best way forward is a new plug-in called ...clang that is just a copy and past of existing llvm with the fixes. I will need to review the patch to see what is best. So the next step would be to submit the contribution via Gerrit. See https://wiki.eclipse.org/CDT/git#Using_Gerrit_for_CDT for step-by-step instructions and come back to me if anything isn't clear enough. Unfortunately we cannot accept patches via any other method.

I will try to answer the rest of your questions later today, but I can't right now. 

Jonah
~~~
Jonah Graham
Kichwa Coders
www.kichwacoders.com


On Sat, 18 Jan 2020 at 12:14, Joost Kraaijeveld <[hidden email]> wrote:
Hi,

At the moment I have:

- created a local bug535565 branch from CDT-9_10_0,
- removed the LLVM-page from Eclipse by removing the LLVM-preference
page in the org.eclipse.cdt.managedbuilder.llvm.ui plugin.xml,
- commented out the code that caused the bugs 535565, 527757 and
probably 500186.

As far as I have tested, it now works for me (Linux, Debian Testing,
i.e. Debian/Bullseye, Clang only). I have attached a patch created by
choosing "Team->Synchronize workspace", select all differences, "create
patch", and then saved it as file.

Some questions...

- Is this enough for getting a minimal real working Clang-support for
the next release of the CDT? If so, what should be my next steps?
If nt: what do you expect?
- Should I remove all the code that is related to the removed LLVM-
preference page?
- What do you hope that I should do more as a minimal effort form
improvement?
- What should be my next steps?

For the future:

- I do not quite understand what Jan means. I do think that rethinking
and documenting some parts of the design would be nice though.
- I do not think that *removing* a separate Clang (LLVM???) plug-in is
the way to go. Although Clang is very compatible with GCC, it also has
specific differences.  Copy&paste re-use and manually maintaining
(structural) similarity comes to mind. E.g. GCC and Clang share a lot

of stuff but seem to be surprising unrelated in terms of the CDT-plugin
- I prefer to leave out the non-Clang(++) stuff because I do not know
anything about the LLVM-stuff. I think that the CDT should compile
C/C++ programs.

FWIW: I really *hate* and don't (maybe want to) understand all the
configuration/declarative XML-stuff. I prefer readable real code. Does
anyone knows a readable tutorial for creating Eclipse plug-ins that
explains the whole extension point stuff?

--
Groeten,

Joost Kraaijeveld
Hommelseweg 123
6821 LD Arnhem
06-51855277
_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
Reply | Threaded
Open this post in threaded view
|

Re: LLVM/Clang toolchain broken

Jonah Graham
In reply to this post by Joost Kraaijeveld
> - I do not quite understand what Jan means. I do think that rethinking and documenting some parts of the design would be nice though.

I believe Jan is a more in depth design question that exceeds the basic managed model design. I will reply to his email separately soon.

> - I do not think that *removing* a separate Clang (LLVM???) plug-in is the way to go. [...]

I agree. We should have separate Clang plug-in with its own wizard entries and settings pages.

As a clang/llvm user, can we call it the clang (what case) plug-in. The existing GCC feature is called "C/C++ GNU Toolchain Build Support" with the wizard/template entries being variants of Linux GCC, Cygwin GCC. etc. Would "C/C++ clang Toolchain Build Support" with the wizard/template entries being variants of Linux clang, Cygwin clang. etc. be a correct mapping? 

> - I prefer to leave out the non-Clang(++) stuff 

Are you recommending having no LLVM stuff other than clang support in CDT? If so, I agree.

 > FWIW: I really *hate* and don't (maybe want to) understand all the [...]

Fair enough - the plugin.xml for extension points in general is a very good system. The toolchain definitions with their options, while powerful, is verbose and fiddly.

> [...] readable tutorial for creating Eclipse plug-ins that explains the whole extension point stuff?


Thanks,
Jonah

~~~
Jonah Graham
Kichwa Coders
www.kichwacoders.com


On Sat, 18 Jan 2020 at 12:14, Joost Kraaijeveld <[hidden email]> wrote:
Hi,

At the moment I have:

- created a local bug535565 branch from CDT-9_10_0,
- removed the LLVM-page from Eclipse by removing the LLVM-preference
page in the org.eclipse.cdt.managedbuilder.llvm.ui plugin.xml,
- commented out the code that caused the bugs 535565, 527757 and
probably 500186.

As far as I have tested, it now works for me (Linux, Debian Testing,
i.e. Debian/Bullseye, Clang only). I have attached a patch created by
choosing "Team->Synchronize workspace", select all differences, "create
patch", and then saved it as file.

Some questions...

- Is this enough for getting a minimal real working Clang-support for
the next release of the CDT? If so, what should be my next steps?
If nt: what do you expect?
- Should I remove all the code that is related to the removed LLVM-
preference page?
- What do you hope that I should do more as a minimal effort form
improvement?
- What should be my next steps?

For the future:

- I do not quite understand what Jan means. I do think that rethinking
and documenting some parts of the design would be nice though.
- I do not think that *removing* a separate Clang (LLVM???) plug-in is
the way to go. Although Clang is very compatible with GCC, it also has
specific differences.  Copy&paste re-use and manually maintaining
(structural) similarity comes to mind. E.g. GCC and Clang share a lot

of stuff but seem to be surprising unrelated in terms of the CDT-plugin
- I prefer to leave out the non-Clang(++) stuff because I do not know
anything about the LLVM-stuff. I think that the CDT should compile
C/C++ programs.

FWIW: I really *hate* and don't (maybe want to) understand all the
configuration/declarative XML-stuff. I prefer readable real code. Does
anyone knows a readable tutorial for creating Eclipse plug-ins that
explains the whole extension point stuff?

--
Groeten,

Joost Kraaijeveld
Hommelseweg 123
6821 LD Arnhem
06-51855277
_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
Reply | Threaded
Open this post in threaded view
|

Re: LLVM/Clang toolchain broken

Joost Kraaijeveld
In reply to this post by Jonah Graham
On Sat, 2020-01-18 at 14:21 -0500, Jonah Graham wrote:
> I will need to review
> the patch to see what is best. So the next step would be to submit
> the contribution via Gerrit. See
> https://wiki.eclipse.org/CDT/git#Using_Gerrit_for_CDT for step-by-
> step instructions and come back to me if anything isn't clear enough.
Following the procedure I am at a point that I have to resolve merge
conflicts with master. I thinks it's because I rebased with master but
I did not pay enough attention to what I really did as I was slavishly
following the instructions.

Should I really rebase to master if I forked from CDT_9_10_ (..but it
should have the latest code (rebased to master)...)?
If so, is it correct that I cannot merge master into my branch but that
I have to merge my branch into master, which is the only thing Eclipse
seems to allow?

Sorry, I really feel helpless with this procedure... And that after >
30 years of software development.

--
Groeten,

Joost Kraaijeveld
Hommelseweg 123
6821 LD Arnhem
06-51855277

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
Reply | Threaded
Open this post in threaded view
|

Re: LLVM/Clang toolchain broken

Jonah Graham
Hi Joost, 

Gerrit has a bit of a learning curve. Generally speaking there is no merges in gerrit workflow - instead you cherry-pick changes between the branch.

The basic steps here are:
1) Checkout master branch
2) Cherry-pick your change from your working branch
3) Submit to gerrit. 

I generally use git command line, so I am not as familiar with the eGit (Eclipse Git Integration) tools. But at the command line I would do this:

git fetch origin
git checkout origin/master -b bug535565_try2
git cherry-pick bug535565
git push origin HEAD:refs/for/master

Later when you are doing updates based on review feedback, you will amend and repush your commit:

# make edits
git add changed files
git commit --amend --signoff
git push origin HEAD:refs/for/master

HTH,
Jonah

~~~
Jonah Graham
Kichwa Coders
www.kichwacoders.com


On Sun, 19 Jan 2020 at 12:37, Joost Kraaijeveld <[hidden email]> wrote:
On Sat, 2020-01-18 at 14:21 -0500, Jonah Graham wrote:
> I will need to review
> the patch to see what is best. So the next step would be to submit
> the contribution via Gerrit. See
> https://wiki.eclipse.org/CDT/git#Using_Gerrit_for_CDT for step-by-
> step instructions and come back to me if anything isn't clear enough.
Following the procedure I am at a point that I have to resolve merge
conflicts with master. I thinks it's because I rebased with master but
I did not pay enough attention to what I really did as I was slavishly
following the instructions.

Should I really rebase to master if I forked from CDT_9_10_ (..but it
should have the latest code (rebased to master)...)?
If so, is it correct that I cannot merge master into my branch but that
I have to merge my branch into master, which is the only thing Eclipse
seems to allow?

Sorry, I really feel helpless with this procedure... And that after >
30 years of software development.

--
Groeten,

Joost Kraaijeveld
Hommelseweg 123
6821 LD Arnhem
06-51855277

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
Reply | Threaded
Open this post in threaded view
|

Future of Managed Build

Jonah Graham
In reply to this post by Jan Baeyens
(I changed the subject because Jan brings up lots of important stuff that extends far beyond LLVM/clang integration.)

Jan,

Your observations are all very accurate. In particular the lack of sensible/easy instantiation of the build model. This is an area that many people have spoken about recently, now the challenge is how to move forward. I think in the Eclipse Ecosystem such a build model should be represented in EMF models. Alexander Federov and William Riley have both been making some discussions in that regard. 

The good makefiles is an interesting point - do other tools such and Meson and CMake make "good makefiles"? At one point Eclipse CDT folk implemented a new builder that (essentially) reimplements Make called the Internal Builder.

What we need is a way forward - and I don't think anyone is excited about evolving as opposed to replacing the existing Managed Make / GNU Makefile Generator code. The current code is very brittle, used by lots and lots of extenders so any change, even small requires a very large amount of work to decide if it is safe. However perhaps I am wrong about this and I should be convinced otherwise.

To answer your questions:
1)Inside the broad CDT world. Does anyone know of other build models like described in the Managed Build System Extensibility Document in the eclipse help->CDT-plugin developer guide->programmers guide->CDT DOM?

(for clarity for others, the MBS Extensibilty Document is not part of the CDT DOM - the link in online help is https://help.eclipse.org/topic/org.eclipse.cdt.doc.isv/guide/mbs/extensibilityGuide/Managed_Build_Extensibility.html?cp=14_0_1)

I don't know of other build models - in the broad CDT world AFAIK almost everyone either uses MBS or has custom Makefiles or other tools to manage build settings.

2)Did you ever build a build model based on "Managed Build System Extensibility Document" or others? Can you share your experiences/thoughts?

Yes, the build model is what you are using in Sleober :-)

3)Suppose I build a buildmodel instantiator (and a makefile generator for testing) does anyone think someone will use it?

This is an area that I cannot answer. I think having a good instantiation of the build model that is easier to use than what we have today would be nice. But as I outlined above, I think that the development effort may best be spent on a step change to new technologies.

4)Could this buildmodel instantiator become part of CDT?

Yes it absolutely can if it is a testable unit of code. If it is useful to you it will probably be useful to others.

Thanks Jan for re-starting this discussion here. There have been lots of discussions on these issues in the past, on the mailing list and some written down in the Wiki*. What is needed is someone to lead this effort and a real commitment of development resources to make it happen.

* I have been reviewing all the Wiki pages. Some pages that I have looked at on this topic are:

These are all on this same broad discussion and nowhere is a full resolution.

Jonah



~~~
Jonah Graham
Kichwa Coders
www.kichwacoders.com


On Sat, 18 Jan 2020 at 08:51, Jan Baeyens <[hidden email]> wrote:

Hi

I'm happy to there is lots of attention for the managed build capability of CDT right now :-).
I have been playing with the gnu makefile generator for some years now and I have been impressed with the capabilities provided by the extension points (called build model below).
I realize I only understand a very small portion of the problem domain. For instance I don't know cMake, LLVM, LLVM-GCC, LLVM + JIT... but I also feel I can contribute.
What I do know is this; Whatever tool you use to build, you will always have to combine source code files with commands (executable and parameters).
The way to combine (and the commands) can be very easy or very complex.

When I look at the gnu makefile generator I think the build model is a very good basis to describe "how to combine source code files with commands".
If I look at the implementation of the gnu makefile generator I see 3 problematic design decisions:

1)No instantiation of the build model.
2)Trying to make "good makefiles"
3)Working with strings

I would like to sell the lack of "instantiation of the build model"  as a current weakness in CDT and the addition of a "instantiation of the build model" as the solution.
And please see my questions below.

1)No instantiation of the build model.

Suppose I have the super simple build model
convert each *.c to *.o with the command g++ -o {output} {input}
convert all *.o to {project}.exe with the command link  {inputs} {output}

With a project myExample, containing the files input.c and test.c; 

A instantiation of the build model would look like

Input files (input.c);output files (input.o);command (g++ -o input.o input.c)
Input files (test.c);output files (test.o);command (g++ -o test.o test.c)
Input files (input.o; test.o);output files (myExample.exe);command (link test.o input.o myExample.exe)

My thinking is that with this instantiation of the build model one can easily

1) make a makefile
2) run the commands
3) make a cmake file
....

Even though gnuMakefileGenrator does all this work; it doesn't instantiate the build model in any way, the code goes from build model directly to makefile in pieces.
If we could agree on a data model for the instantiation of the build model we could avoid lots of duplication and have a great addition to CDT.

2)Trying to make "good makefiles"

The second problem I see with gnuMakefileGenerator is that it tries to generate good makefiles. Hints to this are:
1)The possibility to not expand environment variables in the makefile
2)When using nameproviders the makefile becomes completely different

Though it is noble to make readable/maintainable makefiles, it also requires lots of extra resources (like testing) and a high level of knowledge about makefiles for future maintainers.
Using only one "type" of makefiles one can easily reduce:
1)amount of code
2)amount of tests
3)amount of training

3)Working with strings

The code is all based around strings and doesn't use classes for files and so on. I don't think this is due to lack of skill but a simple consequence of the design decisions I already mentioned.
This however adds to the maintenance nightmare.

My questions

My fingers want to tackle this problem but to avoid needless work I feel it is important for me to have a better understanding of how this fits into the cdt community. As such I have following questions.
1)Inside the broad CDT world. Does anyone know of other build models like described in the Managed Build System Extensibility Document in the eclipse help->CDT-plugin developer guide->programmers guide->CDT DOM?
2)Did you ever build a build model based on "Managed Build System Extensibility Document" or others? Can you share your experiences/thoughts?
3)Suppose I build a buildmodel instantiator (and a makefile generator for testing) does anyone think someone will use it?
4)Could this buildmodel instantiator become part of CDT?

Best regards

Jantje

Op 18/01/2020 om 4:50 schreef Marc-Andre Laperle:
Hi,

From what I remember, the intention of the preference page was to be able to add the system headers which a prebuilt clang did not know about at all and you had to specify manually. This preference page would be global to all projects and configurations. But this is not the way normally managed build projects are configured. In fact, it did this global setting by adding the include paths to every projects and every configuration, before every build and regardless of the toolchain! Also, post-build it would always do a full workspace refresh which I had to comment out because people were complaining quickly that Eclipse was unusable while this plugin was installed.

In the midst of the enthusiasm of having LLVM support, I don’t think this plugin was fully tested or thoroughly reviewed because there were/are glaring issues. I personally ended up using the GCC toolchain and just replacing the command with clang and clang++ which worked fine (especially once many OS/distros had a prebuilt clang fully configured with system paths, etc) so the motivation of fixing the LLVM toolchain was/is low. Still, I think it would be a nice project to go back and clean up the LLVM plugin (…or remove it). A first obvious clean-up would be to remove the preference page and any concept of global include path and libs. Another step would be to remove the many tools and toolchain that I don’t believe people use or are still maintained (LLVM-GCC, LLVM + JIT, etc). Then there is general code clean-up (there shouldn’t be obvious comment for every line).

Cheers,
Marc-André

On Jan 17, 2020, at 12:20 PM, Jonah Graham <[hidden email]> wrote:

Hi Joost,

Thank you for your analysis up until this point.

In my new role as CDT project lead there are a number of areas that I am simply not familiar with (yet!) and this is one of them. 

I can try to provide some background, and I will be very happy to review any patches you provide!

To answer some of your questions:

The LLVM plug-ins were added in Bug 338553 and haven't changed much since then in the particular area of the code you have highlighted. 

From a quick look at this it seems there is a confusion in this code between settings passed to the LLVM compiler and those passed to the CDT indexer and which tool is "in charge". In simple cases that difference probably doesn't matter, but it clearly quickly goes wrong (the case in Bug 527757 is particularly obviously bad).


> - Is there any (high level) design documentation available for the LLVM managedbuilder stuff? Or for GCC?

Have a look at https://wiki.eclipse.org/CDT/designs/MBS it contains design documentation for when managed build was first added and has a number of links that may be useful. 

> - Why is there a CDT LLVM preference page? What is/was it's purpose?

I don't know. There may be answers in the Petri's thesis, but the link to it is dead (see comment).

> There is no such page for GCC?

Because it probably shouldn't exist for LLVM either?

> - I cannot find UI for LLVM/Clang project properties. Can someone
enlighten me?
> - The same for the GCC project properties?

It is in the Project Properties -> C/C++ Build -> Settings. (see attached file to confirm we are talking about the same thing)

Thank you for taking the time to look at this. There is certainly a lot of little bits needed to get a C++ Managed Build project to "just work" and I look forward to supporting you where I can.

Thanks
Jonah




~~~
Jonah Graham
Kichwa Coders
www.kichwacoders.com


On Fri, 17 Jan 2020 at 09:11, Joost Kraaijeveld <[hidden email]> wrote:
Hi,

My immediate personal problem is solved by placing a comment on line 62
of LlvmResourceListener.java to prevent
LlvmPreferenceStore.addStdLibUnix() being called.


I suspect that the same should be done on line 59 but I have no Windows
with MinGW to test it with.

--
Groeten,

Joost Kraaijeveld
Hommelseweg 123
6821 LD Arnhem
06-51855277

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
<Screenshot_2020-01-17_12-16-40.png>_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev


_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
Reply | Threaded
Open this post in threaded view
|

Re: Future of Managed Build

Jan Baeyens

Hi

I was hoping for more responses. I guess I'm the only Jantje on the maillist ;-)

See my comments below

Jantje and not Jan please :-D

Op 20/01/2020 om 19:52 schreef Jonah Graham:
(I changed the subject because Jan brings up lots of important stuff that extends far beyond LLVM/clang integration.)

Jan,

Your observations are all very accurate. In particular the lack of sensible/easy instantiation of the build model. This is an area that many people have spoken about recently, now the challenge is how to move forward. I think in the Eclipse Ecosystem such a build model should be represented in EMF models. Alexander Federov and William Riley have both been making some discussions in that regard.
I agree on the fact that the toolchain description should be a model. That is how I visualize it myself
The components on the model I currently see are files, filters,grouping, tools, states.
However: The current extension points solution has the feature of options in the project properties. I can't recall how it works exactly but this is something I do not want to loose.
One could consider describing the options at the tool level as an extension point referenced by the model. However this would mean that a extension point needs to be instantiated for each tool, which is probably not wanted. Also, in Arduino world the build/compile/archive/link options are locked and decided on by the build options provided by the framework which are completely different from the gnu compile options. So I don't think having a tool extension point will work for that.

I have no experience with emf modelling so I don't know it is capable enough. I also have no experience getting this in java classes. Help will be needed if we go this way. Learning EMF is now on my todo :-)
Is there some doc from the discussion from Alexander Federov and William Riley?

The good makefiles is an interesting point - do other tools such and Meson and CMake make "good makefiles"?
Does CMake make makefiles? I have no clue. I'm wondering myself. That is why I request input. Personally I think it is a waste of resource because I think people who can write makefiles will want to write them themselves. People who can't write makefiles fluently just want to build. I think they don't even care whether it is make, cmake, internal builder.
And as to deploy (as was mentioned in one of the links you shared) ... I don't think we are up to that yet.
At one point Eclipse CDT folk implemented a new builder that (essentially) reimplements Make called the Internal Builder.
From where CDT is/was, that seems pretty logic to me. I wanted to use that with Sloeber so I only had to provide the tools of the toolchain and didn't need to provide a make.exe. However I never got it to work. One of the biggest pro's is that you would use the same make on all oses.
What we need is a way forward - and I don't think anyone is excited about evolving as opposed to replacing the existing Managed Make / GNU Makefile Generator code. The current code is very brittle, used by lots and lots of extenders so any change, even small requires a very large amount of work to decide if it is safe. However perhaps I am wrong about this and I should be convinced otherwise.
As I already said: I like the extension point stuff. But (and this is new) there is quite a lot of makefile contamination in there.
And I do not like you can't use outputs from tool A as input for tool B.
And I only got sloeber to work due to the makefile contamination :-s
After having looked at the problem domain and the current implementations seriously -with evolving on my mind- I think evolving is not feasible
So I'm currently looking for the good and bad things of the managed make and I hope we can leave the bad things behind (but not what we learned) and implement the good things in something new.
To answer your questions:
1)Inside the broad CDT world. Does anyone know of other build models like described in the Managed Build System Extensibility Document in the eclipse help->CDT-plugin developer guide->programmers guide->CDT DOM?

(for clarity for others, the MBS Extensibilty Document is not part of the CDT DOM - the link in online help is https://help.eclipse.org/topic/org.eclipse.cdt.doc.isv/guide/mbs/extensibilityGuide/Managed_Build_Extensibility.html?cp=14_0_1)

I don't know of other build models - in the broad CDT world AFAIK almost everyone either uses MBS or has custom Makefiles or other tools to manage build settings.
I thought autotools was something like that and wasn't there a autotools component in CDT?

2)Did you ever build a build model based on "Managed Build System Extensibility Document" or others? Can you share your experiences/thoughts?

Yes, the build model is what you are using in Sleober :-)
Sorry for the confusion. I'm looking for input on what a managed build system should look like for people willing to "design toolchains" or input for external build tools. This is also why I asked Martin in the Cmake support thread. Basically I'm looking for user requirements.

3)Suppose I build a buildmodel instantiator (and a makefile generator for testing) does anyone think someone will use it?

This is an area that I cannot answer. I think having a good instantiation of the build model that is easier to use than what we have today would be nice. But as I outlined above, I think that the development effort may best be spent on a step change to new technologies.
Thinking about this again I don't think there will be many users. If we build a good internal builder on top of this....there will be even less.
This makes me doubt about EMF being a good way to go. Seems like overkill when taking the little usage I see into account.
Who sees usage opportunities?

4)Could this buildmodel instantiator become part of CDT?

Yes it absolutely can if it is a testable unit of code. If it is useful to you it will probably be useful to others.

Cool.

off-course it must be testable. Sloeber runs several thousands of functional test before releasing. I'm all for testing
My current thinking is that the main module should take a ( targetPath, project files,project,buildModel) and produce a instantiated build model that should be serializable which should be consistent in output. I guess you guys know how to setup a project for testing, I don't.

I have lots of second thoughts about the targetpath, project files and the project. It is just very first thinking and I want to list the important things (I am not talking api right now). From my experiences with gnuMakeFile I want the project in all api calls. I needed to overload and cast the name providers to add the project to be able to read environment variables.
So I know that technically the targetPath and the project files can and should be taken from the project, what I try to express is that even if you have those you'll still need the project to call the name providers and to get build options.

Thanks Jan for re-starting this discussion here. There have been lots of discussions on these issues in the past, on the mailing list and some written down in the Wiki*. What is needed is someone to lead this effort and a real commitment of development resources to make it happen.

If you have resources, feel free to assign them :-) until then I fear you will have to deal with me rofl :-)

I think I'll start with the EMF study and design following toolchains;

Library (debug/release)
executable (debug /release)
Arduino
Please add as crazy as possible ideas

That will be a first go/no go reflection moment

Then I'll try to implement a basic instantiated build model maker and a makefile maker.
This will be a second go/no go reflection moment.

As I'm not a java developer and I'm not a cdt contributor and I'm doing this to get it into CDT I would like someone who knows the eclipse/CDT way to have a look from time to time (candidates please speak up).
I don't yet know how I will handle version control and sharing though (ideas please).


* I have been reviewing all the Wiki pages. Some pages that I have looked at on this topic are:

These are all on this same broad discussion and nowhere is a full resolution.

Jonah



~~~
Jonah Graham
Kichwa Coders
www.kichwacoders.com


On Sat, 18 Jan 2020 at 08:51, Jan Baeyens <[hidden email]> wrote:

Hi

I'm happy to there is lots of attention for the managed build capability of CDT right now :-).
I have been playing with the gnu makefile generator for some years now and I have been impressed with the capabilities provided by the extension points (called build model below).
I realize I only understand a very small portion of the problem domain. For instance I don't know cMake, LLVM, LLVM-GCC, LLVM + JIT... but I also feel I can contribute.
What I do know is this; Whatever tool you use to build, you will always have to combine source code files with commands (executable and parameters).
The way to combine (and the commands) can be very easy or very complex.

When I look at the gnu makefile generator I think the build model is a very good basis to describe "how to combine source code files with commands".
If I look at the implementation of the gnu makefile generator I see 3 problematic design decisions:

1)No instantiation of the build model.
2)Trying to make "good makefiles"
3)Working with strings

I would like to sell the lack of "instantiation of the build model"  as a current weakness in CDT and the addition of a "instantiation of the build model" as the solution.
And please see my questions below.

1)No instantiation of the build model.

Suppose I have the super simple build model
convert each *.c to *.o with the command g++ -o {output} {input}
convert all *.o to {project}.exe with the command link  {inputs} {output}

With a project myExample, containing the files input.c and test.c; 

A instantiation of the build model would look like

Input files (input.c);output files (input.o);command (g++ -o input.o input.c)
Input files (test.c);output files (test.o);command (g++ -o test.o test.c)
Input files (input.o; test.o);output files (myExample.exe);command (link test.o input.o myExample.exe)

My thinking is that with this instantiation of the build model one can easily

1) make a makefile
2) run the commands
3) make a cmake file
....

Even though gnuMakefileGenrator does all this work; it doesn't instantiate the build model in any way, the code goes from build model directly to makefile in pieces.
If we could agree on a data model for the instantiation of the build model we could avoid lots of duplication and have a great addition to CDT.

2)Trying to make "good makefiles"

The second problem I see with gnuMakefileGenerator is that it tries to generate good makefiles. Hints to this are:
1)The possibility to not expand environment variables in the makefile
2)When using nameproviders the makefile becomes completely different

Though it is noble to make readable/maintainable makefiles, it also requires lots of extra resources (like testing) and a high level of knowledge about makefiles for future maintainers.
Using only one "type" of makefiles one can easily reduce:
1)amount of code
2)amount of tests
3)amount of training

3)Working with strings

The code is all based around strings and doesn't use classes for files and so on. I don't think this is due to lack of skill but a simple consequence of the design decisions I already mentioned.
This however adds to the maintenance nightmare.

My questions

My fingers want to tackle this problem but to avoid needless work I feel it is important for me to have a better understanding of how this fits into the cdt community. As such I have following questions.
1)Inside the broad CDT world. Does anyone know of other build models like described in the Managed Build System Extensibility Document in the eclipse help->CDT-plugin developer guide->programmers guide->CDT DOM?
2)Did you ever build a build model based on "Managed Build System Extensibility Document" or others? Can you share your experiences/thoughts?
3)Suppose I build a buildmodel instantiator (and a makefile generator for testing) does anyone think someone will use it?
4)Could this buildmodel instantiator become part of CDT?

Best regards

Jantje

Op 18/01/2020 om 4:50 schreef Marc-Andre Laperle:
Hi,

From what I remember, the intention of the preference page was to be able to add the system headers which a prebuilt clang did not know about at all and you had to specify manually. This preference page would be global to all projects and configurations. But this is not the way normally managed build projects are configured. In fact, it did this global setting by adding the include paths to every projects and every configuration, before every build and regardless of the toolchain! Also, post-build it would always do a full workspace refresh which I had to comment out because people were complaining quickly that Eclipse was unusable while this plugin was installed.

In the midst of the enthusiasm of having LLVM support, I don’t think this plugin was fully tested or thoroughly reviewed because there were/are glaring issues. I personally ended up using the GCC toolchain and just replacing the command with clang and clang++ which worked fine (especially once many OS/distros had a prebuilt clang fully configured with system paths, etc) so the motivation of fixing the LLVM toolchain was/is low. Still, I think it would be a nice project to go back and clean up the LLVM plugin (…or remove it). A first obvious clean-up would be to remove the preference page and any concept of global include path and libs. Another step would be to remove the many tools and toolchain that I don’t believe people use or are still maintained (LLVM-GCC, LLVM + JIT, etc). Then there is general code clean-up (there shouldn’t be obvious comment for every line).

Cheers,
Marc-André

On Jan 17, 2020, at 12:20 PM, Jonah Graham <[hidden email]> wrote:

Hi Joost,

Thank you for your analysis up until this point.

In my new role as CDT project lead there are a number of areas that I am simply not familiar with (yet!) and this is one of them. 

I can try to provide some background, and I will be very happy to review any patches you provide!

To answer some of your questions:

The LLVM plug-ins were added in Bug 338553 and haven't changed much since then in the particular area of the code you have highlighted. 

From a quick look at this it seems there is a confusion in this code between settings passed to the LLVM compiler and those passed to the CDT indexer and which tool is "in charge". In simple cases that difference probably doesn't matter, but it clearly quickly goes wrong (the case in Bug 527757 is particularly obviously bad).


> - Is there any (high level) design documentation available for the LLVM managedbuilder stuff? Or for GCC?

Have a look at https://wiki.eclipse.org/CDT/designs/MBS it contains design documentation for when managed build was first added and has a number of links that may be useful. 

> - Why is there a CDT LLVM preference page? What is/was it's purpose?

I don't know. There may be answers in the Petri's thesis, but the link to it is dead (see comment).

> There is no such page for GCC?

Because it probably shouldn't exist for LLVM either?

> - I cannot find UI for LLVM/Clang project properties. Can someone
enlighten me?
> - The same for the GCC project properties?

It is in the Project Properties -> C/C++ Build -> Settings. (see attached file to confirm we are talking about the same thing)

Thank you for taking the time to look at this. There is certainly a lot of little bits needed to get a C++ Managed Build project to "just work" and I look forward to supporting you where I can.

Thanks
Jonah




~~~
Jonah Graham
Kichwa Coders
www.kichwacoders.com


On Fri, 17 Jan 2020 at 09:11, Joost Kraaijeveld <[hidden email]> wrote:
Hi,

My immediate personal problem is solved by placing a comment on line 62
of LlvmResourceListener.java to prevent
LlvmPreferenceStore.addStdLibUnix() being called.


I suspect that the same should be done on line 59 but I have no Windows
with MinGW to test it with.

--
Groeten,

Joost Kraaijeveld
Hommelseweg 123
6821 LD Arnhem
06-51855277

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
<Screenshot_2020-01-17_12-16-40.png>_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev


_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
Reply | Threaded
Open this post in threaded view
|

Re: Future of Managed Build

Jonah Graham
On Wed, 22 Jan 2020 at 20:15, Jan Baeyens <[hidden email]> wrote:

Hi

I was hoping for more responses. I guess I'm the only Jantje on the maillist ;-)

See my comments below

Jantje and not Jan please :-D


Sorry Jante.
 

Op 20/01/2020 om 19:52 schreef Jonah Graham:
(I changed the subject because Jan brings up lots of important stuff that extends far beyond LLVM/clang integration.)

Jan,

Your observations are all very accurate. In particular the lack of sensible/easy instantiation of the build model. This is an area that many people have spoken about recently, now the challenge is how to move forward. I think in the Eclipse Ecosystem such a build model should be represented in EMF models. Alexander Federov and William Riley have both been making some discussions in that regard.
I agree on the fact that the toolchain description should be a model. That is how I visualize it myself
The components on the model I currently see are files, filters,grouping, tools, states.
However: The current extension points solution has the feature of options in the project properties. I can't recall how it works exactly but this is something I do not want to loose.
One could consider describing the options at the tool level as an extension point referenced by the model. However this would mean that a extension point needs to be instantiated for each tool, which is probably not wanted. Also, in Arduino world the build/compile/archive/link options are locked and decided on by the build options provided by the framework which are completely different from the gnu compile options. So I don't think having a tool extension point will work for that.

I have no experience with emf modelling so I don't know it is capable enough. I also have no experience getting this in java classes. Help will be needed if we go this way. Learning EMF is now on my todo :-)
Is there some doc from the discussion from Alexander Federov and William Riley?

@William - can you share your current state?

@Alexander - do you have some guidance based on your past experiences that you can share?
 

The good makefiles is an interesting point - do other tools such and Meson and CMake make "good makefiles"?
Does CMake make makefiles? I have no clue. I'm wondering myself. That is why I request input. Personally I think it is a waste of resource because I think people who can write makefiles will want to write them themselves. People who can't write makefiles fluently just want to build. I think they don't even care whether it is make, cmake, internal builder.
And as to deploy (as was mentioned in one of the links you shared) ... I don't think we are up to that yet.
At one point Eclipse CDT folk implemented a new builder that (essentially) reimplements Make called the Internal Builder.
From where CDT is/was, that seems pretty logic to me. I wanted to use that with Sloeber so I only had to provide the tools of the toolchain and didn't need to provide a make.exe. However I never got it to work. One of the biggest pro's is that you would use the same make on all oses.
What we need is a way forward - and I don't think anyone is excited about evolving as opposed to replacing the existing Managed Make / GNU Makefile Generator code. The current code is very brittle, used by lots and lots of extenders so any change, even small requires a very large amount of work to decide if it is safe. However perhaps I am wrong about this and I should be convinced otherwise.
As I already said: I like the extension point stuff. But (and this is new) there is quite a lot of makefile contamination in there.
And I do not like you can't use outputs from tool A as input for tool B.

This is one of the challenges - the current code has a general source inputs to one output executable. Anything else is bolted on and was not really in scope of the original implementation IIUC.
 
And I only got sloeber to work due to the makefile contamination :-s
After having looked at the problem domain and the current implementations seriously -with evolving on my mind- I think evolving is not feasible
So I'm currently looking for the good and bad things of the managed make and I hope we can leave the bad things behind (but not what we learned) and implement the good things in something new.


 
To answer your questions:
1)Inside the broad CDT world. Does anyone know of other build models like described in the Managed Build System Extensibility Document in the eclipse help->CDT-plugin developer guide->programmers guide->CDT DOM?

(for clarity for others, the MBS Extensibilty Document is not part of the CDT DOM - the link in online help is https://help.eclipse.org/topic/org.eclipse.cdt.doc.isv/guide/mbs/extensibilityGuide/Managed_Build_Extensibility.html?cp=14_0_1)

I don't know of other build models - in the broad CDT world AFAIK almost everyone either uses MBS or has custom Makefiles or other tools to manage build settings.
I thought autotools was something like that and wasn't there a autotools component in CDT?

There is an autotools component in CDT. It uses MBS too as does the non-Managed Makefile support. Of course both of them don't use Eclipse's GNUMakefileGenerator


2)Did you ever build a build model based on "Managed Build System Extensibility Document" or others? Can you share your experiences/thoughts?

Yes, the build model is what you are using in Sleober :-)
Sorry for the confusion. I'm looking for input on what a managed build system should look like for people willing to "design toolchains" or input for external build tools. This is also why I asked Martin in the Cmake support thread. Basically I'm looking for user requirements.

3)Suppose I build a buildmodel instantiator (and a makefile generator for testing) does anyone think someone will use it?

This is an area that I cannot answer. I think having a good instantiation of the build model that is easier to use than what we have today would be nice. But as I outlined above, I think that the development effort may best be spent on a step change to new technologies.
Thinking about this again I don't think there will be many users. If we build a good internal builder on top of this....there will be even less.
This makes me doubt about EMF being a good way to go. Seems like overkill when taking the little usage I see into account.
Who sees usage opportunities?

4)Could this buildmodel instantiator become part of CDT?

Yes it absolutely can if it is a testable unit of code. If it is useful to you it will probably be useful to others.

Cool.

off-course it must be testable. Sloeber runs several thousands of functional test before releasing. I'm all for testing
My current thinking is that the main module should take a ( targetPath, project files,project,buildModel) and produce a instantiated build model that should be serializable which should be consistent in output. I guess you guys know how to setup a project for testing, I don't.

I have lots of second thoughts about the targetpath, project files and the project. It is just very first thinking and I want to list the important things (I am not talking api right now). From my experiences with gnuMakeFile I want the project in all api calls. I needed to overload and cast the name providers to add the project to be able to read environment variables.
So I know that technically the targetPath and the project files can and should be taken from the project, what I try to express is that even if you have those you'll still need the project to call the name providers and to get build options.

Thanks Jan for re-starting this discussion here. There have been lots of discussions on these issues in the past, on the mailing list and some written down in the Wiki*. What is needed is someone to lead this effort and a real commitment of development resources to make it happen.

If you have resources, feel free to assign them :-) until then I fear you will have to deal with me rofl :-)

I think I'll start with the EMF study and design following toolchains;

Library (debug/release)
executable (debug /release)
Arduino
Please add as crazy as possible ideas

That will be a first go/no go reflection moment

Then I'll try to implement a basic instantiated build model maker and a makefile maker.
This will be a second go/no go reflection moment.

As I'm not a java developer and I'm not a cdt contributor and I'm doing this to get it into CDT I would like someone who knows the eclipse/CDT way to have a look from time to time (candidates please speak up).
I don't yet know how I will handle version control and sharing though (ideas please).


* I have been reviewing all the Wiki pages. Some pages that I have looked at on this topic are:

These are all on this same broad discussion and nowhere is a full resolution.

Jonah



~~~
Jonah Graham
Kichwa Coders
www.kichwacoders.com


On Sat, 18 Jan 2020 at 08:51, Jan Baeyens <[hidden email]> wrote:

Hi

I'm happy to there is lots of attention for the managed build capability of CDT right now :-).
I have been playing with the gnu makefile generator for some years now and I have been impressed with the capabilities provided by the extension points (called build model below).
I realize I only understand a very small portion of the problem domain. For instance I don't know cMake, LLVM, LLVM-GCC, LLVM + JIT... but I also feel I can contribute.
What I do know is this; Whatever tool you use to build, you will always have to combine source code files with commands (executable and parameters).
The way to combine (and the commands) can be very easy or very complex.

When I look at the gnu makefile generator I think the build model is a very good basis to describe "how to combine source code files with commands".
If I look at the implementation of the gnu makefile generator I see 3 problematic design decisions:

1)No instantiation of the build model.
2)Trying to make "good makefiles"
3)Working with strings

I would like to sell the lack of "instantiation of the build model"  as a current weakness in CDT and the addition of a "instantiation of the build model" as the solution.
And please see my questions below.

1)No instantiation of the build model.

Suppose I have the super simple build model
convert each *.c to *.o with the command g++ -o {output} {input}
convert all *.o to {project}.exe with the command link  {inputs} {output}

With a project myExample, containing the files input.c and test.c; 

A instantiation of the build model would look like

Input files (input.c);output files (input.o);command (g++ -o input.o input.c)
Input files (test.c);output files (test.o);command (g++ -o test.o test.c)
Input files (input.o; test.o);output files (myExample.exe);command (link test.o input.o myExample.exe)

My thinking is that with this instantiation of the build model one can easily

1) make a makefile
2) run the commands
3) make a cmake file
....

Even though gnuMakefileGenrator does all this work; it doesn't instantiate the build model in any way, the code goes from build model directly to makefile in pieces.
If we could agree on a data model for the instantiation of the build model we could avoid lots of duplication and have a great addition to CDT.

2)Trying to make "good makefiles"

The second problem I see with gnuMakefileGenerator is that it tries to generate good makefiles. Hints to this are:
1)The possibility to not expand environment variables in the makefile
2)When using nameproviders the makefile becomes completely different

Though it is noble to make readable/maintainable makefiles, it also requires lots of extra resources (like testing) and a high level of knowledge about makefiles for future maintainers.
Using only one "type" of makefiles one can easily reduce:
1)amount of code
2)amount of tests
3)amount of training

3)Working with strings

The code is all based around strings and doesn't use classes for files and so on. I don't think this is due to lack of skill but a simple consequence of the design decisions I already mentioned.
This however adds to the maintenance nightmare.

My questions

My fingers want to tackle this problem but to avoid needless work I feel it is important for me to have a better understanding of how this fits into the cdt community. As such I have following questions.
1)Inside the broad CDT world. Does anyone know of other build models like described in the Managed Build System Extensibility Document in the eclipse help->CDT-plugin developer guide->programmers guide->CDT DOM?
2)Did you ever build a build model based on "Managed Build System Extensibility Document" or others? Can you share your experiences/thoughts?
3)Suppose I build a buildmodel instantiator (and a makefile generator for testing) does anyone think someone will use it?
4)Could this buildmodel instantiator become part of CDT?

Best regards

Jantje

Op 18/01/2020 om 4:50 schreef Marc-Andre Laperle:
Hi,

From what I remember, the intention of the preference page was to be able to add the system headers which a prebuilt clang did not know about at all and you had to specify manually. This preference page would be global to all projects and configurations. But this is not the way normally managed build projects are configured. In fact, it did this global setting by adding the include paths to every projects and every configuration, before every build and regardless of the toolchain! Also, post-build it would always do a full workspace refresh which I had to comment out because people were complaining quickly that Eclipse was unusable while this plugin was installed.

In the midst of the enthusiasm of having LLVM support, I don’t think this plugin was fully tested or thoroughly reviewed because there were/are glaring issues. I personally ended up using the GCC toolchain and just replacing the command with clang and clang++ which worked fine (especially once many OS/distros had a prebuilt clang fully configured with system paths, etc) so the motivation of fixing the LLVM toolchain was/is low. Still, I think it would be a nice project to go back and clean up the LLVM plugin (…or remove it). A first obvious clean-up would be to remove the preference page and any concept of global include path and libs. Another step would be to remove the many tools and toolchain that I don’t believe people use or are still maintained (LLVM-GCC, LLVM + JIT, etc). Then there is general code clean-up (there shouldn’t be obvious comment for every line).

Cheers,
Marc-André

On Jan 17, 2020, at 12:20 PM, Jonah Graham <[hidden email]> wrote:

Hi Joost,

Thank you for your analysis up until this point.

In my new role as CDT project lead there are a number of areas that I am simply not familiar with (yet!) and this is one of them. 

I can try to provide some background, and I will be very happy to review any patches you provide!

To answer some of your questions:

The LLVM plug-ins were added in Bug 338553 and haven't changed much since then in the particular area of the code you have highlighted. 

From a quick look at this it seems there is a confusion in this code between settings passed to the LLVM compiler and those passed to the CDT indexer and which tool is "in charge". In simple cases that difference probably doesn't matter, but it clearly quickly goes wrong (the case in Bug 527757 is particularly obviously bad).


> - Is there any (high level) design documentation available for the LLVM managedbuilder stuff? Or for GCC?

Have a look at https://wiki.eclipse.org/CDT/designs/MBS it contains design documentation for when managed build was first added and has a number of links that may be useful. 

> - Why is there a CDT LLVM preference page? What is/was it's purpose?

I don't know. There may be answers in the Petri's thesis, but the link to it is dead (see comment).

> There is no such page for GCC?

Because it probably shouldn't exist for LLVM either?

> - I cannot find UI for LLVM/Clang project properties. Can someone
enlighten me?
> - The same for the GCC project properties?

It is in the Project Properties -> C/C++ Build -> Settings. (see attached file to confirm we are talking about the same thing)

Thank you for taking the time to look at this. There is certainly a lot of little bits needed to get a C++ Managed Build project to "just work" and I look forward to supporting you where I can.

Thanks
Jonah




~~~
Jonah Graham
Kichwa Coders
www.kichwacoders.com


On Fri, 17 Jan 2020 at 09:11, Joost Kraaijeveld <[hidden email]> wrote:
Hi,

My immediate personal problem is solved by placing a comment on line 62
of LlvmResourceListener.java to prevent
LlvmPreferenceStore.addStdLibUnix() being called.


I suspect that the same should be done on line 59 but I have no Windows
with MinGW to test it with.

--
Groeten,

Joost Kraaijeveld
Hommelseweg 123
6821 LD Arnhem
06-51855277

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
<Screenshot_2020-01-17_12-16-40.png>_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev


_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
Reply | Threaded
Open this post in threaded view
|

Re: Future of Managed Build

Doug Schaefer-4
Great to see so much discussion on this topic and hopefully you can get to a good and sustainable place with CDT build.

On Thu, Jan 23, 2020 at 3:36 PM Jonah Graham <[hidden email]> wrote:
On Wed, 22 Jan 2020 at 20:15, Jan Baeyens <[hidden email]> wrote:

As I already said: I like the extension point stuff. But (and this is new) there is quite a lot of makefile contamination in there. 

And I do not like you can't use outputs from tool A as input for tool B.

This is one of the challenges - the current code has a general source inputs to one output executable. Anything else is bolted on and was not really in scope of the original implementation IIUC

Just to provide a historical perspective, the managed build model was created in the early days of CDT and is very much modelled after the Visual Studio way of doing things, i.e. one build output per project.

Clearly that falls over for anything complicated, with multiple executables especially, and deep toolchains. It's why I became a big fan of CMake which handles this much better and why I created Core Build, to make it easier to use build systems such as CMake without managed build getting in the way.

BTW, managed build did get in the way with Autotools projects. Jeff did a great job getting it to work.

HTH,
Doug

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
Reply | Threaded
Open this post in threaded view
|

Re: Future of Managed Build

Alexander Fedorov
In reply to this post by Jonah Graham
Hello,

We (with my good friends @ArSysOp) are working on Toolchain metamodel prototype that should cover the needs of CDT and will be flexible enough to be applied for sibling areas.
Of course there is a number of EMF usage tricks collected through all that years that we are going to use during this work. We know the model-to-text generation technologies from JET to Acceleo, so we will be able to create whatever we need, not only the default Java classes.

Currently we are polishing things that were presented for Postgres community 1 year ago during the "IDE for stored procedures" talk and discussion. Yes, it may sound curious, but the "toolchain model" idea with all the "target hardware description" and "project structure model" appears to be very similar for a lot of ecosystems.

The plan is to share it on GitHub as a set of Ecore metamodels and then go through several iteration of minor "releases" until we will find good solution. Then we can have it either as a separate Eclipse project or as a part of CDT - this may be decided later.

Regards,
AF

23.01.2020 23:35, Jonah Graham пишет:

I have no experience with emf modelling so I don't know it is capable enough. I also have no experience getting this in java classes. Help will be needed if we go this way. Learning EMF is now on my todo :-)
Is there some doc from the discussion from Alexander Federov and William Riley?
P.S. Well, as we started the introduction session ... actually, my family name is Fedorov, from the Greek "Θεόδωρος " you may know it as the Latin "Theodor", that means "God-given" :)



_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
Reply | Threaded
Open this post in threaded view
|

Re: Future of Managed Build

Jan Baeyens


Op 27/01/2020 om 14:47 schreef Alexander Fedorov:
Hello,

We (with my good friends @ArSysOp) are working on Toolchain metamodel prototype that should cover the needs of CDT and will be flexible enough to be applied for sibling areas.
Of course there is a number of EMF usage tricks collected through all that years that we are going to use during this work. We know the model-to-text generation technologies from JET to Acceleo, so we will be able to create whatever we need, not only the default Java classes.
Great

Currently we are polishing things that were presented for Postgres community 1 year ago during the "IDE for stored procedures" talk and discussion. Yes, it may sound curious, but the "toolchain model" idea with all the "target hardware description" and "project structure model" appears to be very similar for a lot of ecosystems.
This is what I would expect. Basically in toolchain land there are "files" and "tools". There is only a very limited set of combinations you can have with "files" and "tools".
IMHO recursivity at the file level does not exists and tool sequence is pretty obvious at the model level. However: correct me if I'm wrong.

I have been thinking hard about how to model a gcc toolchain and below is a representation of how I see things now

I haven't yet found a good name for what I called router. It is absolutely crucial to understand router to understand the model. All other stuff is pretty obvious
In it's simplest form router will create a list of "input file; output file" (like "input.cpp" "input.cpp.o"). for each item in the list tool can add a command which results in "input file; output file; command" which -in make world- is called a "rule".
A more complicated router may have multiple input files and multiple output files which means that the router produces a list of "list of input files; list of output files".
Also the tool can produce more than one command so the output of a router tool combinations should be a list of "list of input files; list of output files; ordered list of commands".

The router can also filter input files. For instance in the example above the output of the "c to object" tool is send to both the archiver and the linker. It is up to the "collection of routers" to make sure the files are processed at the right place. Given a specific file: the model supports all options (archiver; linker; archiver and linker;none) and assumes the need for coding to support this construction.

The builder can simply go from left to right processing the router/tool columns one by one. Asking the routers the "list of input files; list of output files" checking whether a command needs to be run and if so ask the command to generate the "ordered list of commands" to run. Run these and move on to the next.

Note 1 that the router explicitly names the output file(s).

Note 2 when implementing this model it seems logical to have 2 generic routers ("1 on 1" -fi compile- and "all on 1" -fi archive-) "1 on 1" would need a "name provider" (append .o) and "all on 1" will need a output file (archive.ar). The model above needs "custom build" routers next to the generic ones

Note 3 Looking an the linker input files (which can be object files and archive files) one can argue that the "list of input files' needs to be a "list of lists of input files". One can also argue it is up to the tool to "triage the files". I haven't decided yet what I think is the best.

Note 4: this model assumes there is no order of commands in the column. It behaves as if all the commands from one column can be executed at the same time.

Note 5: the model assumes that all actions in a column must have finished before the next column can be "evaluated/executed". This allows for "non file based commands" like "pre build actions"; "post build actions" to have their own column like demonstrated below

Note 6:I understand that note 4 and 5 are a serious constraint for a more generic modelling tool.


The plan is to share it on GitHub as a set of Ecore metamodels and then go through several iteration of minor "releases" until we will find good solution. Then we can have it either as a separate Eclipse project or as a part of CDT - this may be decided later.
Do you think this will be multi year or multi month project before we get it into CDT?

Regards,
AF

23.01.2020 23:35, Jonah Graham пишет:

I have no experience with emf modelling so I don't know it is capable enough. I also have no experience getting this in java classes. Help will be needed if we go this way. Learning EMF is now on my todo :-)
Is there some doc from the discussion from Alexander Federov and William Riley?
P.S. Well, as we started the introduction session ... actually, my family name is Fedorov, from the Greek "Θεόδωρος " you may know it as the Latin "Theodor", that means "God-given" :)



_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
Reply | Threaded
Open this post in threaded view
|

Re: Future of Managed Build

Alexander Fedorov
Jonah,

After re-reading your vision I have a perception that we have different understanding how the modeling should be applied here. Please don't take it as a criticism, the subject is to big and complex and I may be wrong with my interpretation of your ideas, but let me share my thoughts.

I assume we will use EMF - best in class for today and native to Eclipse ecosystem. And EMF is the most effective for declarative approach. It supports the "methods" for objects but this way will quickly direct you to a swamp of "@generated not".
The good example of the right EMF usage is Sirius: after years of "GMF Gen"-based exercises they found a way to switch from "direct model implementation" to a kind of "interpretation of modeled structures".
May be I'm not so clear in one sentence - this is a big topic more suitable for a book - but it is very important to understand the limits of effective applicability for a particular tool, EMF in our case.

I'll try to describe it on other words: we should not model "how". but "what". I know that you are familiar with Maven it asks us "what" do we have and then applies a set of "how" plugins according to more or less fixed algo. Gradle goes further and provides much more customization, but still with a good focus on declarative approach. But if you remember Ant - it is completely different, it requires to specify each and every step that is really not scalable and hardly maintainable approach.

So, the most important part is to describe carefully and structure effectively the "metadata" that "router" will operate on, not to model a "router" algo. Why? Because "router" who operates on metadata will be most effectively implemented either with OSGi services or by an existing tools, or by a chain of tools, but not with EMF. For example "make" works just perfect and we are not going to re-implement make, right? But we would like to avoid creating make files manually, yes. Well, not a problem, we can generate them if we have sufficient metadata, or generate cmake files, or generate manifests for other build systems. It will be relatively easy if we have good metadata that includes every aspect we may need to care about: host capabilities, target hardware, tool options, project structure, whatever else we need.

Having EMF-based metadata available via OSGi services - we already successfully tested this architecture with Eclipse Passage - "routers" can be developed in parallel according to your pictures. What is important - the same metadata will be available during all the project lyfecycle: create/export/import/refactor/build/run/debug/etc.

Nope, it is not planned to be multi years project before we can _start to_ get it into CDT, it may be done faster. However, it depends on resources, as you can guess this work is not funded ATM.
Also from organizational perspective it is not clear how it can get into CDT as it will have some "volume" with EMF-generated sources and everything:
 - it may be a separate proposal - in this case it will need to go through incubation,
 - or it may be a separate repository (GitHub, please, to get more people involved easily) inside the CDT project to be utilized part-by-part,
 - or you may have other suggestions.

Regards,
AF

30.01.2020 1:56, Jan Baeyens пишет:


Op 27/01/2020 om 14:47 schreef Alexander Fedorov:
Hello,

We (with my good friends @ArSysOp) are working on Toolchain metamodel prototype that should cover the needs of CDT and will be flexible enough to be applied for sibling areas.
Of course there is a number of EMF usage tricks collected through all that years that we are going to use during this work. We know the model-to-text generation technologies from JET to Acceleo, so we will be able to create whatever we need, not only the default Java classes.
Great

Currently we are polishing things that were presented for Postgres community 1 year ago during the "IDE for stored procedures" talk and discussion. Yes, it may sound curious, but the "toolchain model" idea with all the "target hardware description" and "project structure model" appears to be very similar for a lot of ecosystems.
This is what I would expect. Basically in toolchain land there are "files" and "tools". There is only a very limited set of combinations you can have with "files" and "tools".
IMHO recursivity at the file level does not exists and tool sequence is pretty obvious at the model level. However: correct me if I'm wrong.

I have been thinking hard about how to model a gcc toolchain and below is a representation of how I see things now

I haven't yet found a good name for what I called router. It is absolutely crucial to understand router to understand the model. All other stuff is pretty obvious
In it's simplest form router will create a list of "input file; output file" (like "input.cpp" "input.cpp.o"). for each item in the list tool can add a command which results in "input file; output file; command" which -in make world- is called a "rule".
A more complicated router may have multiple input files and multiple output files which means that the router produces a list of "list of input files; list of output files".
Also the tool can produce more than one command so the output of a router tool combinations should be a list of "list of input files; list of output files; ordered list of commands".

The router can also filter input files. For instance in the example above the output of the "c to object" tool is send to both the archiver and the linker. It is up to the "collection of routers" to make sure the files are processed at the right place. Given a specific file: the model supports all options (archiver; linker; archiver and linker;none) and assumes the need for coding to support this construction.

The builder can simply go from left to right processing the router/tool columns one by one. Asking the routers the "list of input files; list of output files" checking whether a command needs to be run and if so ask the command to generate the "ordered list of commands" to run. Run these and move on to the next.

Note 1 that the router explicitly names the output file(s).

Note 2 when implementing this model it seems logical to have 2 generic routers ("1 on 1" -fi compile- and "all on 1" -fi archive-) "1 on 1" would need a "name provider" (append .o) and "all on 1" will need a output file (archive.ar). The model above needs "custom build" routers next to the generic ones

Note 3 Looking an the linker input files (which can be object files and archive files) one can argue that the "list of input files' needs to be a "list of lists of input files". One can also argue it is up to the tool to "triage the files". I haven't decided yet what I think is the best.

Note 4: this model assumes there is no order of commands in the column. It behaves as if all the commands from one column can be executed at the same time.

Note 5: the model assumes that all actions in a column must have finished before the next column can be "evaluated/executed". This allows for "non file based commands" like "pre build actions"; "post build actions" to have their own column like demonstrated below

Note 6:I understand that note 4 and 5 are a serious constraint for a more generic modelling tool.


The plan is to share it on GitHub as a set of Ecore metamodels and then go through several iteration of minor "releases" until we will find good solution. Then we can have it either as a separate Eclipse project or as a part of CDT - this may be decided later.
Do you think this will be multi year or multi month project before we get it into CDT?

Regards,
AF

23.01.2020 23:35, Jonah Graham пишет:

I have no experience with emf modelling so I don't know it is capable enough. I also have no experience getting this in java classes. Help will be needed if we go this way. Learning EMF is now on my todo :-)
Is there some doc from the discussion from Alexander Federov and William Riley?
P.S. Well, as we started the introduction session ... actually, my family name is Fedorov, from the Greek "Θεόδωρος " you may know it as the Latin "Theodor", that means "God-given" :)



_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev


_______________________________________________
cdt-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cdt-dev
123