Performance Of New JDT Indexer

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Performance Of New JDT Indexer

Roland Grunberg
Hello,

I've been playing around with the new JDT indexer and can see it performing a bit better than the older one with some manual tests (through UI). However I'm having difficulty coming up with some basic examples purely through jdt.core API showing it outperform the old indexer. My example is a test that extends
AbstractJavaModelTests for utility methods and is similar to TypeHierarchyTests.
They create a single project and attach jar files to the project classpath.

I test the time it takes for the index to build as well as the actual time to complete a type hierarchy search (against java.lang.Object). I kind of expected the new index would be slower building given that it's roughly 2x as large in most cases, but I'm also consistently seeing the old index outperforming the new one in searches so I must be doing something wrong but I can't see what. I've tested with ~1500 jars, or even 1 jar with ~20000 classes but the end result is the new indexer being anywhere from 2-4x slower.


The tests can be run with :

mvn clean verify -Pbuild-individual-bundles -f org.eclipse.jdt.core.tests.model/pom.xml -DtestClass=org.eclipse.jdt.core.tests.model.IndexerPerformanceTest

Any help/pointers would be really appreciated.


Cheers,
-- 
Roland Grunberg

_______________________________________________
jdt-core-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jdt-core-dev
Reply | Threaded
Open this post in threaded view
|

Re: Performance Of New JDT Indexer

Jesper Steen Møller
Hello,

> On 24 Nov 2017, at 21.19, Roland Grunberg <[hidden email]> wrote:
>
> I've been playing around with the new JDT indexer and can see it performing a bit better than the older one with some manual tests (through UI). However I'm having difficulty coming up with some basic examples purely through jdt.core API showing it outperform the old indexer. My example is a test that extends
> AbstractJavaModelTests for utility methods and is similar to TypeHierarchyTests.
> They create a single project and attach jar files to the project classpath.

Your test case sounds very reasonable, since it resembles (actually is) a real world scenario.
It's a bit disappointing that the new index is slower than the old, and when I tried your test, I also noticed that the indexing doesn't even saturate a single thread:
real 5m26.159s
user 4m55.806s
sys 0m13.217s
(This was for a project of 794 jar files, just running the two newIndex tests). I wonder what the CPU is waiting for. I would have imagined that better use of the CPU can speed up the index-building significantly, such as parallel reading of JAR files.
The I/O was not a bottleneck either, but I must add that I haven't dug into the code to find out why it's so slow.

However, the original design goals didn't actually target on 'faster than the old' for single projects, only but for multiple projects (since it's a per-workspace index, rather than a per-project index). So, in fairness, it guess it should be tested on multiple projects at once. See: https://drive.google.com/open?id=1w3-ufZyISbqH8jxYv689Exjm0haAGufdcSvEAgl2HQ4

The current index has been tuned over many years, and replacing it with something slower seems odd. FWIW, I see a bigger problem in the lack of committer commitment to tune and extend the new index.
(And there's a size issue for multiple workspaces as well, I guess?)

-Jesper

_______________________________________________
jdt-core-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jdt-core-dev
Reply | Threaded
Open this post in threaded view
|

Re: Performance Of New JDT Indexer

Manoj Palat-2
In reply to this post by Roland Grunberg

Hi,
Your best bet is to contact the author Stefan for specific queries on the new indexer.
Rest of the JDT Core committers were tied up with the latest Java release.

Note: if someone is interested to get the new indexer (helpwanted category) up and integrate, it would be good.

Regards,
Manoj

Inactive hide details for Jesper Steen Møller ---11/25/2017 09:24:27 PM---Hello, > On 24 Nov 2017, at 21.19, Roland Grunberg <rJesper Steen Møller ---11/25/2017 09:24:27 PM---Hello, > On 24 Nov 2017, at 21.19, Roland Grunberg <[hidden email]> wrote:

From: Jesper Steen Møller <[hidden email]>
To: "Eclipse JDT Core developers list." <[hidden email]>
Date: 11/25/2017 09:24 PM
Subject: Re: [jdt-core-dev] Performance Of New JDT Indexer
Sent by: [hidden email]





Hello,

> On 24 Nov 2017, at 21.19, Roland Grunberg <[hidden email]> wrote:
>
> I've been playing around with the new JDT indexer and can see it performing a bit better than the older one with some manual tests (through UI). However I'm having difficulty coming up with some basic examples purely through jdt.core API showing it outperform the old indexer. My example is a test that extends
> AbstractJavaModelTests for utility methods and is similar to TypeHierarchyTests.
> They create a single project and attach jar files to the project classpath.

Your test case sounds very reasonable, since it resembles (actually is) a real world scenario.
It's a bit disappointing that the new index is slower than the old, and when I tried your test, I also noticed that the indexing doesn't even saturate a single thread:
real 5m26.159s
user 4m55.806s
sys 0m13.217s
(This was for a project of 794 jar files, just running the two newIndex tests). I wonder what the CPU is waiting for. I would have imagined that better use of the CPU can speed up the index-building significantly, such as parallel reading of JAR files.
The I/O was not a bottleneck either, but I must add that I haven't dug into the code to find out why it's so slow.

However, the original design goals didn't actually target on 'faster than the old' for single projects, only but for multiple projects (since it's a per-workspace index, rather than a per-project index). So, in fairness, it guess it should be tested on multiple projects at once. See:
https://urldefense.proofpoint.com/v2/url?u=https-3A__drive.google.com_open-3Fid-3D1w3-2DufZyISbqH8jxYv689Exjm0haAGufdcSvEAgl2HQ4&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=A51zLdywEjS50U7u2UMWvsDIrUGEJ5IDXskL5MxIEjA&m=P6Yw4Al7mau6mnX7s4th_lEp1CmZQlT3ECQ0_2RR5mM&s=kiXDub2Wk9ilhFR8C6TuS98waFAnAMdExi5MXjXHj4A&e=

The current index has been tuned over many years, and replacing it with something slower seems odd. FWIW, I see a bigger problem in the lack of committer commitment to tune and extend the new index.
(And there's a size issue for multiple workspaces as well, I guess?)

-Jesper

_______________________________________________
jdt-core-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://urldefense.proofpoint.com/v2/url?u=https-3A__dev.eclipse.org_mailman_listinfo_jdt-2Dcore-2Ddev&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=A51zLdywEjS50U7u2UMWvsDIrUGEJ5IDXskL5MxIEjA&m=P6Yw4Al7mau6mnX7s4th_lEp1CmZQlT3ECQ0_2RR5mM&s=C6DA1YRM0stdV5aPqis6SRgFG6PZp3gB_W765oj4wYA&e=





_______________________________________________
jdt-core-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jdt-core-dev
Reply | Threaded
Open this post in threaded view
|

Re: Performance Of New JDT Indexer

Roland Grunberg
In reply to this post by Jesper Steen Møller
On Sat, Nov 25, 2017 at 10:54 AM, Jesper Steen Møller
<[hidden email]> wrote:
> However, the original design goals didn't actually target on 'faster than the old' for single projects, only but for multiple projects (since it's a per-workspace index, rather than a per-project index). So, in fairness, it guess it should be tested on multiple projects at once. See: https://drive.google.com/open?id=1w3-ufZyISbqH8jxYv689Exjm0haAGufdcSvEAgl2HQ4
>
> The current index has been tuned over many years, and replacing it with something slower seems odd. FWIW, I see a bigger problem in the lack of committer commitment to tune and extend the new index.
> (And there's a size issue for multiple workspaces as well, I guess?)

I ended up adding some more parts to the tests to generate an arbitrary
number of projects and split the set of jars among them. I tested for 1,
10, 100 projects with bundles from :
http://download.eclipse.org/releases/oxygen (excluding source jars).

See : https://github.com/rgrunber/eclipse.jdt.core/tree/index_performance

The new indexer certainly improves a lot going from 1, 10, and 100
projects. (total number of classes found is the same, of course).
However with these experiments, the new indexer still doesn't
outperform the old one. It comes pretty close (~1.5x the average
time of the old).

Another thing I observed is that when the old index is freshly built
the first run is always much (~2-3x) slower. I would guess this
difference is the caching referred to by :

https://wiki.eclipse.org/JDT_Core_Index_Programmer_Guide#Rework_the_model_cache_.28Bug_497513.29
https://bugs.eclipse.org/bugs/show_bug.cgi?id=497513

I'll still continue playing around with this from time to time and get a
hold of others that have worked on it. Perhaps looking into actually
disabling caching (for testing) as opposed to just rebuilding the
project/index every time may yield a better comparison. If anyone is
able to point to some tests showing the new index performing better
than the old one, I'm happy to try them out.

Cheers,
Roland Grunberg
_______________________________________________
jdt-core-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jdt-core-dev