Pre-query flush seems not to flush all deletions in the correct order

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Pre-query flush seems not to flush all deletions in the correct order

Mauro Molinari
Hello,
I have the following problem that puzzles me.

I have an entity, let's call it First, which references another entity, let's call it Second.

The relationship is this:

@Entity
public class First {
  // ...
  @OneToOne(
            cascade = { CascadeType.DETACH, CascadeType.PERSIST,
                    CascadeType.REFRESH },
            fetch = FetchType.LAZY,
            optional = false,
            orphanRemoval = false)
    private Second second;
}

The relationship is unidirectional, i.e. there's no @OneToOne relationship mapped in Second towards First.

First also has a @ManyToOne relationship towards a FirstCollection entity. I don't think it should matter at all, but I say it for completeness.

Then I have some code that takes a First instance (let's call it "first") and does some things like:

FirstCollection firstCollection = first.getFirstCollection();
// do some read-only operations on first
// remove first from the FirstCollection related entities
firstCollection.getFirsts().remove(first);
// delete first
entityManager.remove(first);
// delete the related second instance
entityManager.remove(first.getSecond());
// perform a read query on an entity related to first

When the last step is executed, a flush before the execution of the read query is performed; I see something like this in the stack trace:
	at org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush(EntityManagerImpl.java:879)
	at org.eclipse.persistence.internal.jpa.QueryImpl.performPreQueryFlush(QueryImpl.java:967)
	at org.eclipse.persistence.internal.jpa.QueryImpl.executeReadQuery(QueryImpl.java:207)

The problem is that this lfush seems to perform the deletion of second (together with some of its cascaded deleted entities) BEFORE the deletion of First. I mean, the log shows a call to DELETE on the Second table, but no previous delete on the First table. This causes a foreign key constraint failure.

This seems to happen only sometimes. My question is:
  • how is it possible? I'm calling remove on First before calling remove on Second!
  • how to avoid this problem?

Thanks in advance,
Mauro


_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users
Reply | Threaded
Open this post in threaded view
|

Re: Pre-query flush seems not to flush all deletions in the correct order

christopher delahunt
Hello Mauro,

Is there anything else that might reference the first instance?  Could the firstCollection.getFirsts().remove(first) change be being done on an unmanaged instance so that the FirstCollection instance still references the first instance in the context?

This sometimes happens if you have something left in your object model still referencing the first instance with a relationship marked cascade remove or cascade persist, causing the traversal of the object model to 'undo' the remove call.  This may be inconsistent due to lazily fetched relationships, and how the traversal of the model discovers unmanaged instances.

Best Regards,
Chris







On 01/03/2016 12:08 PM, Mauro Molinari wrote:
Hello,
I have the following problem that puzzles me.

I have an entity, let's call it First, which references another entity, let's call it Second.

The relationship is this:

@Entity
public class First {
  // ...
  @OneToOne(
            cascade = { CascadeType.DETACH, CascadeType.PERSIST,
                    CascadeType.REFRESH },
            fetch = FetchType.LAZY,
            optional = false,
            orphanRemoval = false)
    private Second second;
}

The relationship is unidirectional, i.e. there's no @OneToOne relationship mapped in Second towards First.

First also has a @ManyToOne relationship towards a FirstCollection entity. I don't think it should matter at all, but I say it for completeness.

Then I have some code that takes a First instance (let's call it "first") and does some things like:

FirstCollection firstCollection = first.getFirstCollection();
// do some read-only operations on first
// remove first from the FirstCollection related entities
firstCollection.getFirsts().remove(first);
// delete first
entityManager.remove(first);
// delete the related second instance
entityManager.remove(first.getSecond());
// perform a read query on an entity related to first

When the last step is executed, a flush before the execution of the read query is performed; I see something like this in the stack trace:
	at org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush(EntityManagerImpl.java:879)
	at org.eclipse.persistence.internal.jpa.QueryImpl.performPreQueryFlush(QueryImpl.java:967)
	at org.eclipse.persistence.internal.jpa.QueryImpl.executeReadQuery(QueryImpl.java:207)

The problem is that this lfush seems to perform the deletion of second (together with some of its cascaded deleted entities) BEFORE the deletion of First. I mean, the log shows a call to DELETE on the Second table, but no previous delete on the First table. This causes a foreign key constraint failure.

This seems to happen only sometimes. My question is:
  • how is it possible? I'm calling remove on First before calling remove on Second!
  • how to avoid this problem?

Thanks in advance,
Mauro



_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users


_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users
Reply | Threaded
Open this post in threaded view
|

Re: Pre-query flush seems not to flush all deletions in the correct order

Mauro Molinari
Hi Chris,
thank you for your help.

Actually, all the mentioned code is executed within the same transactional method body. No data change is performed before.
firstCollection is obtained directly from first.getFirstCollection().
Between this call and the removal of first there are only other getters calls (which return other related entities that are not modified):
first.getFoo()
firstCollection.getBar()
first.getSomethingElse()
Then, after the deletion of second, and before the query, there's nothing.

The First entity is certainly not referenced by Second, nor by any remove-cascaded entity related to Second (it was an explicit design choice).
In the actual code I'm using Spring Data JPA to perform deletions and queries, but I don't think it should matter.

What I may try to do is to clear the reference between first and second before applying deletions, something like:
Second second = first.getSecond();
first.setSecond(null);
entityManager.remove(first);
entityManager.remove(second);

but it's a shot in the dark. Also, I'm not sure what the "lesson learnt" should be, in order to avoid this problem in the future...

Thanks again,
Mauro

Il 01/03/2016 19:48, christopher delahunt ha scritto:
Hello Mauro,

Is there anything else that might reference the first instance?  Could the firstCollection.getFirsts().remove(first) change be being done on an unmanaged instance so that the FirstCollection instance still references the first instance in the context?

This sometimes happens if you have something left in your object model still referencing the first instance with a relationship marked cascade remove or cascade persist, causing the traversal of the object model to 'undo' the remove call.  This may be inconsistent due to lazily fetched relationships, and how the traversal of the model discovers unmanaged instances.

Best Regards,
Chris







On 01/03/2016 12:08 PM, Mauro Molinari wrote:
Hello,
I have the following problem that puzzles me.

I have an entity, let's call it First, which references another entity, let's call it Second.

The relationship is this:

@Entity
public class First {
  // ...
  @OneToOne(
            cascade = { CascadeType.DETACH, CascadeType.PERSIST,
                    CascadeType.REFRESH },
            fetch = FetchType.LAZY,
            optional = false,
            orphanRemoval = false)
    private Second second;
}

The relationship is unidirectional, i.e. there's no @OneToOne relationship mapped in Second towards First.

First also has a @ManyToOne relationship towards a FirstCollection entity. I don't think it should matter at all, but I say it for completeness.

Then I have some code that takes a First instance (let's call it "first") and does some things like:

FirstCollection firstCollection = first.getFirstCollection();
// do some read-only operations on first
// remove first from the FirstCollection related entities
firstCollection.getFirsts().remove(first);
// delete first
entityManager.remove(first);
// delete the related second instance
entityManager.remove(first.getSecond());
// perform a read query on an entity related to first

When the last step is executed, a flush before the execution of the read query is performed; I see something like this in the stack trace:
	at org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush(EntityManagerImpl.java:879)
	at org.eclipse.persistence.internal.jpa.QueryImpl.performPreQueryFlush(QueryImpl.java:967)
	at org.eclipse.persistence.internal.jpa.QueryImpl.executeReadQuery(QueryImpl.java:207)

The problem is that this lfush seems to perform the deletion of second (together with some of its cascaded deleted entities) BEFORE the deletion of First. I mean, the log shows a call to DELETE on the Second table, but no previous delete on the First table. This causes a foreign key constraint failure.

This seems to happen only sometimes. My question is:
  • how is it possible? I'm calling remove on First before calling remove on Second!
  • how to avoid this problem?

Thanks in advance,
Mauro



_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users



_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users


_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users
Reply | Threaded
Open this post in threaded view
|

Re: Pre-query flush seems not to flush all deletions in the correct order

christopher delahunt
Hello Mauro,

I mistyped when I wrote cascade remove - I meant to write cascade merge or cascade persist.  Any references to 'first' could be causing it to be resurrected, not just ones from the 'second' instance's object graph.  I also noticed you are using cascade refresh and lazy relationships - refresh of the referenced entity over a lazy reference will be delayed until it is accessed, potentially wiping out changes made so far.  Something that causes a refresh of firstCollection for instance might get triggered after the remove call but before the flush, and cause firstCollection to still reference first and cause its resurrection in the persistence unit.  I would detail all the references in your object model to the 'first' class and check that one isn't interfering.

If you haven't already, I'd recommend turning on EclipseLink logging and matching statements up to the calls in your code causing them, and trying to determine what is different when the issue occurs vs when it doesn't.  You might also try delaying or removing the constraint, so that you can see if EclipseLink eventually issues the delete so we can see if it is just an ordering issue or if it never does, and so is a resurrection issue in the application references as mentioned before.

Best of luck,
Chris


You should turn on logging to see the statements executed in relation to your calls, and possibly relax the
On 02/03/2016 4:06 AM, Mauro Molinari wrote:
Hi Chris,
thank you for your help.

Actually, all the mentioned code is executed within the same transactional method body. No data change is performed before.
firstCollection is obtained directly from first.getFirstCollection().
Between this call and the removal of first there are only other getters calls (which return other related entities that are not modified):
first.getFoo()
firstCollection.getBar()
first.getSomethingElse()
Then, after the deletion of second, and before the query, there's nothing.

The First entity is certainly not referenced by Second, nor by any remove-cascaded entity related to Second (it was an explicit design choice).
In the actual code I'm using Spring Data JPA to perform deletions and queries, but I don't think it should matter.

What I may try to do is to clear the reference between first and second before applying deletions, something like:
Second second = first.getSecond();
first.setSecond(null);
entityManager.remove(first);
entityManager.remove(second);

but it's a shot in the dark. Also, I'm not sure what the "lesson learnt" should be, in order to avoid this problem in the future...

Thanks again,
Mauro

Il 01/03/2016 19:48, christopher delahunt ha scritto:
Hello Mauro,

Is there anything else that might reference the first instance?  Could the firstCollection.getFirsts().remove(first) change be being done on an unmanaged instance so that the FirstCollection instance still references the first instance in the context?

This sometimes happens if you have something left in your object model still referencing the first instance with a relationship marked cascade remove or cascade persist, causing the traversal of the object model to 'undo' the remove call.  This may be inconsistent due to lazily fetched relationships, and how the traversal of the model discovers unmanaged instances.

Best Regards,
Chris







On 01/03/2016 12:08 PM, Mauro Molinari wrote:
Hello,
I have the following problem that puzzles me.

I have an entity, let's call it First, which references another entity, let's call it Second.

The relationship is this:

@Entity
public class First {
  // ...
  @OneToOne(
            cascade = { CascadeType.DETACH, CascadeType.PERSIST,
                    CascadeType.REFRESH },
            fetch = FetchType.LAZY,
            optional = false,
            orphanRemoval = false)
    private Second second;
}

The relationship is unidirectional, i.e. there's no @OneToOne relationship mapped in Second towards First.

First also has a @ManyToOne relationship towards a FirstCollection entity. I don't think it should matter at all, but I say it for completeness.

Then I have some code that takes a First instance (let's call it "first") and does some things like:

FirstCollection firstCollection = first.getFirstCollection();
// do some read-only operations on first
// remove first from the FirstCollection related entities
firstCollection.getFirsts().remove(first);
// delete first
entityManager.remove(first);
// delete the related second instance
entityManager.remove(first.getSecond());
// perform a read query on an entity related to first

When the last step is executed, a flush before the execution of the read query is performed; I see something like this in the stack trace:
	at org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush(EntityManagerImpl.java:879)
	at org.eclipse.persistence.internal.jpa.QueryImpl.performPreQueryFlush(QueryImpl.java:967)
	at org.eclipse.persistence.internal.jpa.QueryImpl.executeReadQuery(QueryImpl.java:207)

The problem is that this lfush seems to perform the deletion of second (together with some of its cascaded deleted entities) BEFORE the deletion of First. I mean, the log shows a call to DELETE on the Second table, but no previous delete on the First table. This causes a foreign key constraint failure.

This seems to happen only sometimes. My question is:
  • how is it possible? I'm calling remove on First before calling remove on Second!
  • how to avoid this problem?

Thanks in advance,
Mauro



_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users



_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users



_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users


_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users
Reply | Threaded
Open this post in threaded view
|

Re: Pre-query flush seems not to flush all deletions in the correct order

Mauro Molinari
Hi Chris!
I revisited the whole entity model.
'First' entity is referenced only by:
  • FirstCollection: since I remove the first entity from the first.getFirstCollection().getFirsts() collection before deleting 'first', this relationship shouldn't be causing any issue
  • FirstCollection.thirds which is a @OneToMany relationship towards a Third entity which itself contains a @ManyToMany relationship with First

Substantially, First has:
@ManyToMany(
            cascade = CascadeType.ALL,
            fetch = FetchType.LAZY,
            mappedBy = "firsts")
    private Collection<Third> thirds;

and Third has:
    @ManyToMany(
            cascade = { CascadeType.DETACH, CascadeType.PERSIST,
                    CascadeType.REFRESH },
            fetch = FetchType.EAGER)
    @OrderBy("id ASC")
    private List<First> firsts;

While, concentrating on FirstCollection, this has:

    @OneToMany(
            cascade = CascadeType.ALL,
            fetch = FetchType.LAZY,
            mappedBy = "firstCollection",
            orphanRemoval = true)
    private Collection<Third> thirds;

and Third has:

    @ManyToOne(
            cascade = { CascadeType.DETACH, CascadeType.PERSIST,
                    CascadeType.REFRESH },
            fetch = FetchType.EAGER,
            optional = false)
    private FirstCollection firstCollection;

So: the FirstCollection may be actually resurrecting the First instance if its 'thirds' collection is lazily loaded and that collection contains a Third instance that in turn references 'first'.
However, I'm not reading nor updating any Third instance in my piece of code (neither before the removes, nor after), so I don't even think I should trigger the loading of FirstCollection.thirds, not to say about refreshing.

Anyway, one thing I'm doing wrong is that I should take care of updating the in-memory model in order to remove all Thirds instances from FirstCollection.thirds which reference the soon-to-be-deleted 'first' instance.

However, if we forget this problem for a moment, since the 'thirds' field in First is annotated with "CascadeType.ALL", I would expect the deletion to be cascaded (at least at the database level) to rows corresponding to such Third entities.
In fact, if I look at the SQL generated by EclipseLink when the deletion is successful, I actually see that EclipseLink is doing the right thing:
DELETE FROM `Third_First` WHERE (`first_id` = ?)
DELETE FROM `First` WHERE (`first_id` = ?)
(a lot of DELETE FROM entities related to Second)
DELETE FROM `Second` WHERE ((`id` = ?) AND (`version` = ?))

Instead, when the deletion fails, I do not see the first two DELETEs.
From this, I would even say that the deletion goes WELL when the object graph is certainly complex, and goes BAD when it seems like (but I can't be sure) to be less complex.

However I'm sure it goes well even when the graph is simple. The main problem is that I still could not reproduce the bad behaviour in my test environment: I tried various combinations of data and application paths without success. I only see the problem happening (sometimes) in the production environment... and I could not yet identify the necessary steps.

So, now I'm going to add the necessary code to cleanup firstCollection.getThirds() from Third instances referencing the 'first' instance and see if this will fix the problem. If this goes well, the lesson learnt is "always ALWAYS make sure the in-memory model is FULLY correct before deleting entities"... otherwise hard-to-explain things may happen in certain circumstances.

Thanks again for your help!
Mauro

Il 02/03/2016 15:59, christopher delahunt ha scritto:
Hello Mauro,

I mistyped when I wrote cascade remove - I meant to write cascade merge or cascade persist.  Any references to 'first' could be causing it to be resurrected, not just ones from the 'second' instance's object graph.  I also noticed you are using cascade refresh and lazy relationships - refresh of the referenced entity over a lazy reference will be delayed until it is accessed, potentially wiping out changes made so far.  Something that causes a refresh of firstCollection for instance might get triggered after the remove call but before the flush, and cause firstCollection to still reference first and cause its resurrection in the persistence unit.  I would detail all the references in your object model to the 'first' class and check that one isn't interfering.

If you haven't already, I'd recommend turning on EclipseLink logging and matching statements up to the calls in your code causing them, and trying to determine what is different when the issue occurs vs when it doesn't.  You might also try delaying or removing the constraint, so that you can see if EclipseLink eventually issues the delete so we can see if it is just an ordering issue or if it never does, and so is a resurrection issue in the application references as mentioned before.

Best of luck,
Chris


You should turn on logging to see the statements executed in relation to your calls, and possibly relax the
On 02/03/2016 4:06 AM, Mauro Molinari wrote:
Hi Chris,
thank you for your help.

Actually, all the mentioned code is executed within the same transactional method body. No data change is performed before.
firstCollection is obtained directly from first.getFirstCollection().
Between this call and the removal of first there are only other getters calls (which return other related entities that are not modified):
first.getFoo()
firstCollection.getBar()
first.getSomethingElse()
Then, after the deletion of second, and before the query, there's nothing.

The First entity is certainly not referenced by Second, nor by any remove-cascaded entity related to Second (it was an explicit design choice).
In the actual code I'm using Spring Data JPA to perform deletions and queries, but I don't think it should matter.

What I may try to do is to clear the reference between first and second before applying deletions, something like:
Second second = first.getSecond();
first.setSecond(null);
entityManager.remove(first);
entityManager.remove(second);

but it's a shot in the dark. Also, I'm not sure what the "lesson learnt" should be, in order to avoid this problem in the future...

Thanks again,
Mauro

Il 01/03/2016 19:48, christopher delahunt ha scritto:
Hello Mauro,

Is there anything else that might reference the first instance?  Could the firstCollection.getFirsts().remove(first) change be being done on an unmanaged instance so that the FirstCollection instance still references the first instance in the context?

This sometimes happens if you have something left in your object model still referencing the first instance with a relationship marked cascade remove or cascade persist, causing the traversal of the object model to 'undo' the remove call.  This may be inconsistent due to lazily fetched relationships, and how the traversal of the model discovers unmanaged instances.

Best Regards,
Chris







On 01/03/2016 12:08 PM, Mauro Molinari wrote:
Hello,
I have the following problem that puzzles me.

I have an entity, let's call it First, which references another entity, let's call it Second.

The relationship is this:

@Entity
public class First {
  // ...
  @OneToOne(
            cascade = { CascadeType.DETACH, CascadeType.PERSIST,
                    CascadeType.REFRESH },
            fetch = FetchType.LAZY,
            optional = false,
            orphanRemoval = false)
    private Second second;
}

The relationship is unidirectional, i.e. there's no @OneToOne relationship mapped in Second towards First.

First also has a @ManyToOne relationship towards a FirstCollection entity. I don't think it should matter at all, but I say it for completeness.

Then I have some code that takes a First instance (let's call it "first") and does some things like:

FirstCollection firstCollection = first.getFirstCollection();
// do some read-only operations on first
// remove first from the FirstCollection related entities
firstCollection.getFirsts().remove(first);
// delete first
entityManager.remove(first);
// delete the related second instance
entityManager.remove(first.getSecond());
// perform a read query on an entity related to first

When the last step is executed, a flush before the execution of the read query is performed; I see something like this in the stack trace:
	at org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush(EntityManagerImpl.java:879)
	at org.eclipse.persistence.internal.jpa.QueryImpl.performPreQueryFlush(QueryImpl.java:967)
	at org.eclipse.persistence.internal.jpa.QueryImpl.executeReadQuery(QueryImpl.java:207)

The problem is that this lfush seems to perform the deletion of second (together with some of its cascaded deleted entities) BEFORE the deletion of First. I mean, the log shows a call to DELETE on the Second table, but no previous delete on the First table. This causes a foreign key constraint failure.

This seems to happen only sometimes. My question is:
  • how is it possible? I'm calling remove on First before calling remove on Second!
  • how to avoid this problem?

Thanks in advance,
Mauro



_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users



_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users



_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users



_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users


_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users
Reply | Threaded
Open this post in threaded view
|

Re: Pre-query flush seems not to flush all deletions in the correct order

Mauro Molinari
In reply to this post by christopher delahunt
Hi Chris!
Nope! Even if I'm now fully cleaning the in-memory model by removing any reference to the 'first' instance I'm deleting, today my production environment again threw an exception because of the usual foreign key constraint check failure :-(

Now I'm really out of ideas...

I'll try to dump the whole production data and install it on test to see if the very specific data structure is the key to reproduce...

Mauro

Il 02/03/2016 15:59, christopher delahunt ha scritto:
Hello Mauro,

I mistyped when I wrote cascade remove - I meant to write cascade merge or cascade persist.  Any references to 'first' could be causing it to be resurrected, not just ones from the 'second' instance's object graph.  I also noticed you are using cascade refresh and lazy relationships - refresh of the referenced entity over a lazy reference will be delayed until it is accessed, potentially wiping out changes made so far.  Something that causes a refresh of firstCollection for instance might get triggered after the remove call but before the flush, and cause firstCollection to still reference first and cause its resurrection in the persistence unit.  I would detail all the references in your object model to the 'first' class and check that one isn't interfering.

If you haven't already, I'd recommend turning on EclipseLink logging and matching statements up to the calls in your code causing them, and trying to determine what is different when the issue occurs vs when it doesn't.  You might also try delaying or removing the constraint, so that you can see if EclipseLink eventually issues the delete so we can see if it is just an ordering issue or if it never does, and so is a resurrection issue in the application references as mentioned before.

Best of luck,
Chris


You should turn on logging to see the statements executed in relation to your calls, and possibly relax the
On 02/03/2016 4:06 AM, Mauro Molinari wrote:
Hi Chris,
thank you for your help.

Actually, all the mentioned code is executed within the same transactional method body. No data change is performed before.
firstCollection is obtained directly from first.getFirstCollection().
Between this call and the removal of first there are only other getters calls (which return other related entities that are not modified):
first.getFoo()
firstCollection.getBar()
first.getSomethingElse()
Then, after the deletion of second, and before the query, there's nothing.

The First entity is certainly not referenced by Second, nor by any remove-cascaded entity related to Second (it was an explicit design choice).
In the actual code I'm using Spring Data JPA to perform deletions and queries, but I don't think it should matter.

What I may try to do is to clear the reference between first and second before applying deletions, something like:
Second second = first.getSecond();
first.setSecond(null);
entityManager.remove(first);
entityManager.remove(second);

but it's a shot in the dark. Also, I'm not sure what the "lesson learnt" should be, in order to avoid this problem in the future...

Thanks again,
Mauro

Il 01/03/2016 19:48, christopher delahunt ha scritto:
Hello Mauro,

Is there anything else that might reference the first instance?  Could the firstCollection.getFirsts().remove(first) change be being done on an unmanaged instance so that the FirstCollection instance still references the first instance in the context?

This sometimes happens if you have something left in your object model still referencing the first instance with a relationship marked cascade remove or cascade persist, causing the traversal of the object model to 'undo' the remove call.  This may be inconsistent due to lazily fetched relationships, and how the traversal of the model discovers unmanaged instances.

Best Regards,
Chris







On 01/03/2016 12:08 PM, Mauro Molinari wrote:
Hello,
I have the following problem that puzzles me.

I have an entity, let's call it First, which references another entity, let's call it Second.

The relationship is this:

@Entity
public class First {
  // ...
  @OneToOne(
            cascade = { CascadeType.DETACH, CascadeType.PERSIST,
                    CascadeType.REFRESH },
            fetch = FetchType.LAZY,
            optional = false,
            orphanRemoval = false)
    private Second second;
}

The relationship is unidirectional, i.e. there's no @OneToOne relationship mapped in Second towards First.

First also has a @ManyToOne relationship towards a FirstCollection entity. I don't think it should matter at all, but I say it for completeness.

Then I have some code that takes a First instance (let's call it "first") and does some things like:

FirstCollection firstCollection = first.getFirstCollection();
// do some read-only operations on first
// remove first from the FirstCollection related entities
firstCollection.getFirsts().remove(first);
// delete first
entityManager.remove(first);
// delete the related second instance
entityManager.remove(first.getSecond());
// perform a read query on an entity related to first

When the last step is executed, a flush before the execution of the read query is performed; I see something like this in the stack trace:
	at org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush(EntityManagerImpl.java:879)
	at org.eclipse.persistence.internal.jpa.QueryImpl.performPreQueryFlush(QueryImpl.java:967)
	at org.eclipse.persistence.internal.jpa.QueryImpl.executeReadQuery(QueryImpl.java:207)

The problem is that this lfush seems to perform the deletion of second (together with some of its cascaded deleted entities) BEFORE the deletion of First. I mean, the log shows a call to DELETE on the Second table, but no previous delete on the First table. This causes a foreign key constraint failure.

This seems to happen only sometimes. My question is:
  • how is it possible? I'm calling remove on First before calling remove on Second!
  • how to avoid this problem?

Thanks in advance,
Mauro



_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users



_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users



_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users



_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users


_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users
Reply | Threaded
Open this post in threaded view
|

Re: Pre-query flush seems not to flush all deletions in the correct order

Mauro Molinari
In reply to this post by christopher delahunt
Il 02/03/2016 15:59, christopher delahunt ha scritto:
Hello Mauro,

I mistyped when I wrote cascade remove - I meant to write cascade merge or cascade persist.  Any references to 'first' could be causing it to be resurrected, not just ones from the 'second' instance's object graph.  I also noticed you are using cascade refresh and lazy relationships - refresh of the referenced entity over a lazy reference will be delayed until it is accessed, potentially wiping out changes made so far.  Something that causes a refresh of firstCollection for instance might get triggered after the remove call but before the flush, and cause firstCollection to still reference first and cause its resurrection in the persistence unit.  I would detail all the references in your object model to the 'first' class and check that one isn't interfering.

If you haven't already, I'd recommend turning on EclipseLink logging and matching statements up to the calls in your code causing them, and trying to determine what is different when the issue occurs vs when it doesn't.  You might also try delaying or removing the constraint, so that you can see if EclipseLink eventually issues the delete so we can see if it is just an ordering issue or if it never does, and so is a resurrection issue in the application references as mentioned before.

Best of luck,
Chris

Hi Chris,
I was "lucky enough" to be able today to reproduce this issue locally!! It's very hard to know what is the correct reproduction path, anyway... What I observed by debugging is this:

FirstCollection firstCollection = first.getFirstCollection();
// do some read-only operations on first
// remove first from the FirstCollection related entities
firstCollection.getFirsts().remove(first);
// delete first
entityManager.remove(first);
// added a flush in the hope this fixes the problem
entityManager.flush(); (***)
// delete the related second instance
entityManager.remove(first.getSecond());
// perform a read query on an entity related to first

Stepping over (***) I saw that the only query that was executed was an UPDATE on the FirstCollection table to update the version and lastUpdated fields!!! The DELETE FROM First was not issued at all!
So, this makes me suspect that the problem is what you are saying:
"Something that causes a refresh of firstCollection for instance might get triggered after the remove call but before the flush, and cause firstCollection to still reference first and cause its resurrection in the persistence unit"

So, I removed the cascade=REFRESH from the relationships in FirstCollection and Third that referenced the First entity.
However, I'm not sure whether it will work now or not because I can't reproduce any more.
In fact, I was able to correctly perform the whole deletion once by using this code:

FirstCollection firstCollection = first.getFirstCollection();
// do some read-only operations on first
// remove first from the FirstCollection related entities
firstCollection.getFirsts().remove(first);
// first flush
entityManager.flush();
// delete first

entityManager.remove(first);
// second flush
entityManager.flush();
// delete the related second instance
entityManager.remove(first.getSecond());
// perform a read query on an entity related to first

that is, by performing a first flush before the remove (this caused just the UPDATE on the FirstCollection table!) and a second one after the remove operation (this one actually performed the DELETE FROM First statement!). But having two flushes like this sounded too dirty for me, so I decided to try the removal of the cascade=REFRESH thing.

I will monitor the situation for some days and let you know. Anyway, if the cascade=REFRESH is the actual problem, this is a really bad beast... especially because I'm really doing nothing in between the remove and the flush. Also, I would expect firstCollections.getFirsts() to be already loaded into memory because it has been accessed several times before and the shared cache is enabled, so I don't even expect it to be lazy initialised when I'm calling that code.  So, there's quite a lot of (black) magic here... :-(
And I can't explain exactly why two flushes before and after the remove would fix the problem.

Mauro

_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users
Reply | Threaded
Open this post in threaded view
|

Re: Pre-query flush seems not to flush all deletions in the correct order

Mauro Molinari
In reply to this post by christopher delahunt
Il 02/03/2016 15:59, christopher delahunt ha scritto:
I mistyped when I wrote cascade remove - I meant to write cascade merge or cascade persist.  Any references to 'first' could be causing it to be resurrected, not just ones from the 'second' instance's object graph.  I also noticed you are using cascade refresh and lazy relationships - refresh of the referenced entity over a lazy reference will be delayed until it is accessed, potentially wiping out changes made so far.  Something that causes a refresh of firstCollection for instance might get triggered after the remove call but before the flush, and cause firstCollection to still reference first and cause its resurrection in the persistence unit.  I would detail all the references in your object model to the 'first' class and check that one isn't interfering.

Hi Chris,
just an update. Removing the cascade=REFRESH metadata on the entities referring to First did not solve the problem, actually.
As a last attempt I tried to exchange the order of these two statements, from:
firstCollection.getFirsts().remove(first);
entityManager.remove(first);

to:
entityManager.remove(first);
firstCollection.getFirsts().remove(first);

suspecting that  the removal from the list (which should be mapped to an UPDATE statement on the first instance, since First owns the relationship) might shadow and override the DELETE statement in some way. But this did not solve the problem either.

So, I resigned myself to perform the double flush, immediately before and immediately after the first instance removal:

entityManager.flush();
entityManager.remove(first);
entityManager.flush();

I don't like this at all, but I think I'm really out of ideas now. I really suspect it's an EclipseLink bug, but I really don't know how I could extract a test case for this. As I said, it does not happen always, but often, and I could reproduce it locally just once.

Mauro

_______________________________________________
eclipselink-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/eclipselink-users