Re: (ITS#8300) LMDB mdb_cursor_del data loss

[hyc@symas.com]

This is a multi-part message in MIME format.
--------------090509040809000501070508
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Howard Chu wrote:
> malyn@strangeGizmo.com wrote:
>> Full_Name: Michael Alyn Miller
>> Version: Git head
>> OS: Windows 8.1 x64, NixOS 14.12 x64
>> URL: ftp://ftp.openldap.org/incoming/michaelalynmiller-151104.c
>> Submission from: (NULL) (96.251.78.237)
>>
>>
>> Another issue similar to ITS#8264 and ITS#8299, but this time using
>> mdb_cursor_del inside of the cursor causes corruption.  mdb_del causes
>> corruption as well, but this is the first time that I have seen mdb_cursor_del
>> do the wrong thing.
>>
>> In this case, mdb_cursor_del incorrectly removes one extra key (a key
>> prefixed with 0c000000000000003a...) in addition to correctly removing all of
>> the keys that begin with 0d000000000000003a.
>>
>> mdb_del does not remove that extra key, but instead fails to remove two
>> 0d000000000000003a keys that should have been removed.
>>
>> I am not sure if these are two different bugs or if they are caused by the same
>> underlying issue.
>
> Similar reason, two different mistakes. Fixed now in git.
>>
>> mdb_cursor_del performs correctly on subsequent runs if you do *not* remove the
>> database between tests, whereas mdb_del continues to fail, but stabilizes on a
>> different set of results after the first run.

For future reference, here's a patch to your test program that also checks for 
cursors pointed adjacent to the item being deleted. The previous fix was 
missing a step, fixed now. Note you must build LMDB with -DMDB_DEBUG for the 
mdb_cursor_chk() function to be defined.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

--------------090509040809000501070508
Content-Type: text/plain; charset=UTF-8;
 name="dif.txt"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="dif.txt"

--- /tmp/michaelalynmiller-151104.c	2015-11-06 10:38:44.772490173 +0000
+++ michaelalynmiller-151104.c	2015-11-06 10:18:13.000000000 +0000
@@ -3696,10 +3696,10 @@
 
 void remove_collection12(MDB_env *env, MDB_dbi dbi, int delete_with_cursor)
 {
-	int rc;
+	int i, rc;
 	MDB_txn *txn;
 	MDB_val key, data;
-	MDB_cursor *cursor;
+	MDB_cursor *cursor, *cup, *cdn;
 
 	E(mdb_txn_begin(env, NULL, 0, &txn));
 
@@ -3710,6 +3710,8 @@
 	E(mdb_del(txn, dbi, &key, &data));
 
 	E(mdb_cursor_open(txn, dbi, &cursor));
+	E(mdb_cursor_open(txn, dbi, &cup));
+	E(mdb_cursor_open(txn, dbi, &cdn));
 
 	key.mv_size = 8+1;
 	key.mv_data = "\x0D\x00\x00\x00\x00\x00\x00\x00:";
@@ -3717,13 +3719,19 @@
 	data.mv_data = NULL;
 	E(mdb_cursor_get(cursor, &key, &data, MDB_SET_RANGE));
 
-	for (;;) {
+	for (i=0;;i++) {
 		data.mv_size = 8+1;
 		data.mv_data = "\x0D\x00\x00\x00\x00\x00\x00\x00;";
 		if (mdb_cmp(txn, dbi, &data, &key) < 0) {
 			break;
 		}
 
+		E(mdb_cursor_get(cup, &key, &data, MDB_SET));
+		E(mdb_cursor_get(cdn, &key, &data, MDB_SET));
+		mdb_cursor_get(cup, NULL, NULL, MDB_NEXT);
+		mdb_cursor_get(cdn, NULL, NULL, MDB_PREV);
+
+		printf("Deleting %d\n", i);
 		if (delete_with_cursor) {
 			E(mdb_cursor_del(cursor, 0));
 		} else {
@@ -3731,6 +3739,9 @@
 			data.mv_data = NULL;
 			E(mdb_del(txn, dbi, &key, &data));
 		}
+		mdb_cursor_chk(cursor);
+		mdb_cursor_chk(cup);
+		mdb_cursor_chk(cdn);
 
 		rc = mdb_cursor_get(cursor, &key, &data, MDB_NEXT);
 		if (rc == MDB_NOTFOUND) {

--------------090509040809000501070508--