[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Issue with index in OpenLDAP?
Hi @All,
We meet a performance problem with our OpenLDAP.
We think that we face a problem with the index of the database, and we think that the problem can be resolve by tunning the config (but not sure).
We would like to be sure that our configuration is correct, in order to confirm if we are on a wrong track or not.
[Description]
We have an attribute (cardNumber) which is indexed.
When we request the indexed attribute (cardNumber) with an LDAP Client (Ldapbrowser), we have either fast or very long response time.
For the long response time, the CPU of the server hits 100%.
For example:
Request1: cardnumber=2098001010034 (less than 1sec)
Request2: cardnumber=2090389917486 (nearly 20 sec).
By checking the hit ratio of the attribute, we can see that cache is correctly used (97%).
[Details]
- We are running on a VM with RedHat with 4 process with 24 Go RAM.
- The version of the OpenLDAP is 2.4.16.
- We have 2 500 000 accounts.
[Attachment]
- 201111223_os.txt -> informations about OS and Hardware.
- openldap_version.txt -> informations about the version of OpenLDAP.
- 20111220_stats.txt -> informations about index and perf.
- olcDatabase={1}hdb.ldif.txt -> informations about hdb config.
Do not hesitate if you need some more informations.
Thank you for your help (:
Mathieu
[root@tladiw02 tmp]# uname -a
Linux tladiw02.dktetrix.net 2.6.18-53.el5 #1 SMP Wed Oct 10 16:34:19 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
[root@tladiw02 tmp]# uname -r
2.6.18-53.el5
[root@tladiw02 tmp]# cat /proc/version
Linux version 2.6.18-53.el5 (brewbuilder@hs20-bc1-7.build.redhat.com) (gcc version 4.1.2 20070626 (Red Hat 4.1.2-14)) #1 SMP Wed Oct 10 16:34:19 EDT 2007
[root@tladiw02 tmp]# cat /proc/meminfo
MemTotal: 24686480 kB
MemFree: 133024 kB
Buffers: 103288 kB
Cached: 11080956 kB
SwapCached: 70080 kB
Active: 20305776 kB
Inactive: 3837772 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 24686480 kB
LowFree: 133024 kB
SwapTotal: 16777204 kB
SwapFree: 16259412 kB
Dirty: 8736 kB
Writeback: 0 kB
AnonPages: 12958816 kB
Mapped: 6491600 kB
Slab: 330700 kB
PageTables: 46764 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
CommitLimit: 29120444 kB
Committed_AS: 13813684 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 271320 kB
VmallocChunk: 34359466359 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
Hugepagesize: 2048 kB
[root@tladiw02 libexec]# /opt/openldap/libexec/slapd -V
@(#) $OpenLDAP: slapd 2.4.16 (Apr 29 2009 12:26:56) $
root@tladiw02.dktetrix.net:/home/admin/ALOS2/openldap-2.4.16/servers/slapd
[root@tladiw02 db]# du -c -h *.bdb
14M cardNumber.bdb
618M cn.bdb
478M dn2id.bdb
66M entryCSN.bdb
87M entryUUID.bdb
3.7G id2entry.bdb
1.1G mail.bdb
5.5M objectClass.bdb
8.0K relyingPartyDomain.bdb
8.0K rpCode.bdb
614M sn.bdb
86M uid.bdb
6.6G total
[root@tladiw02 db]# /opt/db4/bin/db_stat -d /opt/openldap/var/oxylane.oxylaneId-data/db/id2entry.bdb
Tue Dec 20 11:27:27 2011 Local time
53162 Btree magic number
9 Btree version number
Little-endian Byte order
Flags
2 Minimum keys per-page
16384 Underlying database page size
4079 Overflow key/data size
3 Number of levels in the tree
2571552 Number of unique keys in the tree
2571552 Number of data items in the tree
545 Number of tree internal pages
3722322 Number of bytes free in tree internal pages (58% ff)
235690 Number of tree leaf pages
570M Number of bytes free in tree leaf pages (85% ff)
0 Number of tree duplicate pages
0 Number of bytes free in tree duplicate pages (0% ff)
0 Number of tree overflow pages
0 Number of bytes free in tree overflow pages (0% ff)
0 Number of empty pages
0 Number of pages on the free list
[root@tladiw02 db]# /opt/db4/bin/db_stat -d /opt/openldap/var/oxylane.oxylaneId-data/db/dn2id.bdb
Tue Dec 20 11:35:13 2011 Local time
53162 Btree magic number
9 Btree version number
Little-endian Byte order
duplicates, sorted duplicates Flags
2 Minimum keys per-page
4096 Underlying database page size
1007 Overflow key/data size
4 Number of levels in the tree
2571580 Number of unique keys in the tree
5143159 Number of data items in the tree
1548 Number of tree internal pages
188066 Number of bytes free in tree internal pages (97% ff)
54575 Number of tree leaf pages
16M Number of bytes free in tree leaf pages (92% ff)
66082 Number of tree duplicate pages
99M Number of bytes free in tree duplicate pages (63% ff)
0 Number of tree overflow pages
0 Number of bytes free in tree overflow pages (0% ff)
0 Number of empty pages
0 Number of pages on the free list
[root@tladiw02 db]# /opt/db4/bin/db_stat -d /opt/openldap/var/oxylane.oxylaneId-data/db/cardNumber.bdb
Tue Dec 20 11:46:12 2011 Local time
53162 Btree magic number
9 Btree version number
Little-endian Byte order
duplicates, sorted duplicates Flags
2 Minimum keys per-page
4096 Underlying database page size
1007 Overflow key/data size
2 Number of levels in the tree
50 Number of unique keys in the tree
382308 Number of data items in the tree
18 Number of tree internal pages
43046 Number of bytes free in tree internal pages (41% ff)
7 Number of tree leaf pages
26954 Number of bytes free in tree leaf pages (5% ff)
1386 Number of tree duplicate pages
289296 Number of bytes free in tree duplicate pages (94% ff)
0 Number of tree overflow pages
0 Number of bytes free in tree overflow pages (0% ff)
0 Number of empty pages
1917 Number of pages on the free list
[root@tladiw02 db]# /opt/db4/bin/db_stat -h /opt/openldap/var/oxylane.oxylaneId-data/ -m
6GB Total cache size
2 Number of caches
2 Maximum number of caches
3GB Pool individual cache size
0 Maximum memory-mapped file size
0 Maximum open file descriptors
0 Maximum sequential buffer writes
0 Sleep after writing maximum sequential buffers
0 Requested pages mapped into the process' address space
51M Requested pages found in the cache (99%)
463317 Requested pages not found in the cache
468 Pages created in the cache
463317 Pages read into the cache
120753 Pages written from the cache to the backing file
0 Clean pages forced from the cache
0 Dirty pages forced from the cache
0 Dirty pages written by trickle-sync thread
463769 Current total page count
463555 Current clean page count
214 Current dirty page count
786418 Number of hash buckets used for page location
51M Total number of times hash chains searched for a page (51541058)
8 The longest hash chain searched for a page
64M Total number of hash chain entries checked for page (64617512)
19 The number of hash bucket locks that required waiting (0%)
2 The maximum number of times any hash bucket lock was waited for (0%)
31 The number of region locks that required waiting (0%)
0 The number of buffers frozen
0 The number of buffers thawed
0 The number of frozen buffers freed
463825 The number of page allocations
0 The number of hash buckets examined during allocations
0 The maximum number of hash buckets examined for an allocation
0 The number of pages examined during allocations
0 The max number of pages examined for an allocation
0 Threads waited on page I/O
Pool File: uid.bdb
4096 Page size
0 Requested pages mapped into the process' address space
7378 Requested pages found in the cache (82%)
1588 Requested pages not found in the cache
5 Pages created in the cache
1588 Pages read into the cache
1495 Pages written from the cache to the backing file
Pool File: sn.bdb
4096 Page size
0 Requested pages mapped into the process' address space
153964 Requested pages found in the cache (85%)
26478 Requested pages not found in the cache
64 Pages created in the cache
26478 Pages read into the cache
28066 Pages written from the cache to the backing file
Pool File: cn.bdb
4096 Page size
0 Requested pages mapped into the process' address space
300413 Requested pages found in the cache (91%)
28872 Requested pages not found in the cache
84 Pages created in the cache
28872 Pages read into the cache
33622 Pages written from the cache to the backing file
Pool File: objectClass.bdb
4096 Page size
0 Requested pages mapped into the process' address space
133787 Requested pages found in the cache (99%)
10 Requested pages not found in the cache
0 Pages created in the cache
10 Pages read into the cache
384 Pages written from the cache to the backing file
Pool File: cardNumber.bdb
4096 Page size
0 Requested pages mapped into the process' address space
18935 Requested pages found in the cache (97%)
485 Requested pages not found in the cache
0 Pages created in the cache
485 Pages read into the cache
1100 Pages written from the cache to the backing file
Pool File: entryCSN.bdb
4096 Page size
0 Requested pages mapped into the process' address space
28429 Requested pages found in the cache (97%)
857 Requested pages not found in the cache
13 Pages created in the cache
857 Pages read into the cache
944 Pages written from the cache to the backing file
Pool File: dn2id.bdb
4096 Page size
0 Requested pages mapped into the process' address space
29M Requested pages found in the cache (99%)
120910 Requested pages not found in the cache
57 Pages created in the cache
120910 Pages read into the cache
463 Pages written from the cache to the backing file
Pool File: mail.bdb
4096 Page size
0 Requested pages mapped into the process' address space
571776 Requested pages found in the cache (92%)
45323 Requested pages not found in the cache
126 Pages created in the cache
45323 Pages read into the cache
51784 Pages written from the cache to the backing file
Pool File: id2entry.bdb
16384 Page size
0 Requested pages mapped into the process' address space
21M Requested pages found in the cache (98%)
236137 Requested pages not found in the cache
108 Pages created in the cache
236137 Pages read into the cache
1378 Pages written from the cache to the backing file
Pool File: entryUUID.bdb
4096 Page size
0 Requested pages mapped into the process' address space
19326 Requested pages found in the cache (87%)
2657 Requested pages not found in the cache
11 Pages created in the cache
2657 Pages read into the cache
1517 Pages written from the cache to the backing file
[root@tladiw02 db]# vi /opt/openldap/var/oxylane.oxylaneId-data/DB_CONFIG
set_cachesize 11 0 2
set_data_dir /opt/openldap/var/oxylane.oxylaneId-data/db
set_lg_regionmax 262144
set_lg_bsize 2097152
set_lg_dir /opt/openldap/var/oxylane.oxylaneId-data/db_log
set_lk_max_locks 3000
set_tas_spins 1
dn: olcDatabase={1}hdb
objectClass: olcDatabaseConfig
objectClass: olcHdbConfig
olcDatabase: {1}hdb
olcSuffix: o=oxylane
olcAccess: {0}to attrs=userPassword by self write by anonymous auth by grou
p/groupOfNames/member.exact="cn=ldapread,ou=group,ou=security,o=oxylane" read
by group/groupOfNames/member.exact="cn=ldapadmin,ou=group,ou=security,o=oxy
lane" read by group/groupOfNames/member.exact="cn=ldapwrite,ou=group,ou=secu
rity,o=oxylane" write by group/groupOfNames/member.exact="cn=administrator,o
u=group,ou=security,o=oxylane" write by * none
olcAccess: {1}to dn.base="ou=people,o=oxylane" by dn.children="ou=people,o=ox
ylane" read by group/groupOfNames/member.exact="cn=ldapread,ou=group,ou=secu
rity,o=oxylane" read by group/groupOfNames/member.exact="cn=ldapadmin,ou=gro
up,ou=security,o=oxylane" read by group/groupOfNames/member.exact="cn=ldapwr
ite,ou=group,ou=security,o=oxylane" write by group/groupOfNames/member.exact
="cn=administrator,ou=group,ou=security,o=oxylane" write by * none
olcAccess: {2}to dn.children="ou=people,o=oxylane" by self write by group/gr
oupOfNames/member.exact="cn=ldapread,ou=group,ou=security,o=oxylane" read by
group/groupOfNames/member.exact="cn=ldapadmin,ou=group,ou=security,o=oxylane
" read by group/groupOfNames/member.exact="cn=ldapwrite,ou=group,ou=security
,o=oxylane" write by group/groupOfNames/member.exact="cn=administrator,ou=gr
oup,ou=security,o=oxylane" write by * none
olcAccess: {3}to dn.base="ou=security,o=oxylane" by group/groupOfNames/member
.exact="cn=ldapadmin,ou=group,ou=security,o=oxylane" read by group/groupOfNa
mes/member.exact="cn=administrator,ou=group,ou=security,o=oxylane" write by
* none
olcAccess: {4}to * by group/groupOfNames/member.exact="cn=ldapread,ou=group,o
u=security,o=oxylane" read by group/groupOfNames/member.exact="cn=ldapadmin,
ou=group,ou=security,o=oxylane" read by group/groupOfNames/member.exact="cn=
ldapwrite,ou=group,ou=security,o=oxylane" write by group/groupOfNames/member
.exact="cn=administrator,ou=group,ou=security,o=oxylane" write by * none
olcAddContentAcl: FALSE
olcLastMod: TRUE
olcLimits: {0}dn.base="cn=replicator,ou=account,ou=security,o=oxylane" size=un
limited time=unlimited
olcLimits: {1}dn.base="cn=backup,ou=account,ou=security,o=oxylane" size=unlimi
ted time=unlimited
olcMaxDerefDepth: 15
olcReadOnly: FALSE
olcRootDN: cn=admin,cn=config
olcSizeLimit: 200
olcTimeLimit: 60
olcDbDirectory: /opt/openldap/var/oxylane.oxylaneId-data
olcDbConfig: {0}set_cachesize 11 0 3
olcDbConfig: {1}set_data_dir /opt/openldap/var/oxylane.oxylaneId-data/db
olcDbConfig: {2}set_lg_regionmax 262144
olcDbConfig: {3}set_lg_bsize 2097152
olcDbConfig: {4}set_lg_dir /opt/openldap/var/oxylane.oxylaneId-data/db_log
olcDbConfig: {5}set_lk_max_locks 3000
olcDbConfig: {6}set_tas_spins 1
olcDbDirtyRead: FALSE
olcDbLinearIndex: FALSE
olcDbMode: 0600
olcDbSearchStack: 16
olcDbShmKey: 0
structuralObjectClass: olcHdbConfig
creatorsName: cn=config
olcSyncrepl: {0}rid=001 provider=ldap://tladiw01.dktetrix.net:389 bindmethod=s
imple timeout=0 network-timeout=0 binddn="cn=replicator,ou=account,ou=securit
y,o=oxylane" credentials="decathlon" starttls=no filter="(objectclass=*)" sea
rchbase="o=oxylane" scope=sub schemachecking=on type=refreshAndPersist retry=
"60 +"
olcSyncrepl: {1}rid=002 provider=ldap://tladiw02.dktetrix.net:389 bindmethod=s
imple timeout=0 network-timeout=0 binddn="cn=replicator,ou=account,ou=securit
y,o=oxylane" credentials="decathlon" starttls=no filter="(objectclass=*)" sea
rchbase="o=oxylane" scope=sub schemachecking=on type=refreshAndPersist retry=
"60 +"
olcMirrorMode: TRUE
olcMonitoring: FALSE
olcDbCacheSize: 5000000
olcDbCheckpoint: 512 30
olcDbNoSync: TRUE
olcDbIDLcacheSize: 15000000
olcDbCacheFree: 100000
olcDbDNcacheSize: 5000000
entryUUID: 43eb8cd8-fb1f-4aa6-a133-2ae2054dbc2d
createTimestamp: 20090713142010Z
olcDbIndex: objectClass eq
olcDbIndex: entryUUID eq
olcDbIndex: entryCSN eq
olcDbIndex: cn eq,sub
olcDbIndex: uid eq
olcDbIndex: memberUid eq
olcDbIndex: sn eq,sub
olcDbIndex: mail eq,sub
olcDbIndex: rpCode eq
olcDbIndex: relyingPartyDomain eq
olcDbIndex: cardNumber eq
entryCSN: 20090714154909.438455Z#000000#000#000000
modifiersName: cn=admin,cn=config
modifyTimestamp: 20090714154909Z