Sleepycat Software's Berkeley DB is the most widely used embedded database manager in the world. The latest release of OpenLDAP (v2.2x) uses Berkeley DB (v2.4x) for it's bdb backend.
With the origina OpenLDAP ldbm backend, database tuning was all done inside the slapd.conf file. Limited configuration of the bdb backend can still be done in slapd.conf, but fine tuning now needs to be done in the standard DB_CONFIG file usually found in the directory specified in slapd.conf by the database section's directory directive (e.g. /var/db/openldap-data).
Right now I'm still experimenting with DB_CONFIG, so my configuration is fairly simple. I got the basic framework from the OpenLDAP FAQ on the subject. Of course to make it effective I'm going to have to rebuild my 10,000 entry test database, but hey, who needs to break for lunch anyway? I'm setting the cache initially at 64 Mb, to see if this gets me around a bulk loading problem I had during a recent db creation session (I wound up using slapadd to complete the process). Here's the config I'm going to use:
# Set the database in memory cache size.
#
set_cachesize 0 64000000 0
#
# Set database flags.
# (for database loading/reindexing)
#set_flags DB_TXN_NOSYNC
#
# Set log values.
#
# set_lg_regionmax 1048576
# set_lg_max 10485760
# set_lg_bsize 2097152
# set_lg_dir /usr/local/var/openldap-logs
Late Note: I used this config with my test db as planned, and it did in fact get me past the bulk loading issue I'd been experiencing with OpenLDAP 2.2, working directly with the db config has been a real eye-openner and would make a great chapter in a book someday.
Slurpd, you can't get it at 7-11
The OpenLDAP man page for slurpd defines it as the "Standalone LDAP Update Replication Daemon". And so it is. Longer in production and more mature than its client-side replication cousin, syncrepl, slurpd provides a server-side "push" of changes from a master LDAP directory to designated replica (or slave) directories.
The directory service I use in my lab at home does not see alot of changes, so issues such as the relative robustness of slurpd over syncrepl are not really a concern there. Given the interest being shown in open source directory solutions in many places, however, I've begun to kick the tires on the daemon to re-familiarize myself with its inner workings.
Slurpd is built by default in a stock "configure, make, make install" of OpenLDAP. On FreeBSD it gets installed as part of the openldap-server port. While I've never had this problem with a source build on Linux, the current FreeBSD port annoyingly hardcodes the daemon so that it will only work when the replication log file location (set using the replogfile directive in slapd.conf) is set to /var/db/openldap-slurp/replica. What you wind up with in this configuration are a pair of replication log files (one for slapd, the other for slurpd) and their corresponding lock files. When a change gets made on the master, this gets recorded in the slapd replog (in my config, /var/db/openldap-slurp/replica/slapd.replog), and then passed over to the slurpd replog (on my server, /var/db/openldap-slurp/replica/slurpd.replog). When an update fails, a record of it gets written to a separate *.rej log. The data in both replog files is written out in clear text, LDIF (LDAP Data Interchange) format with the requisite LDAP commands, and so should be secured with the same attention as the slapd database itself. This particular method of formatting may make it possible to use these files as part of a custom synchronization system that could work with other vendor directories, such as Active Directory (something I've read has been done at the University of Michigan).
The slapd.replog file gets purged once slurpd reads it, while the slurpd.replog file maintains a running history of changes made. The latter, which can grow quite large, is therefore a candidate for log rotation by the newsyslog facility in FreeBSD. I've got mine set to top out at 1000 Kb (1 Mb), which is reasonable for my environment. Larger, more active, installations, may need to grow the file bigger.
The directory service I use in my lab at home does not see alot of changes, so issues such as the relative robustness of slurpd over syncrepl are not really a concern there. Given the interest being shown in open source directory solutions in many places, however, I've begun to kick the tires on the daemon to re-familiarize myself with its inner workings.
Slurpd is built by default in a stock "configure, make, make install" of OpenLDAP. On FreeBSD it gets installed as part of the openldap-server port. While I've never had this problem with a source build on Linux, the current FreeBSD port annoyingly hardcodes the daemon so that it will only work when the replication log file location (set using the replogfile directive in slapd.conf) is set to /var/db/openldap-slurp/replica. What you wind up with in this configuration are a pair of replication log files (one for slapd, the other for slurpd) and their corresponding lock files. When a change gets made on the master, this gets recorded in the slapd replog (in my config, /var/db/openldap-slurp/replica/slapd.replog), and then passed over to the slurpd replog (on my server, /var/db/openldap-slurp/replica/slurpd.replog). When an update fails, a record of it gets written to a separate *.rej log. The data in both replog files is written out in clear text, LDIF (LDAP Data Interchange) format with the requisite LDAP commands, and so should be secured with the same attention as the slapd database itself. This particular method of formatting may make it possible to use these files as part of a custom synchronization system that could work with other vendor directories, such as Active Directory (something I've read has been done at the University of Michigan).
The slapd.replog file gets purged once slurpd reads it, while the slurpd.replog file maintains a running history of changes made. The latter, which can grow quite large, is therefore a candidate for log rotation by the newsyslog facility in FreeBSD. I've got mine set to top out at 1000 Kb (1 Mb), which is reasonable for my environment. Larger, more active, installations, may need to grow the file bigger.
Subscribe to:
Posts (Atom)