eldapo

Eldapo Has Moved!

2007-09-12T22:21:00.001-04:00

Alhough this site will stay up for awhile, I won't be making any new posts here.

For the latest from "Eldapo" you'll need to go to

eldapo.lembobrothers.com

which is a new subdomain of mine set up on a shared host.

I'm still going through all the articles I recently exported from here to that new site, laboriously fixing the formatting of the substantial chunks of source code that are the real treasure this site has to offer, but after completing the conversion of another site this weekend I've decided that there's no time like the present to pick up and continue the work started here over there.

Hope you'll visit soon, and that the content made available there is useful to you.

from blogger to wordpress

2007-09-09T18:45:00.000-04:00

Today was the day I decided to import this blog from Blogger into a copy of WordPress I have running on my new personal site, under the URL eldapo.lembobrothers.com.

While the import itself went smoothly (thanks to a slight hack to the blogger.php file under wp-admin/import (an instance of www2.blogger.com needed to be changed to www.blogger.com), after looking over the formatting of my articles I was disappointed to see that almost every section of text wrapped in the <code> and <pre> tags had been seriously screwed up. Some of the problem was due to the new application trying to be "helpful", completing tags before the code block was, well, complete. It also seemed to have a great deal of trouble getting linebreaks right. Given that eldapo.blogspot.com goes back around 3 years, with a total of 69 mostly code heavy pieces, this means that I’m going to have to go in a reformat almost every article by hand.

Now the thing is, that rendering big chunks of source code in Blogger was never a picnic. In fact, it's been downright aggravating. Unfortunately, it doesn’t look like WordPress is going to be any easier. In fact, I’ve already experienced some of WP’s own eccentricities in this regard and I’m not pleased. There's something to be said for handcoding your own HTLM pages (sorry, I meant to say "XHTML").

Oh joy. Just what I needed. So much for easier, faster, better.

All applications suck. Blogging applications suck most.

(and my current theme in WP does full justification, and a really crummy job of it to boot -- kind of like WordStar for CP/M or IBM DisplayWrite on an old XT clone with a Hercules graphics card ... yuck!)

openldap with an old-style root dn (o=example,c=US)

2007-09-04T09:46:00.000-04:00

I needed to do this during testing of an OpenLDAP proxy, and since examples were so difficult to find, thought I'd post it here.

To make this work your slapd.conf must have an include line for the core.schema (in addition to any others you might need for user entries, like cosine, inetorgperson and nis schemas).

In slapd.conf, under your database line (e.g. database dbd), you need to put a suffix directive like:


database     dbd
suffix       "o=Example,c=US"

Then when you create your db load file for slapadd to create the database, make sure you include your root dn at the top:


dn: o=Example,c=US
objectclass: top
objectclass: organization
o: Example

Notice that there is no 'c' attribute or value in the body of the entry. Under the "standard" organization schema (contained in core.schema) countryName or 'c' is not an allowed attribute. The attribute may be used in a distinguished name, however. DN values are not subject to the schema in the same ways as other attributes within an entry.

openldap as a metadirectory

2007-09-04T08:44:00.001-04:00

Now that I've got some experience using the OpenLDAP proxy backend, I've moved on to sttuggling with getting the meta backend ("slapd-meta" or "back-meta", the former how it's looked up in the man pages, the latter how it's referred to in the doc and on the mail lists).

The doc is really sparse on this. Actually, it doesn't really address it at all. The man pages are, well, man pages. So this is going to be a real "adventure". There's not much on the lists either, for the most part pleas for concrete examples are met with "what are you trying to do" or "why would you want to do that" from various quarters.

Very interesting.

Anyway, I've started off by building my own custom OpenLDAP server from the latest stable source. Here's my configure syntax:


./configure --prefix=/opt/openldap/meta \
--enable-ldap \
--enable-meta \
--enable-sql \
--enable-rewrite \
--enable-rwm
--enable-proxycache \
--enable-glue \
--enable-unique \
--enable-refint \
--enable-dynlist

In the list above only ldap, meta and rewrite should be essential. After some initial experimentation, I found that my efforts to merge together two or more directory trees were only successful after I compiled in the glue overlay.

If you look at my description below of where I'm going with this, I think it will become obvious why I'm including the other modules/overlays listed (except for rwm, that overlay is supposed to replace rewrite in future, so I want to try it out now).

I'm also going to try some different config variations from the slapd-meta man page, which so far hasn't gone so well.

Before I start in earnest though, let me set out what I'm looking for.

One of the big problems that all directory admins have to deal with is the insane amount of data importing, conversion and outright munging necessary to maintain a centralized directory service that's going to be of any use by anyone. While proxying and caching results can help, the most vexing issue remains how to most efficiently consolidate needed information into the directory.

There are other, rather simple, solutions to this dilema. The one most recommended in the early years of LDAP was to use the referral feature to have queries get redirected to another directory that would have the data the inquirer was looking for. Of course, if data were spread out over several different directories (or relational databases with LDAP listeners), that would force an application to make multiple queries and consoliate the information for itself.

What really is needed is a consolidated "view" of multiple sources of data into a single LDAP directory record for a user, all nicely massaged and normalized for use by any application that needs it. That's why I've also compiled in support for the sql backend (an ODBC manager and it's source, like unixODBC and unixODBC-devel are required to compile), proxycache, attribute uniqueness, referential integrity and dynamic groups.

This is my quest. To follow that star. No matter how hopeless. No matter how far.

I hope along the way we'll all learn a few things we can actually use "back at the shop", as my Dad used to say.

What ever happened to x.500? Or, can we get real for a second about "standards"?

2007-09-01T13:43:00.000-04:00

The ubiquitous "dc=example,dc=com" root context hasn't always been the "standard" for LDAP directories. In the beginning (1987), influenced by the .x500 standard, most directories used something like "o=example,c=US" for their directory root. Those were different times, when it looked like networks, the Internet most of all, was going to be big, global, and most of all corporate.

A funny thing happened on the way to the millenium though. As the developers of LDAP had predicted, x.500 and the DAP (Directory Access Protocol) that accompanied it, were too inflexible, too complex, for even the largest companies to cost-justify implementation. The dream of an interconnected, fully interoperable, world-wide directory passed into history. What we were left with was the simpler, more nimble and still barely structured LDAP (Lightweight DAP) protocol and servers to provide standalone directory services for organizations and individuals, first from the University of Michigan team that invented the protocol, and then Netscape and others after these developers moved out into the commercial market.

The "dc=example,dc=com" root dn format, where the "dc" is short for "domainComponent", is only the most visible sign of this shift to a different approach to directory services.

There are many systems that still default to the old x.500 root dn, most notably IBM's Lotus Domino product. This should be a tip-off that things might not work exactly like other directory products that sport the "dc-example,d=com" format. And they don't. This isn't always a good litmus test though. Microsoft has used the new format in it's Active Directory product from the beginning, but AD very definitely has some un-LDAP like qualities that become painfully apparent very soon after you begin working with it.

As they are wont to do, commercial vendors often equate the Internet standard LDAP v3 protocol with it's implementation by one or more vendors. An application written to interface with Lotus Domino may claim to be "LDAP compliant" but in reality it's interoperability with other LDAP servers may be less than seamless. Another may claim "LDAP compatibility" and then give you a choice between Microsoft Active Directory and Sun (often "iPlanet") Directory.

A close look at their code will usually reveal a custom interface for each directory. The MS interface using (or, inexcusably not using) Simple Paged Results Control to page results from deep searches that would otherwise get truncated or fail because of AD's aggressive limits on search results (default 200 entries, hard of 2,000). The Sun interface might instead use Sun's proprietary Virtual List View (again, or not) to achieve the same result. Sometimes you see really stupid, embarressingly stupid, parameters hardcoded in like requiring a single dn for person entries and only providing a level 1 search (one level down from the base given, so searches won't recurse down into subcontainers) -- making the product useless where your DIT (Directory Information Tree) either nests user entries several levels down, or spreads them out several branches across.

second openldap directory on the same server

2007-09-01T13:19:00.000-04:00

If you don't need a second, distinct slapd process (e.g. no need for the directory to run on a different port, listen on a different interface, or use a separate logfile) this actually pretty easy. All you have to do is add another "database" line and define a different suffix than any of the database definitions that came before.

So, if your first database was defined as:


database     dbd
suffix       "dc=example,dc=com"
rootdn       "cn=Manager,dc=example,dc=com"
rootpw       secret
directory    /var/lib/ldap

... with whatever indexing and other command you've got, the next database would be defined with,


database     dbd
suffix       "dc=sample,dc=org"
rootdn       "cn=Manager,dc=sample,dc=org"
rootpw       secret
directory    /var/lib/ldap2

That's all there is to it. You'll connect to this new db using the same port as your other db, but with a different context. So instead of


ldapsearch -x -LLL -b "dc=example,dc=com" -s sub "objectclass=*"

you'd use


ldapsearch -x -LLL -b "dc=sample,dc=org" -s sub "objectclass=*"

Keep in mind that when initializing the new db, you need to specify it's context as one of the parameters for slapadd


slapadd -s /etc/openldap/slapd.conf -b "dc=sample,dc=org" -l sample.ldif

Your new db files will get created in /var/lib/ldap2, or whatever directory you specified within your new database definition.

Don't forget that you must specify indexes, time and size limits for each database defined, even if these will be the same as those already set for previous databases.

Also, when you combine two or more databases in the same slapd.conf, operations from both will get logged together. If you want separate logs you'll need to set up a second slapd.conf and invoke slapd a second time with this new slapd.conf as the config file. Something like this:


slapd -n slapd2 -l LOCAL6 -f /etc/openldap/slapd2.conf -h \
ldap:///:10389 -u ldap -g ldap

In the above line I've set a different syslog facility for my log (and defined it in syslog.conf to write to a different log file than my main slapd, which is running on LOCAL4), I'm also starting it up on a different port (10389).

Configuring RedHat OpenLDAP for Logging

2007-08-29T00:57:00.000-04:00

Out of the box, RedHat's build of the openldap server doesn't provide for a separate log file to track operations performed on the directory. This is easy enough to set up with a couple of config file changes.

First, edit /etc/openldap/slapd.conf and put the following line above where the database definitions go ("# ldbm and/or bdb database definitions"):


loglevel 256

This will enable logging at a level that will give an adequate amount of detail for tracking and troubleshooting.

Next, edit /etc/syslog.conf to add the following line:


local4.*               /var/log/ldap.log

This tells syslog to redirect all messages from the LOCAL4 logging facility to a file called /var/log/ldap.log, slapd being configured by default to send logging information to LOCAL4.

Restart syslog, then slapd (RedHat calls the service "ldap") using /sbin/service and you should see the following in your new ldap.log:


Aug 27 23:36:44 myhost slapd[11929]: @(#) $OpenLDAP: slapd 2.3.30 (Apr 25 2007 04:35:50) $      brewbuilder@hs20-bc2-3.build.redhat.com:/builddir/build/BUILD/openldap-2.3.30/openldap-2.3.30/build-servers/servers/slapd
Aug 27 23:36:44 myhost slapd[11930]: slapd starting

If you've installed your own build of OpenLDAP, or willing to hack the shipping init script, /etc/init.d/ldap, you can use the "-l" switch to specify a different syslog logging facility (LOCAL0 through LOCAL7, or USER or DAEMON).

openldap as a pass-through proxy

2007-08-29T00:23:00.000-04:00

First, you need to download the latest stable source from the OpenLDAP Project. As of this writing that is v2.3.32.

Next do a tar xzf to unarchive the source into a directory for building (I usually use /var/tmp).

For a simple LDAP proxy that can also do dn mapping, use the following configure command:

./configure --prefix=/opt/openldap/proxy --enable-ldap --enable-rewrite

The path specified in "--prefix" can be anything you like, but keep in mind that you probably don't want to step on any existing openldap binaries installed on your distribution, so it's a good idea to segregate it in its own directory. As for almost all 3rd party software, my practice is to install to /opt.

Then do a "make depend", "make", and su-ing to root, a "make install".

Once that is done (which on a recent RedHat family system should complete without any errors), you should change ownership of the new program directory (/opt/openldap/proxy) to the system ldap group and user ("chown -R ldap:ldap /opt/openldap/proxy). For convenience, you should also make everything readable and writable by members of the ldap group ("chmod -R g+rw /opt/openldap/proxy").

My initial slapd.conf to launch an LDAP proxy was pretty simple. Here's the text:


#
# See slapd.conf(5) for details on configuration options.
# This file should NOT be world readable.
#
include  /opt/openldap/proxy/etc/openldap/schema/core.schema
include  /opt/openldap/proxy/etc/openldap/schema/cosine.schema
include  /opt/openldap/proxy/etc/openldap/schema/inetorgperson.schema

pidfile  /opt/openldap/proxy/var/run/slapd.pid
argsfile /opt/openldap/proxy/var/run/slapd.args

access to dn.base="" by * read
access to dn.base="cn=Subschema" by * read
access to *
 by self write
 by users read
 by anonymous auth

loglevel   256

######################################################
# database definitions
######################################################

database ldap
suffix  "dc=mydomain,dc=com"
uri  "ldap://ldap.mydomain.com"
acl-bind bindmethod=simple binddn="cn=proxy,ou=special users,\
dc=mydomain,dc=com" credentials=secret

The important thing to note is that the "database" enabled is "ldap", the proxy back-end. The uri is that of the LDAP server I'm proxing to. The acl-bind directive enables authentication to the target LDAP server for the purpose of applying any access controls in effect there. Without this all binds from the proxy would be as anonymous.

Actually starting up this new slapd can be done with the following syntax:


/opt/openldap/proxy/libexec/slapd -4 -h ldap://myhost.mydomain.com:1389 \
-n slapd-ldap -u ldap -g ldap

The slapd executable is the one we built from source and installed to "/opt". The "-4" directive tells slapd to listen for IP v4 clients only. The "-h" directive specifies the uri of the proxy server. In this case I've given a DNS name, but you could just as easily specify an IP address. You must specify the listening port, or it will use standard LDAP port 389. "-n" lets you name the new slapd process so it can be easily differentiated from any others running on the same box. The "-u" and "-g" specify the system user and group to run the server as.

P.S. If you're having trouble getting slapd to come up, use the "-d" option with some value (I usually just use "1") to get debugging info echoed to the console -- keep in mind that this will cause slapd not to fork and it will only stay up only so long as the console you invoked it in is up. In addition, you can actually log via syslog by specifying "-l LOCAL6" or another logging facility on the command line, and defining the log path in /etc/syslog.conf (be sure to reload syslog with the new config info).

openldap proxy and beyond

2007-08-29T00:11:00.001-04:00

Finally made the time to compile a fresh build of the latest stable openldap (v2.3.32) with the "ldap" backend enabled and configured as a simple pass-through proxy. This was just the beginning of course, because I then did a "make clean" in my build directory and enabled both the "ldap" and "meta" backends, using a different prefix parameter to put the finished product in a different directory (in my case proxy went into /opt/openldap/proxy and meta to /opt/openldap/meta).

What I'm hoping to do with meta is create a "virtual directory" from a couple of different vendor directories. My first try will be with a Fedora Directory and an Active Directory. The idea will be to create a view of selected entries from each of these directories under a single "virtual" DIT (Directory Information Tree) that resides on the metadirectory. This will require some fancy dn mapping, as well as some attribute and attribute value transforms, all of which meta is supposed to be able to do.

I've had mixed success with other LDAP proxy products, and spent a hellish first quarter in 2001 struggling with getting iPlanet Meta-Directory to do anything better than a collection of scripts I wrote myself (I finally got my company to dump Meta-Directory, after receiving lackluster support from iPlanet, and then Sun). Later that year I was at a marketing event where Hal Stern from Sun spoke. When he got to the subject of Meta-Directory, he looked over at me and said, "... or, you could just write your own scripts...". Vindication.

This should prove to be instructive, and may even be useful for at least one project at work.

I'll provide details of my configurations as testing proceeds.

nothing stays the same: adventures in directory rescue

2007-08-21T15:02:00.000-04:00

OK. So this all relates to the downing of my company's primary master directory some weeks ago when the host server's system disk failed. Apparently on it's way out, the system caused some fatal corruption in the directory db that prevented it from starting up. Because that particular master was also the "hub" from which we replicate to our read-only replicas, losing the db also meant that we'd have to rebuild those replicas as well (a drawback of how Netscape engineered replication in it's directory servers). Yup, a pretty bad situation.

When I first started with iPlanet Directory back in '00, I quickly learned the most efficient way to rebuild not only a master, but also a replica, directory in the event the underlying db got corrupted.

For the most part I've continued using the procedures I developed back then until now. For example, to rebuild a replica I normally just did a re-initialization from the master over the wire.

This time, instead of doing an online rebuild, I let the replication agreement wizard create an init file (an LDIF with all the pertinent info), shut down the replica and did an offline init using ldif2db. This solved a couple of problems, the most important of which was that there was a Local Director in front of my replicas that would do a round-robin to a directory as long as it was listening (LD isn't very smart, it can't test to see if the directory is actually responding, just that it's listening). If I kept the directory online, which you must do for an over-the-wire init, apps and users would hit a wall every time they got shifted over to the initializing directory.

It also turns out that with the number of entries I now have in my directory (50,000 plus), the rebuild process also went faster using init files. This was especially true for my heavily indexed primary master, which I rebuilt from the secondary master.

In all it took about an hour to get most everything back online, another hour to get the whole environment back running at peak performance. Because we use clustering in our SSO solution, most apps and their users were unaware of the outtage. We also bought ourselves some breathing space by assigning a CNAME that belonged to the dead master to it's partner, causing all traffic that was hardcoded for that CNAME to go to the functioning directory.

so what is the deal with openldap schema-checking?

2007-08-21T14:13:00.000-04:00

The last few days I've been experimenting with loading some medium sized datasets into OpenLDAP. Basically all I did was transform a dump of all the active entries in my company's directory into a format loadable into OpenLDAP without making any schema changes.

Well, at least that was the plan.

Right off the bat I was tripped up by the fact that the 'c' or 'countryName' attribute, although defined in the core.schema file, is only included in the 'country' objectclass as an allowable attribute. No problem, I just added it to 'organizationalPerson' (hey, I am eldapo, after all).

Then I had some trouble with having a value like "9999999999" for 'facsimiletelephonenumber', with ldapadd screaming at me about invalid syntax (the dreaded "error 21"). When I checked the core.schema file I found 'facsimiletelephonenumber' or 'fax' had a different syntax code that 'telephonenumber'. I fixed that too, adding the EQUALITY and SUBSTR directive from 'telephonenumber' to allow me to index it for eq and sub searches (any good whitepages directory should let you search on fax as well as voice numbers for someone).

There were a few other odd schema violation notices as well, some inexplicable even on careful examination. In the end I did an ldapadd -c to just load the bulk of the entries (which came to just over 15,000).

Of course schema-checking can no longer be disabled in the latest OpenLDAP code (since at least 2.2.28), which Kurt Zeilenga explains was necessary because the feature conflicted with other enhancements being rolled into the code (he adds that it was broken anyway, to which I can attest from personal experience).

When I first started working with OpenLDAP I quickly learned that hacking the schema was an essential skill I needed to develop. For the most part you can get away with adding your own custom objectlasses (usually auxilliary) and attributes to get around the limitations of what ships with the product. Oh yeah, you'll also need to understand the difference between structural and auxilliary objectclasses. For example see this discussion thread (and what preceeded it). A succinct explaination of the matter can be found in this article in the OpenLDAP Faq-O-Matic (which I did not author, though I wish I had).

The aggravating thing is that I've loaded similar data, in fact data that had not had a fraction of the scrubbing, into both Netscape family (Sun, Fedora/RedHat) and Oracle directories without this much trouble -- once again demonstrating that open source is not for amateurs.

forcing the orcladmin password

2007-08-04T11:01:00.000-04:00

Someone asked, so here it is.

Something for all my fellow Oracle Internet Directory admins.

This is fully documented on MetaLink, but in various places.

Here's the situation:

You've either forgotten the cn=orcladmin password, or someone has impolitely changed it without letting anyone know the new one. To add insult to injury, in the latter case the dumb schmuck has also altered the ODS (Oracle Directory Schema) user password on the infrastructure database so that OID won't even start.

The solution:

Go to $ORACLE_HOME/ldap/admin and rename the oidpwdlldap1 and oidpwddr[instance name] files.

Use sqlplus to log into the infrastructure database back-ending OID as SYSTEM.

Reset the password for ODS (the "Oracle Directory Schema" user).

SQL>> alter user ODS identified by [new password]

Then execute

oidpasswd create_wallet=true

and, finally, do a

oidpasswd connect=[connect string] change_oiddb_pwd=true

to set the orcladmin superuser password to the same one you just forced for ODS.

When only direct db access will do

2007-07-18T20:26:00.000-04:00

Over the years I've had a few "close calls" where a corrupted directory db was brought back from the abyss. Had one of those the last couple of days, when one of our newly installed Oracle Internet Directory (OID) instances started misbehaving after I screwed up the acl (access control list) while trying to use the aci (access control instruction) "wizard" in oidadmin to create a new set of permissions.

In seven years of working with LDAP directories. I've come to view the GUI administration consoles that ship with commercial products with deep suspicion. Ditto for the "automatic" configuration routines in almost all 3rd party apps that claim to be "LDAP compliant". In this case I should have known better, but the whole purpose of the work I've been doing the last few months is to learn as much as I can about the new application stack so I can provide useful input on how we go forward.

Anyway, the problem we (Oracle Support and I) were faced with was a cn=Users container that could no longer be searched, even by cn=orcladmin. Now "cn=orcladmin" is the root account for OID, just like "cn=Directory Manager" for the Netscape family of directories (Netscape, iPlanet, Sun, RedHat), or OpenLDAP's "cn=Manager". Of all the login accounts on the directory, this is the one that is supposed to not be subject to access controls. In our case, every time we tried to search the container over LDAP (using ldapsearch) the directory process would die and get restarted by Oracle's monitor process.

After going through a series of diagnostic procedures (stopping and starting different pieces of the stack and then reading the logs), it was finally determined that the way to deal with this was to use Oracle's "bulk" utilities to access the underlying database and restore it to health.

We first issued an "opmnctl shutdown" to kill all running processes, and then used ldifwrite to dump the entire cn=Users container directly from the datatabase to an LDIF file. Then we used bulkdelete to remove the container, and all it's subentries, from the database. I then edited the LDIF file to remove the corrupt aci from the container base. Finally, bulkload was used to re-load the corrected LDIF back into the database. During this process the data was reindexed and subjected to rigorous checking.

A "opmnctl startall" brought the directory and it's associated processes back on line, and after a few simple tests using ldapsearch and ldapmodify we were satisfied all was well again.

Here's the syntax for the commands used during this recovery (during each operation you will be prompted for the root directory user (cn=orcladmin) password, which should be the same as that for the ODS database user):

[In these examples my directory root is "dc=mycompany,dc=com", "testinf" is the $ORACLE_SID for the infrastructure database where directory data is stored and "mydata.ldif" is my LDIF data file. Where "\" appears at the end of a line, it is a "continuation character" indicating that the line should not be broken by a carriage return]

Dumping the container data from the database


$ORACLE_HOME/ldap/bin/ldifwrite connect=testinf basedn="cn=Users, \
dc=mycompany,dc=com" ldiffile=/tmp/mydata.ldif>

Removing the container from the database


$ORACLE_HOME/ldap/bin/bulkdelete connect=testinf basedn="cn=Users, \
dc=mycompany,dc=com"

Reloading the LDIF data directly into the database


$ORACLE_HOME/ldap/bin/bulkload connect=testinf check="TRUE" \
generate="TRUE" append="TRUE" file=/tmp/mydata.ldif

Lest it appear that Oracle is alone in making such lifesaving "direct to database" methods available, Sun/RedHat and OpenLDAP all have similar tools used to directly manipulated their underlying BerkeleyDB hash table managers (BerkeleyDB is now, ironically, owned by Oracle).

For OpenLDAP these are slapcat for dumping data from the db, and slapadd to load data directly into the db. With OpenLDAP there is no way to directly delete just a portion of the directory tree. You've got to dump and reload the whole thing (making whatever changes you need to the LDIF file in between).

Sun/RedHat have a set of command line scripts, db2ldif and ldif2db that you to dump and reload data from the db, but still require you to in essence reload everything if you want to clean it up. For practical purposes you'd want to do this on a Netscape family directory anyway, to make sure all indexes are regenerated correctly (not always a given with their architecture).

OID: Spit install = Splitting Headache

2007-07-14T02:25:00.000-04:00

Alot of people who read this are going to disagree with what I'm about to say. But here it is.

When building an Oracle 10g AS infrastructure most enterprises will do a split install, putting the application server(s) on one box and the metadata repository (Oracle database) on another. In many cases the applications may even be split up, with Oracle Internet Directory (OID) running on one box, SSO another and so on.

This is insane.

OID is the foundation for any Oracle 10g infrastructure. If OID is down, you're hosed.

Restart lsnrctl, or lose your connection with the database either because the db or it's host is being recycled, or because of an errant firewall rule or some other network connectivity transient, and OID will go out to lunch.

As much as I admire and respect all DBA's, in the six months I've worked with OID the only times it's hung or toppled over is when, for some reason, the database sitting on a remote server became unavailable. Most of the time because the instance I needed was down and no one realized it's significance. Actually, make that 2 months. Up until that time all the test installs I used had the database on the same host as the infrastructure and I made damn well sure it was up.

There's nothing more disheartening that having to jump into a VNC session to demonstrate to an experienced DBA that the problem really is a database issue (tnsping is useless for this, the best method is to log on as the system user, make sure your environment is set and fire up sqlplus as ODS).

Many years ago I made a similar argument about locating Microsoft Active Directory database files on a SAN. Given that AD is the heart and soul of the server operating system itself, putting the edb files off on remote storage is clearly asking for trouble.

While using a remote db could have some benefits, like allowing for a high availability configuration such as multiple infrastructure servers connected to a RAC, given the critical place that OID has in the environment I still think it would be better to have the metadata db on the same box as OID. If the environment needs to be highly available, you can still RAC at the app tier or use OID's robust multi-master replication feature.

how do i unlock an EBS account?

2007-07-05T16:55:00.001-04:00

Make the value for END_DATE in FND_USER null, of course.


SQL>UPDATE FND_USER
SQL>SET END_DATE = ''
SQL>WHERE USER_NAME = '[Your User Name]';
...
SQL>COMMIT;

To do it in Perl just use the code I've got below (in mod_ebsdb) and substitute the above SQL for the value of $sql (you might want to change what you print to console in order to prevent confusion). Like this:


my $sql= "UPDATE FND_USER
         SET END_DATE = ''
         WHERE USER_NAME = '$user_name'";

The aforementioned code will commit the transaction, assuming it passes the simple "Does this USER_NAME exist?" test.

indexing attributes on oracle internet directory

2007-07-05T14:04:00.000-04:00

Basic task that we all have to do, no matter what vendor's directory we use.

For OID, like almost everything else, it requires a special tool to reindex the database. The tool is called catalog. It's found under $ORACLE_HOME/ldap/bin.

$ORACLE_HOME/ldap/bin/catalog connect=[oiddbsid] add="TRUE" \
attribute="[attributename]" [or file="filename"]

You can do a bunch of attributes with the optional "file=" parameter instead of "attribute=", where the value would be the path to a text file containing a simple list of the attributes to be indexed, one attribute name per line.

Sometimes the change will be effective right after running the command (Oracle actually recommends shutting down OID before indexing -- which makes sense on a heavily used production system). Other times you'll have to restart OID to see the change take place.

Using Perl to Re-Link EBS GUIDS

2007-07-03T10:07:00.000-04:00

If you've been following the last few posts you've probably guessed that I'm now in the middle of a pretty intense effort to get control of user management in an Oracle Internet Directory (OID) + Enterprise Business Suite (EBS) environment. What's happened is that the consultants, developers and business teams have been out creating accounts on the EBS database ahead of the security team's effort to create users on OID. Now that we've finally begun to integrate EBS with the new Oracle 10g Application Server Single Sign-On Infrastructure, we need to make sure that the user's OID entries are properly linked with their EBS accounts. This happens "automagically" in most cases, but in our situation there was a last-minute decision to change our DIT (Directory Information Tree) structure that required the deletion and re-creation of all our new OID entries AFTER they'd been linked to their corresponding EBS account.

To avoid having to use the tedious procedure Oracle recommends to fix this that I outlined earlier, I decided to put together a script to automate the process. Below is the result, which still contains some (commented) debugging code that I think others may find useful.

#!/usr/bin/perl
# updateguid.pl Updates USER_GUID on EBS FND_USER from orclguid value 
# in corresponding OID user entry.

use strict;
use Net::LDAP;
use Net::LDAP::Entry;
use Net::LDAP::LDIF;
use DBI;


our($OIDhost,$OIDusr,$OIDpw,$ebssid,$ebsappusr,$ebsapppw);

my $HOME = $ENV{'HOME'};

$ENV{'TNS_ADMIN'} = "/etc/oracle";

require "$HOME/etc/orclapp.conf";

my $usrbase = "cn=users,dc=mycorp,dc=com";
my $query = "(givenname=*)"; # Just hit the "real" people
my @attrs = qw(uid orclguid);


fix_guids();

sub fix_guids {

  my $ldap = Net::LDAP->new($OIDhost);
  my $mesg = $ldap->bind($OIDusr, password =>$OIDpw);
  die ("failed to bind with ",$mesg->code(),"\n") if $mesg->code();

  $mesg = $ldap->search( base =>$usrbase,
           filter =>$query,
    scope =>'sub',
    attrs =>\@attrs

 );

  die "Failed to search with ",$mesg->error(),"\n" if $mesg->code();

  while (my $entry = $mesg->shift_entry()) {

      my $dn = $entry->dn();
      my $uid = $entry->get_value('uid');
      my $orclguid = $entry->get_value('orclguid');

      ## Use this code block for testing guid retrieval
      # my $user_name = read_ebsdb($uid);
      #print $user_name, "\n";
      # print $orclguid, " \n";
      # my $user_guid = read_ebsdb($uid);
      # print $user_guid, "\n";

      mod_ebsdb($uid,$orclguid);
    

  }

}



# This subroutine is for testing guid retrieval
sub read_ebsdb {

  my $user_name = @_[0];

  my $dbh = DBI->connect("dbi:Oracle:$ebssid",
   "$ebsappusr",
   "$ebsapppw",

 ) or die "Database not connected: $DBI::errstr";

  my $sql = "SELECT USER_GUID
      FROM FND_USER
      WHERE USER_NAME = '$user_name'";

  my $sth = $dbh->prepare($sql);

  $sth->execute();

  while (my ($user_guid)  = $sth->fetchrow()) {

      return $user_guid;

  }

  $sth->finish;
  $dbh->disconnect;
  
}

# This is the subroutine that commits changes to EBS FND_USER
sub mod_ebsdb {

  my $user_name = @_[0];
  my $orclguid = @_[1];

  # Use AutoCommit =>0 to allow rollback in case of error  
  my $dbh = DBI->connect("dbi:Oracle:$ebssid",
     "$ebsappusr",
   "$ebsapppw",
   {AutoCommit => 0}

  ) or die "Database not connected: $DBI::errstr";

  my $sql = "UPDATE FND_USER
      SET USER_GUID = '$orclguid'
             WHERE USER_NAME = '$user_name'";

  my $sth = $dbh->prepare($sql);

  my $rows_affected = $sth->execute();

  if ($rows_affected > 1) {

    $dbh->rollback();
 print "There are $rows_affected users with ";
 print "the user name $user_name. Transaction cancelled!\n";

  }
  else {
    
      $dbh->commit();
   print "$user_name USER_GUID is now $orclguid\n";

  }

}

__END__;

Perl Script to List Accounts in EBS Database

2007-07-02T16:48:00.000-04:00

Just when you thought it was safe to go in the water!

Here's a quick DBD::Oracle script I cooked up to do what I did using sqlplus in the previous article. I use an external config file to store the Oracle SID (i.e. service name), user (in this case 'APPS') and password. Notice how straightforward it is to plug a standard Oracle query into the code and get results back. The only shortcoming with this script is that the sqlplus page formatting doesn't come through, so if you wanted to do a report you'd need to insert your own.


#!/usr/bin/perl

use strict;
use DBI;

our($orclsid,$orclusr,$orclpw);

my $HOME = $ENV{'HOME'};

$ENV{'TNS_ADMIN'} = "/etc/oracle";

require "$HOME/etc/orclapp.conf";


my $dbh = DBI->connect("dbi:Oracle:$orclsid",
    "$orclusr",
    "$orclpw",

 ) or die "Database not connected: $DBI::errstr";

my $sql = qq[ SELECT USER_NAME FROM FND_USER ];

my $sth = $dbh->prepare($sql);

$sth->execute();

while (my($user_id)  = $sth->fetchrow()) {

    print "$user_id\n";

}

$sth->finish;
$dbh->disconnect;

__END__;

My next task will be to script the linking of OID orclguids to EBS user accounts in FND_USER. Should be fun.

Some resources I used to do this were An Introduction to DBD::Oracle on the Pythian site. I also used my own Oracle Instantclient and DBD::Oracle to properly make and install the module.

Note: Important addition! The line "$ENV{'TNS_ADMIN'} = "/etc/oracle" sets the $TNS_ADMIN environment variable so the script can use an external tnsnames.ora file -- which is handy if you're working in a multiple database environment and want to be able to test your connect strings using sqlplus (or listener status with tnsping).

list accounts on the EBS database

2007-06-29T17:02:00.000-04:00

Kind of related to the previous post.

I'm an sqlplus beginner. So I think this is really important info. Someday soon I'll be embarrassed I even bothered to write this.

Log onto the EBS database as the APPS user.

sqlplus APPS/{appspw]@[dbname]

Then do this:


SQL> SPOOL /tmp/userlist.txt;
SQL> SELECT USER_NAME FROM FND_USER;
...
SQL> SPOOL OFF;
SQL> quit

You'll get a nice list of all the User ID's on the database. The "SPOOL" command will print the output to the file indicated, so you don't have to do a screen scrape.

If you want everything in the table substitute '*' for 'USER_NAME'.

linking an OID orclguid with a USER_GUID in EBS

2007-06-29T16:37:00.000-04:00

Unless you're actually looking for the answer this article provides, you probably won't have any idea what I'm talking about.

Here's the situation. You've put up an Oracle 10g infrastructure, complete with it's own Oracle Internet Directory (OID). By some unknown magic, your 10g infrastructure (including OID and Single Sign-on, SSO) has just been integrated with an existing Oracle Enteprise Business Suite (EBS) Applications instance.

Part of the integration involves setting up automatic provisioning of user accounts in EBS from entries created in OID. You create a new user in OID, and voila! the user gets an account (albeit one with limited access) in EBS.

Your DBAs, developers, consultants, admins and just about everyone who can draw a breath have been haphazardly creating and deleting accounts in both OID and EBS both before and after the integration. Ugh.

Finally, as expected, some of those accounts are now out of sync. You try to create a new account on EBS for an existing OID user and you get the dreaded "account already there" error. Checking the FND_USER table on the EBS database you find that, sure enough, it is there. But no matter how hard you try, you can't seem to get SSO to log you into EBS. The OID entry won't link to the EBS account. Instead of jumping up and down screaming, calm down. Here's what you do.

Log onto the server hosting EBS as the system user (something like "oracle" or "oraebs"). If the .bash_profile for this user doesn't set the environment for you, you'll need to source it -- hopefully from a file you've created like "ebs.env".

First get the orclguid value from the corresponding OID entry. Since this is a system attribute, it won't show up in a normal search. You need to specify it.


ldapsearch -h [oidhost] -D "cn=orcladmin" -w [adminpw] -b "dc=my,dc=corp" \
-s sub "(cn=[userid])" orclguid

This will return a long string of numbers and letters for orclguid. Highlight and copy it.

Now fire up sqlplus and connect to the EBS database as the "APPS" user.

sqlplus APPS/{appspw]@[dbname]

Now follow the following procedure:


SQL> SELECT USER_GUID FROM FND_USER
WHERE USER_NAME='[EBS User ID]';

Where 'USERID' is the ID of the user account you need to fix.

SQL> UPDATE FND_USER
SET USER_GUID = '[orclguid value]'
WHERE USER_NAME = '[EBS User ID]';
...
SQL> COMMIT;
...
SQL>quit

There it is.

ldap browser on linux

2007-06-17T21:41:00.000-04:00

Jarek Gawor's LDAP Browser-Editor has been one of the most popular LDAP tools in use by directory admins for at least 7 years. Never mind that the last release of the software was v2.8.2b2 (that's Beta 2) in 2001, it remains top on my list because, warts and all (not the least of which is that it was programmed using Java Swing), it is still the best at what it does.

Installing and configuring it on Linux has never been hard, unless you want to make it play nice in a multi-user environment. If you don't want to force each user to install their own personal copy, you're going to have to make some changes to the shipping shell script used to launch the app and make a dot directory in your user's home.

My standard setup puts the unarchived app code into a directory called /opt/ldapbrowser, which I normally create by simply copying Browser282b2.tar.gz to /opt and doing a tar czf on it right there.

Next, I backup the shipping lbe.sh and replace it with my own:


#!/bin/sh
JAVA_HOME=/usr/java/jdk1.5.0_11
LBE_ROOT=/opt/ldapbrowser

cd ~/.lbe

${JAVA_HOME}/bin/java -jar ${LBE_ROOT}/lbe.jar $1 $2 $3 $4 $5 $6 $7 $8 $9

Then I run a symlink from the new lbe.sh to /usr/bin/lbe:
ln -s /opt/ldapbrowser/lbe.sh /usr/bin/lbe

Finally, I go and create an .lbe directory in my user's home and copy attributes.config from /opt/ldapbrowser into it. I also run symlinks from /opt/ldapbrowser/help and /opt/ldapbrowser/templates into .lbe, so the resources in those directories will display properly.

Changing Active Directory Passwords Using LDAPS

2007-06-12T14:13:00.000-04:00

This is an article from my old personal site.

Which is the point of what came before ...

Just a brief script to show how to change an Active Directory user's password using LDAPS (LDAP over SSL). Requires that the target Active Directory domain controller be SSL enabled, as described in this article. In Active Directory password data is not stored in userpassword, but instead the hidden "system" attribute, unicodePwd. The script contains a routine used to convert the ASCII string supplied for the password into it's Unicode equivalent.

The "require" line simply imports config info (bind dn and password, etc.) for the script to use.


#!/usr/bin/perl

use Net::LDAP;
use Net::LDAP::Entry;
use Unicode::Map8;
use Unicode::String qw(utf16);

our($adHost,$adAdmin,$adPass);
require "../etc/ldap.inc";

my $adURI = "ldaps://$adHost";
my $basedn = "DC=test,DC=example,DC=com";
my @attrs = qw(cn sn givenname uid mail unicodePwd);
my $query = "(cn=orson)";
my $newpw = "rosebud";

my $charmap = Unicode::Map8-> new('latin1') or die $!;
$newpw = $charmap-> tou('"'.$newpw.'"')-> byteswap()-> utf16();

my $ldaps = Net::LDAP-> new( $adURI ) or die "$@";

my $mesg = $ldaps-> bind($adAdmin, password =>$adPass);

$mesg = $ldaps-> search (
 base => $basedn,
 scope => 'sub',
 filter => $query,
 attrs => \@attrs
 );

while (my $entry = $mesg-> shift_entry()) {
    my $userdn = $entry-> dn;
    print $userdn, "\n";
    # $entry-> dump;
    $entry-> replace('unicodePwd' => $newpw);
    $entry-> update($ldaps);
    

}

$ldaps-> unbind;

__END__;

LDAPS with Net::LDAP

2007-06-12T14:11:00.000-04:00

This is an article from my old personal site.

This script is an example of how to make an LDAP over SSL connection. LDAPS requires a special secure port, usually TCP port 636. The target server in this instance is a Microsoft Active Directory domain controller that has been SSL enabled (a configuration discussed in another article).


#!/usr/bin/perl

use Net::LDAP;

our($adHost,$adAdmin,$adPass);
require "../etc/ldap.inc";

my $basedn = "DC=test,DC=example,DC=com";
my @attrs = qw(cn sn givenname uid mail unicodepwd);
my $query = "(cn=Administrator)";

my $ldaps = Net::LDAP-> new( "ldaps://$adHost" ) or die "$@";

my $mesg = $ldaps-> bind($adAdmin, password => $adPass) or die "$@";

$mesg = $ldaps-> search (
 base => $basedn,
 scope => 'sub',
 filter => $query,
 attrs => \@attrs
 );

while (my $entry = $mesg-> shift_entry()) {

    $entry-> dump;

}

$ldaps-> unbind;

__END__;

SSL Enabling Active Directory

2007-06-12T13:48:00.000-04:00

This is an article from my old personal site.

There are several ways to do this. The easiest is to enable an Enterprise CA and let auto-enrollment propagate SSL certificates to all domain controllers (DC's). This is Microsoft's preference. Most Active Directory (AD) administrators are loathe to go along with this. First, because they believe SSL is an evil virus created by Netscape, and second, because they're afraid to alter the shipping configuration of AD in any way. From their point of view AD is already complicated enough without mucking things up with some Internet standard protocol functionality. As a result of this, most real-world LDAP admins compromise on getting at least one DC enabled for SSL. The most non-intrusive way to do this is to use a 3rd party certificate authority (CA) to sign an appropriately formed certificate request. The resulting certificate is then imported, along with the CA's own certificate, into the target DC's local certificate store.

Official Documentation
How to enable LDAP over SSL with a third-party certification authority

Best Practices for Implementation of a Microsoft Windows Server 2003 Public Key Infrastructure

The first article listed above is the official "how to" from Microsoft on using a 3rd party CA to SSL enable AD. It is written by someone who's obviously done this many times, and because of that is not a very good beginner's guide. Reading through all of the articles linked in the second reference will give you most of what you need. Still, I found that there was no substitute for building my own test AD domain controller and diving right in to the deep end of the pool. Your mileage may vary (please keep in mind that I first "cut my teeth" on Windows NT 3.51 in an enterprise environment over 10 years ago, and had an MCSE number in the low thousands). Even so, on my first go around, I installed Windows 2003 in a VMware Server virtual machine. Happiness is being able to roll back to the last snapshot when bad things happen.

Prerequisites
You'll need a working AD environment, in which all services (especially DNS) are properly configured. You'll also need a 3rd party CA. In my case this was an OpenSSL CA I'd set up on a Linux box.

Generating the Certificate Request
To do this you need to follow the instructions set out in the first article cited above exactly, but for one major bit of misinformation.

In describing the request.inf file used with the certreq utility to generate the request, the article adds a caveat to the effect that some CA's might require additional elements in the "Subject" line -- such as e-mail address, organization name, etc. Adding these elements, however, will guarantee that you won't be able to successfully import the signed certificate. The reason? Because the local computer certificate store (a/k/a the "MY" certificate store) will not have any of these elements and so the distinguished name will not match that of the machine.

Here is the actual request.inf I used to generate my request. Note that the values I used for the different elements should be changed to match your environment:

 ;----------------- request.inf -----------------

[Version]

Signature="$Windows NT$

[NewRequest]

Subject = "CN=dc01.test.mydomain.com"
KeySpec = 1
KeyLength = 1024
Exportable = TRUE
MachineKeySet = TRUE
SMIME = False
PrivateKeyArchive = FALSE
UserProtected = FALSE
UseExistingKeySet = FALSE
ProviderName = "Microsoft RSA SChannel Cryptographic Provider"
ProviderType = 12
RequestType = PKCS10
KeyUsage = 0xa0

[EnhancedKeyUsageExtension]

OID=1.3.6.1.5.5.7.3.1 ; this is for Server Authentication

Once you have your request.inf file, you must run the following command to generate the request. Do this while logged in as the machine Administrator (cn=Administrator, cn=Users, dc=test, dc=domain,dc=com or Administrator@test.domain.com):

certreq -new request.inf newreq.pem

The resulting request file, newreq.pem can now be submitted to a 3rd party CA for signing.

Signing the Request
To sign the certificate request using OpenSSL, you need to transfer the file to where your "demoCA" has been set up. In my case this is at /export/CA on a CentOS Linux box. I like the CA script for this kind of work. It expects the request file to be named newreq.pem by default. As root, I use the following command after changing to /export/CA and making sure that newreq.pem is located there:

/usr/share/ssl/misc/CA -sign

After supplying the CA's secret (password), I accept the defaults and the new signed certificate, by default named newcert.pem is created alongside the newreq.pem file. I usually rename the certificate file to something more unique, in this case dc01.cer.

Importing the Certificate(s)
A careful reader will notice I add a "(s)" to the end of this section title. That is because to make this all work you need to import into Active Directory not only your newly signed certificate, but also the certificate for the CA that signed it. The procedure for importing these certificates differs. Make sure to do it exactly as described. You can use the Certificates Snap-in to import the server certificate, but it will not report some kinds of errors. To be safe, use the command line tool. First transfer the certificate to the DC filesystem, and then, while logged in as Administrator, run the following command:

certreq -accept dc01.cer

Now add the Certificates (not the Certification Authority) Snap-in to your Administrative Tools menu (run mmc, add the snap-in and save as a new console object), being careful to specify that it will manage certificates for the "Computer Account". Use the snap-in to verify that the new server certificate exists under Certificates/Personal/Certificates. At this point the new certificate will be considered suspect, since the system will not recognize the signing CA. To cure this, transfer the CA's public certificate (without the key) to the DC filesystem and import it into the "Third Party Root Certificate Authorities" container using the Certificates Snap-in.

Effect and Verify the Change
Once the above steps are completed, reboot the DC system to make the change effective and enable the LDAPS (LDAP over SSL) listener on the DC. To verify that it is now working, use the ldp.exe from the Windows Support Tools to connect to the local AD instance, by checking off "SSL" and changing the port number to 636.
You can also test the SSL connection using the ldapsearch utility. Before doing this, copy the CA's certificate to the local filesystem (I use a system-wide location of /etc/ssl, you could also use the default /etc/openldap/cacerts) and configure ldap.conf to include the full path to this cert in the TLS_CACERT directive (e.g. TLS_CACERT /etc/ssl/myca.pem). Once you've disposed of these preliminaries, try the following command:

 ldapsearch -x -H ldaps://dc01.test.mydomain.com \
-D "cn=administrator,cn=users,dc=test,dc=mydomain,dc=com -w mypass \
-b "dc=example,dc=com" -s base "objectclass=*"

LDAP over TLS with Net::LDAP

2007-06-12T13:44:00.000-04:00

This is an article from my old personal site.

Following is a simple script using LDAP over TLS (start_tls). Like LDAP over SSL (LDAPS), communications are done over a secure channel. In the case of TLS, however, those communications happen over port 389 instead of a dedicated secure port (LDAPS defaults to port 636). The "require" line is used to import host, userdn and password values from a separate configuration file.

#!/usr/bin/perl

use Net::LDAP;

our($dirHost,$dirUsr,$dirPass);

require "../etc/config.inc";

my $basedn = "dc=example,dc=com";
my @attrs = qw(cn sn givenname uid mail);
my $query = "(cn=*)";

my $ldap = Net::LDAP-> new( $dirHost );
my $mesg = $ldap-> start_tls(verify=> 'require',
       cafile => '/etc/ssl/cacert.pem'
      );
# die $mesg->error() if $mesg-> code();

$mesg = $ldap-> bind($dirUsr, password => $dirPass);

my $mesg = $ldap-> search (
 base => $basedn,
 scope =>'sub',
 filter => $query,
 attrs => \@attrs
 );

while (my $entry = $mesg-> shift_entry()) {

$entry-> dump;

}

$ldap-> unbind;

__END__;

checkfields.pl

2007-06-08T16:18:00.000-04:00

This is one of those works-in-progress that may have some use to others, so I'll share it here.

Oracle DBA's called this morning wanting to know if the length of any of the fields in a delimited file I send them changed (got longer). Told them I had no idea. LDAP doesn't care about field length. Makes it easier to use as a white pages platform if you don't have to keep track of how many characters are in the surname in each entry.

Anyway, because I'm such a nice guy I told them I'd go find out what the largest number of characters were in each field of the file.

Here's another of my (infamously) simplistic Perl scripts that I used to get the info:


#!/usr/bin/perl
# checkfields.pl Audit a delimited LDAP feed
# Read in pipe-delimited feed, count number of chars in each field to get highest
# total. For use in reporting number of chars that import needs to
# accomodate for each field.

use Text::ParseWords;

my $HOME = $ENV{'HOME'};

my $feedfile = "$HOME/tmp/ldap.dat";
my %wholefeed;

my $totfields;


print "Checking $feedfile\n";

open FH, "<$feedfile" or die $!;

while (<FH>) {

 my @line = &parse_line('\|',0,$_);

 $totfields = scalar(@line);

 my $fieldct =0;

 my $index = @line[0];

 foreach my $field(@line) {

  my $fieldno = $fieldct++;
  my $newlen = length($field);

  my $oldlen = $wholefeed{$fieldno};
 
  if ($newlen >$oldlen) {
   $wholefeed{$fieldno} = $newlen;
  
  }
 
 }
 
}

print "Total Fields: $totfields\n";

print "Field,Chars\n";

my $count;

for ($count =0; $count <$totfields; $count++) {
 print $count, ",", $wholefeed{$count}, "\n";
}

__END__;

Let's get a real database

2007-05-30T12:11:00.001-04:00

I just had to share this old post from the Xooglers blog about the evolution of Google's AdWords application. Here are my favortie snippets:

AdWords was built using the MySQL database, which is open-source and therefore available for free. It is by now also nearly as full-featured as the best commercial databases, but back in 2000 this was not the case. MySQL was quite a capable system, but missing a few (what some would consider basic) features. These missing features were obviously not a show-stopper, as we managed to get AdWords to work without them, but in a few cases it did take some extra programming to work around one of these missing features. On the plus side, MySQL was fast and reliable and, as I have already noted, free.
***
After AdWords launched, Jane, the ads group manager, decided that now would be a good time to switch over to a "real" database. "Real" is one of those words that Doug ought to add to his list of words. It means "expensive".
***
We finally decided to go with a commercial database (I won't say which one) over the objections of a number of engineers, including myself.
***
To make a long story short, it was an unmitigated disaster. The new system was slower than molasses in February. Some heroic optimization efforts eventually produced acceptable performance, but it was never as good as the old MySQL-based system had been.
***
It was still that way when I left Google in October of 2001, but I have heard through the grapevine that they eventually went back to MySQL.
***
The moral of the story is that sometimes, and in particular with free software, you get more than what you pay for. There are a lot of companies out there paying dearly for commercial databases (and operating systems for that matter). As far as I'm concerned they might as well be flushing that money down the toilet. Actually, they might be better off. We certainly would have been.
***

Amen.

OpenLDAP Redux

2007-05-29T12:27:00.000-04:00

After several months in "full immersion" mode with Fedora Directory, I just began reconfiguring my home lab network to use OpenLDAP as it's main directory server again. This time around I'm going to also try integrating OpenLDAP as the backend for a new kerberos authentication infrastructure to service both my Linux and Windows machines. It will be interesting to see how easy (or hard) it is to also integrate some of the test JBoss stuff I have out there that's currently working with FDS.

OpenLDAP Proxy

2007-05-29T12:21:00.001-04:00

Surfing various LDAP related mail lists over the weekend I came upon post after post that revealed the great extent to which OpenLDAP has become a common solution as an LDAP proxy. That intrigued me because I've recently been thinking again about both high-availability and data integration issues in which OpenLDAP's proxy and meta backends could provide an answer.

An IT Infrastructure for the Rest of Us

2007-05-29T11:24:00.000-04:00

Because I work in a large Enterprise environment all the time, it's sometimes easy to forget that the kind of heavyweight infrastructure I'm used to isn't something that most people can afford to operate. In fact, the cost of application stacks like those offered by Oracle, IBM or Sun (with "honorable mention" to SAP and others), goes way beyond the license fees charged. Deploying and maintaining those environments also requires a significant investment in time and human resources that is not only beyond the means of most small to mid-sized businesses, but would actually overwhelm them and distract them from whatever their actual business mission is.

From my limited knowledge of what's actually out there, I see two basic options for "the rest of us", companies and firms that aren't on the Fortune 500 list. Microsoft, or open source. Microsoft's cost to benefit numbers are actually pretty good for small scale deployments. Their technology stack is also mostly complete. Here in America they already "own" the desktop, and so building out using their server products makes alot of sense for many businesses. Of course the problem with Microsoft is that basing your infrastructure on their stack will ultimately result in their "owning" you.

Until a few years ago open source couldn't compete with Microsoft because there weren't any vendors who could deliver an integrated application stack to serve the various requirements of this class of businesses. While IBM, in particular, eagerly integrated open source software like Linux and Apache into their solutions, the cost for both licensing and human resources to deploy and maintain kept them outside the reach of most small and mid-sized businesses.

I think Red Hat changed things dramatically with their acquisition of JBoss and improvements to their operating system (as well as accompanying software) in recent years. Red Hat could be doing more in the area of helping small business, especially, learn how to use technology more effectively than the Microsoft Windows/Office paradigm that that's been dominant for the last two decades. Red Hat has been gaining lately, however, by branding solutions that they've developed with partners to serve different business segments like the Red Hat Enterprise Health Care Platform.

What interests me about the Red Hat stack as it exists today is that so much of it is under-the-covers pure open source that's available elsewhere. The thing that makes Red Hat's distribution of this stuff different is their engineering improvements, particularly in the area of manageability. For example, open source kerberos gets high marks as a proven authentication framework that is notoriously difficult to implement and integrate with the rest of an infrastructure. Red Hat's "re-spin" of kerberos in it's Enterprise Linux product includes integration points with common services like the Apache HTTP server and the OpenLDAP directory. With what Red Hat gives me in a few basic Enterprise Linux subscriptions I can now build an entire infrastructure that works as well if not better than what you can get from any of the vendors who are focused on Enterprise sales at a fraction of the cost -- and I can do it without the proprietary "lock-in" that would result from using Microsoft.

Don't even get me started on the galaxy of Perl and PHP modules that are available as rpms, and how well integrated these are with Red Hat's Apache and MySQL (now finally at v5 after an excruciating number of years stuck in v3) builds.

Although I continue to believe that the price points for Red Hat subscriptions need adjustment in a downward direction, I recognize that this is an easy opinion for someone to have who isn't trying to keep up with the exponential increase in the demand for their services that Red Hat faces. Hopefully they will be able to properly balance profits with reinvestment so that the ultimate cost of a subscription will continue to reflect the real value provided to their customers.

ssh tunnel to your directory

2007-05-21T15:52:00.002-04:00

This is a little trick I learned out of self-preservation. While I have the utmost respect for my brothers in network security, there comes a time in every directory admin's life when the IT bureaucracy (the "hidden" 8th layer in the OSI Model) causes a collision between your need for access and change control lead times.

It is for such a time that I believe ssh tunneling was invented.

In a nutshell, if you can reach the remote box over ssh (TCP port 22), then you can set up a "tunnel" between your workstation and that remote box to connect with whatever port you want over there.

Here's the magic command,

ssh -L 1389:remotehostname:389 remoteuserid@remotehostname

Basically, that's it. Now you can connect to the remote directory server with something like,

ldapsearch -x -h localhost -p 1389 -b "" -s base "objectclass=*"

What you've done is assign port 1389 on your workstation as your local end of the tunnel that connects to port 389 on the remote box.

This works with most ssh setups because by default sshd, the ssh service daemon, allows TCP port forwarding. That can be changed by a particularly, well, security-conscious, security admin, but rarely is.

Fedora Directory init script

2007-05-04T12:50:00.000-04:00

Here's a very basic example, ripped off from Red Hat's script for OpenLDAP. After creating this in /etc/init.d (I usually call the file "nsslapd"), add and enable it using chkconfig.


#!/bin/bash
# processname: ns-slapd
# nsslapd  Start and stop Fedora Directory
# chkconfig: - 38 62
# description: Script to control Fedora Directory server
# Derived from /etc/init.d/ldap, Copyright by Red Hat, Inc.

DSHOME=/u01/app/sun/directory52
HOST=myhost
function start() {
        # Start daemons
        $DSHOME/slapd-$HOST/start-slapd
        $DSHOME/start-admin

}

function stop() {
        # Stop daemons.
        $DSHOME/stop-admin
        $DSHOME/slapd-$HOST/stop-slapd
}
# See how we were called.
case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    restart)
        stop
        start
        ;;
esac



Oracle E-Business Suite 11i Links
2007-04-26T02:29:00.000-04:00
Yeah, I know this isn't directly LDAP-related -- but sometimes it helps to know something about the apps your ID management infrastructure is protecting.

Anyway, I'm now well into the design phase of my company's ERP implementation (or should I say, (plural) implementations, which currently assumes we'll be going with 11i apps.

Here are some links to stuff that looks like it will be useful going forward:

Become an Oracle Apps DBA
Oracle Apps Blog
Shayam the Apps DBA
Oracle Applications DBA Blog

... and let's not forget the official Oracle doc:

Oracle 11i Virtual Documentation Library.


init script for Oracle infrastructure
2007-04-24T13:52:00.000-04:00
I went crazy the other day looking for this, sure that I'd posted it somewhere. I was wrong, I had not. This script should be copied into /etc/init.d on a Red Hat Enterprise/CentOS system, given execute permissions and then enabled using chkconfig. This is a companion to another script I did that starts/stops the underlying database (this script still needs some code to test that the DB is live before it tries to start everything up, I'm working on that...).


#!/bin/bash
#
# chkconfig: 29 99 1
# description: init script to start/stop oracle AS 10g
#
#
# match these values to your environment:
export ORACLE_HOME=/u01/app/orainfra/product/10.1.4/inft
export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/opmn/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$LD_LIBRARY_PATH
export JAVA_HOME=$ORACLE_HOME/jdk
export ORACLE_SID=inft
export ORACLE_USER=orainfra

# see how we are called:
case $1 in
    start)
    echo "Starting Oracle AS ..."
    su - ORACLE_USER << EOS
    opmnctl startall
    emctl start iasconsole

EOS
    ;;

    stop)
    echo "Stopping Oracle AS ..."
    su - $ORACLE_USER << EOS
    emctl stop iasconsole
    opmnctl stopall

EOS
    ;;

    *)
    echo "Usage: $0 {start|stop}"
    ;;
esac



Using an Existing Database for Oracle IDM
2007-04-20T12:21:00.000-04:00
Oracle Identity Management can use an existing Oracle 10g database for it's Metadata Repository.

The instructions are in the Oracle Application Server Metadata Repository Creation Assistant User's Guide.


Oracle Identity Management Resource Library
2007-04-09T15:50:00.000-04:00
The Oracle Identity Management Resource Library has some very interesting and useful stuff that warrants attention. It's all, of course, geared to sell you Oracle product, and therefore is for the most part very "high level". Having said that, the issues and strategies presented should resonate with anyone who has struggled with the mechanics of IDM (there, another acronym -- acronyms are good, they usually lead to higher salaries, in direct proportion to the obscurity of the acronym).

Give a listen (while you work, the slides can safely be ignored) to some of the webcasts, especially the customer discussion panel. If your management is impressed by such things, you might also want to grab the latest Gartner "Magic Quadrant" and some of the other analysts reports there (take what you read there with a grain of salt, most of these "experts" wouldn't know a code 27 from a code 28 to save their own lives -- which explains why many commercial products don't either).


Managing LDAP book list
2007-03-29T11:00:00.000-04:00
I've got a Listmania  list on Amazon called "Managing LDAP", my picks for books that every directory admin should have on their bookshelf.


Referential Integrity
2007-03-26T16:50:00.000-04:00
Referential integrity is one of those essential functions that every directory needs, but on whose implementation there is no agreement among directory product vendors. The major use of referential integrity is to make sure that the user dns listed in groups actually exist on the directory. Without referential integrity a user could be deleted but the reference to their entry in a particular group (say, "cn=Administrators"), would not. This would be characterized as "A BAD THING (TM)" by most directory admins.

Recognizing the need for such a feature, Netscape included a "referential integrity postoperation plugin" in all of it's directory products from at least iPlanet 4. This plugin has survived to this day in both the latest Sun and Red Hat/Fedora Directory products (see the Red Hat Directory Administrator's Guide). It is disabled by default, but preconfigured to check on uniquemember and member. It can be enabled and re-configured through the GUI directory console or by editing dse.ldif under the directory server root. Once enabled, the plugin is invoked whenever an entry is deleted from the directory. You do take a performance hit with this feature turned on, which is probably the reason for shipping it disabled (you wouldn't want unnecessarily hurt benchmark performance, never mind how unreal-world the shipping configuration is -- kudos to the Fedora Directory team for finally ridding their product of the allidsthreshhold).

Oracle also provides a referential integrity solution. The instructions for enabling and configuring it are contained in the Oracle   Internet Directory Administrator's Guide. The procedure detailed there steps you through the process of editing and compiling a Java object and modifying directory entries to enable the feature. It then shows how to edit a script file and run it against the underlying Oracle database. Finally, the manual recommends running the process at frequent intervals via cron or some other scheduler.

Comparing this to the Netscape/Red Hat/Fedora facility, the first word that came to mind was:

P-R-I-M-A-T-I-V-E

and really illustrates the difference in between the two product architectures. The Netscape family directories were optimized over time for use in building flexible directory services, built by directory developers and admins for directory developers and admins. The roots of the Oracle product are clearly in Oracle's relational (object) database product. At almost every turn you can see this legacy. Having to compile some java code, edit a script file and run a PL-SQL command to enforce referential integrity is just one of the more obvious examples.

Of course, to be fair, at least Oracle provides a solution. OpenLDAP doesn't. Referential integrity isn't even on their roadmap. Microsoft actually seems to do the best at this, referential integrity for things like groups is integral to the base product (and is probably one reason it doesn't perform as well as the default Netscape or Oracle configurations on delete and write operations).


Checking Active Directory for Duplicate E-Mail Addresses
2007-03-21T16:56:00.000-04:00
Here's an application using Net::LDAP's module for Simple Paged Results Control.

Anyone whose environment depends on Microsoft Exchange for messaging knows what a pain duplicate e-mail addresses can be. Following is a script I wrote to walk the Active Directory tree and do a quick lookup for duplicate addresses. It's kind of raw, but it works. Uses a config file for host name, bind dn and password. Iterates through an array of the target containers specified. In this particular case we've got a separate container for what used to be known as "custom recipients", users who don't log into the domain but whose e-mail addresses we want to be searchable in the Exchange GAL. The script only reports on exact matches on the value of the 'mail' attribute in entries other than the current subject.


#!perl
# Check Active Directory for duplicate e-mail addresses.
# Created 03/12/07 by P Lembo

use strict;
use Net::LDAP;
use Net::LDAP::Entry;
use Net::LDAP::Control::Paged;
use Net::LDAP::Constant qw( LDAP_CONTROL_PAGED );


our($adHost,$adUsr,$adPass);
require "c:/apps/etc/app.conf";
my $errfile = "../addupchk.log";
open LOGZ, ">$errfile" or die $!; 

my $count=0;
my $found=0;

my $contactbase = "ou=addressbook,dc=mydomain,dc=mycompany,dc=com";
my $adroot = "dc=mycdomain,dc=mycompany,dc=com";
my @allbases = ( "ou=users,dc=mydomain,dc=mycompany,dc=com",
     "ou=others,dc=mydomain,dc=mycompany,dc=com",
     "ou=addressbook,dc=mydomain,dc=mycompany,dc=com",
     );
     
my $query = "(mail=*)";
my @attrs = qw(mail displayname cn);
my $timestamp = localtime();
print LOGZ "$timestamp\tBegin Checking AD for duplicate e-mails\n";

my $ldap = Net::LDAP->new($adHost) or die $!;
my $mesg = $ldap->bind($adUsr, password =>$adPass);

foreach my $base(@allbases) {

 my $page = Net::LDAP::Control::Paged->new( size => 1000 ) or die $!;
 my @args = ( base => $base,
           scope => 'sub',
           filter => $query,
           attrs => \@attrs,
           control => [ $page ],
      );
   
 my $cookie;
 
 while (1) {
 
  $mesg = $ldap->search ( @args ) or die $!;
   while (my $entry = $mesg->shift_entry()) {
      my $entrydn = $entry->dn();
      my $mail = $entry->get_value('mail');
      my $displayname = $entry->get_value('displayname');
      my $cn = $entry->get_value('cn');
  
      if($mail !~ /.+\@.+/g) {
      print "$entrydn not mail-enabled\n";
      }
      else {
         my @hits = dup_search($entrydn,$mail,$displayname,$cn);
         foreach my $hit(@hits) {
             if($hit =~ /$entrydn/) {
             # Same entry, move on
             }
             else {
                 print "\t$displayname $entrydn mail same as\n";
                 print "\t$hit\n\n";
                 print LOGZ "\t$displayname $entrydn mail duplicate found\n";
                 print LOGZ "\t$hit should be deleted from contacts\n\n";
   
                 $count++;
             }
        }
      } # else
  } # while
  my ($resp) = $mesg->control(LDAP_CONTROL_PAGED) or last;
  $cookie = $resp->cookie or last;
  $page->cookie($cookie);
 } # while (1)
 
 if ($cookie) {
  $page->cookie($cookie);
  $page->size(0);
  $ldap->search( @args );
 }
} # foreach
close FH;
$ldap->unbind;

$timestamp = localtime();
print "Number of hits: $count\n";
print LOGZ "$timestamp\tChecking completed\n";
close LOGZ;

sub dup_search {

 my $addn = @_[0];
 my $admail = @_[1];
 my $displayname = @_[2];
 my $cn = @_[3];
 print "$addn $admail\n";
 my @matchdns;
 my $nldap = Net::LDAP->new($adHost) or die $!;
 my $nmesg = $nldap->bind($adUsr, password =>$adPass);
 my $nquery = "(mail=$admail)";
 my @nattrs = qw(mail displayname cn);
 my $npage = Net::LDAP::Control::Paged->new( size => 1000 ) or die $!;
 
 my @nargs = ( base => $contactbase,
         scope => 'sub',
         filter => $nquery,
         attrs => \@nattrs,
         control => [ $npage ],
     );
 
 my $ncookie;
 
 while (1) {
  $nmesg = $nldap->search ( @nargs ) or die $!;
   while (my $nentry = $nmesg->shift_entry()) {
   my $nentrydn = $nentry->dn();
   my $ndisplayname = $nentry->get_value('displayname');
   my $acn = $nentry->get_value('cn');
   push @matchdns, $nentrydn;
  }
  my ($nresp) = $nmesg->control(LDAP_CONTROL_PAGED) or last;
  $ncookie = $nresp->cookie or last;
  $npage->cookie($ncookie);
 }
 if ($ncookie) {
   $npage->cookie($ncookie);
   $npage->size(0);
   $nldap->search( @nargs );
 }
  return @matchdns;
  $nldap->unbind;
 }

__END__;



DST Problem on OIM Release 3 Install
2007-03-21T15:50:00.000-04:00
Last couple of days was frustrated in trying to install Oracle Identity Management Release 3 in a VM on my work machine. Since at the OS level it's identical to my home system, I figured there'd be no problem following my expert install procedure (written during my last install on March 9 of this year).

Ha! Did 3 abortive installs, with a trip home in between to re-download and re-burn the CDs (and copy them from disk mounted inside the VM -- I try to be kind to my fellow employees in the bandwidth poor sales branch I work out of), all with the result that dbconsole failed to come up during setup. On the fourth attempt I let the installer complete and then tried starting dbconsole. Clocked.

Damn. DST. Freaking H.R. 6, the "Energy Policy Act of 2005". Signed into law on August 8, 2005. Thank you SO much 109th Congress for wasting my time and my employer's money by changing a 20 year old standard that's been burned into every damn computer system on the planet.

So I shut everything down, grabbed patches 5865568 and 5632264 (READ THE DIRECTIONS!), did the "opatch apply" thing for each and, of course, everything now works.

I'm still waiting for someone to explain to me why it took almost every commercial software vendor two freaking years to come out with patches incorporating the shift from April 1 to March 11 into their products, and why any of them still have bits for download -- especially distribution disks -- that don't include it.

Morons.

For those of you out there who haven't been paying attention, below I've spelled out what the 2007 dates for DST would have been here in the U.S., and what the law now requires.

Would be: April 1 - October 28
Is now: March 11 - November 4

So if anyone out there thought they were clever and took the short cut of just manually advancing their clocks, they're going to be in for a surprise on April 1. And on October 28.

You have been warned.


Simple Paged Results with Net::LDAP
2007-03-13T17:24:00.000-04:00
Not sure if I've posted on this before, but I wanted to capture it now. Useful for doing big searches against an Active Directory (default restriction on search results returned is 200). Using Net::LDAP.


#!/usr/bin/perl
use Net::LDAP;
use Net::LDAP::Entry;
use Net::LDAP::Control::Paged;
use Net::LDAP::Constant qw( LDAP_CONTROL_PAGED );

my $adHost = "detroit.domain.com"
my $adUsr = "cn=ldapadmin,cn=users,dc=company,dc=domain,dc=com";
my $adPass = "1234xyz";

my $base = "cn=users,dc=company,dc=domain,dc=com";
my $query = "(mail=*)";
my @attrs = qw(cn displayname mail);

my $ldap = Net::LDAP->new($adHost) or die $!;
my $mesg = $ldap->bind($adUsr, password =>$adPass);

my $page = Net::LDAP::Control::Paged->new( size => 1000 ) or die $!;
 
my @args = ( base => $base,
             scope => 'sub',
             filter => $query,
             attrs => \@attrs,
             control => [ $page ],
           );
   
my $cookie;
 
while (1) {
 
     $mesg = $ldap->search ( @args ) or die $!;
 
     while (my $entry = $mesg->shift_entry()) {
 
         my $entrydn = $entry->dn();
         my $mail = $entry->get_value('mail');
         my $displayname = $entry->get_value('displayname');
         my $cn = $entry->get_value('cn');

         print "\"$displayname\",\"$mail\"\n";

     } # while
  
  
 my ($resp) = $mesg->control(LDAP_CONTROL_PAGED) or last;
 $cookie = $resp->cookie or last;
 $page->cookie($cookie);
  
} # while (1)
 
if ($cookie) {
 $page->cookie($cookie);
 $page->size(0);
 $ldap->search( @args );
}

$ldap->unbind;



Installing Oracle OIM_OIF 10g R3
2007-03-10T00:45:00.000-05:00
... otherwise known as Oracle Identity Management 10g, Release 3 (10.1.4.0.1), which is a mouthful, even for an Oracle product.

Note: This article will be updated from time to time as I learn more about how these Oracle products fit together.

Previous versions of 10g Application Server included the Identity Management Infrastructure. With Release 3, it gets split off into it's own distribution on 2 separate CD's.

There are lots of little improvements that go into the new release, not the least of which is the update to database 10.1.0.5 and Application Server 10.1.2.0.2 that relieves you of the time-consuming task of patching up to these versions.

Installing is pretty straightforward. If you use CentOS 4 as I do, you'll need to modify the /etc/redhat-release file by deleting the line referring to CentOS and substituting "Red Hat Enterprise Linux AS release 4" so that the installer will not abort because your system isn't certified.

Assuming my install will be called "infra1", this is how I would lay out the filesystem:

Oracle Base /u01/app/orainfra
Oracle Inventory /u01/app/orainfra/oraInventory
Oracle Home /u01/app/orainfra/product/10.1.4
Database files /u02/oradata/infra1

(the db instance was named "infra1", which is also the SID)

Make sure to have root privileges on the server. There are a couple of scripts that need to be run as root during the install.

There may be kernel parameter changes that need to be done. These can be done dynamically using the sysctl utility, without rebooting the system. The Quick Install Guide has  a nice section on how to do this under "Kernel Parameter Settings for OracleAS Metadata Repository".

Thanks to the Energy Policy Act of 2005, every system on the planet needs to be patched to account for DST coming early this year, March 11 to be exact.

Oracle is no exception, and even though it was only release a little while ago, Identity Manager R3 also requires patching.

There are two basic patches that are needed, both only available to customers with support contracts, via MetaLink:
5865568, updates the Oracle JVM used by the App Server with the new timezone data.

5632264, replaces the old timezone data for the database and database clients.

Ah, let the games begin!


Using round robin DNS to load balance LDAP
2007-02-11T09:50:00.000-05:00
This is so simple in BIND 9, it's scary. Don't let anyone tell you it can't/shouldn't be done. Remmember, if LDAP is unavailable because the host your apps are hardcoded to point at is down -- it's your rear end.

Here's how I do it (at home, I have NO control over DNS at work -- their loss).

Add an additional interface on each LDAP box to dedicate to the round robin process. If you've got your LDAP server tied down to a single interface, add this new interface to the mix (or open LDAP up to listen on all interfaces -- the default for Sun and Red Hat Directory Server).

Now add a new A record in your forward zone file for each of these two new interfaces, giving them the same host name. On my home network it looks something like this:

ldap A 192.168.2.101
ldap A 192.168.2.102

Finally, update the serial number on your forward zone file and reload the config. For Red Hat/CentOS/Fedora you can do a

/sbin/service named reload.

That's all there is to it. The default behavior with BIND 9 will be to give a different address as the first in the list returned each time. As a result you'll see:


[root @bigserver ~] nslookup ldap
Server:         192.168.0.112
Address:        192.168.0.112#53

Name:   ldap.mydomain.com
Address: 192.168.0.101
Name:   ldap.mydomain.com
Address: 192.168.0.102

the first time and


[root @bigserver ~] nslookup ldap
Server:         192.168.0.112
Address:        192.168.0.112#53

Name:   ldap.mydomain.com
Address: 192.168.0.102
Name:   ldap.mydomain.com
Address: 192.168.0.101

on the second try. Your client will use the first address provided each time.

Now you have no excuse.

That is all.

Installing Fedora Directory eldapo's way

2007-02-07T10:15:00.000-05:00

A colleague (the infamous Gary Utz, "Worlds Greatest SysAdmin") has been asked to install Fedora Directory on a Red Hat Enterprise 3 VMWare guest, and needs some guidance on how to approach it. Since he's been my lifeline in more tight spots than I can remmember, there wasn't any question that I would help.

So here, for both him and my vast Internet audience (both of you), is my own take on how to install Fedora Directory on Red Hat Enterprise Linux.

Note to The Experienced
First, if you happen to be an experienced Netscape/Sun Directory admin, keep in mind that the Red Hat team has been through 4 minor version updates since the purchase of Netscape Directory 6. There are differences in the Red Hat product that you should keep an eye out for.

System Minimums
Minimum system requirements to install and run Fedora Directory are simple. First, although you can try to compile from source for other platforms, at the moment binary packages are only available in rpm format for Red Hat Enterprise and Fedora Linux. As a RHEL clone, CentOS works fine too and is highly recommended at least for test environments (it's what I use in my home lab). You'll need at least 1 Gb RAM and 1.5 Gb swap. CPU should be above 1 GHz. Dual core, or dual CPUs, are better than single core. Like all LDAP servers, FDS is a cycle hog. The software itself needs about 300 Mb of space when it first installs, last time I checked, but I'd give it at least 1 Gb because once you start running you're going to need it for the database files.

Get the Software
To begin, you need to download the appropriate rpm from Fedora Directory Project Download. The latest as of this writing is v1.0.4, and is available for every Red Hat platform starting with Fedora Core 2/RHEL 3 and up to Fedora Core 6. Starting with Fedora Core 4 there is also a version for x86_64. For Red Hat Enterprise 3, you need to use the Fedora Core 2/RHEL 3 version, for Red Hat Enterprise 4 the Fedora 3/RHEL4 version and so on.

These rpms all install to /opt. Sorry, but that's how it is. The next version (1.0.5?) may change that to an install into the base filesystem in accordance with the latest "Filesystem Hierarchy Standard" specification. For more, see the the notes on FHS Packaging. For now, DO NOT try to change this by recompiling with the srpm. It won't work. Believe me, I've tried.

Installing the RPM
Before installing the rpm, you'll need to check on a couple of major system dependencies and install the required software, these are:

Red Hat's version of the Apache web server (use Red Hat's rpm for httpd-server).

Sun's Java 1.4 or above (I use Sun's J2SE SDK 1.5 rpm for this, off java.sun.com, which installs to /usr/java/jdk1.5.0_x). You can use the alternatives system for configuring Java (see my article on how to do this).

Installing the rpm is as easy as doing a

rpm -Uvh fedora-ds-1.0.4-1.RHEL3.i386.opt.rpm

to install the latest version for Red Hat 3, for example.

This will put the setup binaries and utilities under /opt/fedora-ds.

Gather Further Requirements
Next, you need to go over to /opt/fedora-ds and execute the idsktune utility as root in order to determine what needs to be done in preparation for setup.

Pay close attention to what it reports. I would recommend executing like this, idsktune >idsktune.log so that you can scroll through the output in a text editor.

Take care of the package dependancies first. Most should be easily cured with up2date (or if you're blessed with installing on CentOS or Fedora Core, yum).

In addition to some additional software, you will probably also need to make some kernel configuration changes.

Everything I'm about to say on that, and more, is contained in an article on Peformance Tuning up on the Fedora Directory Wiki, which I consider essential reading.

At a miniumum you're going to want/need to:

Increase the number of local ports available with

echo "ipv4.ip_local_port_range = 1024 65000" >> /etc/sysctl.conf

and running /sbin/sysctl -p to effect the change.

Make the following changes to the number of available files and file descriptors available:

echo "fs.file-max = 64000" >> /etc/sysctl.conf

echo "* soft nofile 8192" >> /etc/security/limits.conf

echo "* hard nofile 8192" >> /etc/security/limits.conf

Effect these changes by a sysctl -p and a ulimit -n 8192.

Running Setup
Do this with a /opt/fedora-ds/setup/setup as root.

Before getting into the details of how to respond to the prompts, here are some quick design pointers that may be of some help in getting the overall picture:

Red Hat Directory, like it's predecessors from Netscape and Sun, consists of two parts, an Administration Server and at least one Directory Server. The Administration Server provides access to the Directory environment via the Directory Console, a gui managment utility that connect over http on a custom port. It stores its configuration in the first Directory installed. Red Hat has modified the code so that the Admin Server now uses the system httpd binary and system Java.

I used to set up the initial Directory instance on a non-standard port and make it the configuration directory for that machine. This allowed me to start, stop, delete and create additional user and application directory instances using the gui console, which can be a real time saver for a neophyte. In the interests of simplicity I now put the initial instance on port 389 and let it serve double-duty as my primary user directory.

For clarity I also used a function-based directory instance naming scheme, "slapd-[hostname]-admin" for the Administration Server directory and "slapd-[hostname]-user", for first Master user directory if these are separate. This is alot more descriptive than the "slapd-[hostname]1", the default. Where there's only one directory instance on the box (which is how I usually set things up nowadays), I just leave off the "-admin" or "-user" qualifier.

Apart from the above, the defaults presented by the installer are actually pretty reasonable.

Step-by-Step
The Fedora Directory installer is still (thank God) curses based. That means you don't need to fire up X to use it, and the install proceeds pretty briskly along without the usual hesitation (and smearing) that those of us who have to install the latest Sun and Oracle directories have to deal with (there is actually a way to do both of the latter from the command line, but the interfaces are ... clumsy).

The Red Hat Installation Guide is still a pretty good reference for this part of the job. As I said above, I used to set up separate administration and user directories on each server, to provide greater flexibility in managing the directory environment with the GUI Directory Console. Two important drawbacks to this approach were that it required the Console user have an account on the administration directory or use Directory Mananger, and, that it forces you to use a separate Console session for each directory server (since every directory server has it's own unique admin server). In my old age I now prefer a really simple, single, directory environment.

The installer will begin by asking you to accept the license. Answer yes.

Select "install mode" 2 - Typical.

Accept the hostname presented, unless none is found -- in which case quick enter something in /etc/hosts and use that (the directory constantly checks for the host name where it lives).

Server user and group to use. I usually accept "nobody".

Accept the defaults for Directory Manager dn, enter a password. Do the same for the Administration Server admin.

Set the directory suffix, this should be the Internet domain name for your host, unless you want it to be something different. I usually accept the default (which is the domain name split up like "dc=blogger,dc=com").

When asked about the configuration directory, use the default -- you want a new one on this machine (unless your enterprise architecture calls for a central config directory -- not a good idea in my opinion, but to each his own).

Port numbers. For the first directory (which will be your main directory for both config and users, if you follow my practice), use standard LDAP port 389. For the Admin Server, again pick something high that won't interfere with other stuff. I like something in the 3000's (usually port 3389).

For the user directory location, I always choose the default -- a new one on this machine.

Administration domain: default (usually the domain of the machine you're installing to, unless you've got an extensive and complicated architecture calling for multiple admin domains -- God help you).

At some point you'll get through all of this and setup will finally start the directory and admin servers.

To set up additional directories the best procedure is to fire up the gui Directory Console in an X session, /opt/fedora-ds/startconsole and connect to the Admin Server using it's unique URL (e.g. http://hostname.domain.com:port), which should have been echoed to the screen when the Admin Server first came up.

For good documentation on how to actually create and manage additional directories, look at the Red Hat Administrator's Guide and the extensive How-Tos on the Directory Project site.

Don't forget to create and install an init script to automatically start your environment. A sample is here.

So there you go. That's all there is to it!

Stay tuned for my next post, where I'll cover the dirty business of creating a new user directory from scratch from existing data, including tweaking some important performance settings and pre-loading indexing and access control instructions.

Fedora Directory Revisited

2006-12-16T10:21:00.000-05:00

I've spent the last few months immersed in Oracle's identity management products as part of my preparation for working on a team bringing Oracle ERP to my company. These products are surprisingly good, particularly the core Oracle Internet Directory (OID). OID is basically an LDAP listener for an Oracle database. I've been quite impressed with it's performance and features (for example it supports both Simple Paged Results Control and Server Side Sort). Of course, like all directories except those from the Netscape lineage (Sun and Red Hat's in the current market), it failed my super-secret stress test -- predictably crashing when I did a ridiculously large bulk load imto it. That's not Oracle's fault, it's just a consequence of inefficiencies in the LDAP protocol.

In the little spare time I've had during the last week I was able to install and begin working with the latest release of Fedora Directory Server. I've been considering moving my home environments over to FDS for some time now. While reluctant to give up the simplicity and rock-solid stability (well, most of the time) of OpenLDAP, going to full immersion mode with FDS has been a long time coming. One of the differences between this conversion and the brief forays of the past is that I'm pairing it with a working deployment of JBoss.

While I try to keep an open mind about the appropriate uses of closed source software, I continue to believe that open source development provides better, albeit sometimes imperfectly executed, results. The open-sourcing of the former Netscape Directory codebase was a major, revolutionary, step for the identity management business. Warts and all, that codebase has consistently delivered superior results during it's lifetime. The Fedora team has made some significant improvements. One of my favorites was replacing the old Netscape web server code for the Administration server with Apache.

Because of this continuing effort to improve the codebase I am much more comfortable with plans by the developers to more tightly integrate FDS with Fedora Core, and presumably, Red Hat Enterprise Linux. While the OpenLDAP SDK and tools have served the community well, there's no reason FDS can't be positioned to provide a high quality alternative. From a system administrator's perspective, OpenLDAP's ubiquitous libraries and tools have provided a kind of standard that should not be taken for granted. As we've seen with other open source projects, the ultimate solution might be a convergence of the code where OpenLDAP and FDS adopt the best features from each others source.

Perhaps the greatest contribution by the FDS developers so far has been the additional documentation they've published. The Documentation Wiki provides information and techniques that I've found helpful even in my company's Sun Directory environment. I'm looking forward to using it to build my own reference environment in the home lab.

Look for this blog to me a little more active from this point forward, as I explore FDS in more depth and note my progress here. Hopefully

New LDAP Stuff and Musings on System Integration

2006-05-21T15:48:00.000-04:00

I've got some new articles posted over on lembobrothers.com under my personal home page. Two of them deal with using the Net::LDAP perl module to connect to LDAP directories over TLS or SSL. The latter includes a script I wrote to connect to an SSL enabled Active Directory. Speaking of Active Directory, I've also got a short piece on SSL Enabling Active Directory and another on forcing AD passwords using Net::LDAP from a Unix box.

My work in directory services continues to be more and more concentrated on integration with other systems, particularly COTS (Commercial Off-The-Shelf) software. Contrary to what most CIO's seem to believe, dropping COTS (also called "packaged software") into an environment can cost as much in integration expense as writing something in house from the ground up, A big difference between the two approaches is that closed-source COTS software can continue to generate unanticipated expense when patches or upgrades "break" existing integration solutions. Fear of these kinds of hard costs will often lead internal project teams to shy away from promising any level of integration at all, resulting in a net loss of functionality for users. To be honest, I don't blame internal IT. The fact is that many -- no, most -- integration solutions from commercial vendors are "Rube Goldberg" affairs whose fragility make it a sure bet that eventually there will be a "crash and burn".

That's too bad. Open source software and community support tools like wikis, forums and plain old cvs could revolutionize how IT gets the job done, and lead to huge improvements in both the quantity and quality of what the end user receives.

An Admin's Life

2006-04-03T21:03:00.000-04:00

As I write this one of my colleagues is still at work, waiting for a very important group of MS Exchange accounts in a particular Active Directory container to get rebuilt. The problem started earlier today when I did a quick mod to a production script that we thought would clean up some cruft in that container. Instead it (quite predictably, in hindsight) whacked all the user accounts in the container, about 100 in all. After a couple of hours a strategy was agreed upon to repair the damage, and the process begun. So here we sit and wait. My friend, who is the lead Exchange admin, in front of a console at work and me, here in my basement waiting for the phone to ring.

Everyone who's ever worked as a sysadmin knows these kinds of humbling moments. The good part is that you tend to find out who your friends really are. While it doesn't quite erase the sense of defeat and hopelessness that surround such situations, it is nice to see the team at work together on a problem. On that note I count myself privileged to be part of this team. Although alot has changed over the last 8 years, the one thing that hasn't is the sometimes overwhelming generosity and kindness that sets my co-workers here apart from those in other places I've worked.

Where OpenLDAP Fits

2006-03-25T08:46:00.000-05:00

With all the buzz about Sun and Red Hat's latest offerings, OpenLDAP has not received much attention lately. There has in fact been a steady improvement in the product, and it's had significant penetration into the enterprise over the last two years. The most significant commercial deployment has been at HP, where Katik Subbrao & Co. have made Symas's Connexitor Directory the core of their company's Identity Management infrastructure. The nice thing for the OpenLDAP community is that all of the new functionality built into Connexitor for HP can eventually wind up as part of OpenLDAP, as Symas is a major contributor to the project.

Although its nice to see OpenLDAP getting such a big boost over at a big company like HP, I see it being a strong contender in small to medium sized environments. As shipped with Red Hat Enterprise Linux 4, OpenLDAP is a reliable, seamlessly integrated and well-documented operating system service that can serve as the basis for a no frills authentication and white pages directory solution. When combined with Kerberos, it can provide a secure, standards-based identity management infrastructure. In an alternate universe where vendor and consultancy hype (as well as the corporate IT penchant for consuming licenses as easily as my co-workers do pretzels) I could easily see an enterprise the size of my own company (approximately 10,000 users) easily satisfying its identity management requirements with this software stack.

Linux Journal only recently completed yet another series focused on deploying OpenLDAP and Kerberos as an Identity Management solution. These articles got me thinking again about all the opportunities this software provides. One of my personal projects for the balance of this year will involve exploring how the latest syncrepl "pull" based replication can provide the basis for a highly available service infrastructure.

OpenLDAP is such a good fit in so many situations that I hope Red Hat will continue to ship it as an integrated part of their OS, and not deprecate it in favor of their more complex Fedora/Red Hat/Netcape Directory product. If it were any other software vendor, I'd say the odds would be against it -- but we're talking about Red Hat here, the company that bowed to pressure from its customers to finally start shipping more current MySQL packages (previously Red Hat had only shipped packages based on a legacy version), and so I'm pretty sure that OpenLDAP will continue to be an optional part of their shipping product.

Forking with Sun - Almost

2006-03-25T07:46:00.000-05:00

With their latest version releases, the Netscape family directory products of Sun and Red Hat have finally forked significantly enough to make seamless interoperability a thing of the past. The different algorithms used for replication now make it impossible for these latest directories to replicate with each other. This was to be expected. Intellectual property law concerns really demanded it, especially if Sun's version was to remain a proprietary product.

At my company we have opted to remain with Sun, primarily because by doing so we can avoid the steep initial cost of purchasing Red Hat's commercially supported product (I opted not to push for implementation of the free Fedora Directory product sans support, which would have been too radical a step for them). I've already deployed the latest release of Sun Java Systems Directory (v5.2) in development, and hope to move quality and production over to it in good order. The demand for high availability has finally forced us to adopt a multi-master model, which is not supported by OpenLDAP, and the need to simplify sign-on has led us to begin testing Sun's Identity Synchronization for Windows, for which their directory product (among other things) is a dependancy. Red Hat/Fedora could have satisfied both these requirements with their slightly different solutions, but unfortunately the startup cost of licensing the necessary instances was prohibitive. As a long-time Sun customer our cost to implement Sun's solution was ... $0.

I don't see this as necessarily a loss for Red Hat. It may very well be that Red Hat does not have the resources to do this yet, or that they aren't yet convinced that directory services should be part of their core business. There's nothing wrong with that. Deployment of directory services has become just one step in building an Identity Managemnt infratructure, and Red Hat does not really have its own IM offering. The big players in that space are CA (with the former Netegrity SiteMinder product), Sun (with its Java Systems Identity Management Suite), Oracle and Microsoft. Of these, only CA and Sun's offerings are real end-to-end solutions, and I would argue that Sun is the more complete of these two.

Of course we run the CA solution in my work environment -- which at the time seemed to be the way to go (Sun's entire product line was in serious flux at the time). Time will tell whether any other vendor will be able to make a dent in this market. I doubt that Red Hat will try. Their best strategy is to continue concentrating on their core compentency in providing the best server OS platform available (not that I don't expect some movement toward a compelling desktop product soon -- more on that in my other blog).

The upshot of all of this is that now that we've forked with the Sun product, there's really no easy was to turn back. For all intents and purposes my company won't be exploring deployment of Red Hat Directory going forward. Barring the collapse of Sun, or its abandonment of its own directory, we're now stuck with them. There is the possibility that Sun will participate in the Fedora Directory project and contribute their own changes to it. I don't see that happening in the near term, although I think it would be a huge benefit to the entire OSS community.

Fedora Directory Replica of Sun Directory Master

2005-06-04T00:57:00.000-04:00

I've already posted this to the Fedora Directory User List. After downloading the Fedora Directory rpms to a Red Hat Enterprise 3 on HP Vectra desktop yesterday, I did a quick install of the server as a consumer directory. I then went up to my company development LDAP master, which runs Sun One Directory 5.1 SP2 on Solaris/Sparc (an E220, I think), and set up a replication agreement to push the directory out to the new Fedora consumer. That was it. Within 25 minutes of running rpm -ivh *RHEL4*.rpm, I was able to make queries against the consumer and to verify that changes on the master were flowing out to it. Not bad.

Today I had a chance to play with this setup some more, and deploy the Fedora DS to a couple of Fedora Core 3 machines. All had already been preinstalled by me with Sun's Java SDK 1.4.2, configured through the alternatives system, and had their own little java.sh script in /etc/profiles.d to make the Java environment universal. About the only problem I've had is an inability to launch the Directory Console in a remote X session. While other Java apps run fine remotely, the DS Console is stubbornly refusing the play ball. I had similar problems with the Sun One 5.x Console though (don't remmember it with the 4.x one), so I'm not letting it get me down. At home I've configured my two test LDAP servers to use multi-master replication, and hope to be able to experiment some with the DSML Gateway and try my hand at some ad hoc Samba integration. At work the bosses are asking about the included Windows Password Sync, but I'm going to try to hold them up for a firm commitment on some new x86 hardware to migrate the present environment to Red Hat.

In the meantime, I've begun planning my "sabbatical", 8 weeks of paid leave that employees at my company get after 7 years of service. Our current plans will take us down to the North Carolina shore for a week, up inland and then back North through Virginia and Maryland before returning to Long Island. I've already told my bosses that I'm going to be leaving not only my pager behind (a sacred tradition for those on sabbatical), but also my company laptop. The kids were happy to hear that, that laptop is too often a rival for my attention. Over a decade ago I bailed out of the legal profession in part to be able to spend more time with my family. Although IT has been very good to us, sometimes it can be as jealous a mistress as the law ever was. Keeping the job in its proper place is one of those constant battles, like mosquito control, you just have to keep plugging away at.

Red Hat Directory Server Announced

2005-06-02T02:08:00.000-04:00

Not really news at this point, but today Red Hat announced the open sourcing of the former Netscape Directory Server as the "Red Hat Directory Server". Like it's Enterprise Linux product, the directory would be available on a subscription basis. Further development will be conducted within a new Directory Project within the Fedora Project. I knew something was up when I stumbled across the newly rebranded Red Hat product documentation on their site last week. How big a deal is this? Well, last I checked (about 1 minute ago) the new Fedora Directory site is not responding, obviously as the result of being hammered by a multitude of engineers like myself who are looking the long-awaited source code for this directory product. Will this sideline OpenLDAP? I certainly hope not. That open source rival has improved so much in performance and reliability over the last 2 years that I and many others think it's more than ready for the enterprise. Benchmarks released by Symas Corporation, show the overall performance of their commercial OpenLDAP based directory matching that of the Netscape product that Red Hat purchased. I intend to continue developing solutions with OpenLDAP, while at the same time taking the plunge into the new Fedora Directory as soon as I can get the code.

Red Hat Directory Coming Soon

2005-04-03T14:12:00.000-04:00

The press release in which Red Hat announced it's purchase of the Netscape Security Solutions products, including Directory Server, stated in part that:

"Red Hat plans to start marketing these products as part of its Open Source Architecture over the next 6 to 12 months."

Then in January InternetNews had an article quoting a Red Hat spokesperson that the company would by releasing the Directory source under the GPL in the next 12 months.

Chris Blizzard of Red Hat wrote about the subject in a January blog entry. After making reaffirming his employer's commitment to open source and to taking this particular code into the community, he went on to say:

"We'll be open sourcing components as we can. We've still got to talk to our customers and figure out what they would like to see. We need to try to talk to the larger community and try to come up with a structure that works to build a large, outward-facing and successful project. And we need to get the source code ready. None of these tasks are small. But we're working on them. Stay tuned."

Later, in a February entry, he expanded further on what was going on behind the scenes:

"For the Directory Server we've selected a license for the project and have started working on getting the source together for a release... From the strictly technical standpoint, we've got some large hurdles... The build system that it uses right now is pretty far from autoconf... the locations for a lot of the libraries that the Directory Server expects to build against are hard wired into the build system... I'm not sure if we'll be in a buildable state at the time the source is released, but I'm hoping that we can get there. It's just a long road."

Red Hat now hosts the documentation for the acquired Netscape products. In talking about the release of Red Hat Enterprise Linux 4 Server, one of the product team members noted that they were able to successfully install the Directory Server on it (something that can't be said for Sun's latest version of the same product). What this all means is that we'll probably see an early opening up of the source tree for the Directory very soon, with SRPMS and RPMS to follow awhile later.

Federated Directories

2005-01-16T10:43:00.000-05:00

The nice thing about finally getting some help in operating the LDAP environment is that I can restart some long dormant LDAP related research projects, like designing an enterprise configuration that uses OpenLDAP. One idea I have is to try using the meta backend to provide an aggregated view of our Active Directory and regional Notes directories. This would result in a "federated" directory infrastructure that gets administered at the local level. Basically, the meta backend would do all the translation for things like differing attribute names and tree structures so that the actual directory holding the data would not matter to the end user (or consumer application). It's going to take some time to get it right, but from what I've read in the OpenLDAP doc and on the mail lists it should be more than doable. The labor intensive part will be in creating the schema files to support all those different directories, since the native schemae are not in OpenLDAP schema format. The mapping part should not be too difficult, I've already documented most of it though, so its just a matter of making the time to get it done.

Life Beyond LDAP

2005-01-16T10:41:00.000-05:00

One big change that's been happening at work is I've finally gotten a designated "apprentice" to train to do my LDAP job when I go on sabbatical this summer (in keeping with the saying "two there are always, a master and his apprentice"). As a practical matter this means I should begin having time to do more non LDAP stuff as summer draws near. One of the areas I'm interested in pursuing is system security, and right now I'm looking around for some security training courses. The Red Hat class, emphasizing as it does hands on configuration, is a good possibility. I've been spending more time with the server consolidation team lately, yapping about our nascent Linux infrastructure, so there is a practical benefit to my employer. Another area I'm going to be spending more time on is source control for our internal admin applications like Perl and shell scripts. The new monitoring guy has put up a cvs tree and we're now getting all the sysadmins in the department to contribute their code, wherever it might happen to be. I'll be configuring a webcvs front end for it soon, to make it easier to browse the code base.

Forward with Fedora Core

2005-01-16T09:49:00.000-05:00

Been pretty busy through the New Year. One major change was migrating from FreeBSD to Fedora Core 3 as my home desktop and server O/S. There's just one FreeBSD box left, my 5.2.1 file server, that also does DNS and DHCP for the internal network in the basement downstairs. Now that I've finally got another reasonably big Fedora file server up (80 Gb HDD), I'll be moving the data over from the BSD box and reformatting it for either Fedora or Red Hat Enterprise Server 4 Beta. My guess is that Enterprise 4 will be in release by the time I get around to taking the RHCE again, so I might as well get up to speed on it.

Back to Red Hat

2004-12-22T02:05:00.000-05:00

Last week I took the Red Hat Certified Engineer "Crash Course", a five day marathon that resulted in my receiving the lesser Red Hat Certified Technician (RHCT) credential after the hardest test I've ever taken (including the NY Bar Exam). Given the time and pain invested in both the course and the test, I've decided to go the total immersion route with things Red Hat for awhile. That's already resulted in wiping the disks of my primary desktops at home and work, and installing Fedora Core 3 on them. I've also put up a Red Hat Enterprise Server 3 box at each location for further practice. For now, the main file server at home remains on FreeBSD 5.2.1, but will probably get reinstalled at a Red Hat box as well. My original foray into UNIX systems was on Red Hat Linux 5.1 (Manhatten), and I was a die hard devotee of Red Hat 6.2 (Apollo) long after most of the world moved on to the 2.4 kernel. The last Linux I ran before switching to BSD was Red Hat 7.3 (Valhalla), which would become the basis for Red Hat Enterprise 2.1 Advanced Server (I'm not counting the brief experience I had with the Marist College distribution of Linux for S/390 I installed as a POC for work).

One thing I learned a few months ago when I started working with Red Hat Enterprise 3 was that in many cases compiling from source yielded better results than using an rpm. This turned out to be especially true for multimedia apps on both Workstation and Fedora Core. Mplayer, an app I've grown fond of because of its uncanny ability to do everything that the Quicktime and Windows Media players can do, is a case in point. Another app I'll probably be building from source is OpenLDAP, since even the latest rpm was compiled in such a way that most of the "good stuff" is not enabled. My instructor for the RHCE class suggested I try building my own rpm for OpenLDAP, something I'm now researching as a way to make possible future maintenance easier.

Sun Directory 5.1 SP2 on FreeBSD 5.2.1

2004-11-11T09:46:00.000-05:00

Well, not quite. I have finally managed to get the Directory Console to install and run, though. This is a good thing on many levels, not the least of which is I finally can run it on a stable platform for day to day management tasks. Sure, it still goes all "smeary" sometimes, but, I've noticed that the GUI actually seems to snap back quicker than on Windows -- much like it does when running it on Solaris/Sparc.

The big challenge is still to get something working on Red Hat Enterprise 3 (maybe 4, by the time my company takes the plunge). So far I haven't had any luck getting the Sun Admin Server to install cleanly, although ns-slapd does work. Problem there is that directory operations are now so tied up with the GUI that the Admin Server is a must-have. I did get a response to my post on the Sun Developer Forum that led me to a patch, but have not yet had the time to test it.

My hope is that Red Hat itself gets an open sourced edition of the Netscape Directory server out the door sooner than later. Assuming it passed my internal quality testing, I'd be inclined to recommend that as a solution going forward. Not that I'll abandon experimenting with OpenLDAP, which I continue to believe will be the long term winner so long as they stay on the track they've been on (simple and truly lightweight, rather than complex and bloated).

LDAP Books

2004-10-30T11:26:00.000-04:00

Every once and awhile someone asks me to recommend a book or two about LDAP. Fortunately all of these requests so far have been from people looking for a good introduction to the topic, and not for something providing advanced techniques along the lines of Mastering Regular Expressions(O'Reilly) or the Perl Cookbook(O'Reilly). Introductory texts on LDAP abound, most follow the pattern of spending their first half explaining the protocol, its accompanying API, and giving a few examples. This is in stark contrast to the LDAP-derived Active Directory, which now can boast several advanced tomes, including my favorites, Robbie Allen's Active Directory Cookbook(O'Reilly) and his older Managing Enterprise Active Directory Services (Addison-Wesley).

About the best introductory book I've found is also one of the more recent, Gerry Carter's LDAP Administration(O'Reilly), which covers alot of ground for a book that's supposed to focus mostly on OpenLDAP (here's a secret, while OpenLDAP is used as an example LDAP directory through the book, Gerry actually provides a pretty balanced presentation of generic LDAP, including some space on Active Directory). For IT managers, the recently updated Understanding and Deploying LDAP Directory Services (Addison-Wesley) provides the best resource for architectural issues. Those managing a Sun Directory environment will find the Sun Blueprints series title LDAP in the Solaris Operating Environment somewhat helpful, although most of the material there appears to be a rehash of the manuals (which are actually pretty good as software manuals go, the Deployment Guide and Command Reference, are must-reads).

For a directory administrator like myself, unfortunately, there is currently no advanced book to help with the practical day to day problems of synchronizing (which is really just a function of "transforming" and then "diffing" entries) directories, writing connectors to non-LDAP systems and best practices for data management. While all the books currently out there provide examples of code using the Netscape SDKs and PerLDAP, these tools are now pretty long in the tooth, and don't have the price/performance punch of the current Sun Java/JNDI and the pure Perl Net::LDAP (a/k/a perl-ldap) offerings. To Gerry Carter's credit Net::LDAP gets good coverage in his book (as it does in Robbie Allen's older book, only marketing forces can explain why he went pure VBScript in his newer writings). For OpenLDAP administrators there still is no book with coverage of tuning the Berkeley DB backend, which as I've pointed out elsewhere on this blog is a critical task if that product is to be used in an enterprise environment. What is missing, and sorely needed, is an "LDAP Administrator's Cookbook" and a companion "Advanced LDAP Directory Management" to cover best practices.

I am, of course, open to any offers.

Using Spreadsheet::ParseExcel

2004-10-19T16:28:00.000-04:00

In my own experience as a directory administrator only about 25% of my time is spent managing the servers and software that support the environment. The lion's share of the effort in my job is managing the import and export of data through the directory environment, in most cases working with groups outside the IT department across the globe. Recently we had some weird results when a csv file with employee data from our offices in Brazil was imported into the directory. The short version of the story is that in several attributes (surname, locality, state), some of the characters used didn't quite match up with a known character set. Our assumption is that the data was originally created in Excel and then dumped to CSV format, probably using a localized copy of the program and the Portugese (Brazil) keyboard template. Among other things this got me going in a direction I'd avoided for a long time -- learning how to parse Excel spreadsheets with Perl.

My problem was that the doc for Spreadsheet::ParseExcel was opaque to me, as are most man pages on UNIX systems. Maybe it's just that I'm dense, or don't have enough patience. Anyway, after much experimentation, I finally "cracked the code" and was able to come up with the following little script to convert an Excel spreadsheet into a delimited text file.

#!/usr/bin/perl
# parse_excel.pl Read an Excel spreadsheet and print its contents to
# a bar delimited file
# Created 10/19/04 by P Lembo

use strict;
use Spreadsheet::ParseExcel;

my $HOME = "/u01/home/ldapcon";
my $xlsFile = "$HOME/data/admin/ArrowOIDRegistry.xls";
my $txtFile = "$HOME/data/admin/oid-registry.txt";

my $wkBook = Spreadsheet::ParseExcel::Workbook->Parse($xlsFile);

open FH, ">$txtFile" or die $!;

my $nuSheets = $wkBook->{SheetCount};
print "There are $nuSheets sheets in this workbook\n";

# Iterate through worksheets
for ( my $iSheet=0; $iSheet < $wkBook->{SheetCount}; $iSheet++ ) {

my $wkSheet = $wkBook->Worksheet($iSheet);
my $wksName = $wkSheet->{Name};

print "Name of worksheet is $wksName\n";

# Now iterate through a worksheet by row
for (my $iRow=0; $iRow < $wkSheet->{MaxRow}; $iRow++ ) {

my $colNu = $wkSheet->{MaxCol};

# Loop through the columns in the row
for (my $iCol=0; $iCol < $wkSheet->{MaxCol}; $iCol++ ) {

# Pull out the cell value
my $wksCell = $wkSheet->{Cells}[$iRow][$iCol];

# Print each value
print FH $wksCell->Value, if ($wksCell);

# Print a delimiter after each value except the last in the row
print FH "\|", if ($iCol < ($colNu - 1));

}

# Print a newline after each row
print FH "\n";

}
}

print FH "\n":

close FH;

__END__;

Mixing Metaphors: LDAP and ADSI at work

2004-10-13T23:06:00.000-04:00

I just finished writing a cgi to provide a barebones web based password reset facility for Active Directory as a backup to my groups "approved" COTS solution. As I noted in my personal blog, working with ADSI through Perl's' Win32::OLE module was a challenge. For now the code I'll be posting here will provide a standby capability for production. If I have time later on I'd like to eliminate the Win32::OLE pieces altogether and do everything with Net::LDAP. That won't be possible until SSL is enabled in our Active Directory environment, something that probably won't be entertained by the Windows engineering team until next year.

So, without further ado, here is the code:


#!perl -w
# adpassreset.cgi Reset Active Directory passwords
# 10/1/04 by P Lembo
# Give choice of unlocking, forcing or both (default)
# version 0.01

use strict;
use CGI;
use CGI::Carp qw(fatalsToBrowser);
use URI::URL;
use Net::LDAP;
use Net::LDAP::Entry;
use Win32::OLE;
$Win32::OLE::Warn = 3;

my $q = CGI->new;

require "../etc/ldapapp.conf";

our ($adsTestHost,$adsQualHost,$adsProdHost,$adsTestPath,
$adsQualPath,$adsProdPath,$adsUsr,$adsPass);

my @adshosts = ("$adsTestHost","$adsQualHost","$adsProdHost");
my @adsbases = ("$adsTestPath","$adsQualPath","$adsProdPath");
my $newpass = 'yellow';

my $webhost = "localhost";
my $webport = "80";


if ($q->param('Review')) {

 give_status($q);

}

elsif ($q->param('Reset Account')) {
 
 reset_user($q);

}


else {

    show_main($q);
}

sub show_main {

  print $q->header;
  print $q->start_html(-title=>'Active Directory Account Reset',
     -style=>{'src'=>'/styles/ldapadmin.css'}
     );
  print $q->h1("Active Directory Account Reset");
   
  my $action = $q->url;
  print $q->start_form(-method=>'POST',
      -action=>$action,
            );

  print $q->h4("Select Server");
  print $q->popup_menu(-name=>'adshost',
     -values=>\@adshosts
     );
  
  print $q->h4("Choose Domain");
  print $q->popup_menu(-name=>'adsbase',
    -values=>\@adsbases
       );
       
  print $q->h4("Enter User ID");
  print $q->textfield(-name=>'userid',-size=>5);
  print $q->p();
  print $q->submit('Review');
  print $q->reset("Cancel");

  print $q->end_form;
  print $q->p();
  print $q->a( { -href => "/acctadmin/index.html" },
 "Back To Active Directory Tools");
  print $q->end_html();

 
} # main


sub give_status {

  
  print $q->header(-charset=>'UTF-8');
  print $q->start_html(-title=>'Active Directory Account Reset',
  -style=>{'src'=>'/styles/ldapadmin.css'}
      );
  print $q->h1("Active Directory Account Reset");
  print $q->h3("Confirm User Information");
  my $action = $q->url;
  print $q->start_form(-method=>'POST',
                 -action=>$action,
           );
  
  my $adshost = $q->param('adshost');
  my $adsbase = $q->param('adsbase');
  my $userid = $q->param('userid');
 
  
  print $q->hidden(-name=>"adshost",value=>"$adshost");
  print $q->hidden(-name=>"adsbase",value=>"$adsbase");
  print $q->hidden(-name=>"userid",value=>"$userid");
  
  print $q->p("Working on $adshost");
  print $q->p("$adsbase");
 
  $adsUsr = "$adsUsr,$adsbase";

  my $basedn = $adsbase;
  my @attrs = qw(displayname cn userprincipalname samaccountname);
  my $filter = "(|(samaccountname=$userid)(cn=$userid))";
  
  my $ldap = Net::LDAP->new($adshost, version =>'3');
  my $mesg = $ldap->bind($adsUsr, password => $adsPass);
  
  $mesg = $ldap->search(base => $basedn,
    scope => 'sub',
    filter => $filter,
    attrs => \@attrs
     );
    
    if ( $mesg->count <1 ) {
        
        print $q->h4("No results returned!");
    
    } # if

  elsif ( $mesg->count >1 ) {
  
  print $q->p("Please choose from one of the entries listed and retry");
  
  while (my $entry = $mesg->shift_entry()) {
   my $userdn = $entry->dn;   
   my $cn = $entry->get_value('cn');
   my $displayname = $entry->get_value('displayname');
     print $q->p("$userdn");
     print $q->p("UserID: $userid",$q->br, "Full Name: $displayname");
      }
         } 
   
  else { 
    
               my $entry = $mesg->shift_entry(); 
      my $userdn = $entry->dn;
      print $q->hidden(-name=>"userdn",value=>"$userdn");
      my $cn = $entry->get_value('cn');
        print $q->hidden(-name=>"cn",value=>"$cn");
        my $displayname = $entry->get_value('displayname');
        print $q->hidden(-name=>"displayname",value=>"$displayname");
        my $userprincipalname = $entry->get_value('userprincipalname');
      print $q->hidden(-name=>"userprincipalname",value=>"$userprincipalname");

        print $q->p("$userdn");
        print $q->p("UserID: $userid",$q->br, "Full Name: $displayname");
   
       
      # Bind using ADSI to check flags
        my $ldapObj = Win32::OLE->GetObject('LDAP:');
        my $usrObj = $ldapObj->OpenDSObject("LDAP://$adshost/$userdn","$adsUsr", "$adsPass", 1);
 
          if ($usrObj->AccountDisabled == 1) {
       print $q->p("Account is disabled, contact System Administrator");
      }

      if ($usrObj->{IsAccountLocked} == 1) {
                print $q->p("Account is locked, reset will unlock");
        }
        else {
         print $q->p("Account not locked");
        }

      print $q->p();
            print $q->submit('Reset Account');
                    print $q->reset("Cancel");
 
     }
  
  $ldap->unbind;

  print $q->end_form;
 
  print $q->p();
    
  print $q->a( { -href => '/acctadmin/adacctreset.cgi' }, 'Try Again');
  print $q->p();
  print $q->a( { -href => "/acctadmin/index.html" }, "Back To Help Desk Tools");
  print $q->end_html;
  
} # status
  
  
sub reset_user {
  
  
  print $q->header(-charset=>'UTF-8');
  print $q->start_html(-title=>'Active Directory Account Reset',
    -style=>{'src'=>'/styles/ldapadmin.css'}
        );
  print $q->h1("Active Directory Account Reset");
  print $q->h3("Confirming Changes");
  
  my $adshost = $q->param('adshost');
  my $adsbase = $q->param('adsbase');
  my $userdn = $q->param('userdn');
  my $cn = $q->param('cn');
  my $displayname = $q->param('displayname');
  my $userprincipalname = $q->param('userprincipalname');
    
  print $q->p("Working on $adshost");
  
  print "$userdn\n";
  print "$displayname
\n";
  
  $adsUsr = "$adsUsr,$adsbase";

  # Bind using ADSI to reset password
  my $ldapObj = Win32::OLE->GetObject('LDAP:');
  my $usrObj = $ldapObj->OpenDSObject("LDAP://$adshost/$userdn","$adsUsr", "$adsPass", 1);

  # Check for disabled account
  if ($usrObj->AccountDisabled == 1) {
     print $q->p("Account is still disabled, contact System Administrator");
  }
  
  # Unlock account
  if ($usrObj->{IsAccountLocked} == 1) {
      $usrObj->{IsAccountLocked} = 0;
      $usrObj->SetInfo;
      print $q->p("Account unlocked");
  }
  else {
      print$q->p( "Account not locked");
  }
  
  # Reset password
  $usrObj->SetPassword($newpass);
  $usrObj->SetInfo;
  print $q->p("Password reset to $newpass");

  # Forces change password on next logon 
  $usrObj->Put('pwdLastSet', 0);
  $usrObj->SetInfo;  
  print $q->p("User must change password at next logon");

  
  print $q->p();
  
  print $q->a( { -href => '/acctadmin/adacctreset.cgi' }, 'Try Again');
  print $q->p();
  print $q->a( { -href => "/acctadmin/index.html" }, "Back To Active Directory Tools");
  print $q->end_html;
 
   
} # reset

__END__;

Red Hat buying Netscape Directory, Certificate Servers

2004-10-01T20:21:00.000-04:00

It's official. Red Hat has posted a press release announcing a deal to acquire Netscape's Directory and Certificate servers from AOL/Netscape. This is real good news for those of us in the LDAP business. The Slashdot post on this from 9/30/04 had over 200 replies, which just goes to show there is indeed intelligent life in the universe.

As a fairly satisfied user of Sun's rebranded version of the Netscape Directory server (which they acquired after the dissolution of the iPlanet partnership), I've kept a careful eye on the Netscape version (which has sometimes been hard to find, buried somewhere among a pile of broken links to technologies Netscape no longer supports). It's been pretty clear for a long time that AOL wasn't even trying to market these Netscape properties, and at least from a "maintaining balance in the universe" standpoint its good to see they will be going to a home where they won't be wasted.

The other good part is that these products will soon be open sourced, and made available for deployment on advanced Linux systems like the upcoming Red Hat Enterprise 4. AOL/Netscape did not support Linux on its latest versions, and Sun has only paid lipservice to the open source O/S (They claim their latest version, rebranded "Java System Enterprise Directory", works on the now antiquated Red Hat Enterprise 2.1. Right.). At work this has been an issue because we're now engineering a probable move of most of our web tier to Linux on Intel. Up until now it looked like the directories would have to stay on Solaris/Sparc.

I'll take a careful look at Red Hat's next release of the Netscape Directory, especially because the schemae and access controls in the Sun and Netscape products are almost identical and as a result the transition in our enterprise would be fairly painless. Interesting contrast here. Some companies. like Oracle, AOL and Novell (remmember WordPerfect? Or UNIXWare? DRDOS?), gobble up others and run them into the ground (beware, PeopleSoft and SuSE customers), others, like Red Hat, give them new life (think Cygwin here).

Tuning the Berkeley DB

2004-09-21T22:48:00.000-04:00

Sleepycat Software's Berkeley DB is the most widely used embedded database manager in the world. The latest release of OpenLDAP (v2.2x) uses Berkeley DB (v2.4x) for it's bdb backend.

With the origina OpenLDAP ldbm backend, database tuning was all done inside the slapd.conf file. Limited configuration of the bdb backend can still be done in slapd.conf, but fine tuning now needs to be done in the standard DB_CONFIG file usually found in the directory specified in slapd.conf by the database section's directory directive (e.g. /var/db/openldap-data).

Right now I'm still experimenting with DB_CONFIG, so my configuration is fairly simple. I got the basic framework from the OpenLDAP FAQ on the subject. Of course to make it effective I'm going to have to rebuild my 10,000 entry test database, but hey, who needs to break for lunch anyway? I'm setting the cache initially at 64 Mb, to see if this gets me around a bulk loading problem I had during a recent db creation session (I wound up using slapadd to complete the process). Here's the config I'm going to use:

# Set the database in memory cache size.
#
set_cachesize 0 64000000 0
#
# Set database flags.
# (for database loading/reindexing)
#set_flags DB_TXN_NOSYNC
#
# Set log values.
#
# set_lg_regionmax 1048576
# set_lg_max 10485760
# set_lg_bsize 2097152
# set_lg_dir /usr/local/var/openldap-logs

Late Note: I used this config with my test db as planned, and it did in fact get me past the bulk loading issue I'd been experiencing with OpenLDAP 2.2, working directly with the db config has been a real eye-openner and would make a great chapter in a book someday.

Slurpd, you can't get it at 7-11

2004-09-05T16:25:00.000-04:00

The OpenLDAP man page for slurpd defines it as the "Standalone LDAP Update Replication Daemon". And so it is. Longer in production and more mature than its client-side replication cousin, syncrepl, slurpd provides a server-side "push" of changes from a master LDAP directory to designated replica (or slave) directories.

The directory service I use in my lab at home does not see alot of changes, so issues such as the relative robustness of slurpd over syncrepl are not really a concern there. Given the interest being shown in open source directory solutions in many places, however, I've begun to kick the tires on the daemon to re-familiarize myself with its inner workings.

Slurpd is built by default in a stock "configure, make, make install" of OpenLDAP. On FreeBSD it gets installed as part of the openldap-server port. While I've never had this problem with a source build on Linux, the current FreeBSD port annoyingly hardcodes the daemon so that it will only work when the replication log file location (set using the replogfile directive in slapd.conf) is set to /var/db/openldap-slurp/replica. What you wind up with in this configuration are a pair of replication log files (one for slapd, the other for slurpd) and their corresponding lock files. When a change gets made on the master, this gets recorded in the slapd replog (in my config, /var/db/openldap-slurp/replica/slapd.replog), and then passed over to the slurpd replog (on my server, /var/db/openldap-slurp/replica/slurpd.replog). When an update fails, a record of it gets written to a separate *.rej log. The data in both replog files is written out in clear text, LDIF (LDAP Data Interchange) format with the requisite LDAP commands, and so should be secured with the same attention as the slapd database itself. This particular method of formatting may make it possible to use these files as part of a custom synchronization system that could work with other vendor directories, such as Active Directory (something I've read has been done at the University of Michigan).

The slapd.replog file gets purged once slurpd reads it, while the slurpd.replog file maintains a running history of changes made. The latter, which can grow quite large, is therefore a candidate for log rotation by the newsyslog facility in FreeBSD. I've got mine set to top out at 1000 Kb (1 Mb), which is reasonable for my environment. Larger, more active, installations, may need to grow the file bigger.

HP and OpenLDAP

2004-08-30T10:46:00.000-04:00

Earlier this month the OpenLDAP Project had their 3rd annual Developer's Day. While I missed the conference myself (I don't travel, and even if I did my employer would have never paid for this one), I found the slide presentations archived on the project site very interesting. One that especially caught my eye was this:

www.openldap.org/conf/odd-sandiego-2004/Neil.pdf

The big news to me was that HP is moving off Sun's directory and onto OpenLDAP as their core enterprise directory in October. In fact, as the slides show, they've already made the switch for their extranet. The slides pretty much confirmed my own view of the readiness of OpenLDAP for the enterprise, with most of the same caveats I've raised to my management when we've discussed it. HP's approach was to work with OpenLDAP leader Symas Corporation to develop the enhancements they required. HP's team are the same guys whose homegrown LDAP sync tool, used during the HP and Compaq merger to sync their disparate directories, can be now purchased by outsiders for $20,000. Two of my remaining reservations, internal application reliance on proprietary features of the Netscape/Sun Directory series (mainly, Server Side Sort) and write performance (OpenLDAP used to be abyssmal, while Sun was always on top), are not addressed in the Developer Day presentation. The former is really a matter for internal standards enforcement, the latter may not be the issue it once was, now that OpenLDAP 2.2 has gone stable with its new Berkeley DB backend.

Samba and LDAP

2004-08-04T00:06:00.000-04:00

Someone with an eye for detail might have noticed that the smb.conf I had in an earlier post was configured to use the tdbsam instead of the ldapsam backend. While tdbsam is a good alternative to the older smbpasswd backend that mimics a Windows LSA (Local Security Authority) database, it is understandable that one would assume that eldapo would opt for the LDAP backend. Although I've tried this in previous installations of both Samba 2 and Samba 3 (while the latter was still in development), my goal this time around was to get the service up and running as quickly and cleanly as possible.

With a little time on my hands today, I in fact did take the next step and convert over to the ldapsam backend. After pouring over the doc for awhile I thought I understood what to do, which I did -- pretty much.

First, I added the samba 3 schema file to my OpenLDAP instance's schema subdirectory and made the necessary changes to my slapd.conf (including adding an access control and indexes to support the new objectclass and attributes needed by Samba). Then I edited my smb.conf to add the LDAP related directives. Finally, I moved the Samba account information for my existing users over from tdbsam to LDAP.

For slapd.conf, the following lines got added:

At the top of the file:

include /usr/local/etc/openldap/schema/samba.schema

Above the database settings section:

# Limited access to Samba account attributes
access to attr=sambalmpassword,sambantpassword,sambapwdlastset,sambaacctflags,
sambalogontime,sambalogofftime,sambakickofftime,sambapwdcanchange,
sambapwdmustchange,sambahomepath,sambahomedrive,sambaprofilepath,
sambauserworkstations,sambaprimarygroupsid,sambadomainname,sambasid
by group/groupofuniquenames/uniquemember="cn=Administrators,dc=lembo,dc=ny,dc=us" write
by self write

At the end of the database settings section:

# Samba v3 schema attributes
index sambasid eq
index sambaprimarygroupsid eq
index sambadomainname eq

I ran the command slapindex -f /usr/local/etc/openldap/slapd.conf to create the new indexes on the directory.

Here are the additional lines in the global section of smb.conf:

passdb backend = ldapsam
ldap admin dn = "cn=manager,dc=lembo,dc=ny,dc=us"
ldap ssl = start tls
ldap delete dn = no
ldap user suffix = ou=People
ldap group suffix = ou=Groups
ldap machine suffix = ou=Computers
ldap idmap suffix = ou=Domains
ldap suffix = dc=lembo,dc=ny,dc=us
ldap filter = (uid=%u)

Pretty standard stuff. I did have to use smbpasswd as root to add the "cn=manager" root directory account's password to the database before switching over. To migrate the existing tdbsam data to ldapsam I used the Samba pdbedit tool thusly:

pdbedit -i tdbsam -e ldapsam

For those users who did not have a previous account on tdbsam I used the smbpasswd utility to add them, which then wrote the requisite information into the user's corresponding LDAP entry.

Note that for now at least I have not enabled "ldap sync" to maintain both Samba and LDAP passwords automatically. This is mostly due to initial caution on my part. I know this feature does in fact work. What I'm really interested in in synching the Samba and UNIX passwords. Previously I was running FreeBSD 4.x, which does not support PAM (Pluggable Authentication Modules), a critical part of that feature. Now that I've moved everything to FreeBSD 5.2, I'm going to be researching how to configure PAM for this purpose. For now, I'll continue to maintain the 3 different passwords independantly, using native tools (passwd for UNIX, ldappasswd for LDAP and smbpasswd).

A sample slapd.conf

2004-08-02T06:58:00.000-04:00

This is a sample slapd.conf for those who might be interested in a bdb back-end, monitor, some basic access controls and indexes (Note: the stock stylesheet that I use on this site cuts off some of the line ends unless you maximize your browser):



include  /usr/local/etc/openldap/schema/core.schema
include  /usr/local/etc/openldap/schema/cosine.schema
include  /usr/local/etc/openldap/schema/inetorgperson.schema
include  /usr/local/etc/openldap/schema/nis.schema
include  /usr/local/etc/openldap/schema/samba.schema

# Define global ACLs to disable default read access.

# Do not enable referrals until AFTER you have a working directory
# service AND an understanding of referrals.
#referral ldap://root.openldap.org

pidfile  /var/run/openldap/slapd.pid
argsfile /var/run/openldap/slapd.args

# Load dynamic backend modules:
# modulepath /usr/local/libexec/openldap
# moduleload back_bdb.la
# moduleload back_ldap.la
# moduleload back_ldbm.la
# moduleload back_passwd.la
# moduleload back_shell.la

# Sample security restrictions
# Require integrity protection (prevent hijacking)
# Require 112-bit (3DES or better) encryption for updates
# Require 63-bit encryption for simple bind
# security ssf=1 update_ssf=112 simple_bind=64

# Sample access control policy:
# Root DSE: allow anyone to read it
# Subschema (sub)entry DSE: allow anyone to read it
# Other DSEs:
#  Allow self write access
#  Allow authenticated users read access
#  Allow anonymous users to authenticate
# Directives needed to implement policy:
#
# if no access controls are present, the default policy is:
# Allow read by all
#
# rootdn can always write!

# Custom access controls - created 12/31/03 by P Lembo
# Read access to root dse by all
access to dn=""
 by * read

# Access by all to schema
access to dn.base="cn=subschema"
 by * read

# Access to monitor by all
access to dn.base="cn=monitor"
   by * read

# Limited access to password
access to attr=userpassword
 by group/groupofuniquenames/uniquemember="cn=Administrators,dc=company,dc=com" write
 by self write
 by users auth
 by anonymous auth

# Limited access to Posix account attributes
access to attr=uidnumber,gidnumber,homedirectory,loginshell,gecos
 by group/groupofuniquenames/uniquemember="cn=Administrators,dc=company,dc=com" write
 by self read

# Limited access to Samba account attributes
access to attr=lmpassword,ntpassword,pwdlastset,acctflags,logontime,logofftime,kickofftime,
pwdcanchange,pwdmustchange,smbhome,homedrive,profilepath,userworkstations,
primarygroupid,rid
 by group/groupofuniquenames/uniquemember="cn=Administrators,dc=company,dc=com" write
 by self read

# User access to organizational attributes
access to attr=o,ou,c,description,uniquemember
 by group/groupofuniquenames/uniquemember="cn=Administrators,dc=company,dc=com" write
 by users read

# Access to public attributes by all
access to attr=uid,c,title
 by group/groupofuniquenames/uniquemember="cn=Administrators,dc=company,dc=com" write
 by users read
 by anonymous read

# Self write access to public attributes  
access to attr=cn,displayname,sn,givenname,mail,telephonenumber,facsimiletelephonenumber
 by group/groupofuniquenames/uniquemember="cn=Administrators,dc=company,dc=com" write
 by self write
 by users read
 by anonymous read

# Self write and limited access to private attributes
access to attr=homephone
 by group/groupofuniquenames/uniquemember="cn=Administrators,dc=company,dc=com" write
 by self write
 by users read

# Self write and read access to all other attributes
access to *
 by group/groupofuniquenames/uniquemember="cn=Administrators,dc=company,dc=com" write
 by self write
 by users read
 by anonymous read

password-hash {SHA}

allow bind_v2
sizelimit   200
timelimit   120
# TLS/SSL directives
TLSCertificateFile /usr/local/etc/openldap/newcert.pem
TLSCertificateKeyFile /usr/local/etc/openldap/newkey.pem
TLSCACertificateFile /usr/local/etc/openldap/newca.pem

#######################################################################
# ldbm database definitions
#######################################################################
# Monitor backend
database        monitor

# User database
database bdb
suffix  "dc=company,dc=com"
rootdn  "cn=Manager,dc=company,dc=com"
# Cleartext passwords, especially for the rootdn, should
# be avoid.  See slappasswd(8) and slapd.conf(5) for details.
# Use of strong authentication encouraged.
rootpw  {SHA}xxxxxxxxxxxxxxxxxxxxxxxx 
# The database directory MUST exist prior to running slapd AND
# should only be accessible by the slapd and slap tools.
# Mode 700 recommended.
directory /var/db/openldap-data

# Cache directives
cachesize    2000
idlcachesize 4000
# Indices to maintain
index objectclass eq
index cn  eq,pres,sub
index sn  eq,pres,sub
index givenname eq,pres,sub
index mail  eq,pres,sub
index c  eq,pres
index uid  eq,pres
index displayname  eq,pres,sub
index uidnumber  eq
index gidnumber  eq
index memberuid  eq
# Samba v2 schema attribs
index rid  eq

# Samba v3 schema attributes
# index sambasid eq
# index sambaprimarygroupsid eq
# index sambadomainname eq

FreeBSD-Windows Interoperability Using Samba

2004-08-02T06:43:00.000-04:00

In the past I've set up Windows machines to talk lpd and NFS to UNIX boxes, but recently I decided to give Samba another try. Many years ago I used Samba 1.x with some success, and was at first frustrated by 3.0, until I stopped using SWAT and other GUI's and did everything from the command line. Here's my smb.conf:

# Global parameters
[global]
workgroup = WORKGROUP
server string = Samba Server
passdb backend = tdbsam
log file = /var/log/samba/log.%m
max log size = 50
os level = 33
preferred master = yes
domain master = yes
wins support = yes
printing = bsd
load printers = yes
show add printer wizard = yes
use client driver = no

[printers]
comment = All Printers
path = /var/spool/samba
printable = yes
browseable = yes
public = yes
guest ok = yes

[share]
path = /usr/local/share
read only = No

[lp]
comment = gimp/pcl-4;r=300x300;q=medium;c=gray;p=letter;m=auto
path = /var/spool/samba
read only = No
printable = Yes
printer name = lp
use client driver = Yes
oplocks = No
share modes = No

eldapo on vacation

2004-08-02T06:28:00.000-04:00

... for the first time since Y2K. It's hard to believe, but it actually has been 5 years since I took a whole week off at a time. I was a desktop guy during Y2K, testing software and creating images to be deployed on 6,000+ PC's in my company. After a very brief respite, I volunteered to work on our new LDAP directory project. That turned into a full time job, involving way more dedication and hard work than any of us imagined. This summer, though, I thought I owed it to my family -- and myself -- to take some time off. There are a bunch of guys back at the shop who are covering for me this week. Hopefully the population, and the systems, won't be too hard on them. Especially because I'll be wanting them to do it all again at the end of August, when I take my next week off.

OpenLDAP 2.2 Now Standard for FreeBSD Ports

2004-07-26T22:30:00.000-04:00

In recognition of OpenLDAP 2.2 going stable, the FreeBSD project has now made it the standard version for the -CURRENT ports tree going forward. This will relieve many of us of the extra work needed to hack dependant ports so they would accept 2.2 in place of the previous standard, 2.1 (which is still what you'll get -- for now -- in the FreeBSD -STABLE branch or in the packages that shipped with -RELEASE). Eldapo thanks the talented engineers over at the FreeBSD project for this most intelligent decision.

OpenLDAP 2.2 STABLE

2004-07-13T16:55:00.000-04:00

OpenLDAP version 2.2 has gone STABLE! While I've been running it for a few months, its finally good to be able to recommend it for production use. Among the features it now offers to OpenLDAP users for the first time are LDAP Sync and Simple Paged Results Control. Still missing are Server Side Sort and Multi-Master replication, but the project leads continue to insist that this is A Good Thing (TM). Being in the midst of moving my own app dev people away from SSS, I wholeheartedly agree on the former. Why should I give up precious RAM and CPU cycles so the app server will look faster than it really is? MMR is another thing, but so far I am inclined to the view that it is not a critical feature, and that finite developer time would be more effectively used in other areas for now. Microsoft and Novell marketing have been beating Sun up for years on this, but like so many other "value-adds" from closed-source vendors, it really is something we can mostly do without.

LDAP Sync is another matter. While implemented in 2.2 as a way for consumer directories to "pull" updates from suppliers, it has the potential for allowing syncrhonization between directories of different vendors -- something that has up to now been impossible. It also invites the possibility of client-side synchronization by mail clients and PDAs. ELDAPO is pleased.

OpenLDAP Access Controls

2004-07-13T10:33:00.000-04:00

At work we don't use OpenLDAP -- yet. But that doesn't keep me from continuing to experiment with what I consider the best LDAP directory software available today.

Perhaps one of the most daunting tasks of a neophyte OpenLDAP administrator is properly setting up access controls. Although acknowledged by all to be "powerful", the regex style ACL syntax used by OpenLDAP are a real stumbling block to many. The primary rule to be observed is that once an attribute or dn has had rights to it defined, any subsequent attempt to define access will be ineffective. Failure to define the access of a particular category of user (users, self, anonymous) will result in a denial of access. I found that ordering my rules from specific to general according to logical groupings of dns or attributes to be the most effective strategy. These groupings usually relate to who should be able to see and write to particular attributes or dns. For example, the following control

access to attr=uid,c,title
by group/groupofuniquenames/uniquemember=
"cn=Administrators,dc=example,dc=com" write
by users read
by anonymous read

will result in title only being writable by admins, and only being readable by authenticated users (write implies read, which also implies search). Users cannot write to the attribute. Everyone else (anonymous) will be unable to see it at all. Note the unintuitive syntax for defining a group's rights -- I have the author of this WLUG Wiki to thank for leading me to the correct string to use. Also note that in an actual access
control the "by" line would not contain a line break.

Following is a default set of ACLs I recently implemented in production. Keep in mind that for formatting purposes I've put line breaks here. Lines beginning with "access to" or "by" should continue without breaking in an actual slapd.conf file.

# Custom access controls - created 12/31/03 by P Lembo
# Read access to root dse by all
access to dn=""
by * read

# Access by all to schema
access to dn.base="cn=subschema"
by * read

# Access to monitor by all
access to dn.base="cn=monitor"
by * read

# Limited access to password
access to attr=userpassword
by group/groupofuniquenames/uniquemember="cn=Administrators,dc=company,dc=com" write
by self write
by users auth
by anonymous auth

# Limited access to Posix account attributes
access to attr=uidnumber,gidnumber,homedirectory,loginshell,gecos
by group/groupofuniquenames/uniquemember="cn=Administrators,dc=company,dc=com" write
by self read

# Limited access to Samba account attributes
access to attr=lmpassword,ntpassword,pwdlastset,acctflags,logontime,logofftime,

kickofftime,pwdcanchange,pwdmustchange,smbhome,homedrive,profilepath,

userworkstations,primarygroupid,rid
by group/groupofuniquenames/uniquemember="cn=Administrators,dc=company,dc=com" write
by self read

# User access to organizational attributes
access to attr=o,ou,c,description,uniquemember
by group/groupofuniquenames/uniquemember="cn=Administrators,dc=company,dc=com" write
by users read

# Access to public attributes by all
access to attr=uid,c,title
by group/groupofuniquenames/uniquemember="cn=Administrators,dc=company,dc=com" write
by users read
by anonymous read

# Self write access to public attributes
access to attr=cn,displayname,sn,givenname,mail,telephonenumber,facsimiletelephonenumber
by group/groupofuniquenames/uniquemember="cn=Administrators,dc=company,dc=com" write
by self write
by users read
by anonymous read

# Self write and limited access to private attributes
access to attr=homephone
by group/groupofuniquenames/uniquemember="cn=Administrators,dc=company,dc=com" write
by self write
by users read

# Self write and read access to all other attributes
access to *
by group/groupofuniquenames/uniquemember="cn=Administrators,dc=company,dc=com" write
by self write
by users read
by anonymous read