Replacing NIS with Kerberos and LDAP HOWTO



What's New

February 13, 2004
Updated instructions from OpenLDAP 2.0.x to 2.1.x. OpenLDAP 2.1 has been out for over a year and is considered stable. 2.0 is no longer supported by the OpenLDAP developers.

Introduction

NIS is a name service for UNIX directories like the passwd map, ethers map, etc. NIS is easy to setup and administer, scales reasonably well, is supported by nearly all forms of UNIX, and is thus very popular. Unfortunately, it is also completely insecure. Weakly encrypted passwords, as well as everything else, are sent over the network in the clear. NIS is difficult to firewall. Clients have no way to ensure that the server they are talking to is actually an official server. Of course this has been known for many years, but NIS is still widely used because a resonable alternative hasn't been available. Sun developed NIS+, but it was difficult to setup and administer (requiring a variety of difficult to remember commands and command-line options) and was adopted slowly (or not at all) by Open Source versions of UNIX.

In the last couple of years there has been a lot of buzz about LDAP. The Lightweight Directory Access Protocol was designed as a way to access directories containing all matter of information. It didn't take long for someone to consider storing UNIX information in a directory and using LDAP to access it. In theory this is a great idea, you can store all of the information you are used to storing in NIS (user ids, group ids, home directories, etc.) as well as other information like titles, office numbers, telephone numbers, etc. The system administrators can access the information they need, HR can get the info they need, etc. It is all stored in a central database, eliminating data replication. And most LDAP server implementations support pretty good security through SSL for authentication and transport encryption, fine grained access controls, etc.

So why hasn't LDAP already replaced NIS? It suffers many of the same problems as NIS+: it is difficult to set up a secure server, the command line utilities are numerous and take many obtuse options and client support has been slow to appear. Unfortunately Internet security has gotten to the point that we can no longer wait for a NIS replacement that is just as easy to manage. Hopefully this page will take some of the pain out of setting up a server and administering things. Client support exists if your client OS supports PAM and NSS, as well as a few other options if it doesn't.

What about Kerberos? Kerberos isn't necessary. You can store encrypted passwords in LDAP just like you can in NIS. It can even be made reasonably secure with SSL and proper access controls. However, you're still putting private information into a directory designed to hold public data. Kerberos was designed to solve this problem. It has been around for a long time, it is relatively easy to setup, and client support is fairly widespread. And Kerberos is even more secure than LDAP, because in a properly designed Kerberos environment even encrypted passwords are almost never transmitted across the network. Kerberos is also useful for things like secure NFS (coming in NFSv4). As such, this page is based on using Kerberos for authentication and LDAP for directory services. If you want to use LDAP for authentication, you'll want to look into pam_ldap.


Alphabet Soup

The world of Kerberos and LDAP is filled with various standards, each of which comes with its own acronym (or several). Here are some of the common terms:

LDAP
Lightweight Directory Access Protocol. As the name implies, LDAP is just an access protocol, each implementation is free to use any sort of database on the backend.
SASL
Simple Authentiction and Security Layer. More info . A protocol framework for doing client-server authentication over the network. Provides a common way for clients and servers to negotiate a method of authenticating and then perform that authentication. Common SASL plugins are plain-text passwords, a SASL database of passwords allowing transmission in hashed format, or GSSAPI (usually used for Kerberos).
GSSAPI
Generic Security Services Application Programming Interface. More info . A generic API for doing client-server authentication. Differs from SASL in that SASL defines a network protocol, GSSAPI defines a programming interface. In fact, GSSAPI is a supported SASL authentication method, with Kerberos being the only commonly used GSSAPI implementation. Thus, an LDAP client and server would use SASL to negotiate GSSAPI as the authentication method (as opposed to plain-text passwords, etc.) and then the GSSAPI plugin for SASL would be invoked on each end to speak Kerberos to one another. Crazy, eh? Generally when you see GSSAPI you can just think Kerberos.
PAM
Pluggable Authentication Module. A system for plugging various authentication methods into applications without having to recompile. Typical authentication methods include the UNIX password database (either from local files or NIS), Kerberos and SMB (Windows).
NSS
Name Service Switch. A system for configuring lookups of UNIX directories from various sources. Typical sources include local files, DNS, NIS and LDAP.
Principal
The term for an entry in a Kerberos database
Distinguished Name
The term for an entry in an LDAP database

Software

You'll need the following software. Hopefully your OS comes with pre-compiled packages for everything, if not you'll need to compile in the order listed.

Server

(* Implementation used in this documentation)

If you happen to be using Fedora Linux, you can get all of the above with the following RPMs:

Older versions of Red Hat (Red Hat Enterprise Linux 3 for example) come with OpenLDAP 2.0.x, so you'll need to compile your own copy of OpenLDAP 2.1.x.

Solaris 8 and later come with SEAM, the Sun Enterprise Authentication Mechanism, an implementation of Kerberos 5. The OS ships with the client software, and the server software is available for free download. The client interoperates with MIT Kerberos but the server does not. As such, I recommend you only use the SEAM server in an all Sun environment. However, using the SEAM clients (kinit, pam_krb5, etc.) against an MIT server is fine. You may find that you want to also install MIT Kerberos in an out of the way location like /usr/local/krb5 so that you have a functional kadmin on your Solaris boxes (the SEAM version of kadmin speaks the SEAM server protocol and thus doesn't work with an MIT server). If you do that, you need to symlink /etc/krb5.conf to /etc/krb5/krb5.conf so that both SEAM and the MIT programs can find it. Additionally, if you link any programs against the MIT Kerberos libraries, and those programs need a keytab, you'll want to symlink /etc/krb5.keytab to /etc/krb5/krb5.keytab.

Client

Everything on the server, plus:

If you happen to be using Red Hat Linux, you can get all of the above with the following RPMs:

Turbo Fredriksson has some (now fairly dated) information about compiling all of this stuff which might prove helpful in a pinch.

If your client doesn't support NSS, PADL's nss_ldap also supports something called IRS (Information Retrieval Service), which I don't know much about.

If your client doesn't support PAM, the other option is to replace /bin/login with the Kerberized version which comes with MIT Kerberos.


Configuration

Note on Example Hostnames

Throughout this HOWTO I'll use the following set of sample hostnames for the servers: foo.example.com and bar.example.com are the actual hostnames of the two servers as reported by the hostname command and by reverse DNS lookups of the server IP addresses. ldap1.example.com and kerberos1.example.com are aliases for foo.example.com and can be either CNAMEs or additional A records in DNS. Similarly ldap2.example.com and kerberos2.example.com are aliases for bar.example.com. ldap.example.com is the hostname for a third IP address that is used by the LDAP load balancer.

Kerberos Servers

The first thing to setup is your Kerberos servers (called KDCs). Paths shown are for Red Hat Linux, they may differ on your system. You should have at least two KDCs, one of which will be the primary (accepting password changes, new principals, etc.) and the rest will get regular updates from that primary KDC. Start with setting up the primary KDC, then when that is working move on to setting up the secondaries. Here are the steps to getting your primary KDC running:

*/admin@EXAMPLE.COM     *

Testing

At this point you should be able to kinit username where username is one of the principals you've created already. You should be prompted for a password. Then run klist to make sure everything looks OK.

Replication

Copy krb5.conf and kdc.conf from the primary KDC to the secondary KDCs. Then follow the instructions in the Kerberos documentation. Here is a sample kpropd.acl and a sample script to do the replication as described in the documentation. Note that kpropd can also be run in standalone mode with the -S switch if you aren't running inetd. Red Hat comes with a kprop init script already setup to do this, just enable it with chkconfig.

Packet Filters

In case you have packet filters between any of the elements of your Kerberos system, here are the ports you'll need to open up. More information can be found in the Kerberos FAQ.

Clients -> KDCs:

Clients -> Primary KDC (system running kadmind)

Primary KDC -> Secondary KDCs

Logs

Kerberos logs can be found in the following files unless configured otherwise in krb5.conf:

Kerberos Clients

Once you have a working set of Kerberos servers, you'll probably want to be able to log into your system using your Kerberos password. Since we don't have LDAP working yet, you should add a local entry for your username to the passwd and shadow files, but set your crypted password in /etc/shadow to *K*, the community standard to indicate that the password comes from Kerberos.

For both Red Hat and Solaris, the various console login methods (/bin/login, CDE, etc.) have PAM support. So we need to modify the PAM configuration to use Kerberos. First off, if the system you are setting up isn't one of the KDCs themselves, you'll need to copy krb5.conf from one of them. Then configure PAM:

Testing

If the system you are setting up isn't one of the KDCs, do a kinit first to make sure that basic Kerberos is working. Then try to log in on the console. Solaris can be made to syslog tons of PAM debugging info simply by touching /etc/pam_debug. Make sure to remove it once you have things working, since it generates a lot of log data.

Other applications

Now you'll want any application that does authentication to talk to Kerberos. You can either have the application talk directly to Kerberos (if it supports it), or use PAM. Using PAM allows you to have Kerberos support in any application that supports PAM, as opposed to having to have native Kerberos support in each of those applications. Since PAM support is more widespread than native Kerberos support, this is generally a good idea. However, you do lose the advantages of native Kerberos authentication, namely the ability to authenticate once and have a ticket for 8 hours. Some applications (like OpenSSH) can support both. You'll have to figure out what is available and what makes the most sense for your applications.

If you are only using applications supplied by your OS vendor, and your OS supports PAM, then you probably don't need to worry about this. But if you compile your own applications, you'll need to make sure that you compile in PAM support. Sudo and xlockmore are a couple that I had to recompile on a few systems, specifically enabling PAM when running configure. Running ldd on an application will show you if it is linked against libpam or not, a quick test to see if it was compiled with PAM support.

See also the Kerberos Application Servers section in a little bit for information about network services (SSH, etc.) which use Kerberos for authentication.

Restricting Access

With NIS, it was common to keep users off a machine by putting something like the following in /etc/passwd:

+::::::/bin/false

This allowed the machine to have all of the account information from NIS but not allow the users to log in. Useful on something like a print server, etc. With Kerberos and LDAP, the easiest place to do something similar is PAM. This, admittedly, is somewhat of a misuse of PAM, since PAM stands for Pluggable Authentication Modules and thus shouldn't be doing authorization. However, it seems to be the best way to do it since configuring each application with proper authorization settings is difficult at best. There is no standard mechanism and many applications (/bin/login, xdm, etc.) simply lack authorization features altogether.

Linux provides a variety of PAM modules for doing authorization. Peruse /usr/share/doc/pam-*/txts/* on a Red Hat box to see all of the available options. The one that seemed to be the best fit for me was pam_access. First you add an entry like this to /etc/pam.d/system-auth:

account    required     /lib/security/pam_access.so

Then add appropriate entries to /etc/security/access.conf. Here's a sample:

# Allow only a few users to login
-:ALL EXCEPT root joe bob jane:ALL

On Solaris we initially tried to do something similar using the pam_projects module that comes with Solaris. Unfortunately we discovered that it can only filter on the user's primary group, and we wanted to filter on supplementary groups. As such, we ended up porting pam_access to Solaris. This was pretty simple, just a few minor changes. If you want to go the pam_projects route, consult the project(4), pam_projects(5) and getdefaultproj(3exacct) man pages. It isn't very well documented...

Windows Kerberos Clients

Windows 2000, and possibly newer versions of Windows, can be Kerberos clients of a UNIX Kerberos server. This doesn't work as well as having a Windows domain, since the accounts are still local to each box and thus you can't centrally manage group memberships, etc. However, it might be sufficient for a small operation. The following set of steps worked for me for setting up a Windows 2000 Professional workstation to talk to a MIT Kerberos server. The definitive document for this (although it is not much more helpful) is this document from Microsoft. There is also an interesting paper from LISA-NT a few years ago about the concepts of Windows and UNIX Kerberos interoperability. And this is a very interesting presentation by the Microsoft Kerberos project manager and a guy from MIT (requires Windows Media Player and probably IE).

If you make a mistake in the mapuser command, there is no way to change it except editing the registry. The Kerberos configuration is stored in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\Kerberos

Kerberos Application Servers

For any network service which accepts native Kerberos authentication (instead of taking a username and password and authenticating the user via PAM), you need to have a service principal stored in a keytab. Some common ones are:

OpenSSH, Solaris 9 pam_krb5 host/hostname
LDAP ldap/hostname
Kerberized NFS nfs/hostname

Note that if the service is not directly accepting Kerberos tickets, but rather is taking a username and password and authenticating the user via pam_krb5, you may not need a service principal. For Red Hat's pam_krb5 it is optional. Solaris 8's pam_krb5 doesn't use it, but for Solaris 9's pam_krb5 it is required that you have a host/hostname principal stored in /etc/krb5/krb5.keytab. An additional note, Solaris 9 apparently doesn't support the des3-hmac-sha1:normal Kerberos key type that is one of the two types specified in the kdc.conf I use. Rather than just ignore that key and use the des-cbc-crc:normal key (which Solaris 9 does support), it refused to work. So when creating the host principals for Solaris 9, use commands like the following (which only create des-cbc-crc keys, instead of both like normal).

First off, if you haven't done so on the machine already, copy /etc/krb5.conf (/etc/krb5/krb5.conf on Solaris) from a KDC and make sure you can kinit.

Create and store a service principal like this:

Note that if the application daemon does not run as root, you'll need to store the service principal in a file other than /etc/krb5.keytab that can be made readable by the application daemon. For an example, see the instructions for LDAP below.

Kerberos Wrap-up

At this point you should have a fully functional Kerberos setup. Make sure everything is working perfectly before you move on to configuring LDAP.

LDAP Servers

Once you have Kerberos setup, you can configure your LDAP servers. The first thing to do is to create a user and group dedicated to the LDAP server if your OS doesn't already come with one (Red Hat does if you've installed the openldap-servers RPM). I recommend calling both ldap.

The next thing you'll need is an SSL certificate for each of your LDAP servers. You could purchase these, but making your own is easy and doesn't cause any extra headaches like making your own Web SSL certificates does. Instructions for making certs can be found here and many other places on the web, or you can use my JavaCA tool. When creating the cert, you should use the official hostname (as reported by a reverse DNS lookup) in the CN field. You can add other names to the certificate by adding them to the subjectAltName field in the usr_cert section of openssl.cnf. (The OpenLDAP tools require that the hostname you specify when running them match either the CN or subjectAltName fields of the SSL cert supplied by the server.) The format is like this:

subjectAltName=DNS:ldap2.example.com,DNS:ldap.example.com

Once you've got the cert, you'll need to make sure that the key portion of it (some certs have both the key and the cert in one file, others have them seperated) is readable by the server that the ldap user will run as, but preferably not owned or writeable by that user, nor readable by anyone else. Something like chown root:ldap key.pem; chmod 640 key.pem should do the trick.

Next up you need to create a Kerberos service principal for the LDAP server and extract it to a keytab file (but not the system keytab in /etc/krb5.keytab because then you'd have to give the LDAP server permission to read that file which you don't want to do). The permissions on this keytab file should be the same as on the SSL key. Be careful with the hostname here, it needs to match what you get for a reverse lookup of the server's IP address. If you have the short (i.e. not fully qualified) hostname listed first in /etc/hosts you should probably change that.

Next up is creating your slapd.conf. Here's a sample one. Some of the things you need to edit are:

Starting with OpenLDAP 2.1 the Berkeley DB backend is used more like a real database. This provides some performance and reliability improvements, but it also means that you have to start treating Berkeley DB as a real database. The biggest issue is that you have to tune Berkeley DB to your environment. This caused me a very big headache and is a frequent problem discussed on the OpenLDAP mailing list. Some of the Berkeley DB settings can be changed in slapd.conf (see the sample slapd.conf above for examples). However, the major performance related settings go in a file called DB_CONFIG that needs to be located in your slapd database directory. See this FAQ entry for more details. Here's my DB_CONFIG. However, you'll need to do some research and determine the appropriate settings based on your environment. Searching the openldap-software mailing list archives for postings by Howard Chu about DB_CONFIG will be very informative.

# Increase the cache size to 8MB
set_cachesize 0 8388608 1

Then configure slapd to be started at boot time and start it. The command line should look like the following. This presumes that the default group for the ldap user is the ldap group. If not, you'll need to add a -g ldap to make sure that slapd is running as group ldap so it can read the Kerberos keytab and SSL key. The KRB5_KTNAME points the Kerberos libraries at a different keytab. The -h option tells slapd to listen on both the standard and ldaps ports. For debugging purposes, you can add a -d 1 and slapd will run in the foreground, printing out info about its actions. This can be very helpful if things aren't working. See the slapd man page for info about values other than 1 to -d to get even more info.

KRB5_KTNAME=/etc/openldap/ldap.keytab /usr/sbin/slapd -u ldap -h "ldap:/// ldaps:///"

Next up is the initial population of your database. Here's a sample LDIF file. Import it with the following command (make sure to specify the rootdn that is actually in slapd.conf).

Presuming that was successful, you should now comment out the rootdn and rootpw entries out of slapd.conf and restart slapd. All future modifications should be authorized via the ACLs defined in slapd.conf.

Testing

First skip ahead to the LDAP Clients section below and create an /etc/openldap/ldap.conf as directed there. Starting with OpenLDAP 2.1 the TLS_CACERT entry in that file is necessary when connecting with some form of SSL because OpenLDAP actually checks the server's certificate chain. There doesn't seem to be a way to specify the path to your CA cert on the command line.

Test your LDAP server in stages, as you'll likely have problems at each step. Make sure that the hostname you use is one specified in the CN or subjectAltName of your LDAP SSL cert. The OpenLDAP tools are picky about that. See the troubleshooting section below for error messages and their meanings. In each case, since you aren't specifying a filter, you should see the entire contents of the database, i.e. everything you added with the ldapadd command above.

Simple auth, no encryption
ldapsearch -H ldap://hostname/ -b dc=example,dc=com -x
Simple auth, SSL via LDAPS
ldapsearch -H ldaps://hostname/ -b dc=example,dc=com -x
Simple auth, SSL via StartTLS
ldapsearch -H ldap://hostname/ -ZZ -b dc=example,dc=com -x
SASL auth, no encryption
ldapsearch -H ldap://hostname/ -b dc=example,dc=com
SASL auth, SSL via LDAPS
ldapsearch -H ldaps://hostname/ -b dc=example,dc=com
SASL auth, SSL via StartTLS
ldapsearch -H ldap://hostname/ -ZZ -b dc=example,dc=com

Once you have all of those working, you are much closer to being done!

Replication

The way that replication works with OpenLDAP is that you configure slapd to write all changes to a log file. Then you run a seperate daemon (slurpd) which watches for changes to that log file and sends them (via LDAP) to all of the replicas it has been told about. Note that this means slurpd is an LDAP client and thus needs a Kerberos ticket (as opposed to a stored service principal like slapd) to authenticate to the replica LDAP servers. It also means that whatever principal that slurpd will be using must be authorized via the ACLs in the replicas' slapd.confs to write to all objects.

slurpd runs as root, so you might as well let it use the host/hostname Kerberos principal already stored in /etc/krb5.keytab. Just add a cronjob to root's cron to keep a ticket active for this principal. Presuming you use the standard 8 hour timeout for Kerberos tickets, I'd recommend having the cron job run every 4 hours. Here's a sample script to run:

#!/bin/sh

KRB5CCNAME=/var/run/slurpd.krb5cache /usr/kerberos/bin/kinit -k host/foo.example.com

Next you need to point slurpd at this ticket cache file by adding that KRB5CCNAME variable to the environment when slurpd is run. For Red Hat, add the following to /etc/sysconfig/ldap. For other OSs, modify the appropriate init script.

export KRB5CCNAME=/var/run/slurpd.krb5cache

Now we need to make sure that this principal is allowed to write to objects on the replica LDAP servers. See the sample slapd.conf above for example ACLs.

Next, we need to add a couple of entries to slapd.conf on the replicas so that they know which principal will be making changes and to refer everyone else to the master server for changes.

updatedn "uid=host/foo.example.com,cn=GSSAPI,cn=auth"
updateref ldaps://ldap1.example.com/

At this point, if slapd is running on the master server you should shut it down to avoid possible loss of data. Then copy the database from the master to the replicas. The most reliable way to do this is to use slapcat on primary server to dump the database out to an LDIF file and then use slapadd on the replica to read it back in. Either run slapadd as the ldap user or chown the database directory to ldap:ldap when you're done running slapadd.

Then add an entry to slapd.conf on the master server to enable writing of the log file for replication:

# Create a replication log in /var/lib/ldap for use by slurpd.
replogfile      /var/lib/ldap/master-slapd.replog

And another entry to configure slurpd:

replica host=ldap2.example.com:389 tls=critical
        bindmethod=sasl saslmech=GSSAPI
        authcId=host/foo.example.com@EXAMPLE.COM

At this point you should be ready to start everything back up. slapd and slurpd should be running on the master server and slapd on each of the replicas. The Red Hat ldap init script will automatically run slurpd if it finds a replogfile line in slapd.conf, for other systems you may need to modify/add an init script to start slurpd. It runs as root and doesn't need any command line arguments other than the environment variable mentioned above.

Test it by changing something on the master and make sure it gets propogated to the replicas.

Packet Filters

All standard and TLS LDAP traffic occurs over port 389/tcp. LDAPS traffic occurs over port 636/tcp. Connections are always initiated by the client.

Backup Recommendations

In addition to backing up your LDAP configuration and data though normal channels (i.e. to tape) I recommend that you have a cron job do a slapcat on a regular basis (nightly) to dump your LDAP database to an LDIF file. I have the cron job save 30 copies, which means I have 30 days of database backups easily accessible if something happens.

Troubleshooting

The LDAP command line utilities cough up a variety of error messages, none of which make much sense. Here are the ones I've seen:

ldap_sasl_interactive_bind_s: Unknown authentication method
ldap_sasl_interactive_bind_s: No such attribute
On Red Hat, the cyrus-sasl-gssapi RPM was not installed. (I'm not sure how I got two error messages for the same cause, but that's what I have in my notes. I know the first one is correct, but maybe the second one isn't.)

ldap_sasl_interactive_bind_s: Local error
ldap/hostname service principal not set up
or your Kerberos ticket is expired

ldap_sasl_interactive_bind_s: Unknown error
    additional info: GSSAPI: gss_acquire_cred: Miscellaneous failure; Permission denied;
slapd can't read /etc/openldap/ldap.keytab

ldap_sasl_interactive_bind_s: Can't contact LDAP server (when using SASL)
ldap_bind: Can't contact LDAP server (when using plain-text authentication, i.e. -x with ldapsearch)
The hostname you specified did not match the CN or subjectAltField fields of the SSL cert on the server. Make sure that the hostname you specified occurs in one of those fields of the SSL cert on the server.

ldap_result: Can't contact LDAP server
Bug in OpenLDAP 2.0.11 and possibly other versions, upgrade to a newer version.

Here are some useful commands when troubleshooting. Also, as mentioned earlier, you can run slapd in debug mode by specifying -d 1 when starting it. This is frequently handy.

Show a bunch of info about an SSL cert
openssl x509 -in file.pem -noout -text

Connect to an SSL server
openssl s_client -CAfile /etc/ssl/ca.pem -connect ldap1.example.com:636

LDAP Clients

PADL nss_ldap

The configuration file for the PADL nss_ldap module is /etc/ldap.conf. The sample file that comes with nss_ldap (and is installed on Red Hat) has extensive comments. However, I've found that you need only a few options. Here's a fully functional example:

host               ldap1.example.com ldap2.example.com
base               dc=example,dc=com
ssl                start_tls
tls_checkpeer      yes
tls_cacertfile     /etc/ssl/ca-cert.pem

The configuration file for the OpenLDAP utilities (ldapsearch, ldapmodify, etc.) is /etc/openldap/ldap.conf. You'll generally only want to list your primary server in this file because the OpenLDAP command line utilities don't chase referals. If you're using them to make changes you'll want to make sure they always talk to your primary server. Here's an example:

URI         ldaps://ldap1.example.com/
BASE        dc=example,dc=com
TLS_CACERT  /etc/ssl/ca.pem

Solaris nss_ldap

The Solaris nss_ldap is configued via the /var/ldap/ldap_client_file configuration file. Solaris has a very complicated system for doing automatic configuration, but I don't really recommend it. More info if you are interested can be found in Sun's documentation and here. The easy route is to just manually edit ldap_client_file. Here are samples for Solaris 8 and Solaris 9.

The Solaris 9 file was actually generated using the new manual option to ldapclient: ldapclient manual -a defaultSearchBase=dc=example,dc=com -a defaultServerList="ldap2.example.com ldap1.example.com". This worked ok, except that it changed the hosts: line in /etc/nsswitch.conf to hosts: ldap [NOTFOUND=return] files and thus it couldn't find the LDAP servers until I fixed nsswitch.conf by hand. (The ldapclient command hung until I did so.)

Solaris 9 SSL/TLS configuration

Note: I don't have this working yet, these are notes on what I've figured out so far.

Solaris 9 supports TLS (although it still seems to use LDAPS instead of StartTLS) for securing connections to the LDAP servers. The client needs to have the CA certificate in a cert7.db file. This is the file format used by the Netscape (and Mozilla) browser. It appears that the only way to create these files is actually to use Netscape (or possibly Mozilla). Apparently you point your browser at the LDAPS port on your LDAP server (it has to be LDAPS, not LDAP w/ StartTLS) and accept the certificate when prompted. Be sure to accept the certificate forever and not just for this session. An appropriate URL to enter into the browser would look like: https://ldap.example.com:636/ Note that this may present a problem with the way I've suggested creating certificates, with the server's FQDN as the CN in the cert and the alias (ldap.example.com) as a subject alternative name. Netscape and Mozilla won't accept those type of certificates (they only check the hostname against the CN), and since I suspect the Solaris client is using the same code, it probably won't either. I'm also concerned about what other certificates might be in that file. The docs also mention that you need the key3.db file from Netscape although it shouldn't have anything in it.

Once you have created cert7.db and key3.db with Netscape, they should be copied to /var/ldap and chmod'd 444. Then it looks like you would add -a authenticationMethod=tls:simple to the ldapclient command line given above.

HP-UX and LDAP-UX

HP has an addon package for HP-UX 11 called LDAP-UX which adds LDAP support. It can be downloaded for free.

Similar to Solaris, LDAP-UX uses a complicated method of storing client configuration in the LDAP directory itself. Unfortunately, unlike Solaris, the LDAP-UX configuration file is binary and can't easily be generated by hand. LDAP-UX comes with a tool to configure the necessary schema and directory entries, but it only supports Netscape/iPlanet Directory Server and Windows Active Directory.

My co-worker Walter Marchuk was able to work out the necessary schema file. We added that to the list of schema files in slapd.conf. We then added a user account just for LDAP-UX (uid=profile_manager) which we gave a real userPassword to and added ACLs to slapd.conf to give that user write access to an ou=profiles area. (Note that if you're using Kerberos and haven't restricted access to the userPassword attribute before now, you'll want to add ACLs to do that now.) After that, Walter was able to run the LDAP-UX configuration tool, tell it we had a Netscape server and to use the uid=profile_manager DN and password and it populated the ou=profiles tree.

From there you should be able to configure the clients following the standard HP instructions.

Load Balancer

In our environment, we eventually pointed all of the clients (both Linux and Solaris) at a load balancer. We initially did this for the Solaris clients because Solaris 8 doesn't seem to ever actually fail over if the first server fails. We later discovered that the decision to fail over in the Linux client is not very smart: if the client can make a TCP connection to the server it assumes the server is up. As we added more and more clients, we eventually overloaded our primary server and it started to crash on a regular basis. However, the server would still accept a TCP connection (but then immediately disconnect before serving any data) so the Linux clients wouldn't fail over. This generally results in locked up client boxes that won't recover even after the server is restarted.

I highly recommend setting up a load balancer. It's a little extra effort, but your servers will have more balanced usage and your environment will be more resilient to server failures. The added resiliency comes from being able to verify that each server is actually serving valid data and not just accepting TCP connections. If a server isn't giving out good data, you tell the load balancer to stop sending new connections to it.

I use the balance load balancer and a secondary script I wrote called balance_ecv to monitor the individual servers and tell balance to stop sending clients to a particular server if it isn't sending back valid data.

To prevent the load balancer from becoming a single point of failure, I clustered two boxes together using heartbeat. heartbeat then has a virtual interface (i.e. eth0:1) with a seperate IP address that it activates on whichever node is primary. (This IP would generally be mapped to something like ldap.example.com in DNS.) If you happen to be using your LDAP servers themselves to run the load balancer and cluster software (as I do), you'll need to modify the slapd init script to tell slapd to only bind to localhost and the normal IP address on the system. Otherwise it will bind to all IP addresses, including the virtual IP. Then when you try to bind balance to the virtual IP it will fail. The difference between the stock Red Hat ldap init script and mine is:

< 	    daemon ${slapd} -u ldap -h '"ldap:/// ldaps:///"' $OPTIONS $SLAPD_OPTIONS
---
> 	    daemon ${slapd} -u ldap -h "\"ldap://localhost/ ldap://`hostname`/ ldaps://localhost/ ldaps://`hostname`/\"" $OPTIONS $SLAPD_OPTIONS

The rest of the related files are as follows. The balance and ecv init scripts which go in /etc/ha.d/resource.d/. The ecv init script requires another script called ecv_loop which is what actually loops and runs the balance_ecv script mentioned above every 2 minutes. Also we have /etc/ha.d/ha.cf and /etc/ha.d/haresources. You'll also need to create /etc/ha.d/authkeys but the format should be obvious from the heartbeat documentation.

Testing

The last step in configuring the clients is to add ldap to appropriate lines in /etc/nsswitch.conf.

Running id username or getent passwd username are two of the easiest commands to test nss_ldap functionality. If things don't work, try turning off checkpeer, and then SSL altogether. Those are major sources of potential problems. If things still don't work, go back and make sure all of the ldapsearch commands in the server testing section work. nss_ldap essentially does not log anything, so troubleshooting can be a bit difficult.


Management

Kerberos

Keytab files

Keys can be added and removed from keytabs via the ktadd and ktremove subcommands within kadmin.

ktutil is the command to use to list the contents of a keytab.

Passwords

Centralized user management tool

If you have a centralized tool for users and admins to change passwords (for example, we have one to simulataneously update NIS, the Windows domain, Kerberos and LDAP), you'll want to integrate Kerberos into it. Here's how we did it.

Our user tool has two parts: one that is run as root for adding users, changing passwords, etc. and one that is run by the users to change their password or shell. As such, we created two Kerberos principals: menu and usermenu. Then we added the following entries in kadm5.acl on the Kerberos master:

# The menu program can do anything to normal principles
menu@EXAMPLE.COM     *     *@EXAMPLE.COM
# And may list all principals (used to generate the discrepancy report)
menu@EXAMPLE.COM  l

# The user menu program can change passwords for normal principles
usermenu@EXAMPLE.COM     c     *@EXAMPLE.COM

The *@EXAMPLE.COM prevents either principal from changing service principals like host/foo.com@EXAMPLE.COM since the asterix will not match the /. See the kadmind man page for details on the ACL syntax.

The menu principal was stored in /etc/krb5.keytab and the usermenu principal was stored in another keytab that was readable by the users. Then we added entries like the following to our tools:

kadmin -p menu -k -q 'addprinc -pw $password $username'
kadmin -p menu -k -q 'delprinc -force $username'

and

kadmin -p usermenu -k -t /etc/krb5.keytab.usermenu -q 'cpw -pw $password $username'
Users use the passwd command to change their password

If you have a PAM environment you should make sure you have an entry like the following in your PAM configuration:

password   required     /lib/security/pam_krb5.so

Whether it should be required, sufficient, etc. and the order relative to other PAM modules will depend on your environment.

If you don't have a PAM environment then users should use kpasswd to change their password.

LDAP

The page up one level has some links to some Perl scripts and modules I wrote to help with LDAP maintenance.

$Id: howto.html,v 1.18 2004/06/11 00:12:06 jheiss Exp $