Hybrid one-time authentication on Ubuntu Server

I just received my order of a couple of Yubikeys. They’re nifty little devices: you can program each with two passwords that it will spit out on command when you plug the key into a computer and touch the button.

Among other interesting keying methods, the new yubikeys support OATH-HOTP, one of the authentication mechanisms of the Open Authentication initiative. The key and the server are initialized with a shared secret, and a counter initialized at 0. The secret is repeatedly hashed with the incrementing counter to generate unguessable one-time passwords. There’s a little more to it, but that’s the gist of it.

So, I want to use my yubikey to authenticate to my dedicated linux server, with OATH-HOTP. Since my server is shared with people who don’t have yubikeys (and are garrisoned in locked down accounts :) ), I needed a hybrid solution that allows traditional logins for some, and OTP logins for others.

I was successful, and here’s how to do it.

First, let’s define more specifically what we want. We want two types of users, a low security user and I high security user.

  • When a low security user authenticates, they enter their unix password, and everything works just fine.

  • When a high security user authenticates, they enter a password, and without hitting enter, they trigger their yubikey, causing an OTP to be appended to the static password. The authentication checks both the OTP and the static password before succeeding.

  • If a high security user loses their OTP token, they can instead authenticate using a very long and very random emergency password. This is nice for situations like the administrator losing his OTP token to log in as root, thus blowing his ability to regenerate his own OTP credentials.

Note that this setup enforces something often forgotten about OTPs: the OTP is a component in 2 factor authentication. If users are prompted for just an OTP, it would simply prove that whoever is authenticating has stolen a yubikey. Combining the static password with an OTP associates something the user knows and something the user has, which requires the unlikely simultaneous theft of both to be of use to the attacker.

Okay, so, what do we need?

PAM OTP

First, we need to get the server to understand OTPs. This requires installing the pam_hotp module, which is not yet widely available. Furthermore, we need to install my slight fork of the module, which fixes some nasty bugs that got in the way of doing what I’m about to describe.

So, clone my copy of hotp-toolkit, and install it on the server. You’ll need the autoconf/automake build tools, and a few other things. On ubuntu, the packages you need are build-essential, autoconf, automake, gengetopt and libpam-dev.

$ git clone git://github.com/danderson/hotp-toolkit.git
$ cd hotp-toolkit
$ autoreconf --install
$ ./configure && make && sudo make install

This will install the PAM module at /usr/local/security/pam_hotp.so. You need to symlink that into /lib/security so that PAM can find it.

# cd /lib/security && ln -s /usr/local/security/pam_hotp.so .

Now, let’s install the module in the authentication chain, and give it a test drive. To be safe, we won’t immediately hook it into all of the system’s authentication, so that we don’t lock ourselves out. We’ll practice on su’s configuration only. That way, sudo would still let us in without an OTP in an emergency (configure your sudoers!) and we get to hang onto any root terminals we already have.

Edit /etc/pam.d/su. You should have something like this (comments removed for brevity):

auth sufficient pam_rootok.so
@include common-auth
@include common-account
@include common-session

This means: if root wants to become root, let him, and pass anything else onto the common authentication setup for the system. We want to subvert that temporarily, and instead invoke pam_hotp for all authentication business.

auth sufficient pam_rootok.so

auth sufficient pam_hotp.so usersfile=/etc/users.hotp window=10 digits=8 debug
auth requisite pam_deny.so

#@include common-auth
@include common-account
@include common-session

If you’re not familiar with what “sufficient” and “requisite” mean in the context of PAM, I suggest catching up around now with the PAM manual, because we’re going to be doing quite a bit of it as we perfect the setup.

To summarize, here we’re saying: if pam_hotp successfully authenticates the user, then that authentication is all we need, and there’s no need to try other modules. However, if we hit pam_deny (which fails no matter what), then the authentication attempt should fail completely. Even more summarized: try to authenticate with an OTP, and reject the attempt if that doesn’t work out.

An aside on the window parameter: this tells PAM how many OTPs in the sequence it should try before failing. Here, we say that your key and the server can be desynchronized by up to 10 OTPs. So, if you accidentally output a couple OTPs too many with your yubikey, as long as the key isn’t leading the server by more than 10 OTPs, the server will resync the next time you authenticate.

Before we can test, we also need a user file, which defines the OTP secrets. So, create the file /etc/users.hotp:

HOTP/E/8 root - 00000000000000000000000000000000

The format of the user file is quite simple: there is one line per user, in the form <OTP type> <username> <password> <HOTP secret>. We said in the PAM configuration that the OTP is 8 digits, which means we have to specify a type of HOTP/E/8 here. There is also HOTP/E/7, HOTP/E/6, and HOTP which is shorthand for HOTP/E/6. What they do should be pretty obvious.

We currently don’t define a password, just to get started, but that is the important part of making the 2-factor authentication actually 2-factor. And, finally, the secret is a 32 character hex string that the server and yubikey have in common. Here I just put in all zeros for testing, we’ll see how to generate a better one later on.

Oh, and don’t forget to chmod 600 and chown root:root the user file. If someone gets their hands on the OTP secret, they can generate valid OTPs, which would be bad.

Now, all we’re missing for testing are a couple of OTPs. Instead of configuring the yubikey right away, we’ll use hotptool, which came with pam_hotp, to generate a couple of them:

$ hotptool -w5 -d8 00000000000000000000000000000000
35328482
30812658
41073348
81887919
72320986
76435986

The tool just gave us the 5 first OTPs for a secret of all zeros. Please don’t use all zeros as your production secret :).

Now, finally, we can test. As a normal user, try to su to root with that first OTP. You should be successful, with lots of spammy debug output from pam_hotp.

$ whoami
dave
$ su
[pam_hotp.c:parse_cfg(118)] called.
[pam_hotp.c:parse_cfg(119)] flags 0 argc 4
[pam_hotp.c:parse_cfg(121)] argv[0]=usersfile=/etc/users.hotp
[pam_hotp.c:parse_cfg(121)] argv[1]=window=10
[pam_hotp.c:parse_cfg(121)] argv[2]=digits=8
[pam_hotp.c:parse_cfg(121)] argv[3]=debug
[pam_hotp.c:parse_cfg(122)] debug=1
[pam_hotp.c:parse_cfg(123)] alwaysok=0
[pam_hotp.c:parse_cfg(124)] try_first_pass=0
[pam_hotp.c:parse_cfg(125)] use_first_pass=0
[pam_hotp.c:parse_cfg(126)] usersfile=/etc/users.hotp
[pam_hotp.c:parse_cfg(127)] digits=8
[pam_hotp.c:parse_cfg(128)] window=10
[pam_hotp.c:pam_sm_authenticate(157)] get user returned: root
One-time password (HOTP) for `root': 
[pam_hotp.c:pam_sm_authenticate(232)] conv returned: 35328482
[pam_hotp.c:pam_sm_authenticate(291)] OTP: 35328482
[pam_hotp.c:pam_sm_authenticate(302)] authenticate rc 0 
                                      last otp Sat Jun 27 01:30:32 1931
[pam_hotp.c:pam_sm_authenticate(325)] done. [Success]
# whoami
root
#

Now, if you try to reauthenticate with the same OTP, you will be denied, but the next one in the sequence works. Congratulations, you’ve set up OTP authentication for su! Take a look in /etc/users.hotp, you’ll see the original line has grown, and now contains the counter state that lets PAM keep track of which OTPs are valid.

Going 2-factor

Okay, so we’ve got the basics: we know how to get prompted for an OTP, and validate it. Now let’s claw our way up to the requirements we had at the beginning, starting with making the authentication 2-factor instead of 1-unusual-factor.

For that, simply fill in the password field of the HOTP user file, as follows. Note that I removed all the fields our authentication attempt added, which resets the counter to 0 and lets me reuse the same five OTPs while debugging. On a production system, you shouldn’t delete that state, since it would make past OTPs valid again.

HOTP/E/8 root supersecret 00000000000000000000000000000000

Now, to authenticate, the user has to type in the password, immediately followed by the OTP. So, for the all-zeros secret, the correct answer to the first su invocation would be “supersecret35328482”, the second would be “supersecret30812658”, and so on. Give it a whirl, and check that the debug output says that both password and OTP are found:

[pam_hotp.c:pam_sm_authenticate(232)] conv returned: supersecret35328482
[pam_hotp.c:pam_sm_authenticate(273)] Password: supersecret 
[pam_hotp.c:pam_sm_authenticate(291)] OTP: 35328482

That’s nice, but not terribly nice. Cleartext passwords aren’t ideal here. Fortunately, my fork of hotp-toolkit supports crypt(3) passwords, which lets us use a password hash instead of cleartext. It’s slightly annoying to generate said password hash, but here’s a bit of Python that does it for you:

#!/usr/bin/env python

from crypt import crypt
from getpass import getpass
from sys import exit

user = raw_input('Username: ')
passwd = getpass('Password:')
repeat = getpass('Repeat password:')
if passwd != repeat:
print 'Passwords do not match'
exit(1)
rand = open('/dev/urandom')
salt = rand.read(8).encode('hex')
secret = rand.read(16).encode('hex')
hash = crypt(passwd, '$6$%s$' % salt)
print 'HOTP/E/8 %s %s %s' % (user, hash, secret)

The script actually generates a complete line for the users file, including a password hashed with random salt and a random OTP secret. If you use it to generate a new configuration for root (and obviously use hotptool to generate the corresponding OTPs, if you don’t use all zeros), you should still be able to authenticate with the combination of your static password and the correct OTP.

Login for legacy users

All we’re missing now on the server side is graceful integration with non-OTP users. Right now, if we were to deploy our current configuration site-wide, only OTP users would be able to log in. So, we need to configure PAM to allow either of a Unix password or an OTP password + OTP to authenticate. To do this, modify the configuration for su like this:

auth sufficient pam_rootok.so

auth sufficient pam_unix.so nullok_secure
auth sufficient pam_hotp.so usersfile=/etc/users.hotp window=10 digits=8 use_first_pass
auth requisite pam_deny.so

#@include common-auth
@include common-account
@include common-session

Note that I removed the debug from pam_hotp’s line, but feel free to add it back while testing this setup, to see what’s going on.

So, here we’ve added an extra preliminary step: we first give a chance to pam_unix, which will prompt the user for a password, and try to validate it against the usual /etc/shadow. If that works, then authentication is complete.

However, if it fails, PAM falls through to pam_hotp, where we’ve cunningly added use_first_pass. This directive tells pam_hotp to reuse the password that was provided to pam_unix. That way, all users get a single “Password:” prompt. Normal users enter their password, OTP users enter the password+OTP phrase, and either method will work if you have the right credentials.

An additional twist to this is that even OTP users have the escape hatch of the unix account: root can authenticate with an OTP, but if he enters the correct Unix password, he can still log in and shunt the OTP part.

This allows us to implement the third part of our initial requirements: by setting the Unix password to something long and random, we give ourselves an escape hatch from the OTP system, in case the OTP token is stolen. If it’s not stolen, the OTP password is much less complex, which encourages the user to not be lazy (if the unix password were too simple, he’d just not use 2-factor authentication).

I recommend pwgen -nys 32 for suitably horribly long, random, complex, impossible to use every day passwords for the unix account. This backup password is meant to be written down and stored somewhere safe.

The client side

Now that we have the server side nailed down, let’s move to the client side. How do we configure a yubikey to generate the correct OTPs?

First, we need the yubikey personalization tool, to reprogram the key. You need a very recent version of the tool (HOTP support is recent). If you want a quick way to build from source on Ubuntu, you can use this script that I wrote. You’ll need the whole repository (for support libraries), and you’ll also need to source the paths.sh file into your shell, so that it can find the tool and associated libraries. Check that ykpersonalize -h has oath-hotp in the list of options.

Now, you need the random OTP secret you generated just above with the python script. This is the part where you synchronize the server and the key. Let’s say that you ran the script, and it gave you 336e6a7b93d5f87a0c84ab9c7c8cdeab as the OTP secret. To program your yubikey, plug it in, and run:

$ ykpersonalize -1 -ooath-hotp -ooath-hotp8 \
    -ouid=000000000000 -a336e6a7b93d5f87a0c84ab9c7c8cdeab

Firmware version 2.1.2 Touch level 1859 Program sequence 3
Configuration data to be written to key configuration 1:

fixed: m:
uid: h:000000000000
key: h:336e6a7b93d5f87a0c84ab9c7c8cdeab
acc_code: h:000000000000
ticket_flags: APPEND_CR|OATH_HOTP
config_flags: SHORT_TICKET|OATH_HOTP8

Commit? (y/n) [n]: y
$

Now, if you trigger your yubikey, with the above secret, it should output 47059834. Trigger it again, it says 79242647. Again, 42319411. Does hotptool agree?

$ hotptool -w3 -d8 336e6a7b93d5f87a0c84ab9c7c8cdeab
47059834
79242647
42319411

Bingo! Try using your yubikey with su. Don’t forget to type in your static password first, before triggering the key.

For serious now!

Now that we’re all ready, we just need to move the PAM config from su’s configuration to the global authentication configuration. Copy the three custom auth lines from /etc/pam.d/su into /etc/pam.d/common-auth, before any existing uncommented auth statements. Leave the existing block alone, apt will want to mangle it if you ever apt-get install PAM modules. Also revert /etc/pam.d/su to its original configuration.

And, finally, the moment of truth, try logging into your server, first with a legacy non-OTP account, then with an OTP account using the yubikey, and finally with the horrible backup unix password for that account.

Successful? Congratulations, you are now using 2-factor authentication with a physical key to generate OTPs! You gain 20 nerd points, unlock the paranoid geek achievement, and are somewhat more metal than you used to be. Enjoy!

Fun with prime numbers

While messing around in Haskell, I came across a rather delightful little nugget of code.

It started out with helping a friend debug code for a primality testing routine. I then started to wonder how I would define such a function in Haskell. It’s a reasonably simple exercise in thinking pure thoughts in Haskell, and if you want to give it a try I suggest trying now, before reading on.

Still here? So, we want to define a function isPrime, having a type of Integer -> Bool. We’ll use the straightforward primality testing algorithm for now: a number N is prime iff no prime number up to sqrt(N) divides it.

So, how would we implement this in Haskell? First, let’s make an assumption. Let’s assume that we have at our disposal an ordered list of prime numbers, from 2 up to infinity and beyond.

primes :: [Integer]

As an aside for my readers who are not familiar with Haskell, I should point out that Haskell is a so-called lazy language. This means, among other things, that you can define infinite lists without having your program hang for ever. While the list is technically infinite, the lazy nature of the language means that its elements will not be computed until they are actually needed.

Now, you might call me out here for begging the question: if I already have a list of all the primes, then I have no need to test numbers using the algorithm above. However, let’s disregard that for a second and see where it takes us.

Given the list of all primes, we need only the primes from 2 to sqrt(N):

candidateFactors :: [Integer]
candidateFactors n = takeWhile (\x -> x*x <= n) primes

This reads reasonably well as English: take elements from the primes list, until an element causes the lambda function to fail. Next, we need to check whether any of these cleanly divides N. Fortunately, the Haskell Prelude has just what we need, and we can without further ado define isPrime:

isPrime :: Integer -> Bool
isPrime n = all (\x -> n `rem` x /= 0) (candidateFactors n)

Again, this should be fairly straightforward to read: N is prime iff, for all candidate factors X, the integer division of N by X is not clean (that is, X is not a factor of N).

This definition of primality is very clean and concise, and not very far removed from the mathematical definition. This is fairly common in Haskell.

However, we must now go back and figure out what to do with our initial bluff: this nice primality predicate assumes the existence of a list of prime numbers from which to draw candidate factors. But this list doesn’t exist, and unless we’re prepared to enter all primes from 2 to infinity by hand, we need a way of programmatically defining that list.

Let’s now reverse our assumption. Assume that we have an isPrime function, with an implementation based on magic, which can tell us whether or not a number is prime without using the primes list. Given isPrime, it is of course trivial to define primes:

primes :: [Integer]
primes = 2 : filter isPrime [3,5..]

This should be read in two parts, centered around the colon: first, on the right, we take the infinite list of all positive odd integers, and filter it using isPrime to retain only the prime numbers. Then, we tack 2, the only even prime, onto the front of that list, and voilà! One infinite list of all the prime numbers.

We are currently high in the spheres of academic code: nice, but quite useless, because we assume too big a part of the solution. This is where it gets fun: let’s remove the two assumptions, smush the two code snippets together, and see what remains:

primes :: [Integer]
primes = 2 : filter isPrime [3,5..]

isPrime :: Integer -> Bool
isPrime n = all (\x -> n `rem` x /= 0) candidates
where candidates = takeWhile (\x -> x*x <= n) primes

Here’s the kicker: the above code compiles, and correctly produces an infinite list of all the prime numbers.

Why? isPrime relies only on primes up to sqrt(N) to decide if N is prime. Examine the base case of determining the primality of 3 given a list containing only 2, and you’ll find that it works correctly. After that, the length of the primes list grows faster than the length of the prefix that isPrime needs to give a primality verdict for the following number (compare f(x) = x and f(x) = sqrt(x)).

I find this definition of prime numbers absolutely delightful. It has a wonderful zen-like quality to it, and yet successfully gets away with defining the list of prime numbers in terms of itself. Even better, it turns out that this implementation is quite efficient as well! The Haskell compiler is capable of turning this roundabout definition into a fast prime number calculator.

After posting this to #haskell to share this delightful definition (which, of course, most of them had already seen - the joys of being new to a language!), I was directed to a research paper by Melissa O’Neill, in which she goes over various Sieve of Eratosthenes implementations, and shows how this beautiful definition can be made even more efficient. Another paper on my stack of books, papers, manuals and novels to read…

How easily does your language of choice let you define a reasonably efficient Sieve of Eratosthenes?

New site!

I’ve rebuilt my website and blog using Hakyll. Thanks Zwe for the many years of faithful service, but I’ve finally gone static.

One side-effect is that comments are disabled, but given the post and comment traffic before this change, I don’t think it’ll be too missed :-).

If you haven’t changed yet, please update your feeds if they still point to natulte.net, as I’m going to be putting another site up there shortly. The correct feed URL is in the related links for the blog: http://blog.natulte.net/atom.xml.

The blog is not yet complete from the layout POV. Most notably, I’m missing easy access to the archives and to the tag cloud. Unfortunately, the site got pressed into service earlier than I planned, when I accidentally nuked my server and had to reinstall everything. Watch this space.

How to write a good changelog

I’ve just read through the Python 3.1 changelog, and it has encouraged me to write up something that I’ve wanted to say for a while, but never quite got round to: dear open source developer, please write good changelogs for your project.

What do I mean by changelogs? Historically, the term has meant something akin to “Our version control system sucks, we need to keep track of logical changes by hand”. This leads to a “changelog” that looks something like this:

1999-03-24  Stan Shebs  address@removed

	* configure.host (mips-dec-mach3*): Use mipsm3, not mach3.

	Attempt to sort out SCO-related configs.  

	* configure.host (i[3456]86-*-sysv4.2*): Use this instead of
	i[3456]86-*-sysv4.2MP and i[3456]86-*-sysv4.2uw2*.
	(i[3456]86-*-sysv5*): Recognize this.

	* configure.tgt (i[3456]86-*-sco3.2v5*, i[3456]86-*-sco3.2v4*):
	Recognize these.

If you’re like any developer not involved with the project, that changelog entry is a complete parse error. You have no idea what the actual change is, whether you should care, whether the change was cosmetic or semantic… A compiler analogy would be that if the version control logs are machine code, this is a barely decorated disassembly.

Humans generally don’t want to read a disassembly of version control logs. They want to know what changed at a higher, human level, units of change that can be grokked in terms of new tools available to them, new syntax, new libraries, what they need to change in their data to ensure compatibility…

A good changelog doesn’t just disassemble version control logs. It presents an executive summary of a whole slew of changes, in terms that make it clear why I, the end user, should be giving a damn.

Thanks for your attention.

Don't panic

It’s Towel Day. Know where your towel is, and remember Douglas.

I’m heading out to California for a week or so. My towel is in my travel bag. Where is yours?