Add This Directory to Your PATH

You’ve probably read this instruction quite a few times in online tutorials about installing command-line tools. What is this PATH for? Why should we add directories “to” it? This is the subject of this post.

How the Shell Finds Executables

When you type something like ls, your shell has to find what is this ls program you’re trying to run. Typical shells only have a handful predefined commands like cd; most of the commands you use everyday are standalone programs.

In order to find that ls program; your shell looks in a few directories. Those directories are stored in an environment variable called PATH; you can look up its value in most shells using the following command:

echo $PATH

You can see it contains a list of paths separated by colons. Finding a program in these is just a matter of checking each directory to see if it contains an executable with the given name. This lookup is implemented by the which program. The algorithm is pretty simple and goes like this:

cmd = "ls"
for directory in $PATH.split(":") {
    candidate = $directory/$cmd ;
    if candidate exists and is executable {
        return candidate ;
    }
}

This is pseudo-code, but there are implementations in various languages: C, Go, Ruby, Dart, etc.

As you can see, the order of these directories matters because the first matching candidate is used. That means if you have a custom ls program in /my/path, putting /my/path at the beginning of the PATH variable will cause ls to always refer to your version instead of e.g. /bin/ls because /bin appears after /my/path.

You can perform this lookup using the which command. Add -a to get all matching executables in the order in which they’re found. This is what which python gives on my machine:

$ which python
/usr/local/bin/python

$ which -a python
/usr/local/bin/python
/usr/bin/python

You can see I have two python’s but the shell picks the one in /usr/local/bin instead of /usr/bin because it appears first in my PATH. You can bypass this lookup by giving a path to your executable. This is why you run executables in the current directory by prefixing them with ./:

./my-program

This tells the shell you want to run the program called my-program in the current directory. It won’t search in the PATH for it. It also works with absolute paths. The following command runs my python in /usr/bin regardless of what’s in my PATH variable:

/usr/bin/python

For performance reasons a shell like Bash won’t look executables up all the time. It’ll cache the information for the current session and will hence do this lookup only once per command. This is why you must reload your shell to have your PATH modifications taken into account. You can force Bash to clear its current cache with the hash builtin:

hash -r

Now that we know how our shell find executables; let’s see how this PATH variable is populated.

Where Does That PATH Come From?

This part depends on both your shell and your operating system. Bash reads /etc/profile when it starts. It contains some setup instructions, including initial values for the PATH variable. On macOS, it executes /usr/libexec/path_helper which in turns looks in /etc/paths for the initial paths.

The file looks like this on my machine:

$ cat /etc/paths
/usr/bin
/bin
/usr/sbin
/sbin

The actual code to set the PATH variable (or any variable for that matter) in Bash is below:

PATH="/one/directory:/another/directory:/another/one"

By default, Bash doesn’t pass its variables to the child processes. That is, if you set a variable in Bash then try to use it in a program it’ll fail:

$ myvar=foo
$ ruby -e 'puts ENV["myvar"] || "nope :("'
nope :(

Bash allows one to mark a variable as exported to subprocesses with the export builtin command:

$ myvar=foo
$ export myvar
$ ruby -e 'puts ENV["myvar"] || "nope :("'
foo

This is usually done when setting the variable:

$ export myvar=foo
$ ruby -e 'puts ENV["myvar"] || "nope :("'
foo

Technically Bash doesn’t need you to export the PATH variable to use it but it’s better if for example a program you use executes another program; in this case the former must be able to find the latter using the correct PATH.

How Do We Modify It?

Each shell has its own file in the user directory to allow per-user setup scripts. For Bash, it’s ~/.bash_profile, which often sources ~/.bashrc. You can use this file to override the default PATH. It’ll be loaded when starting a session; meaning you have to either reload your shell either re-source this file after modifying it.

We saw in the previous section how to set the PATH variable; but most of the time we don’t want to manually set the whole directories list; we only want to modify its value to add our own directories.

We won’t dive into the details in that post, but Bash has a syntax to get the value of a variable by prefixing it with a dollar symbol:

echo myvar  # prints "myvar"
echo $myvar # prints "foo", i.e. myvar's value

Bash also supports string interpolation using double quotes: You can include the value of a variable in a double-quotes string by just writing $ followed by its name:

echo "hello I'm $myvar"  # prints "hello I'm foo"

We use this feature to append or prepend directories to the PATH variable: prepending means setting the PATH’s value to that directory followed by a colon followed by the previous PATH’s value:

PATH="/my/directory:$PATH"

You usually don’t need to re-mark this variable as exported but using export at the beginning of the command doesn’t hurt.

Wrapping Things Up

Modifying the PATH is not something we do very often because most tools are installed in standard locations—already in our PATH. Most package managers install executables in their own location and need the user to modify their PATH. Homebrew, for example, installs them under /usr/local/bin and /usr/local/sbin by default. If those are not already in the PATH, one needs to add them:

# In e.g. ~/.bash_profile
export PATH="/usr/local/bin:/usr/local/sbin:$PATH"

This means the shell will first look in these directories for executables. It allows one to “override” existing tools with more up-to-date ones.