#16: Introduction to lsof and fuser

You've probably heard of lsof and fuser. Since on UNIX systems everything is a file, these tools are absolutely important to know for each system administrator. They can make your work easier and they can also help identifying attacks on your system. lsof and fuser are both designed to show open files. lsof stands for list open files and fuser for file user. So both do about the same job and often fuser is referred to as the to more primitive lsof, but that's actually not the case. fuser has a slightly different approach than lsof, so both tools have their right of existence and the question “lsof vs. fuser” is not a question of which tool is great and which is bad. But what's the difference between these tools except that lsof can do more? Well, lsof can do everything fuser can do except one thing and that is to kill the processes which occupy a file. So that's one difference. Therefore, if you want to kill all processes which are making use of a file, you could either run

kill -KILL `lsof -t myfile`

which passes all processes listed for that file to kill, which then sends a SIGKILL to all of them. Alternatively you may simply run

fuser -k myfile

This is only one command. So for this purpose you would prefer fuser over lsof. By the way, if you want to send, e.g., a SIGINT instead of SIGKILL, you could run either of these commands:

kill `lsof -t myfile`
fuser -k -INT myfile

So that's one difference in addition to the complexity. But what's the major one? The main difference is that fuser is designed to be an easy of use tool for displaying processes which use a specific file, whereas lsof can also list processes occupying files with only specifying their type. Therefore, lsof is also able to list all open UNIX sockets on your system or established TCP connections etc.

If you use fuser for a file or directory, by default only a list of process numbers is shown with trailing characters for the type of access. These characters are:

• c: stands for current working directory so this directory is the CWD for the process with this PID
• e: the file is an executable being run in this process
• f: the file is currently opened for reading
• F: the file is currently opened for writing
• r: this is the root directory
• m: this is a memory mapped file or shared library

With the parameter -v fuser also shows some more information for each process like the owner of this process and the command. That's basically it, some more options are mentioned in the man page. You see: fuser is very easy to use and that's its main advantage. Now let's come to lsof.

When you run lsof, you normally get something like this:

COMMAND  PID  USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
bash    5455  test  cwd    DIR    9,1     1024    2 .
lsof    7515  test  cwd    DIR    9,1     1024    2 .
lsof    7516  test  cwd    DIR    9,1     1024    2 .

I won't explain all columns of this output but the main ones. First there are the command, the PID and the owner/user of this process (note: this is not always the owner of the actual file!). The next column FD reports the type of access (FS stands for file descriptor). This is mainly what fuser shows with its trailing characters except that it's far more complex. This column can have more values and may also consist of two parts, not just one. In this example we have cwd for current working directory. Executables being run are marked with txt, the root directory with rtd and memory mapped files with mem. For read access lsof may append r as a second part to the file descriptor or w for write access or u for read and write access. These are just the values fuser shows you as well. For a complete list please read the man page.

The next column contains the type of the file (whose name is mentioned in the last column NAME, here it is . which is the current directory). This can be one of many values. The most important are IPv4 for IPv4 sockets, IPv6 for IPv6 sockets, unix for UNIX sockets, DIR for a directory, FIFO for a FIFO/named pipe and REG for a regular file.

That was the easy part. We now come to the more complicated things for lsof and that are its parameters. If you know (most) of them this is really the fun part, but it may take a while until you know the most important commands by heart. You don't really have to know all commands since you can look them up in the man page (which is a very good one, really!) but it's good to know at least some of them by rote. I'll just list the parameters which I think are most important to know (by the way, if you run lsof without any parameter, not even a filename, all open files on your system are listed).

• lsof -t: Show only PIDs (we already used this before).
• lsof -c bla: Show processes starting with bla
• lsof -c /bla/: If you start and end with a slash, bla is assumed to be a regular expression. Only processes matching this pattern are displayed. lsof -c /^bla/ is equivalent to lsof -c bla.
• lsof +d dir/: Show all opened instances of this directory and the files it contains (mind the prefix +!). This option is equivalent to lsof dir/ dir/*
• lsof +D dir/: Does the same but scans the directory recursively. This may be very slow.
• lsof -d: Specify a comma separated list of file descriptors (column FD) to include. If you prefix a value with ^, it will instead be excluded from view.
• lsof -u user: List files opened by user user. You may pass multiple users separated by commas. Users prefixed with ^ will instead be excluded. This parameter also accepts raw UIDs instead of login names.
• lsof -U: Show open UNIX sockets.
• lsof -i[46][protocol][@hostname|hostaddr][:service|port]: This one is very interesting and that's why I listed all its optional parts. Executed with -i, lsof lists all established Internet connections and listening server daemons. This is invaluable to see where your computer is connecting to, including established connections from a remote client to a local server daemon.
  Instead of showing all connections, you may also list only IPv4 or IPv6 hosts (either lsof -i4 or lsof -i6), only TCP or UDP connections (lsof -iTCP or lsof -iUDP), only connections from one particular host (e.g. lsof -i@example.com or lsof -i@192.0.32.10) or connections by port (e.g. lsof -i:imap or lsof -i:143). For your information: named ports are listed in /etc/services. You may also specify everything together: lsof -i4TCP@example.com:imap. This will show you all IPv4 connections over TCP to example.com on the IMAP port.
  Don't be shocked if you try lsof -i on your local machine! If you have opened a browser and maybe a mail client, you'll see many connections, but that's okay. Particularly IMAP mail clients keep a lot of ports open.
• lsof -n: No DNS resolution, may speed up host lookups by only showing plain IP addresses instead of hostnames. This is mainly used with -i.
• lsof -P: Don't resolve port names, display only plain port numbers instead of their names as defined in /etc/services. This is handy in combination with -i if something is wrong with port name resolution or if you don't know which port number a name is mapped to.
• lsof -a: Connect all parameters with an AND operator. By default, all parameters are connected by OR. So to list all open UNIX sockets belonging to user test, run lsof -U -u test -a. Without -a, this command would list all files which are either UNIX socket OR belong to user test.
• lsof -l: Display UIDs instead of login names for file users.
• lsof -p: Comma separated list of PIDs to include or to exclude (again, prefix ^ excludes PIDs). If you want to see all files opened by a specific process, just run lsof -p 123 without any further parameter (of course replace 123 with the particular PID).

That's a long list but actually these are only a few commands. There are many more, but these listed parameters should really burn into your memory. Knowing them makes handling lsof a lot easier and less painful.

One last thing to mention concerning security: by default lsof allows all users to list all files. To avoid this, make sure to compile lsof with the HASSECURITY option. To apply this restriction only to non-socket files, you may also compile with HASNOSOCKSECURITY. Another compile-time option to note is HASDCACHE, which enables creation of device cache files. This may also be a security issue. Enabling device cache files will improve the speed but disabling it will give you more security.

Read more about lsof and fuser:

• About.com: lsof man page
• About.com: fuser man page