Find missing files in Linux

The other day, I ran into the age-old situation where a Python package failed to build because the Python.h header file was missing. This happens a lot, so I hardly think about it anymore, but it used to be a big problem.

What is this file? Where did it come from? Who is providing it? Who should be providing it? The easiest solution is to paste the error into StackOverflow, but before you do that, try these steps first to save some time and learn about your system's layout in the process.

When you already (sort of) know where it is

Sometimes, the easiest way to search for something is to have an idea where it might be in the first place. This happens when, for example, the thing you're looking for is in a standard location.

It turns out that standards are (mostly) a thing in Linux, and this is why the Filesystem Hierarchy Standard is important. It lets you know where something should be so you can start your search and end it quickly. Some examples:

• If it's the kernel, it's probably in /boot/.

• If it's a system library, it's probably in something like /lib/, /usr/lib/, etc.

• If it's a system config file, it's probably in /etc/.

• If it's a user config file, it's probably in ~/.config/ or similar.

• If it's a file you created, it's probably somewhere in your home directory.

• If it's a log file, it's probably in /var/log/.

• If it's a temporary file downloaded from the internet, it's probably in /tmp/.

Using these rules of thumb, and having an understanding of your filesystem layout, is probably the fastest way to find the things you're looking for. Do a quick ls in the right directory and find what you need.

However, we'll soon see that this doesn't work for everything.

When it's already in the PATH (binaries only)

Sometimes there's a command on your system—one of your favorites that you use all the time—and you just want to where it's installed. Or maybe you've done something strange to your system PATH and want to find out what directory the tool is being sourced from. Either way, the quickest way to find this info is with the which command:

which COMMAND

$ which python3
/usr/bin/python3

Note that the which command may not be available on every platform, so when it's not, the less-convenient but still very usable type command can be used instead:

type COMMAND

$ type python3
python3 is /usr/bin/python3

When it's on your local drive

This is where the find command really shines. It does an awful lot more than what we're using it for here, but it is also an ideal and simple tool for everyday filesystem searching.

Here's the basic incantation:

find PATH [-name PATTERN]

To find all files in the current working directory (denoted by .):

find .

To find every file on the system named Python.h:

find / -name Python.h

To find every file with Python in the filename under the directory /usr/share/:

find /usr/share/ -name \*Python\*

To find every file on your system with the .h extension that's not a directory or other special file type (commonly known as a regular file):

find / -name \*.h -type f

When it was installed by your package manager

When you've been using a system for awhile, you start to get a feel for the likely origins of certain files.

For example, you may have noticed that executables tend to live in /usr/bin/ and resource files in /usr/share/; this is fairly common because the system package manager by default puts those files in those respective directories.

You may have also noticed that custom installations live in /usr/local/ or /opt/. In many cases with Linux, the local sysadmin is probably you yourself, so unless you did otherwise, custom installations have a high likelihood of being found in those places. For everything else, it was probably installed by the package manager.

So if you think a file should exist, and it doesn't, the local package manager is a good place to start looking for it.

Debian / Ubuntu

dpkg -S is an excellent tool for apt-based systems to find what you need:

dpkg -S FILE

So when Python.h cannot be found, you can quickly determine if it's present on the system by doing the following:

dpkg -S Python.h

In some situations, the returned results will be too long, so you can use a quick grep to narrow the results:

dpkg -S ldd | grep ldd$

Fedora / RHEL

RHELish systems have rpm -qf to determine what package a file came from:

rpm -qf FILE

However, as it requires a full path, you'd need to run find anyway to get that information before this command is useful:

rpm -qf $(find / -name Python.h)

Here it is in action:

$ find / -name Python.h
/usr/include/python3.11/Python.h
$ rpm -qf $(find / -name Python.h)
python3-devel-3.11.1-1.fc37.x86_64

When it's available from your package manager

It could be that the file you're looking for hasn't been installed yet, but you are confident that it can be installed via your local package manager. This is encountered frequently when you're first installing a system or when you're trying to package a container.

Luckily, package managers generally have tools that allow you to search their package contents directly.

Debian / Ubuntu

For Debian-based systems, apt-file is available. Annoyingly though, not only is it not installed by default, but you also need to build its index before you can use it. Oh, well. A temporary pain:

sudo apt-get install apt-file
sudo apt-file update

But now you'll be in apt-file nirvana:

apt-file find FILE

Let's find our Python development header:

apt-file find Python.h

You'll end up with a lot of results, so adding some targeted grep commands would be useful:

apt-file find Python.h | grep '/Python.h$' | grep python3

$ apt-file find Python.h | grep '/Python.h$' | grep python3
libpython3.10-dbg: /usr/include/python3.10d/Python.h
libpython3.10-dev: /usr/include/python3.10/Python.h
libpython3.11-dbg: /usr/include/python3.11d/Python.h
libpython3.11-dev: /usr/include/python3.11/Python.h

Now you know where it should be installed, even though you haven't installed it!

Fedora / RHEL

For RHELish systems, dnf has the provides subcommand:

dnf provides FILE

Similarly to rpm -qf, it wants a full path (ugh), but unlike rpm -qf, dnf provides has a wildcard escape hatch so that you can get on with your life.

Here's how we can search for our Python.h header:

dnf provides '*/Python.h'

grep would have been useful, but the output of this command does not lend itself to parsing very easily:

python3.6-3.6.15-12.fc37.i686 : Version 3.6 of the Python interpreter
Repo        : fedora
Matched from:
Filename    : /usr/include/python3.6m/Python.h

Piping the output to less is more convenient here:

dnf provides '*/Python.h' | less

This will open the output in a pager and allow you to search through it by pressing forward slash (/) and typing a search query.

When you have no idea where to look

If you're not sure where to look for something, chances are, you also don't know what you are looking for. This is probably the right time to go on StackOverflow, or better yet, ask a trusted friend.

Knowing what you seek is the first step to finding it. 🤯