Τρίτη 19 Απριλίου 2011

frelink: restoring deleted open or loop-mounted files in Linux

Has it ever happened to you that an important file has been deleted (by yours or someone else's mistake) but this file is still open by a process or loop-mounted somewhere?

In the "still open by a process" case, if the file is relatively small, as well as static or append only, you can use the well-known tricks of
cp /proc/<pid>/fd/<fd> /new/file/path
or
tail -c +0 -f /proc/<pid>/fd/<fd> > /new/file/path
to bring it back from the land of the "unlinked" by copying to someplace safe.

But what happens if the file is too big to copy or changes in a random fashion so that "tail -f" can not ensure that the copy will have the latest updates? Even worse, what happens if the file is loop-mounted in which case there is not even an fd to copy it from?

The most suitable solution in this case is probably to do the "undelete" process at VFS-level in the kernel. Such a solution has the potential of being compatible with all linux filesystems (since they all go through VFS).

The proposal to introduce a system call (flink) for this use-case seems to appear every now and then in LKML (e.g., this post) but every time it gets rejected for (pretty legitimate) security reasons.

Therefore, the only reasonable way to implement the functionality is probably via a loadable module + userspace tool.

frelink is a "proof-of-concept" implementation of how this process can work, from the viewpoint of the user. It has been based on the code from the fdlink project, an earlier effort in this area.

It should already be able to undelete both from /proc fd links and from loop devices, if your kernel is supported.

Unfortunately, especially for the loop mount case we currently have to use a very ugly hack (a jprobe if you are wondering) to get the necessary information. The code is also in a "pre-alpha" stage from a robustness perspective, with very little testing so far.

Nevertheless, it has already proved quite useful in some "extreme" cases in the last few days, so I thought it would be good to share it (you should read its big fat warning before using it of course).

Since the tool is now in the "it works" phase, what remains is to make it "work right" (making it fast is pretty much pointless in this case). This will probably require changes in the kernel infrastructure like e.g., in the loop module.

In the meantime, testing and comments will be happily accepted here or at my pktoss address at the mail service from google ;)

-Pantelis







4 σχόλια:

  1. This is useful to recover recently viewed Flash videos from website such as Youtube.
    The latest Flash player (or Flash video viewer) does save the .flv file in /tmp, but as soon as it creates it, it deletes it. So, you can recover by looking into the /proc filesystem.

    Discussion at ubuntu-gr forum on recovering the Flash videos.

    ΑπάντησηΔιαγραφή
  2. For this case I would choose the cp method I showed first, because these files are small and static (once fully downloaded). With the cp method you don't need to install anything and you also don't have the risk of messing with kernel structures. But think about recovering a big VMWare virtual machine for example. The file size could be too big to copy, given the free disk space and random parts of it could also be changing so tail doesn't help either.

    ΑπάντησηΔιαγραφή
  3. https://ebalaskas.gr/blog/2009/07/02/file-recover-via-lsof/

    ΑπάντησηΔιαγραφή
  4. @ebalaskas: I wrote in the beginning of the post that for static files you can (and should better) use the cp method you also describe in that post.

    Infact you don't even need lsof. e.g., find /proc/*/fd -lname "myfile" -exec cp {} $HOME/ \;

    But if the file is too large, or changes randomly (running VM image, innodb tablespace, etc etc) cp won't save you.

    Even worse, what about mount myfile.iso tmp -o loop && rm -f myfile.iso

    In that case there is nothing in /proc to copy from, as far as I know at least

    This was why frelink was written

    ΑπάντησηΔιαγραφή