Restoring a wiped out dpkg status file

2009.08.13 18:23 by Leo Antunes - 6 Comments

UPDATE: Nevermind, nothing to see here, move along… Steve Kemp helpfully pointed out to me the obvious thing I had missed: status-old isn’t the only backup, there’s /var/backups/dpkg.status.*! Oops… doubly stupid.

After seeing madduck’s post and feeling my dpkg was certainly not as quick as it used to be, I decided to try the suggested status-file cleanup.

Since madduck is an old-time Debian developer (actually just one year longer than I am, but certainly way more active) I just assumed he’d have better understanding of dpkg’s inner workings than I do, so I blindly – and wrongly – copy-pasted his code and managed to erase my dpkg status file.
No backups were made out of sheer trust-induced stupidity and the status-old file was somehow also nuked, probably because of something I ran before noticing the problem.

It’s obviously not his fault that I didn’t pay attention to what I was doing, but by then I had royally messed up my machine, basically nuking dpkg, so I had to come up with this ugly little script to try and rebuild the status file.
If someone else managed to do the same absurd mistake, this could help.

#!/bin/bash
 
ls -1 /var/lib/dpkg/info/*.list | sed -r 's/.*\/(.*).list$/\1/' | while read PACKAGE; do 
	echo Package: $PACKAGE
	if [ -d /usr/share/doc/$PACKAGE ]; then 
		if [ -f /usr/share/doc/$PACKAGE/changelog.Debian.gz ]; then
			VERSION=`zcat /usr/share/doc/$PACKAGE/changelog.Debian.gz | head -1 | sed -r 's/.*\((.*?)\).*/\1/'`
		elif [ -f /usr/share/doc/$PACKAGE/changelog.Debian ]; then # downfall of the ia32-* hack
			VERSION=`head -1 /usr/share/doc/$PACKAGE/changelog.Debian | sed -r 's/.*\((.*?)\).*/\1/'`
		elif [ -f /usr/share/doc/$PACKAGE/changelog.gz ]; then
			VERSION=`zcat /usr/share/doc/$PACKAGE/changelog.gz | head -1 | sed -r 's/.*\((.*?)\).*/\1/'`
		else
			VERSION=0 # force upgrade, probably locally-built
		fi
 
		echo Status: install ok installed
		echo Version: ${VERSION}
		STRIP_FIELDS="Version|Package|Size|Filename|MD5sum|SHA1|SHA256|Tag|Task"
		CONFFILES_LIST=conffiles
	else
		echo Status: deinstall ok config-files
		STRIP_FIELDS="Package|Size|Filename|MD5sum|SHA1|SHA256|Tag|Task"
		# in this case we treat all remaining files as conffiles
		# (if there are no docs, there's no .conffiles either)
		CONFFILES_LIST=list
	fi
	if [ -s /var/lib/dpkg/info/${PACKAGE}.${CONFFILES_LIST} ]; then
		echo Conffiles:
		cat /var/lib/dpkg/info/${PACKAGE}.${CONFFILES_LIST} | while read CONFFILE; do
			if [ -f $CONFFILE ]; then
				echo " $CONFFILE `md5sum $CONFFILE | awk '{ print $1 }'`"; 
			fi
		done
	fi
	# "grep-available --invert-show" doesn't work as expected, so we grep the fields out
	(grep-aptavail --mmap -P $PACKAGE -X || (echo package $PACKAGE not found, leaving minimal entry 1>&2 ; echo -e "Version: 0+borkdstatus\n")) | grep -Ev "^($STRIP_FIELDS):"
done

Known limitations:

  • Needs root access to create some conffile md5sums
  • Can’t detect virtual packages, like nvidia-kernel-* stuff and probably everything created with module-assistant. A reinstall of the created packages in /usr/src/*.deb should suffice
  • Some packages do some version black-magic in the changelog (gcc-defaults comes to mind). I couldn’t find a better way to get the actual installed version, but a dist-upgrade after restoring the file should just re-install the affected packages.
  • Won’t correctly detect packages in an intermediary state like “unpacked, but not configured” (haven’t tested this)
  • Can’t detect purged packages at all, though this shouldn’t be a problem

Did I miss something? Is there a tool or a dpkg option that does this in a smarter/easier way?

  1. Maybe /var/backups/dpkg.status would have helped you out?

    Reply

    Oh… so much for my little ugly script I guess…

    I looked everywhere inside /var/lib/{apt,dpkg}, figuring something like this would be there, and didn’t even bother to read the source looking for it.
    Well, nice quick exercise at least! :)

    Reply

    It’s a shame the contents of /var/backups aren’t more discoverable. But I’m pleased I did find them somehow – more than once I’ve rescued a system’s passwd/shadow file.

    These days I tend to auto-backup mysql + /etc/ beneath /var/backups/local, even if that is a little naughty. It doesn’t always help but its a simple enough to be worth doing regardless.

    Reply

  2. Surely you didn’t manage to delete /var/backups/ too? you should have lots of slightly old status files in there to start from.

    Reply

    Yeah, Steve Kemp mentioned that as well.
    I initially assumed status-old was the only backup and didn’t bother to look further.

    For the record I filled two bugs (here and here) to better document the existence of these backups and to actually shift around the code that creates them from cron to dpkg. Hopefully this could avoid another blind guy like me failing to notice the obvious!

    Reply

  3. Hi – How do I solve the problem of BOTH available & available-old missing ? I do have status & the backups though.
    I figured out that formatting will be lost, so also posted the entire issue here-
    http://askubuntu.com/questions/460658/both-available-old-available-missing-from-dpkg-dpkg-error-failed-to-open

    Reply