Pages

Sunday 5 February 2012

Upgrading Debian From i386 To amd64

                                                          Upgrading Debian From i386 To amd64

Check wether your Intel/AMD processor supports the 64-bit "long mode"


grep -q '^flags.* lm ' < /proc/cpuinfo && echo yes

If you compile you own 64-bit kernel on the still 32-bit system you just have to install the package gcc-multilib and to start make with :

ARCH=x86_64 make

At the end your 64-bit kernel will smile in arch/x86_64/boot/.
Setup your bootloader to start it with and reboot your system.

Create a 64-bit mini system

mkdir /chroot64
apt-get install debootstrap
debootstrap --arch amd64 wheezy /chroot64 http://ftp.debian.org/debian

"Login" into your mini system and add some more packages:
chroot /chroot64
apt-get install file bzip2 openssh-client lftp
<CTRL-D>

Create a list of the 64-bit binaries of the mini system


The mini system is what you want as your main system apart from the facts that a)it comprise a lot less packages and b) it's not you main system just a sandbox now. We will disperse this sand onto the main, but only the executables, all other files (e.g. your adored config files in /etc remain intact), so collect them:

cd /chroot64
find . -depth | while read f
do
if file -L "$f" | grep -q ELF
then
echo "$f" >> /tmp/bins.txt
fi
done
echo "./etc/ld.so.cache" >> /tmp/bins.txt

Note1: you could run the commands in the chroot too, but then you have to go to its root 'cd /' at first and your 'bins.txt' come off in /chroot64/tmp.
Note2: the last command, adding the ld.so.cache is crucial! It's not an executable, but needed by the dynamic linker (ld.so).

Upgrade your 32-bit system closest to the mini system


In order to have your 32-bit system as much as compatible (from the configuration files POV) to your mini 64-bit system, upgrade all your packages you have installed in the 64-bit mini root.

#chroot /chroot64
#dpkg --get-selections|awk '{print $1}' > /tmp/minipackages.txt
#<CTRL-D>
#apt-get update
#apt-get install `cat /chroot64/tmp/minipackages.txt`

Boot your rescue disk


One step must be done from an external system (Live Linux CD), because you cannot change the very base binaries beneath youself. Or at least it would be quiet risky. So, get a system rescue linux CD or USB stick based distro. One of my favourite is [http://blag.chaox.net/|Chaox]. It has a very recent kernel, it's tiny, it has all the important system tools and even an X interface with nvidia proprietary driver and works both from CD and from USB out of the box.
After you booted it mount your main system eg. under /main:

#mkdir /main
#mount /dev/sda1 /main
#cd /main
#ls   # check if it's really the root of your main system
#cat etc/fstab # see all your other filesystems, eg: /usr
#mount /dev/sda8 /main/usr
#mount /dev/sda11 /main/tmp
...

Overwrite your 32-bit base system


Overwrite the binaries of your main system from the 64-bit mini root:
#cd /main/chroot64
#cpio -pVdu /main < /main/tmp/bins.txt
Unmount all your mounted partitions:
#umount /main/tmp
#umount /main/usr ...
#umount /main

Reboot to your 64-bit system


Now since the all the minimalistic, most important part of your system (binaries and libraries) are 64-bit, you can boot it! Of course the 99% of your system is still 32-bit. They will most probably not work, because the rudimentary libraries are already 64-bit and the dynamic loader will complain it about. You will have a terrible boot :-), but you will able to login and use the base system.
Once in this point I got the following friendly message:

run-init: /sbin/init: No such file or directory
Kernel panic - not syncing: Attempted to kill init!

I booted from USB again and saw that the /sbin/init is there and it's a good 64-bit exec. Kernel lies! Or don't? Just a bit. The init is there but init wanted to load the libc6.so. Of course the 64-bit version which resided in /lib64. On the mini root system there were a symlink: /lib -> /lib64. So I created the symlink (sill using the rescue image) and the next boot succeeded, sigh.

Reinstall all the packages

apt-get install -f

you will need your own creativity, because the issues you will have are always differ. I could not do two migration with the same problems. I can therefore tell you about some common case you will probable experince too. After each fix I'll write here below about you can repeat your previous installation step and it will goes further.

  • libc will not be upgraded automatically
Internal Error, Could not perform immediate configuration (2) on libc6

E: Could not perform immediate configuration on 'libc6'.Please see man 5 apt.conf under APT::Immediate-Configure for details. (2)

Apt downloaded libc6-amd64, but don't dare to exchange it, because it's another architecture. (It does not know, that it is already changed, so it would do nothing just update its own database.) So, you have to help it:

#cd /var/cache/apt/archives
#dpkg -i libc6*deb
Then you can continue the 'apt-get install -f'.
  • clashing libs
The dynamic loader (at the time of writing) does not support to have the 32-bit and the 64-bit version of the same library (soname) in the same directory. If you see warnings like:

ldconfig: libraries libz.so.1.2.3.0 and libz.so.1.2.3.3 in directory /usr/lib have same soname but different type.


you should go there and check, which is the 32-bit version. It won't be needed any more, so you can simply delete it:

# cd /usr/lib
# ls -l libz.so*
lrwxrwxrwx 1 root root    15 Dec 30 21:25 libz.so -> libz.so.1.2.3.4
lrwxrwxrwx 1 root root    15 Dec 30 21:25 libz.so.1 -> libz.so.1.2.3.4
-rw-r--r-- 1 root root 93936 Dec 28 20:10 libz.so.1.2.3.4
-rw-r--r-- 1 root root 93936 Jun 18 11:14 libz.so.1.2.3.0

# file libz.so.1.2.*
libz.so.1.2.3.0: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, stripped
libz.so.1.2.3.4: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, stripped

# rm libz.so.1.2.3.0
# ldconfig   # now you won't get complains about libz


Such warnings are frequent if your old system lags more from the current unstable one, because then the new version of the same libs could have another version, so they won't be simply overwritten.
  • missing "base" packages
Many poinstall scripts want to start 'menu' and 'pyhon'. But apt-get not enough clever to start the package reinstallation with them, you can help again. If you see messages 'update-menus failed', that's about the package 'menu', and 'update-menus' is called from a dpkg hook.

#cd /var/cache/apt/archives
#dpkg -i menu*
#dpkg -i python*

  • debsums does not work
In case you get errors because not working perl module MD5, which is actually a binary perl module, you can install it manually too. Sometimes a package reinstallation complains, because in the older packages some files belonged to another package and then dpkg don't want to overwrite them, you have to force it:

#cd /var/cache/apt/archives
#dpkg -i --force-overwrite perl*

  • reinstall of doc-base failed
Doc base uses ndbm database files (files.db and status.db) which are seemingly architecture specific. The update procedure of the 64-bit doc-base cannot read the remained 32-bit ones. Don't care, they will be regenerated if you delete them:

#cd /var/lib/doc-base/info
#rm files.db status.db

Resinstall all remained 32-bit packages


Sometimes after 'apt-get install -f' succeeds everything is done, your system is migrated. But sometimes a dozen of packages remains 32-bit. They either work (if they find their 32-bit libs) or not. But in any case it's better to reinstall them too. To list what packages are they do the following query:

dpkg-query -W -f '${Package} ${Architecture}\n' | awk '/i386$/ {print $1}'


I like this migration method also because at this point a lot of package could crop up which are not more exists (installed 5 years ago, not needed any more and you haven't even know what they are. You can purge them now gloatingly.
If you want to reinstall all i386 packages:

apt-get install `dpkg-query -W -f '${Package} ${Architecture}\n' | awk '/i386$/ {print $1}'`


But of course it will stop if there are no more existing packages. Then you can "apt-get purge" the really not needed ones, and try the reinstallation command again.
If you system do have tons of such not upgradable packages then you could find this procedure tiring, then you can decide you will care for them later, just the let upgradeable be upgraded, you can perform the command for each package separately:

for p in `dpkg-query -W -f '${Package} ${Architecture}\n' | awk '/i386$/ {print $1}'`
do
  apt-get -y install $p
done


it is slower (reads the apt db for every single package), but it skips the not upgradable ones.
And when you have time and are in the humor to entertain yourself with the remained i386 "what the hell are they" packages, you can list them with their description and judge on their life or death:

# dpkg-query -W -f '${Architecture} ${Package} ${Description}\n' | awk '/^i386/ {print; getline; for(;$0 ~ /^ /;getline) print; print "\n";}'
i386 pbzip2 parallel bzip2 implementation
 pbzip2 is a parallel implementation of the bzip2 block-sorting file
 compressor that uses pthreads and achieves near-linear speedup on SMP
 machines. The output of this version is fully compatible with bzip2
 v1.0.2 (ie: anything compressed with pbzip2 can be decompressed with
 bzip2).


i386 pcscd Middleware to access a smart card using PC/SC (daemon side)
 The purpose of PC/SC Lite is to provide a Windows(R) SCard interface
 in a very small form factor for communicating to smart cards and
 smart cards readers.
 .
 The PC/SC daemon is used to dynamically allocate/deallocate reader
 drivers at runtime and manage connections to the readers.
...
I've done this procedure many times beleive it or not, it's safe, real geeks could even enjoy it. And if you failed (but you would not!) then you could still install the system in the way you would done without this guide.