Revision [29934]
This is an old revision of howPuppyWorks made by darkcity on 2014-01-28 11:57:47.
| cn::de::es::fr::gr::hu::it::ja::kr::nl::pl::pt::ru::se::us::vn:: |
HomePage > ComponentHowTo Components and HowTos
How Puppy Works
Latest Information
Note, information on this page may be outdated. For the latest version see How Puppy Works (PuppyLinux.com)
 | The information on this page may be out of date. Any help updating appreciated, more info on the WikiIndex Wiki Index page. |
Details
One thing to understand is that Puppy is unique. Barry built Puppy from scratch, file by file, so the architecture is not based on any other distro.puppy boot process
layout when the live-CD boots: vmlinuz, the Linux kernel, loads into RAM, followed by image.gz, which loads into a fast "ramdisk" . The file is ".gz"-compressed, so it is uncompressed into the ramdisk and then the ramdisk becomes the very basic Puppy filesystem, that is, designated "/" and called top or root directory together with all subdirectories /bin, /sbin, /lib, /dev, /tmp, etc.
A fundamental objective of Puppy is that everything should load into ramdisk, thus freeing up the CD drive so that you can use the CD drive for other purposes eject Also, if everything is in RAM, application startup and running speed is stunning. Besides, a computer without spinning media (CD, hd) is much more rugged for use in boats, offroad-cars asf..
When the power goes off, RAMdisk content is lost. To retain your settings, such as configuration changes, email, browser history asf Puppy during bootup looks for a suitable hard drive partition, and if one is found, creates a file on it called "pup001". I arbitrarily chose "pupxxx" where the "xxx" is three numeric digits. The three numeric digits have no particular meaning either, except we are currently using pup001 for the live-CD and pup100 for USB installations of Puppy.
pup001 is a single file (easy to backup), but internally is a complete ext2 filesystem. After creating this file, Puppy mounts it, by means a "loopback device", onto /root directory. This directory /root is your home directory -- all your personal stuff goes in there. Thus, when you shutdown, all will still be there next time you start Puppy.
I do need to explain the bootup steps properly -- Puppy first has to mount the hard drive partition onto /mnt/home. This must be done before Puppy can get access to pup001 and mount it on /root. So you can see two mount points in the diagram.
usr_cram.fs file
Most of the files in Puppy, in any Linux distro for that matter, are in /usr - furthermore there is normally no need to write to anything inside /usr (that is, no need to edit, create or delete files). So the entire content of /usr is compressed as a single file, named usr_cram.fs and mounted as-is on /usr, by means of a "loopback device" -- the actual technique is not the main issue here, but you need to be aware that all of /usr is really only a single file, containing a compressed and read-only filesystem. Utilizing union-fs, however, changes in /usr can be made at run-time while usr_cram.fs remains unchanged.
part of the diagram in an orange color to illustrate the different places where the file usr_cram.fs may be located. Those four numbers, 1, 2, 3, and 4, are the search order in case of booting off CD:
1. At bootup, Puppy will first look to see if usr_cram.fs∞ is at "/" in the ramdisk (which meant that it was inside image.gz originally)
2. failing that, Puppy will look in /root (that is, inside pup001)
3. else in /mnt/home (in the hard drive partition)
4. finally Puppy will look on the CD (the slowest media of these)
Even if Puppy falls through to option 4, it will do his best not to leave usr_cram.fs on the slow CD. We really want to copy it to the ramdisk, or failing that to pup001 in the hard drive. If we left it on the CD and just mounted it on /usr, Puppy will run slowly and the CD drive will not be available for other purposes. With enough RAM usr_cram.fs copies to "/" in the ramdisk, then mounts it on /usr.
How much RAM is "enough"? Puppy is happiest on a PC with at least 128M RAM. 128M is a good size -- it is enough for image.gz (uncompressed) and usr_cram.fs to both reside on ramdisk.
What if your PC has only 64M, 48M, or even only 32M RAM? Well, usr_cram.fs is not going to fit in the ramdisk. Just ball-park figures, image.gz uncompressed is about 10M, usr_cram.fs is about 50M.
Puppy does have a card up his sleeve, so to speak -- if the PC has a swap partition Puppy will automagically use that, to increase the virtual size of the ramdisk. If you have ever installed another Linux distro on the PC, chances are it created a swap partition, so it is already sitting there ready for Puppy to use. If your PC has 128M of RAM, Puppy will allocate 62M of that to ramdisk, however if the PC also has a 250M (for example) swap partition, then the effective size of the ramdisk becomes 62+250 = 312M!
However, I made a decision with the startup logic of Puppy, only to copy usr_cram.fs into ramdisk if there is at least 62M of physical RAM allocated to the ramdisk. So, for PCs with less physical RAM than 128M, usr_cram.fs will never be copied into ramdisk, even if there is a swap partition.
Another issue is the search order, those four numbers 1, 2, 3, and 4. When you have booted up Puppy from the CD, if you copy usr_cram.fs from the CD or /, to /root (ie., inside pup001), that will be the second place Puppy looks next time he boots. Or, copy it onto "/" on the hard drive -- that's the 3rd place Puppy will look. So even on a PC with only 32M RAM, you can bootup from a live-CD and have the CD drive freed up for other purposes.
PCs with less than 128M really do need a swap partition for Puppy to work properly. Even though Puppy can boot up from live-CD on a PC with only 32M RAM, some of the applications are memory hogs -- Mozilla for example. Mozilla is not viable on a PC with less than 128M RAM. However, as mentioned above, a swap partition increases the effective size of the ramdisk, so you can get Mozilla to work on PCs with very little RAM, albeit slowly. It is also a good idea to add a swap file, if you have some spare space on the hard drive - and your partition tables allows it.
Think about that search order a bit more... #3 is interesting. Puppy looks on the hard drive for usr_cram.fs, but how would the file get there? You might have put it there manually or run the install-to-hard-drive-option-1 script.
However, when you boot a newer version of Puppy from CD, Puppy will find the old usr_cram.fs ! ...not so good. Or, an old usr_cram.fs if you put it in /root, for that matter.
To get around that problem Puppy has a technique for checking that usr_cram.fs is the correct version. Puppy checks this, and will ignore any usr_cram.fs files it finds that have the wrong version number. The technique used is when usr_cram.fs is created, its size, in bytes, is recorded in /etc/sizeusrcram, which can then be used to check against usr_cram.fs at boot-time.
with puppy 1.0.1+ it is often desirable to pursue the "everything on one piece of media" policy, but you may split puppy to overcome space limits on that media (CD-RW or USB stick).
Package installation
A key point about this architecture is that /usr is read-only. After downloading a .tar.gz "tarball" or RPM package you cannot install it into Puppy because most packages want to install part of themselves into /usr.
However, version 1.0.1 has revolutionised package management in Puppy by using UNIONFS to make /usr read-write. Any files written to /usr are actually stored in /root/.usr, but look as if they are in /usr to the RPM packs and such. A package manager with non-volatile installation, removal and dependency checking named PupGet is feasible thus !
PupGet offers the packages of the Puppy Unleashed package suite. All of these packages are available for download -- from the Puppy menu choose "Setup -> PupGet package manager", and follow the steps in the GUI.
The Unleashed packages are all on the Internet, and PupGet can download them individually to install them, however Linux developers can create their own live-CDs from the complete Unleashed package suite. The live-CD that you are using now was created from Puppy Unleashed.
Puppy Unleashed enables you to put together your own custom Puppy boot-CD from a choice of hundreds of packages (applications). For further information, see the Puppy Unleashed page.
Booting from USB
Flash technology
A 128M Flash drive matches perfectly with Puppy. Consider, Puppy is only about 50-60M, so will fit on a 128M USB drive leaving over 70M free for personal data. Yes, a complete portable operating system and personal data, all on the one tiny non-spinning noiseless device!
The downside to Flash technology is limited number of write-cycles. After so many times, the stick will collapse. I have seen figures as low as 50,000 writes - see FlashDetail .
Anyway, the good news is that Puppy is especially designed to have no writes to the Flash drive during a session, enormously extending its life span. Take a look at this diagram:
liveusb image
When Puppy boots from USB, the steps are much the same as for the live-CD. The kernel vmlinuz is loaded into RAM, image.gz is uncompressed and loaded into a ramdisk.
puppy boot from USB
Puppy then searches for usr_cram.fs, first looking in "/" in the ramdisk, then in "/" in the USB partition. The first case would only be if Puppy had been created with usr_cram.fs inside image.gz, which is not the normal situation. So it falls back to number 2. Puppy will find usr_cram.fs in the USB partition and will mount it on /usr.
Puppy will attempt an optimisation. If there is enough RAM, Puppy will copy usr_cram.fs into the ramdisk and then mount it on /usr. This takes a little while but improves running speed. However, even if Puppy does leave usr_cram.fs on the USB drive and mounts it from there onto /usr, that is not a problem as /usr is read-only. There will be no writes to /usr, so the lifetime of the Flash drive is not compromised (Puppy version 1.0.1 has UNIONFS which makes /usr read-write, but all writes actually go to /root/.usr, which is in the ramdisk, as described below).
The very first time that Puppy boots from USB, the file pup100 does not yet exist -- Puppy has to create it. Puppy creates it, and mounts it directly on /root - physically on the stick - by means of a loopback device. So far, the architecture is basically the same as for the live-CD.
The lifetime of the Flash device is compromised by writes to /root, that is, to pup100. This is your home directory, and applications store all kinds of stuff in it. A lot of writing will be happening to /root. The Linux kernel does cache the writes, but even so, the cache will be flushed frequently.
A significant difference occurs the second time that Puppy is booted from USB. If there is enough RAM (remembering that the ramdisk can also use a swap partition if it exists, so the available space is effectively the size of RAM plus size of the swap partition), then Puppy will mount pup100 onto /mnt/pupxxx directory in order to copy all the files from /mnt/pupxxx to /root.
Thus, a complete copy of pup100 is in /root, and no writing will occur to physical pup100 during a session. At shutdown, all the files in /root are copied back to /mnt/pupxxx, thus updating pup100. We could, in theory, backup to pup100 at regular intervals, rather than only at shutdown, but that is not implemented in the current version of Puppy -- quite frankly, I have very rarely had Puppy crash, and the only situation that would cause backup not to occur would be a power failure.
Puppy is highly intelligent, and the optimisations described above are only implemented if there is enough RAM (+swap). If there is not enough RAM, then obviously pup100 cannot be copied into /root in the ramdisk. During boot puppy tells you whether it manages to get all into RAM.
To give an idea of what is "enough" RAM, a PC with 256M RAM matches well with a 128M Flash drive. The pup100 file will be about 60M, and there's enough RAM for everything to load into ramdisk. On the otherhand, a 256M Flash drive would have a 180M pup100 file which would be too big. With a 256 flash-drive 512MB RAM are good to keep all in RAM. If the PC has a swap partition, Puppy will automatically use this to increase the effective maximum size of the ramdisk. Whatever the situation on your PC, Puppy will work out the best configuration, totally automatically.
Updating, archiving, configuring
As you can see, Puppy consists of only 4 files. (actually, it is possible to have usr_cram.fs inside image.gz, reducing Puppy down to just 3 files -- the web download sites may have that configuration available. It is convenient for booting off a network to have just the two files vmlinuz and image.gz to worry about)
To update to the latest version of Puppy, you just need to get the latest vmlinuz, image.gz and usr_cram.fs. Updating is so very simple, but with v0.9.8 I made it even simpler by building an update option into the installation scripts. That is, boot up the latest live-CD and run the install-to-USB, install-to-Zip or install-to-hard-drive script and choose "update" rather than "new installation".
It's so simple, you could do it manually if you wish.
Puppy does have a couple of archive programs, including one that I wrote that backs up only the changed files from /root each time. However, all your personal data, really, a snapshot of the entire the state of the system, is in the pup001 or pup100 file, so you can save a complete system snapshot just by making a copy of the pup001/100 file. Then compress it, like this (assuming the copy is named pup100-12dec04):
gzip pup100-12dec04
and you will have pup100-12dec04.gz. Archive that anywhere you want.
You can mount an archived pup100 file at any time, and view its contents:
losetup-FULL /dev/loop2 pup100-12dec04
mount -t ext2 /dev/loop2 /mnt/data
You can also view the contents of pup100-12dec04 from Windows, by using the Explore2fs (ViewPup001FromWindows) program. Having many pup100 e.g. on a DVD-RW makes sense to fully utilize all DVD space while maintaining the "1 media policy" and full "all-in-RAM speed" by keeping a moderately sized individual pup100. From the menu, pup001 on hd can be enlarged (you may know this word from your spam mail).
Booting steps
(section not finished)
Booting Puppy Linux (obsolete but educational)
Step 1
The first things that happen is the Linux kernel in file "vmlinuz" loads into RAM and file "image.gz" is uncompressed and loaded into /dev/ram0, a 11'264 K fixed-size ramdisk.
In the case of a "big image" Puppy, that is, with file usr_cram.fs inside image.gz, there has to be the kernel boot parameter "ramdisk_size=" to specify a suitable size to hold all of image.gz (uncompressed) and usr_cram.fs plus some spare space. For example:
ramdisk_size=63488
For the "big image" situation, /dev/ram0 will be large, in the above example 63488 Kbytes.
Step 2
The program "/sbin/init" is executed. In the case of the "big image" Puppy, /sbin/init is just a link to /bin/busybox, as init is a program built-in to busybox. The normal situation though, is that /sbin/init is a shell script. Here it is:
#!/bin/sh
#v0.9.8 booting up in /dev/ram0, move root / to tmpfs...
INITARGS="`echo -n "$@" | sed -e 's/\/dev\/ram0/tmpfs/g'`"
PATH=/bin:/sbin
mount -t proc none /proc
mount -o remount,rw / #-n option not needed with busybox mount.
STARTUPFSSIZE=`cat /root0/.etc/ramdiskfssize` #currently set to 11264K (file gets updated below)
#specifying ram1 here as it isn't in use...
SIZERAMDISKM=`disktype /dev/ram1 | grep "Block device" | cut -f 4 -d " "`
SIZERAMDISKK=`expr $SIZERAMDISKM \* 1024`
#total ram, less any shared video...
PCRAMSIZE=`free | head -n 2 | tail -n 1 | tr -s " " | cut -f 3 -d " "`
#...note, busybox free output format different from standard free program.
if [ $PCRAMSIZE -gt 220000 ];then #256M
SIZEFILLK=`expr $PCRAMSIZE \/ 2` #half of ram.
else
if [ $PCRAMSIZE -gt 117000 ];then #128M
SIZEFILLK=67584 #63488
else
if [ $PCRAMSIZE -gt 60000 ];then #64M
SIZEFILLK=16384 #leaves about 7M free in f.s.
else
SIZEFILLK=11264 #only about 2M free in f.s.
fi
fi
fi
#precaution...
if [ $SIZEFILLK -lt $STARTUPFSSIZE ];then
SIZEFILLK=$STARTUPFSSIZE
fi
PHYSICALFILLK="$SIZEFILLK"
#want to know if there is a swap partition available...
SWAPPART= SWAPINFO="`fdisk -l | grep "Linux swap" | head -n 1`" if [ ! "$SWAPINFO" = ];then
#we can make the ramdisk real big now...
SWAPPART="`echo "$SWAPINFO" | cut -f 1 -d " "`"
SWAPSIZE="`fdisk -s $SWAPPART`"
#SWAPSIZE=`expr $SWAPSIZE \/ 2` #let's use half of the swap partition.
#...nah, grab the whole lot. Puppy can create a swap file later if reqd.
SIZEFILLK=`expr $SIZEFILLK + $SWAPSIZE`
#0.9.8... but have to delay swapon until rc.sysinit running... see below...
fi
mkdir /new_root
mount tmpfs /new_root -t tmpfs -o size=${SIZEFILLK}k;check_status $?
cp -a /bin /new_root/
cp -a /dev /new_root/
cp -a /etc /new_root/
cp -a /lib /new_root/
cp -a /mnt /new_root/
cp -a /root /new_root/
cp -a /root0 /new_root/
cp -a /sbin /new_root/
cp -a /usr /new_root/
cp -a /proc0 /new_root/proc
cp -a /var0 /new_root/var
mkdir /new_root/tmp
chmod 777 /new_root/tmp
sync
#0.9.8R2 rc.sysinit will start the swap partition running...
if [ ! "$SWAPPART" = "" ];then
echo -n "$SWAPPART" > /new_root/root0/.etc/swappartition1
fi
#i want to know the max size if not going to use the swap partition...
echo -n "$PHYSICALFILLK" > /new_root/root0/.etc/ramdiskphysicalfssize
echo -n "$SIZERAMDISKK" > /new_root/root0/.etc/ramdisksize
echo -n "$SIZEFILLK" > /new_root/root0/.etc/ramdiskfssize #size of f.s.
cat /etc/fstab | sed -e 's/\/dev\/ram0/tmpfs/g' > /new_root/etc/fstab
#if ever run install-to-hd script, /sbin/init must be normal symlink...
mv -f /new_root/sbin/init /new_root/sbin/init_ORIG
ln -s /bin/busybox /new_root/sbin/init
cd /new_root
mkdir old_root
umount /proc
#have code in /etc/rc.d/rc.sysinit which will umount /dev/ram0...
pivot_root . old_root
exec chroot . sh -c "exec /bin/busybox init $INITARGS" dev/console 2>&1
In a nutshell, what this script does is copy all of /dev/ram0 to a new tmpfs ramdisk, which has the advantage of variable size and it can use a swap partition to create a very large effective size for the ramdisk. After the swap, /dev/ram0 is no longer being used.
Notice the very last line of the script. After the swap-over (chroot), "/bin/busybox init" is executed. This causes the init program built into Busybox to execute, the same as if /sbin/init was a link to /bin/busybox and init had been executed.
Step 3
(Coming soon)
