When I first started to use icsci on RHEL5, isciadm seemed like a very complicated command. This turned out not to be the case, but the syntax is a bit verbose and can be hard to remember if you don’t use it frequently. I put together a wrapper script to make it easier to remember.
One on server I adminster, users run workflows that occasionally leave zombie processes behind. A zombie describes a processes that has finished but has not had its exit status read by its parent process. I keep this shell script around to generate a report on zombie children and their negligent parents that I can send to the workflow developers.
Applying old updates via yum
When new software packages are released, you don’t want to blindly apply them to all your systems. You should have a test environment, or at least a set of less-critical systems, where you test things first. If you normally stick to a consistent update schedule, such as once a week, you can use the date a package was created to determine whether it should be updated in your production environment. I couldn’t find an way to do this directly through yum, so I created a shell script named update_production to do this. As written, it applies all updates, except for packages released to CentOS or EPEL in the last week. It’s designed to be easy to tweak the repositories you delay updates from or how long the delay is. Ideally this functionality should be available as a plugin to yum, but that looked much more complicated to develop.
Converting software raid
If you accidentally set up a server with the wrong software raid type, as long as you used LVM you can use mirroring to temporarily move the filesystem off the RAID device, recreate the RAID properly, then move the filesystem back onto it. Needless to say if things go wrong during this process you’ll be reinstalling. Below is an example where md1 on sda2 and sdb2 is being converted from RAID0 to RAID1, and an iSCSI volume serves as the temporary storage. In it VG is the volume group name. LV is the only logical volume being moved, and DEVICE is the device linux creates for the iSCSI target. If doing this yourself, you’ll want to frequently check the sanity of what you’re doing with commands like ‘pvs’ and ‘lvs -a -o +devices’. If feasible, you can save a lot of time by turning off swap, removing its logical volume during the conversion, then recreating it afterwards.