Monday, December 24, 2012

CentOS6 Disk encryption with remote password entering

Disk Encryption in CentOS6 with remote password entering

I played a bit with disk encryption in CentOS6. For this, I checked the "encryption" checkbox in the Anaconda installer during installation, which encrypts at the PV (Physical Volume) partition level. Therefore, all partitions except /boot are encrypted.
Disadvantage is, that you need to enter the decryption password on the local console during boot.

Inspired by RedHat Bug #524727 for Fedora, I setup a "Early-SSH" functionality which allows ssh login to the system at the earliest stage (before the decryption password is asked). With this, I'm able to ssh into a freshly started system and enter the decryption pw without local access.

Early-SSH is a initramfs hook which installs Dropbear SSH server into the initramfs image and starts it at an early stage during boot (before the disks are mounted), so you can perform many things there (Unlock encrypted disks, checking file systems, etc.). The hook gets installed as a Dracut module, therefore it is ensured that every time you update the kernel, the early-SSH module gets installed automatically.

(As usual, try at your own risk)

Remote password entering (early-SSH)

  • Normally you can only continue booting after you entered the decryption password on the local console. 
  • We will add support for early-ssh to be able to enter the decryption pw remotely.
  • There are some instructions available for Fedora Core and Debian, but not for RHEL/CentOS. So here I explain this feature on CentOS6 (tested on 6.3 x86_64)
  • See for details on Fedora.

Backup your initrd file

First, backup your current initrd image file to be able to fall back to it in case something goes wrong:
cp /boot/initramfs-`uname -r`.img /boot/initramfs-`uname -r`.img_
If you need to fall back to the original inited file, simply add an "_" to the Grub line pressing "e" during Grub boot stage.

Create dracut module

Install EPEL repository (for Dropbear)

rpm -Uvh
Install dropbear

yum install dropbear
Install compiler

yum install gcc
Create dracut module
cd /usr/share/dracut/modules.d/
mkdir 40earlyssh
cd 40earlyssh

# Generate public/private DSS and RSA host keys for the server.
dropbearkey -t dss -f dropbear_dss_host_key
dropbearkey -t rsa -f dropbear_rsa_host_key

# Create config files
echo 'multi on' > host.conf
echo -en '\tlocalhost\n::1\tlocalhost\n' > hosts
echo 'root:x:0:0:root:/home/root:/bin/sh' > passwd
echo '/bin/sh' > shells

Create install file

  • CentOS uses an old Dracut utility (Version 004). Due to this, we must separate check, installkernel and install into separate scripts. But we keep them also together in the "install" script as functions if the upstream vendor switches to a newer Dracut version, later.
  • If on newer Dracult version (>= 008), this file is named ""
  • Updated: 2014-10-20 (pkill and added)
    vi install
    # On newer Dracult versions, this file is named "".
    # These functions are currently ignored by CentOS6 Dracut.
    check() { # do not add this module by default: return 255 return 0 } depends() { return 0 } installkernel() { instmods eth0
        instmods vmxnet3   # vmware ethernet driver in this example.
    # CentOS6 uses an old version of Dracut.Therefore, install() is not called, so we comment the function definition out for now, so statements are always called.
    #install() {
        dracut_install -o ip
        # Need to use inst_library insead of dracut_install for libnsl as it is a symlink
        inst_library   /usr/lib64/
        dracut_install /lib64/
        dracut_install /lib64/
        dracut_install /lib64/
        dracut_install -o dropbear
        inst_dir "/etc/dropbear"
        inst_simple "${moddir}/dropbear_dss_host_key" "/etc/dropbear/dropbear_dss_host_key"
        inst_simple "${moddir}/dropbear_rsa_host_key" "/etc/dropbear/dropbear_rsa_host_key"
        inst_dir "/home"
        inst_dir "/home/root"
        inst_dir "/home/root/.ssh"
        inst_simple "${moddir}/authorized_keys" "/home/root/.ssh/authorized_keys"
        inst_simple "/etc/localtime"
        inst_simple "${moddir}/nsswitch.conf" "/etc/nsswitch.conf"
        inst_simple "${moddir}/resolv.conf" "/etc/resolv.conf"
        inst_simple "${moddir}/host.conf" "/etc/host.conf"
        inst_simple "${moddir}/hosts" "/etc/hosts"
        inst_simple "${moddir}/passwd" "/etc/passwd"
        inst_simple "${moddir}/shells" "/etc/shells"
        inst_simple "${moddir}/local.conf" "/etc/modprobe.d/local.conf"
        inst_hook pre-trigger 01 "$moddir/"
        inst_hook pre-trigger 01 "$moddir/"
        inst_binary "${moddir}/auth" "/bin/auth"
        inst_binary "${moddir}/tiocsti" "/bin/tiocsti"
        # Binaries
        dracut_install -o ps find lsof grep egrep sed less more cat tac head tail true false mkdir rmdir rm strace touch vi ip ping ping6 traceroute ssh scp pkill
        # fsck tools so you can check disks when logged in....
        # dracut_install -o fsck fsck.ext2 fsck.ext4 fsck.ext3 fsck.ext4dev fsck.vfat e2fsck

Create check file

vi check

# add this module by default
exit 0

Create installkernel file

You must add the Network kernel module for your network card and place an alias to eth0. Use "ethtool -i eth0" to get the driver name (use your device name if other than eth0)
vi installkernel

# install kernel module script for older dracut.
# Note: You must add your network module, here.
instmods eth0
instmods vmxnet3

Create local.conf (modprobe) file

vi local.conf
# /etc/modprobe.d/local.conf
# device to name mapping.
# Call "ethtool -i eth0" to get your driver name
# Note: You MUST add the network module to installkernel file in this dir, also!

alias eth0 vmxnet3

Alternatively, you can use the "biosdevname" utility, but I haven't done that.

Create nsswitch.conf file

vi nsswitch.conf
passwd:     files
shadow:     files
group:      files
initgroups: files

hosts:      files dns

bootparams: files

ethers:     files
netmasks:   files
networks:   files
protocols:  files
rpc:        files
services:   files

automount:  files
aliases:    files

Create resolv.conf file

vi resolv.conf 
search example

Create file

This script is called during boot. It sets up the IP-Address, default gateway and starts the dropbear ssh daemon.
Note: When the system switches context, the initramfs and all processes are closed, therefore security is not harmed after successful boot. But it is absolutely necessary to keep your ssh private key for early-stage ssh secure!

# Setup network card static IP and SSH server on port 222
# In this example with gateway

/sbin/ip link set dev lo up
/sbin/modprobe eth0
/sbin/ip addr add broadcast + dev eth0
/sbin/ip link set dev eth0 up
/sbin/ip route add default via

mkdir -p /var/log
> /var/log/lastlog

/usr/sbin/dropbear -E -m -s -p 222 -a -K 600

Create file

This script is called on exit of dracut after disk decryption key was entered. It clears the set IP-Address and frees the interface. (added 2014-10-20)
# Unsetup network card and kill SSH daemon
/sbin/ip link set dev lo down
/sbin/ip link set dev eth0 down
/sbin/ip addr delete broadcast + dev eth0
/sbin/ip route del default via
[ -f /tmp/ ] || exit 0
read main_pid </tmp/
kill -STOP ${main_pid} 2>/dev/null
pkill -P ${main_pid}
kill ${main_pid} 2>/dev/null
kill -CONT ${main_pid} 2>/dev/null

Create authorized_keys

vi authorized_keys
<Add all public ssh keys you want be able to login into this ssh server as you normally do in your regular authorized_keys file for ssh>

Compile some helper programs


vi auth.c
#include <sys/ioctl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <fcntl.h>
#include <stdio.h>
#include <termios.h>
#include <unistd.h>
#include <string.h>

int main (int argc, const char * argv[]) {
  char *passphrase;
  const char *prompt="Passphrase: ";
  int i;

  if (argc != 2) {
    printf("Usage: auth 'passwd'\n");
    return 1;

  int fd = open("/dev/console", O_RDONLY);
  if (fd < 0) {
    return 2;


  for (const char * str = passphrase; *str; ++str){
    ioctl(fd, TIOCSTI, str);
  ioctl(fd, TIOCSTI, "\r");
  // clear string immediately
  int len=strlen(passphrase);
  for (i=0;i<len;i++) { 
  return 0;


vi tiocsti.c
// gcc -std=gnu99 -O2 -Wall tiocsti.c -o tiocsti
#include <sys/ioctl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <fcntl.h>
#include <stdio.h>
#include <termios.h>
#include <unistd.h>

void stuff(int fd, const char * str) {
  printf("stuff [%s]\n", str);
  for (; *str; ++str) {
    // printf("(%c)", *str);
    int rv = ioctl(fd, TIOCSTI, str);
    if (rv < 0) perror("ioctl(TIOCSTI)");

int main (int argc, const char * argv[]) {
  if (argc < 3) {
    printf("Usage: tiocsti /dev/ttyX text string\n");
    return 1;

  int fd = open(argv[1], O_RDONLY);
  if (fd < 0) {
    return 2;

  for (int i = 2; i < argc; ++i) {
    if (i != 2) stuff(fd, " ");
    stuff(fd, argv[i]);

  return 0;

Compile helper programs

gcc -std=gnu99 -O2 -Wall tiocsti.c -o tiocsti
gcc -std=gnu99 -O2 -Wall auth.c -o auth

Fix permissions

You need to ensure the permissions of the files are correct, otherwise dropbear and the helper files are not included in the initramfs (Thanks to Ben Curtis):

chmod 755 check install installkernel
chmod 600 authorized_keys

Build new initramfs

So you end up with the following files in /usr/share/dracut/modules.d/40earlyssh:

















Create new initrd image:
dracut --force

Reboot, test

  • Now reboot. 
  • Once the machine is booted and at it's waiting for the password, it should be pingable from the network.
  • Login remotely:
ssh -p 222 root@<machine_name_or_its_ip>
  • Enter the preboot password:
tiocsti /dev/console "$(echo -n '<your_decryption_password>\r')"
# example: tiocsti /dev/console "$(echo -n 'verySecurePw!\r')"

Thursday, November 29, 2012

Repairing HP 8180 All-In-One printer

A friend of mine called me yesterday.
His HP Model C8180 Photosmart All-In-One Printer/Scanner/Copier  was faulty and rebooted all the time after switching it on. See this video with exactly the same fault.

It seems this was hit by crapy electrolytic capacitors (aka capacitor plague).
So I removed the logic board and saw the defect immediately.
3 caps were defect and the other one was suspect. Therefore I replaced all of them with better ones (one voltage level higher).

C614 and C662 (680uF at 6.3V, replace them with 680uF/10V)
C613 and C660 (330uF at 10V, replace them with 330uF/16V) 

Cost: Under $2. (Good chance that shipping cost will be higher than the parts cost)

Here are some pics:

Remove 2 screws on the right upper front
Remove 1 screw on the rear right. Afterwards, remove the 2 covers.
Now you see the logic board.

It might be a good idea to take some photos of the cabling now.
You need to remove 12 connectors.
Unscrew the logic board (4 screws) and take it out. 
There we have it. Look at the 4 capacitors.
If the vents are open or dried, they need to be replaced.
There we see that 3 are defect and one also does not look promising.
Unsolder all of them. Watch out the polarity of the new ones.
If you insert it the wrong way, you get a nice firecracker..
After you replaced them, reconnect the connectors and reinstall the board.
Then test your printer.

Friday, November 23, 2012

Good bye Eclipse, Hello Idea!

Used Eclipse since IBM contributed the first version in several flavors: First pure Eclipse, then MyEclipse and the last years SpringSource STS.
Projects went bigger and migrated from Ant to Maven to Gradle. The last projects were 100% Grails.

Grails development itself is great. Fast, nice and ... fast.
But Eclipse went worse and worse with every new revision released. Spring Plugin had to be disabled as working was impossible with it. Using Eclipse at the end was a pain.

Due to this, a few weeks ago I tried IntelliJ IDEA 11.
And what could I say? It is great. Fast, feature rich and supports things I didn't ever think about.

Small example: Resource bundles.
You see miss-spelled rb key names in all files using them. All files? Yes! Java, Groovy, JSF, JSP, etc.
You also see unused rb keys in the rb files!

Debugging is much better than in Eclipse. Javascript debugging is simply great.
Spring support? Bean validation, Spring Webflow charts, etc. work (I didn't notice and speed reduction while using it)

Groovy? Much better than current Eclipse plugin.
Gradle? Great as well! Including validation, which is missing in Eclipse.

Code completion works like a charm, even in Groovy and Gradle build files.

Eclipse is years behind.

Ok, it is not free of charge. Personal license is $199 and Commercial license start at $499.
This means for me, it will take only days until I got this amount back, as I'm so much faster in developing now.

Wednesday, November 14, 2012

Creating an Apple-Fusion like drive on your own

If you have a SSD drive, you can create your own Fusion like drive

See here: fusion-drive-on-older-macs

Friday, March 30, 2012

Using Postfix SASL Mail Relay in CentOS6

One day I setup a new CentOS6 machine with Postfix mail relay with SASL auth to a provider.

At the start, I received lots of "no worthy mechs" errors in the mail log.

Reason: you must install paket "cyrus-sasl-plain" for it.

yum install cyrus-sasl-plain

Here is an example postfix config excerpt:

# enable SASL-AUTH
smtp_sasl_auth_enable = yes
# Sasl Users/passwords in this file.
# NOTE: you must compile the file on every change with "postmap smtp_auth"
smtp_sasl_password_maps = hash:/etc/postfix/smtp_auth
# never use anonymous login
smtp_sasl_security_options = noanonymous
# relayhost with SASL-AUTH
relayhost = <your_providers_mail_relay_server_name>

The smtp_auth file contains the mail-relay-hostname, account and password:

relayhost.your-provider.example  example-user:example-password