CentOS6 Disk encryption with remote password entering

Disk Encryption in CentOS6 with remote password entering

I played a bit with disk encryption in CentOS6. For this, I checked the "encryption" checkbox in the Anaconda installer during installation, which encrypts at the PV (Physical Volume) partition level. Therefore, all partitions except /boot are encrypted.
Disadvantage is, that you need to enter the decryption password on the local console during boot.

Inspired by RedHat Bug #524727 for Fedora, I setup a "Early-SSH" functionality which allows ssh login to the system at the earliest stage (before the decryption password is asked). With this, I'm able to ssh into a freshly started system and enter the decryption pw without local access.

Early-SSH is a initramfs hook which installs Dropbear SSH server into the initramfs image and starts it at an early stage during boot (before the disks are mounted), so you can perform many things there (Unlock encrypted disks, checking file systems, etc.). The hook gets installed as a Dracut module, therefore it is ensured that every time you update the kernel, the early-SSH module gets installed automatically.

(As usual, try at your own risk)

Remote password entering (early-SSH)

• Normally you can only continue booting after you entered the decryption password on the local console. 
• We will add support for early-ssh to be able to enter the decryption pw remotely.
• There are some instructions available for Fedora Core and Debian, but not for RHEL/CentOS. So here I explain this feature on CentOS6 (tested on 6.3 x86_64)
• See  https://bugzilla.redhat.com/show_bug.cgi?id=524727 for details on Fedora.

Backup your initrd file

First, backup your current initrd image file to be able to fall back to it in case something goes wrong:

cp /boot/initramfs-`uname -r`.img /boot/initramfs-`uname -r`.img_

If you need to fall back to the original inited file, simply add an "_" to the Grub line pressing "e" during Grub boot stage.

Create dracut module

Install EPEL repository (for Dropbear)

rpm -Uvh http://ftp-stud.hs-esslingen.de/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

Install dropbear

yum install dropbear

Install compiler

yum install gcc

Create dracut module

cd /usr/share/dracut/modules.d/
mkdir 40earlyssh

cd 40earlyssh

# Generate public/private DSS and RSA host keys for the server.
dropbearkey -t dss -f dropbear_dss_host_key
dropbearkey -t rsa -f dropbear_rsa_host_key

# Create config files
echo 'multi on' > host.conf
echo -en '127.0.0.1\tlocalhost\n::1\tlocalhost\n' > hosts
echo 'root:x:0:0:root:/home/root:/bin/sh' > passwd
echo '/bin/sh' > shells

Create install file

• CentOS uses an old Dracut utility (Version 004). Due to this, we must separate check, installkernel and install into separate scripts. But we keep them also together in the "install" script as functions if the upstream vendor switches to a newer Dracut version, later.
• If on newer Dracult version (>= 008), this file is named "module-setup.sh"
• Updated: 2014-10-20 (pkill and remote-ssh-delete.sh added)

  vi install
  

  #!/bin/bash
  # On newer Dracult versions, this file is named "module-setup.sh".

  # These functions are currently ignored by CentOS6 Dracut.
  
  
  check() {
      # do not add this module by default: return 255
      return 0
  }
  
  depends() {
      return 0
  }
  
  installkernel() {
      instmods eth0

      instmods vmxnet3   # vmware ethernet driver in this example.
  }
  
  # CentOS6 uses an old version of Dracut.Therefore, install() is not called, so we comment the function definition out for now, so statements are always called.
  
  
  

  #install() {
      dracut_install -o ip
      

      # Need to use inst_library insead of dracut_install for libnsl as it is a symlink
      inst_library   /usr/lib64/libnsl.so
      dracut_install /lib64/libnss_compat.so.2
      dracut_install /lib64/libnss_files.so.2
      dracut_install /lib64/libnss_dns.so.2
      dracut_install -o dropbear
  
      inst_dir "/etc/dropbear"
      inst_simple "${moddir}/dropbear_dss_host_key" "/etc/dropbear/dropbear_dss_host_key"
      inst_simple "${moddir}/dropbear_rsa_host_key" "/etc/dropbear/dropbear_rsa_host_key"
  
      inst_dir "/home"
      inst_dir "/home/root"
      inst_dir "/home/root/.ssh"
      inst_simple "${moddir}/authorized_keys" "/home/root/.ssh/authorized_keys"
  
      inst_simple "/etc/localtime"
      inst_simple "${moddir}/nsswitch.conf" "/etc/nsswitch.conf"
      inst_simple "${moddir}/resolv.conf" "/etc/resolv.conf"
      inst_simple "${moddir}/host.conf" "/etc/host.conf"
      inst_simple "${moddir}/hosts" "/etc/hosts"
      inst_simple "${moddir}/passwd" "/etc/passwd"
      inst_simple "${moddir}/shells" "/etc/shells"
      inst_simple "${moddir}/local.conf" "/etc/modprobe.d/local.conf"
      inst_hook pre-trigger 01 "$moddir/remote-ssh.sh"

      inst_hook pre-trigger 01 "$moddir/remote-ssh-delete.sh"
  
  
      inst_binary "${moddir}/auth" "/bin/auth"
      inst_binary "${moddir}/tiocsti" "/bin/tiocsti"
  
      # Binaries
      dracut_install -o ps find lsof grep egrep sed less more cat tac head tail true false mkdir rmdir rm strace touch vi ip ping ping6 traceroute ssh scp pkill
      # fsck tools so you can check disks when logged in....
      # dracut_install -o fsck fsck.ext2 fsck.ext4 fsck.ext3 fsck.ext4dev fsck.vfat e2fsck
  #}
  

Create check file

vi check

#!/bin/sh

# add this module by default
exit 0

Create installkernel file

You must add the Network kernel module for your network card and place an alias to eth0. Use "ethtool -i eth0" to get the driver name (use your device name if other than eth0)

vi installkernel

#!/bin/bash

# install kernel module script for older dracut.
# Note: You must add your network module, here.
instmods eth0
instmods vmxnet3

Create local.conf (modprobe) file

vi local.conf

# /etc/modprobe.d/local.conf
# device to name mapping.
# Call "ethtool -i eth0" to get your driver name
# Note: You MUST add the network module to installkernel file in this dir, also!

alias eth0 vmxnet3

Alternatively, you can use the "biosdevname" utility, but I haven't done that.

Create nsswitch.conf file

vi nsswitch.conf

passwd:     files
shadow:     files
group:      files
initgroups: files

hosts:      files dns

bootparams: files

ethers:     files
netmasks:   files
networks:   files
protocols:  files
rpc:        files
services:   files

automount:  files
aliases:    files

Create resolv.conf file

vi resolv.conf 

search example
nameserver 192.168.3.100
nameserver 192.168.3.200

Create remote-ssh.sh file

This script is called during boot. It sets up the IP-Address, default gateway and starts the dropbear ssh daemon.
Note: When the system switches context, the initramfs and all processes are closed, therefore security is not harmed after successful boot. But it is absolutely necessary to keep your ssh private key for early-stage ssh secure!

vi remote-ssh.sh

#!/bin/sh
# Setup network card static IP and SSH server on port 222

# In this example 192.168.3.10/24 with gateway 192.168.3.1

/sbin/ip link set dev lo up
/sbin/modprobe eth0
/sbin/ip addr add 192.168.3.10/24 broadcast + dev eth0
/sbin/ip link set dev eth0 up
/sbin/ip route add default via 192.168.3.1

mkdir -p /var/log
> /var/log/lastlog

/usr/sbin/dropbear -E -m -s -p 222 -a -K 600

Create remote-ssh-delete.sh file

This script is called on exit of dracut after disk decryption key was entered. It clears the set IP-Address and frees the interface. (added 2014-10-20)

vi remote-ssh-delete.sh

#!/bin/sh

# Unsetup network card and kill SSH daemon




/sbin/ip link set dev lo down

/sbin/ip link set dev eth0 down

/sbin/ip addr delete 192.168.121.3.10/24 broadcast + dev eth0

/sbin/ip route del default via 192.168.3.1




[ -f /tmp/dropbear.pid ] || exit 0

read main_pid </tmp/dropbear.pid

kill -STOP ${main_pid} 2>/dev/null

pkill -P ${main_pid}

kill ${main_pid} 2>/dev/null

kill -CONT ${main_pid} 2>/dev/null

Create authorized_keys

vi authorized_keys

<Add all public ssh keys you want be able to login into this ssh server as you normally do in your regular authorized_keys file for ssh>

Compile some helper programs

auth

vi auth.c

#include <sys/ioctl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <fcntl.h>
#include <stdio.h>
#include <termios.h>
#include <unistd.h>
#include <string.h>

int main (int argc, const char * argv[]) {
  char *passphrase;
  const char *prompt="Passphrase: ";
  int i;

  if (argc != 2) {
    printf("Usage: auth 'passwd'\n");
    return 1;
  }

  int fd = open("/dev/console", O_RDONLY);
  if (fd < 0) {
    return 2;
  }

  passphrase=getpass(prompt);

  for (const char * str = passphrase; *str; ++str){
    ioctl(fd, TIOCSTI, str);
  }
  ioctl(fd, TIOCSTI, "\r");

  // clear string immediately
  int len=strlen(passphrase);
  for (i=0;i<len;i++) { 
   passphrase[i]=0; 
  };
  return 0;
}

tiocsti

vi tiocsti.c

// gcc -std=gnu99 -O2 -Wall tiocsti.c -o tiocsti
#include <sys/ioctl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <fcntl.h>
#include <stdio.h>
#include <termios.h>
#include <unistd.h>

void stuff(int fd, const char * str) {
  printf("stuff [%s]\n", str);
  for (; *str; ++str) {
    // printf("(%c)", *str);
    int rv = ioctl(fd, TIOCSTI, str);
    if (rv < 0) perror("ioctl(TIOCSTI)");
  }
}

int main (int argc, const char * argv[]) {
  if (argc < 3) {
    printf("Usage: tiocsti /dev/ttyX text string\n");
    return 1;
  }

  int fd = open(argv[1], O_RDONLY);
  if (fd < 0) {
    perror("open");
    return 2;
  }

  for (int i = 2; i < argc; ++i) {
    if (i != 2) stuff(fd, " ");
    stuff(fd, argv[i]);
  }

  close(fd);
  return 0;
}

Compile helper programs

gcc -std=gnu99 -O2 -Wall tiocsti.c -o tiocsti
gcc -std=gnu99 -O2 -Wall auth.c -o auth

Fix permissions

You need to ensure the permissions of the files are correct, otherwise dropbear and the helper files are not included in the initramfs (Thanks to Ben Curtis):

chmod 755 check install installkernel remote-ssh.sh
chmod 600 authorized_keys

Build new initramfs

So you end up with the following files in /usr/share/dracut/modules.d/40earlyssh:

authauth.c

authorized_keys

check

dropbear_dss_host_key

dropbear_rsa_host_key

host.conf

hosts

install

installkernel

local.conf

nsswitch.conf

passwd

README

remote-ssh.sh

resolv.conf

shells

tiocsti

tiocsti.c

Create new initrd image:

dracut --force

Reboot, test

• Now reboot. 
• Once the machine is booted and at it's waiting for the password, it should be pingable from the network.
• Login remotely:

ssh -p 222 root@<machine_name_or_its_ip>

• Enter the preboot password:

tiocsti /dev/console "$(echo -n '<your_decryption_password>\r')"

# example: tiocsti /dev/console "$(echo -n 'verySecurePw!\r')"

Repairing HP 8180 All-In-One printer

A friend of mine called me yesterday.

His HP Model C8180 Photosmart All-In-One Printer/Scanner/Copier  was faulty and rebooted all the time after switching it on. See this video with exactly the same fault.

It seems this was hit by crapy electrolytic capacitors (aka capacitor plague).
So I removed the logic board and saw the defect immediately.

3 caps were defect and the other one was suspect. Therefore I replaced all of them with better ones (one voltage level higher).

Parts: 

C614 and C662 (680uF at 6.3V, replace them with 680uF/10V)

C613 and C660 (330uF at 10V, replace them with 330uF/16V) 

Cost: Under $2. (Good chance that shipping cost will be higher than the parts cost)

Here are some pics:

Remove 2 screws on the right upper front

Remove 1 screw on the rear right. Afterwards, remove the 2 covers.

Now you see the logic board.

It might be a good idea to take some photos of the cabling now.
You need to remove 12 connectors.

Unscrew the logic board (4 screws) and take it out. 

There we have it. Look at the 4 capacitors.
If the vents are open or dried, they need to be replaced.

There we see that 3 are defect and one also does not look promising.
Unsolder all of them. Watch out the polarity of the new ones.
If you insert it the wrong way, you get a nice firecracker..
After you replaced them, reconnect the connectors and reinstall the board.
Then test your printer.

Good bye Eclipse, Hello Idea!

Used Eclipse since IBM contributed the first version in several flavors: First pure Eclipse, then MyEclipse and the last years SpringSource STS.
Projects went bigger and migrated from Ant to Maven to Gradle. The last projects were 100% Grails.

Grails development itself is great. Fast, nice and ... fast.
But Eclipse went worse and worse with every new revision released. Spring Plugin had to be disabled as working was impossible with it. Using Eclipse at the end was a pain.

Due to this, a few weeks ago I tried IntelliJ IDEA 11.
And what could I say? It is great. Fast, feature rich and supports things I didn't ever think about.

Small example: Resource bundles.
You see miss-spelled rb key names in all files using them. All files? Yes! Java, Groovy, JSF, JSP, etc.
You also see unused rb keys in the rb files!

Debugging is much better than in Eclipse. Javascript debugging is simply great.
Spring support? Bean validation, Spring Webflow charts, etc. work (I didn't notice and speed reduction while using it)

Groovy? Much better than current Eclipse plugin.
Gradle? Great as well! Including validation, which is missing in Eclipse.

Code completion works like a charm, even in Groovy and Gradle build files.

Eclipse is years behind.

Ok, it is not free of charge. Personal license is $199 and Commercial license start at $499.
This means for me, it will take only days until I got this amount back, as I'm so much faster in developing now.

Creating an Apple-Fusion like drive on your own

If you have a SSD drive, you can create your own Fusion like drive

See here: fusion-drive-on-older-macs

Using Postfix SASL Mail Relay in CentOS6

One day I setup a new CentOS6 machine with Postfix mail relay with SASL auth to a provider.

At the start, I received lots of "no worthy mechs" errors in the mail log.

Reason: you must install paket "cyrus-sasl-plain" for it.

yum install cyrus-sasl-plain

Here is an example postfix config excerpt:

# enable SASL-AUTH
smtp_sasl_auth_enable = yes
# Sasl Users/passwords in this file.
# NOTE: you must compile the file on every change with "postmap smtp_auth"
smtp_sasl_password_maps = hash:/etc/postfix/smtp_auth
# never use anonymous login
smtp_sasl_security_options = noanonymous
# relayhost with SASL-AUTH
relayhost = <your_providers_mail_relay_server_name>


The smtp_auth file contains the mail-relay-hostname, account and password:


relayhost.your-provider.example  example-user:example-password