Michael,

I found this blog post, but no date, http://blog.miketoscano.com/?p=307. It seems the Ubuntu installer can be used to set up raid1 and lvm. I was surprised at the blogger used lvm and the entire drive for /, but to each his own. I assume one could still use the Ubuntu installer and add some partitions on the drive.

Your script is obviously more detailed, and and I am not sure the Ubuntu installer does any of the tuning you do, such as vm.swappiness=1, which I have read about in other posts as well.

I am also curious about other viewpoints on this!

Thanks,

Mark


On Fri, Sep 5, 2014 at 6:32 AM, Michael Butash <michael@butash.net> wrote:
Not to discourage your learning, but here's how I build my ssd's on both my desktop and laptops now universally (assuming I can cram 2 disks in).  This I've built over several years of trial and error with ssd's and various os. I made a variation for uefi booting too my asus that wouldn't do legacy, but this should work for any non-uefi/mbr build.

I wouldn't mind some peer review on the process anyways, it's well notated just why I did things so I can remember later.  This was for my last stab at ubuntu, never was fully successful, then just applied it to mint, and my sanity was much better for it.

<doc>

## boot the ubuntu desktop cd, when at desktop, hit ctrl-alt-t and spawn a console

## if you need to wipe the disks, use a security erase on them
## you will sometimes need to unfreeze drives, a suspend and awaken works

sudo hdparm -I /dev/sda | grep froz
sudo hdparm -I /dev/sdb | grep froz

## make sure these are the right disks, they will be wiped.

hdparm --user-master u --security-set-pass PasSWorD /dev/sda
hdparm --user-master u --security-erase PasSWorD /dev/sda

hdparm --user-master u --security-set-pass PasSWorD /dev/sdb
hdparm --user-master u --security-erase PasSWorD /dev/sdb

## at console, issue the following to install mdadm:

sudo apt-get install mdadm

## next, issue fdisk to partition the disks from terminal:

## http://askubuntu.com/questions/8592/how-do-i-align-my-partition-table-properly

## block size (file system block size, ex. 4096 or 4k)
## erase head size (usually 4096 or 4k)
## stripe size (same as mdadm chunk size, set. 128k)
## stride: stripe size / block size (ex. 128k / 4k = 32)
## stripe-width: stride * #-of-data-disks (ex. 2 disks RAID 1 is 1 data disks; 32*1 = 32)

sudo fdisk -S32 -H32 -u /dev/sda
n
p

+250M

n
p



a
1
t
1
da
t
2
da
p
w

sudo fdisk -S32 -H32 -u /dev/sdb
n
p

+250M

n
p
2



a
1
t
1
da
t
2
da
p
w

## build the raid now using mdadm

mdadm --create /dev/md0 --auto=yes --force --name=boot0 --level=1 --chunk=128 --raid-devices=2 /dev/sda1 /dev/sdb1
mdadm --create /dev/md1 --auto=yes --force --name=spv0 --level=1 --chunk=128 --raid-devices=2 /dev/sda2 /dev/sdb2

## or if building with one missing

mdadm --create /dev/md0 --auto=yes --force --name=boot0 --level=1 --chunk=128 --raid-devices=2 /dev/sda1 missing
mdadm --create /dev/md0 --auto=yes --force --name=spv0 --level=1 --chunk=128 --raid-devices=2 /dev/sda2 missing

## create secure physical volume, 0, change $pw-user*.  This provides slot 0-7, 0-3 admin, 4-7 user.
## we control slots 0-3, users are given 4-7 (assuming multi-user or l-user)

## http://java-hamster.blogspot.com/2012/04/aligning-partitions-lvm-and-encrypted.html
## http://newspaint.wordpress.com/2012/09/21/full-disk-encryption-on-xubuntu-precise-12-04/
## http://security.stackexchange.com/questions/40208/recommended-options-for-luks-cryptsetup

## default, uses essiv-256, cpu-intensive

## cryptsetup --align-payload=8192 luksFormat /dev/md/spv0
## intelni optimized, sha256, 256bit

cryptsetup --align-payload=8192 -c aes-xts-plain64 -h sha256 -s 256 luksFormat /dev/md1
YES
$pw-slot0-diskmaster0

## add secondary user key, change $pw-user*

cryptsetup luksAddKey --key-slot 4 /dev/md1
$pw-slot4-user0

## confirm there are two slots, the master (0) and user (4)

cryptsetup luksDump /dev/md1

## remove a slot

cryptsetup luksRemovekey --key-slot 4 /dev/md1

## unlock the spv0

cryptsetup luksOpen /dev/md/spv0 spv0

<pass>

## create pv for lvm on spv0

## http://java-hamster.blogspot.com/2012/04/aligning-partitions-lvm-and-encrypted.html

pvcreate --dataalignment 4m /dev/mapper/spv0

## create volgroup $hostname-vg0 on spv0 - use the hostname of the local device (tpm locked anyways in bios - theoretically)

## http://java-hamster.blogspot.com/2012/04/aligning-partitions-lvm-and-encrypted.html

vgcreate $hostname-vg0 -s 4m /dev/mapper/spv0

## create your logical volumes

lvcreate --size 3G --name root0 $hostname-vg0
lvcreate --size 3G --name swap0 $hostname-vg0
lvcreate --size 9G --name usr0 $hostname-vg0
lvcreate --size 3G --name var0 $hostname-vg0
lvcreate --size 1G --name varlog0 $hostname-vg0
lvcreate --size 64G --name home0 $hostname-vg0
lvcreate --size 64G --name ext0 $hostname-vg0

## make ext4 partitions, match stripe/stride with md chunks
## need to add bit about setting inode counts here...

mkfs.ext2 -b 4096 -E stride=32,stripe-width=32 /dev/md0
mkfs.ext4 -b 4096 -E stride=32,stripe-width=32 /dev/mapper/$hostname--vg0-root0
mkfs.ext4 -b 4096 -E stride=32,stripe-width=32 /dev/mapper/$hostname--vg0-usr0
mkfs.ext4 -b 4096 -E stride=32,stripe-width=32 /dev/mapper/$hostname--vg0-var0
mkfs.ext4 -b 4096 -E stride=32,stripe-width=32 /dev/mapper/$hostname--vg0-varlog0
mkfs.ext4 -b 4096 -E stride=32,stripe-width=32 /dev/mapper/$hostname--vg0-home0
mkfs.ext4 -b 4096 -E stride=32,stripe-width=32 /dev/mapper/$hostname--vg0-ext0

tune2fs -c0 -i0 /dev/md0
tune2fs -c0 -i0 /dev/mapper/$hostname--vg0-root0
tune2fs -c0 -i0 /dev/mapper/$hostname--vg0-usr0
tune2fs -c0 -i0 /dev/mapper/$hostname--vg0-var0
tune2fs -c0 -i0 /dev/mapper/$hostname--vg0-varlog0
tune2fs -c0 -i0 /dev/mapper/$hostname--vg0-home0
tune2fs -c0 -i0 /dev/mapper/$hostname--vg0-ext0

## mkswap

mkswap /dev/mapper/vg0-swap

## make/mount target dir
mkdir /target
mount /dev/mapper/$hostname--vg0-root0 /target
mkdir /target/boot
mkdir /target/usr
mkdir /target/var
mkdir /target/home
mkdir /target/mnt
mkdir /target/mnt/ext0

mount /dev/mapper/$hostname--vg0-usr0 /target/usr
mount /dev/mapper/$hostname--vg0-var0 /target/var
mount /dev/mapper/$hostname--vg0-home0 /target/home
mount /dev/mapper/$hostname--vg0-ext0 /target/mnt/ext0
mount /dev/md0 /target/boot
mkdir /target/var/log
mount /dev/mapper/$hostname--vg0-varlog0 /target/var/log

## continue the installer and get to partition, use "manual"

## enable all the partitions and set the mount structure

## set "yes" to boot failed raid

## continue installing

## at grub, use mbr to install

## before rebooting, vi the /etc/crypttab file and add contents for uuid

ls -la /dev/disk/by-uuid | grep md1 | awk '{ print $9 }' >> /target/etc/crypttab

<>
# <target name> <source device>         <key file>      <options>
## example
## spv0 UUID=5f694a41-f8c6-4da1-8679-8263e8642eb1       none luks,retry=1,discard
spv0    UUID=$uuid-here     none    luks,retry=1,discard

<>

## if you need/want to remount chroot to install or fix, add device dir's to chroot and enter

mount --rbind /proc /target/proc
mount --rbind /sys /target/sys
mount --rbind /dev /target/dev
mount --rbind /run /target/run

chroot /target
bash

apt-get update
apt-get install mdadm cryptsetup lvm2

## make your fstab and modify the boot disk to be the correct uuid from the table

ls -la /dev/disk/by-uuid | grep md0 | awk '{ print $9 }' >> /etc/fstab
vi /etc/fstab

<>
# UNCONFIGURED FSTAB FOR BASE SYSTEM
proc    /proc    proc    defaults                        0 0
tmpfs    /tmp    tmpfs    defaults,noatime,nodiratime,mode=1777         0 0
/dev/mapper/$hostname--vg0-root0    /    ext4 defaults,noatime,nodiratime    0 1
UUID=218b2c98-3f7e-4008-950c-b99e3d6dabab    /boot    ext2 defaults,noatime,nodiratime    0 1
/dev/mapper/$hostname--vg0-usr0    /usr    ext4 defaults,noatime,nodiratime    0 2
/dev/mapper/$hostname--vg0-var0    /var    ext4 defaults,noatime,nodiratime    0 2
/dev/mapper/$hostname--vg0-varlog0    /var/log    ext4 defaults,noatime,nodiratime    0 2
/dev/mapper/$hostname--vg0-home0    /home    ext4 defaults,noatime,nodiratime    0 2
/dev/mapper/$hostname--vg0-ext0    /mnt/ext0    ext4 defaults,noatime,nodiratime,commit=600    0 2
/dev/mapper/$hostname--vg0-swap0    none    swap    sw,discard 0 0

<>

## ensure these are correct too

vi /etc/crypttap
vi /etc/mdadm/mdadm.conf

update-initramfs -k all -t
grub-install /dev/sda
grub-install /dev/sdb
update-grub
grub-install /dev/sda
grub-install /dev/sdb

## set and make a udev rule for setting scheduler to deadline

echo deadline > /sys/block/sdb/queue/scheduler

vi /etc/udev/rules.d/60-ssd-scheduler.rules

<>
# set deadline scheduler for non-rotating disks

ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/rotational}=="0", ATTR{queue/scheduler}="deadline"
<>

## edit lvm.conf to change discards to =1

vi /etc/lvm/lvm.conf

<>
issue_discards = 1
<>

# issue vm.swappiness to +1 now

<>
sysctl -w vm.swappiness=1
<>

## add into sysctl

vi /etc/sysctl.d/ssd-optimization.conf

<>
# Added so Linux kernel no longer attempts to enlarge the cache by paging applications out
# http://rudd-o.com/linux-and-free-software/tales-from-responsivenessland-why-linux-feels-slow-and-how-to-fix-that

vm.swappiness=1
<>

## test the disks, get them closed to zero

sync
echo 3 > /proc/sys/vm/drop_caches
time dd if=/dev/zero of=/mnt/ext0/incoming/testfile count=1 bs=900M

sysctl -w vm.vfs_cache_pressure=100
find / > /dev/null
cp /mnt/ext0/incoming/testfile /mnt/ext0/incoming/testfile2
time find / > /dev/null

sysctl -w vm.vfs_cache_pressure=50
find /  > /dev/null
cp /mnt/ext0/incoming/testfile2 /mnt/ext0/incoming/testfile3
time find / > /dev/null

rm -f /mnt/ext0/incoming/testfile /mnt/ext0/incoming/testfile2 /mnt/ext0/incoming/testfile3

## add permenantly cat into sysctl under prior entry

vi /etc/sysctl.d/ssd-optimization.conf

<>

## add for filesystem caching

vm.vfs_cache_pressure=50
<>

</doc>

-mb


On 09/04/2014 10:19 AM, Mark Phillips wrote:
Michael,

Thanks again for your comments, they are very helpful. I have been googling RAID1 and LVM and finding lots of good information.

I really like your idea of a RAID1 for the two SSDs. Does it matter if one is msata and one is not?

I am trying to decide on the merits of using LVM with the RAID1, since I only have 1 disk and I normally don't partition it so I don't have to worry about running our of space until the disk is almost full. Could you explain to me the benefit of using LVM + RAID1 for these two drives? How would you partition the drives? My current drive has about 420 GB of data in /home, about 9GB in /opt, and some misc stuff in /var, all of which I need to transfer that to the new system.

Thanks,

Mark

P.S. One benefit of using both LVM and RAID1 is learning something new! ;)