Monday, March 16, 2009

Hot-Swapping in Software RAID Works in Linux, Apparently

I need to try this out when I get a software RAID going:

When will software RAID hotswap work?

Some tips from the one-sided discussion from "Robotbeat" is to make sure the kernel supports hotswapping and that the BIOS set to use AHCI for the sata drives.

Another link on the topic.

Wikipedia entry on AHCI has some useful information like what motherboards have problems with AHCI and how to get AHCI working on Linux, copy-pasted below.

Common problems switching to AHCI under Linux

  • The AHCI controller does not work on AMD/ATI RS400-200, and RS480 HBA; and Nvidia nForce 560 chipset[citation needed] when MSI is enabled due to a hardware error. In order for AHCI to work users must provide the "pci=nomsi" kernel boot parameter. With MSI disabled in this way, the PCIe bus can only act as a faster PCI bus with hotplug capabilities.
  • The AHCI controller on AMD/ATI SB600 HBA can't do 64-bit DMA transfers. 64-bit addressing is optional in AHCI 1.1 and the chip claims it can do them, but, in reality, it can't so it is disabled. After that it will be forced to do 32-bit DMA transfers. Thus DMA transfers will occur in the lower 4 GiB region of the memory, and bounce buffers must be used sometimes if there is more than 4 GiB of RAM.[9]
  • The VIA VT8251 South bridge suffers the same fate but it can be circumvented with the "pci=nomsi" option to force detection of the chip. This has been tested to work on 2.6.26, 2.6.24 and 2.6.20 kernels.
  • Under RHEL, CentOS and similar, if you change your BIOS to AHCI mode and do not have the ahci drivers in your initrd then you will not be able to boot. To solve this
    • Set you BIOS to IDE/ATA/Original setting mode
    • boot into linux
    • edit /etc/modprobe.conf and add the line alias scsi_hostadapter2 ahci
    • run mkinitrd -f /boot/initrd-`uname -r`.img `uname -r`
    • reboot in to you BIOS and set to AHCI/RAID mode
    • boot into linux
    • check your vendors manual for more details

Thursday, October 9, 2008

stty -echo

Note to self:

use stty -echo to hide keyboard input on terminal, stty echo to undo hide.

Source: Read in passwords with bash

Wednesday, October 8, 2008

.local Host Name and Multicast DNS

While installing and configuring OpenVPN on Ubuntu, I ran into a network problem I've been intermittently encountering on Windows.

Not much was required in installing and configuring openvpn. I just installed the prebuilt package via apt. Configuration files were downloaded from the Astaro firewall running the openvpn server. The only change needed was to include

up /etc/openvpn/update-resolv-conf
down /etc/openvpn/update-resolv-conf

in the .ovpn file to apply/undo the nameserver settings the openvpn server pushes to the client upon connect/disconnect. "update-resolv-conf" is a script that came with the openvpn package that parses the environment variables for the DNS settings the openvpn server pushes and the client exports to "foreign_variable_#" variables before calling the up/down scripts. And "resolvconf" package needed to be installed since "update-resolv-conf" uses it.

Even after configuring things this way, though, I couldn't connect to machines on the VPN. I could retrieve the correct IP addresses via nslookup, but when trying to connect any application them resulted in the hosts the names not resolving correctly. Connecting by specifying the IP address manually worked.

After some searching, I found some online discussions referring to this problem. "/etc/nsswitch.conf" was configured to kill the name resolution process when the mDNS could not resolve a ".local" name.

hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4

... was changed to ...

hosts: files dns mdns4


I suppose one way to look at it is that the a ".local" domain name was chosen for host names on the VPN even though mDNS was not set up. Astaro apparently doesn't support the protocol.

If I were in charge of managing the network, I would transition all hosts to use a different domain. But I'm not. The administrator insists that we continue using ".local" because Astaro should support mDNS. I think he should put his love for all things Apple aside for once and face the messy reality.

The underlying cause of the problem may be the same for the Windows machines. Though the fact that the problem crops up only occasionally with Windows is odd. For Windows, I've specified the IP addresses of the few hosts I need to connect to on the VPN in the "hosts" file. This has so far worked out for me.

TOra with Oracle Support

Used these instructions to build and install TOra with Oracle support.

Will try out and update.

Saturday, July 19, 2008

New Hard Drive - Determining Best Parameters for ext3 File System

I've install a new hard drive on my laptop. Got a 320 GB Western Digital Passport and followed instructions here to extract the 320 GB SATA hard drive and replace my old 100 GB hard drive. I ended up mangling a plastic fastener in the hard drive enclosure like the author of the instructions did. But it still closes fine and now holds the old hard drive.

EDIT: Another guide (with pictures!) for disassembling this external drive. Might be better to follow this page.

In moving contents of the old hard drive to the new, I decided to stick with ext3. I've read JFS is faster, but I really like being able to access my linux partitions while using Windows. Also, this thread seems to suggest that ext3 is better at recovering from disk errors.

I want good parameters for partitions and file systems on the new drive. So, using tools like "df" and "tune2fs", I found out some statistics on the files residing on my old partitions.

I use two non-swap partitions for linux. A partition for "/home" and a partiton for everything else ("/"). "/" partition used 4.2 GB with 209893 items (files, directories, symbolic links...). In the "/home" partition, 61681 items took up 30.8 GB. From this, I've determined that "/" partition will be 20 GB and "/home" will be 140 GB on the new hard drive.

On the "/" partition, 42% (1,250,630/3,166,498) of blocks and 14% (209,301/1,609,344) of inodes were used. On the "/home" partition, 88% (10,061,413/12,016,356) of blocks and 2% (86,576/6,087,360) of inodes were used. I imagine the disparity in blocks and inodes used for "/home" results from large files I stored there. As both partitions used 4 KiB blocks, the ratio of bytes to inode were 8,059 for "/" and 8.085 for "/home". Clearly, past usage shows "/home" partition could do with larger bytes/inode ratio. As the laptop will continue to be used exclusively by me, I feel it's safe to change the ratio to 10,000 for "/" and 200,000 for "/home" without worry of running out of inodes.

Also, I'll set inode size to 256 in both partitions. 256 byte inodes improve performance for some applications and allow smooth upgrade to ext4. While older kernels won't be able to use the file system and grub had trouble booting on ext3 partitions with 256 byte inodes. But I'm not concerned as I won't use older kernels and grub has been patched. Ext2 IFS, the Windows driver I had been using to access the ext3 partitions, doesn't seem to support this setting so I may go with Crossmeta or ext2fsd as latest version of those drivers should support 256 byte inodes.

To determine the best blocks size, I ran some "find" commands to find out the distribution of the items based on their size:

ubuntu@ubuntu:/media/disk-1$ sudo find | wc -l
209893
ubuntu@ubuntu:/media/disk-1$ sudo find -size +1k | wc -l
125940
ubuntu@ubuntu:/media/disk-1$ sudo find -size +2k | wc -l
101253
ubuntu@ubuntu:/media/disk-1$ sudo find -size +3k | wc -l
88251
ubuntu@ubuntu:/media/disk-1$ sudo find -size +4k | wc -l
58733
ubuntu@ubuntu:/media/disk-1$ sudo find -size +5k | wc -l
53168
...


Resulting data:

Item Distirbution by Size
size"/""/home"
All20989386567
≤1KiB8395317953
>1KiB, ≤2KiB246875951
>2KiB, ≤3KiB130024934
>3KiB, ≤4KiB295186902
>4KiB, ≤5KiB55652536
>5KiB, ≤6KiB42101860
>6KiB, ≤7KiB33941429
>7KiB, ≤8KiB28951232
>8KiB, ≤9KiB26091054
>9KiB, ≤10KiB2787955
>10KiB, ≤11KiB2285767
>11KiB, ≤12KiB2133878
>12KiB, ≤13KiB1793803
>13KiB, ≤14KiB1509852
>14KiB, ≤15KiB1313942
>15KiB, ≤16KiB13881110
>16KiB, ≤32KiB1069614588
>32KiB, ≤64KiB793810158
>64KiB, ≤128KiB39243666
>128KiB20977997


More than half the files in "/" partition were 2 KiB or smaller. In the "/home" partition, however, only around 28% fell into this category and only 41% were smaller than 4KiB.

I ran the commands again with "-type d" option to find size distribution of directories. Because both of the old partitions used 4 KiB block sizes, the size attributed to the directories are all multiples of 4 KiB. This explains how the distribution for both partitions seem to deviate from the decreasing pattern at 3 KiB to 4 KiB range. Most of the directories uses a single 4 KiB block and causes the spike.

Though "/" may benefit from smaller block size, 4 KiB doesn't seem bad. I'll leave that as is.

I've decided to preserve some of other configuration from the old file systems. So following are the resulting mkfs commands I'll be running:

For "/", mkfs.ext3 -c -b 4096 -i 10000 -I 256 /dev/sda3

For "/home", mkfs.ext3 -c -b 4096 -i 200000 -I 256 /dev/sda4

Other options specified in the "/etc/mke2fs.conf" file seemed good enough (sparse_super,filetype,resize_inode,dir_index). Hopefully all goes well.

UPDATE: Output from running mkfs is below. Might come in handy for data recovery.

ext3 -c -b 4096 -i 200000 -I 256 /dev/sda4;
mke2fs 1.40-WIP (14-Nov-2006)
Warning: 256-byte inodes not usable on older systems
warning: 380 blocks unused.

Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
2000176 inodes, 4882432 blocks
244140 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=0
149 block groups
32768 blocks per group, 32768 fragments per group
13424 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000

Checking for bad blocks (read-only test): done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 31 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
mke2fs 1.40-WIP (14-Nov-2006)
Warning: 256-byte inodes not usable on older systems
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
691488 inodes, 33709204 blocks
1685460 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=0
1029 block groups
32768 blocks per group, 32768 fragments per group
672 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872

Checking for bad blocks (read-only test): done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 38 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.


UPDATE2: After doing some reading, I've decided to create an ext2 boot partition. It seems the only sure-fire way to have grub boot and have the other partitions be ext3 with 256 byte inodes.

Thursday, April 17, 2008

Boys Who Don't Play Video Games (or Play Violent Games Exclusively) at Risk

Authors of "Grand Theft Childhood" were interviewed by G4 (seems a bit rehearsed) on video games and violence. They suggest video games are not cause of violence. And though playing violent games excessively is associated with higher risk of getting into trouble, so is not playing video games at all.

Summary:

Playing violent video games is not a cause of real world violence.

Exclusively playing violent M-rated games and for 15 hours or more a week is a risk marker - associated with statistically significant risk of getting in trouble. This is true for both boys and girls.

But for boys, not playing any video games at all is also a risk marker. The researchers suppose that playing video games is a sign of social competence for boys and not playing at all is a sign that something is wrong.

A whole lot of people play games (63% of US). Yet we're still doing well and we'd see that if we stop paying attention to a causal link that does not exist.

Violent media does not cause crime. But if a person is at risk of violent behavior, an underlying factor may also cause them to consume violent media excessively. Or if they are socially isolated they may get into more trouble, and video games are a big part of how boys socialize now.

Seems like mainstream media has got cause and effect backwards. If the world survived Jazz and Rock & Roll, I'm sure it will survive video games.

Sunday, April 13, 2008

Ignorance is Bliss to None but Fools

It's funny how I thought I was a decent-looking guy. Didn't even notice my face was so asymmetrical and misshapen until I took a few pictures of myself to draw self-portraits recently.

Maybe it's a recent development. I've seen plenty of my baby pictures and pictures from high school, but I never noticed this.

I'll probably stop minding it soon, but it disturbs me that I failed to notice it soooner. What else about myself did I fail to perceive accurately?