Solutions Log » Solaris

Replace Pipes with Tabs in a Delimited File

Dan — Fri, 04 Dec 2009 20:44:37 +0000

Issue:
It is often necessary to replace delimiters in a file with a form the receiving party expects. sed is my favorite method of meeting the need.

Solution:
Replace double pipes with tabs

sed 's/||/ /g' file.in > file.out

Note: If you find the tab key simply does not work, try CTRL+V+I from your terminal.
Reference: http://forums.devshed.com/unix-help-35/replacing-tabs-with-spaces-372623.html

Caveats:
Special consideration (and a regex) will be required for data where delimiters are present in the data itself. Consider your use case and apply appropriately.

Remove a Range of Unwanted ZFS Snapshots from the Command Line

Dan — Thu, 26 Nov 2009 05:51:41 +0000

Remove a range of unwanted snapshots, from the command line, if necessary.
In the following example, all automatic snapshots in the bash shell are removed.

for s in $(zfs list -H -o name -t snapshot | grep @zfs-auto-snap); do zfs destroy $s; done

One could add a secondary pipe and grep statement to filter on individual file systems.

Reference: How to Manage the Automatic ZFS Snapshot Service

Disable X in OpenSolaris

Dan — Sat, 10 Oct 2009 03:18:46 +0000

From the shell

pfexec svcadm disable gdm

Create a pidof command to find PID numbers easily

Dan — Wed, 07 Oct 2009 20:30:44 +0000

Most UNIX environments include the pidof command which is put to use whenever one needs to quickly determine, by name, the pid of a running program. Apple's Mac OS X lacks the pidof command, however, one may approximate its function with the following shell script:

#!/bin/sh
ps axc|awk "{if (\$5==\"$1\") print \$1}";

Save the script as /bin/pidof and be sure to set its executable bit:

chmod a+x /bin/pidof

Strip empty (null) lines from a file

Dan — Tue, 06 Oct 2009 18:12:06 +0000

sed meets the need; the recipe follows:

sed '/^$/d' filename

Create a Large File For Testing

Dan — Thu, 13 Aug 2009 15:23:33 +0000

Issue:
Often you need a set of variable sized files for testing a particular scenario. Generating test data is a painless endeavor.

Resolution:
The Unix dd command is perfectly suited to dispatch this need.

dd if=/dev/zero of=~/testfile.txt bs=1m count=5

The above command will create a 5 megabyte file full of zeroes. Lovely. You may adjust the count (or blocksize) to achieve the results you desire. This data also achieves stellar compression ratios based on its content.

One could also create a test file full of pseudo random data by pointing if to /dev/urandom.

dd if=/dev/urandom of=~/testfile.txt bs=1m count=5

Explanation of /dev/urandom

Remote Mirroring Using nc and dd

Dan — Mon, 10 Aug 2009 16:58:06 +0000

You can use the dd and nc commands for exact disk mirroring from one server to another. The following commands send data from Server1 to Server2:

1 2	Server2# nc -l 12345 \| dd of=/dev/sdb Server1# dd if=/dev/sda \| nc server2 12345

Make sure that you issue Server2's command first so that it's listening on port 12345 when Server1 starts sending its data.

Unless you're sure that the disk is not being modified, it's better to boot Server1 from a RescueCD or LiveCD to do the copy.

Reference: http://www.linuxjournal.com/content/tech-tip-remote-mirroring-using-nc-and-dd

PowerDNS continually dies after creating a slave zone

Dan — Wed, 29 Jul 2009 17:19:20 +0000

Issue:
When PowerDNS is configured for a sqlite or sqlite3 backend and a slave zone is added, the PowerDNS daemon continually dies and respawns. This loop persists until the daemon is forcibly terminated.

My logs showed the following:

Jul 29 12:12:38 localhost pdns[14465]: PowerDNS 2.9.22 (C) 2001-2009 PowerDNS.COM BV (Jul 29 2009, 04:47:43, gcc 4.3.3) starting up
Jul 29 12:12:38 localhost pdns[14465]: PowerDNS comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it according to the terms of the GPL version 2.
Jul 29 12:12:38 localhost pdns[14465]: DNS Proxy launched, local port 22293, remote 192.168.8.1:53
Jul 29 12:12:38 localhost pdns[14465]: Launched webserver on 0.0.0.0:8081
Jul 29 12:12:38 localhost pdns[14465]: Master/slave communicator launching
Jul 29 12:12:38 localhost pdns[14465]: Creating backend connection for TCP
Jul 29 12:12:38 localhost pdns[14465]: gsqlite3: connection to '/var/lib/powerdns/pdns.db' succesful
Jul 29 12:12:38 localhost pdns[14465]: gsqlite3: connection to '/var/lib/powerdns/pdns.db' succesful
Jul 29 12:12:38 localhost pdns[14465]: About to create 3 backend threads for UDP
Jul 29 12:12:38 localhost pdns[14465]: gsqlite3: connection to '/var/lib/powerdns/pdns.db' succesful
Jul 29 12:12:38 localhost pdns[14465]: 1 slave domain needs checking
Jul 29 12:12:38 localhost pdns[14465]: Domain domain.local is stale, master serial 349, our serial 0
Jul 29 12:12:38 localhost pdns[14465]: Initiating transfer of 'domain.local' from remote '192.168.8.40'
Jul 29 12:12:38 localhost pdns[14465]: gsqlite3: connection to '/var/lib/powerdns/pdns.db' succesful
Jul 29 12:12:38 localhost pdns[14465]: AXFR started for 'domain.local', transaction started
Jul 29 12:12:38 localhost pdns[14465]: Communicator thread died because of error: Database failed to start transaction: Error while retrieving SQLite query results

Cause:
The PowerDNS sqlite and sqlite3 backends do not support slave zones as of v.2.9.22

Resolution:
Configure PowerDNS to use a different database backend: MySQL, PostgreSQL, Oracle, etc.

rsync failed to set times on [filename]

Dan — Fri, 10 Jul 2009 02:34:52 +0000

This error occurs because the version of rsync on the system cannot preserve modified times for directories.

Run rsync with the following arguments to suppress this warning:

 rsync -avrPO ./source/* ./destination

Explanation of switches:

1
2
3
4
5

a -> Archive mode (do not preserve hard links, ACLs, or extended attributes)
v -> Verbose (I like to know what is happening)
r -> Copy directories recursively
P -> Equivalent to --partial --progress (for long transfers that may be interrupted)
O -> Omit directories from times

Enable opportunistic locking with Sun SMB service on Solaris Nevada

Dan — Wed, 08 Jul 2009 23:13:01 +0000

The primary reason for implementing this is performance.

From Microsoft:

Opportunistic locking (oplock) is a mechanism that allows a server to tell a client process that a requested file is only being used by that process. The client can safely do read-ahead and write-behind as well as local caching, knowing that the file will not be accessed or changed in any way by another process while the opportunistic lock is in effect. The server notifies the client when a second process attempts to open or modify the locked file.
Reference: http://msdn.microsoft.com/en-us/library/dd327670.aspx

The snippet:

svccfg -s smb/server setprop smbd/oplock_enable=boolean: true