Replace Pipes with Tabs in a Delimited File
Issue:
It is often necessary to replace delimiters in a file with a form the receiving party expects. sed is my favorite method of meeting the need.
Solution:
Replace double pipes with tabs
Note: If you find the tab key simply does not work, try CTRL+V+I from your terminal.
Reference: http://forums.devshed.com/unix-help-35/replacing-tabs-with-spaces-372623.html
Caveats:
Special consideration (and a regex) will be required for data where delimiters are present in the data itself. Consider your use case and apply appropriately.
Remove a Range of Unwanted ZFS Snapshots from the Command Line
Remove a range of unwanted snapshots, from the command line, if necessary.
In the following example, all automatic snapshots in the bash shell are removed.
One could add a secondary pipe and grep statement to filter on individual file systems.
Create a pidof command to find PID numbers easily
Most UNIX environments include the pidof command which is put to use whenever one needs to quickly determine, by name, the pid of a running program. Apple's Mac OS X lacks the pidof command, however, one may approximate its function with the following shell script:
ps axc|awk "{if (\$5==\"$1\") print \$1}";
Save the script as /bin/pidof and be sure to set its executable bit:
Strip empty (null) lines from a file
sed meets the need; the recipe follows:
Create a Large File For Testing
Issue:
Often you need a set of variable sized files for testing a particular scenario. Generating test data is a painless endeavor.
Resolution:
The Unix dd command is perfectly suited to dispatch this need.
The above command will create a 5 megabyte file full of zeroes. Lovely. You may adjust the count (or blocksize) to achieve the results you desire. This data also achieves stellar compression ratios based on its content.
One could also create a test file full of pseudo random data by pointing if to /dev/urandom.
Remote Mirroring Using nc and dd
You can use the dd and nc commands for exact disk mirroring from one server to another. The following commands send data from Server1 to Server2:
1 2 | Server2# nc -l 12345 | dd of=/dev/sdb Server1# dd if=/dev/sda | nc server2 12345 |
Make sure that you issue Server2's command first so that it's listening on port 12345 when Server1 starts sending its data.
Unless you're sure that the disk is not being modified, it's better to boot Server1 from a RescueCD or LiveCD to do the copy.
Reference: http://www.linuxjournal.com/content/tech-tip-remote-mirroring-using-nc-and-dd
PowerDNS continually dies after creating a slave zone
Issue:
When PowerDNS is configured for a sqlite or sqlite3 backend and a slave zone is added, the PowerDNS daemon continually dies and respawns. This loop persists until the daemon is forcibly terminated.
My logs showed the following:
Jul 29 12:12:38 localhost pdns[14465]: PowerDNS 2.9.22 (C) 2001-2009 PowerDNS.COM BV (Jul 29 2009, 04:47:43, gcc 4.3.3) starting up Jul 29 12:12:38 localhost pdns[14465]: PowerDNS comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it according to the terms of the GPL version 2. Jul 29 12:12:38 localhost pdns[14465]: DNS Proxy launched, local port 22293, remote 192.168.8.1:53 Jul 29 12:12:38 localhost pdns[14465]: Launched webserver on 0.0.0.0:8081 Jul 29 12:12:38 localhost pdns[14465]: Master/slave communicator launching Jul 29 12:12:38 localhost pdns[14465]: Creating backend connection for TCP Jul 29 12:12:38 localhost pdns[14465]: gsqlite3: connection to '/var/lib/powerdns/pdns.db' succesful Jul 29 12:12:38 localhost pdns[14465]: gsqlite3: connection to '/var/lib/powerdns/pdns.db' succesful Jul 29 12:12:38 localhost pdns[14465]: About to create 3 backend threads for UDP Jul 29 12:12:38 localhost pdns[14465]: gsqlite3: connection to '/var/lib/powerdns/pdns.db' succesful Jul 29 12:12:38 localhost pdns[14465]: 1 slave domain needs checking Jul 29 12:12:38 localhost pdns[14465]: Domain domain.local is stale, master serial 349, our serial 0 Jul 29 12:12:38 localhost pdns[14465]: Initiating transfer of 'domain.local' from remote '192.168.8.40' Jul 29 12:12:38 localhost pdns[14465]: gsqlite3: connection to '/var/lib/powerdns/pdns.db' succesful Jul 29 12:12:38 localhost pdns[14465]: AXFR started for 'domain.local', transaction started Jul 29 12:12:38 localhost pdns[14465]: Communicator thread died because of error: Database failed to start transaction: Error while retrieving SQLite query results
Cause:
The PowerDNS sqlite and sqlite3 backends do not support slave zones as of v.2.9.22
Resolution:
Configure PowerDNS to use a different database backend: MySQL, PostgreSQL, Oracle, etc.
rsync failed to set times on [filename]
This error occurs because the version of rsync on the system cannot preserve modified times for directories.
Run rsync with the following arguments to suppress this warning:
rsync -avrPO ./source/* ./destination
Explanation of switches:
1 2 3 4 5 | a -> Archive mode (do not preserve hard links, ACLs, or extended attributes) v -> Verbose (I like to know what is happening) r -> Copy directories recursively P -> Equivalent to --partial --progress (for long transfers that may be interrupted) O -> Omit directories from times |
Enable opportunistic locking with Sun SMB service on Solaris Nevada
The primary reason for implementing this is performance.
From Microsoft:
Opportunistic locking (oplock) is a mechanism that allows a server to tell a client process that a requested file is only being used by that process. The client can safely do read-ahead and write-behind as well as local caching, knowing that the file will not be accessed or changed in any way by another process while the opportunistic lock is in effect. The server notifies the client when a second process attempts to open or modify the locked file.
Reference: http://msdn.microsoft.com/en-us/library/dd327670.aspx
The snippet:
svccfg -s smb/server setprop smbd/oplock_enable=boolean: true