xinetd per_source limit issues
Issue:
Users note availability issues when accessing services backed by xinetd (subversion, rsync, etc.)
Identification:
Syslog on the affected server will present multiple lines containing daemon per_source_limit for 0.0.0.0.
Cause:
You have exceeded per_source_limit defaults imposed by your xinetd configuration. Many distributions include per_source limits that may not be suitable for your use case. Evaluate your needs carefully.
Resolution:
Modify the default setting for per_source in /etc/xinetd.conf or modify the service specific configuration (recommended) under /etc/xinet.d. per_source limits may be set as follows:
per_source may be set to an integer or UNLIMITED (the number represents the number of connections allowed per host). A sensible fixed value is always better than UNLIMITED.
Reference: xinetd.conf(5)
Replace Pipes with Tabs in a Delimited File
Issue:
It is often necessary to replace delimiters in a file with a form the receiving party expects. sed is my favorite method of meeting the need.
Solution:
Replace double pipes with tabs
Note: If you find the tab key simply does not work, try CTRL+V+I from your terminal.
Reference: http://forums.devshed.com/unix-help-35/replacing-tabs-with-spaces-372623.html
Caveats:
Special consideration (and a regex) will be required for data where delimiters are present in the data itself. Consider your use case and apply appropriately.
Create a pidof command to find PID numbers easily
Most UNIX environments include the pidof command which is put to use whenever one needs to quickly determine, by name, the pid of a running program. Apple's Mac OS X lacks the pidof command, however, one may approximate its function with the following shell script:
ps axc|awk "{if (\$5==\"$1\") print \$1}";
Save the script as /bin/pidof and be sure to set its executable bit:
Strip empty (null) lines from a file
sed meets the need; the recipe follows:
Create a Large File For Testing
Issue:
Often you need a set of variable sized files for testing a particular scenario. Generating test data is a painless endeavor.
Resolution:
The Unix dd command is perfectly suited to dispatch this need.
The above command will create a 5 megabyte file full of zeroes. Lovely. You may adjust the count (or blocksize) to achieve the results you desire. This data also achieves stellar compression ratios based on its content.
One could also create a test file full of pseudo random data by pointing if to /dev/urandom.
Remote Mirroring Using nc and dd
You can use the dd and nc commands for exact disk mirroring from one server to another. The following commands send data from Server1 to Server2:
1 2 | Server2# nc -l 12345 | dd of=/dev/sdb Server1# dd if=/dev/sda | nc server2 12345 |
Make sure that you issue Server2's command first so that it's listening on port 12345 when Server1 starts sending its data.
Unless you're sure that the disk is not being modified, it's better to boot Server1 from a RescueCD or LiveCD to do the copy.
Reference: http://www.linuxjournal.com/content/tech-tip-remote-mirroring-using-nc-and-dd
PowerDNS continually dies after creating a slave zone
Issue:
When PowerDNS is configured for a sqlite or sqlite3 backend and a slave zone is added, the PowerDNS daemon continually dies and respawns. This loop persists until the daemon is forcibly terminated.
My logs showed the following:
Jul 29 12:12:38 localhost pdns[14465]: PowerDNS 2.9.22 (C) 2001-2009 PowerDNS.COM BV (Jul 29 2009, 04:47:43, gcc 4.3.3) starting up Jul 29 12:12:38 localhost pdns[14465]: PowerDNS comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it according to the terms of the GPL version 2. Jul 29 12:12:38 localhost pdns[14465]: DNS Proxy launched, local port 22293, remote 192.168.8.1:53 Jul 29 12:12:38 localhost pdns[14465]: Launched webserver on 0.0.0.0:8081 Jul 29 12:12:38 localhost pdns[14465]: Master/slave communicator launching Jul 29 12:12:38 localhost pdns[14465]: Creating backend connection for TCP Jul 29 12:12:38 localhost pdns[14465]: gsqlite3: connection to '/var/lib/powerdns/pdns.db' succesful Jul 29 12:12:38 localhost pdns[14465]: gsqlite3: connection to '/var/lib/powerdns/pdns.db' succesful Jul 29 12:12:38 localhost pdns[14465]: About to create 3 backend threads for UDP Jul 29 12:12:38 localhost pdns[14465]: gsqlite3: connection to '/var/lib/powerdns/pdns.db' succesful Jul 29 12:12:38 localhost pdns[14465]: 1 slave domain needs checking Jul 29 12:12:38 localhost pdns[14465]: Domain domain.local is stale, master serial 349, our serial 0 Jul 29 12:12:38 localhost pdns[14465]: Initiating transfer of 'domain.local' from remote '192.168.8.40' Jul 29 12:12:38 localhost pdns[14465]: gsqlite3: connection to '/var/lib/powerdns/pdns.db' succesful Jul 29 12:12:38 localhost pdns[14465]: AXFR started for 'domain.local', transaction started Jul 29 12:12:38 localhost pdns[14465]: Communicator thread died because of error: Database failed to start transaction: Error while retrieving SQLite query results
Cause:
The PowerDNS sqlite and sqlite3 backends do not support slave zones as of v.2.9.22
Resolution:
Configure PowerDNS to use a different database backend: MySQL, PostgreSQL, Oracle, etc.
rsync failed to set times on [filename]
This error occurs because the version of rsync on the system cannot preserve modified times for directories.
Run rsync with the following arguments to suppress this warning:
rsync -avrPO ./source/* ./destination
Explanation of switches:
1 2 3 4 5 | a -> Archive mode (do not preserve hard links, ACLs, or extended attributes) v -> Verbose (I like to know what is happening) r -> Copy directories recursively P -> Equivalent to --partial --progress (for long transfers that may be interrupted) O -> Omit directories from times |
Create thumbnails en-masse from a bash prompt
A simple one-liner and ffmpeg, basename, and cut gets this done.
1 | for i in *.f4v; do ffmpeg -i `basename $i` -s 320x240 `basename $i | cut -d'.' -f1`.jpg; done |
Thumbnail output size is configurable with the -s switch.