Replace Pipes with Tabs in a Delimited File
Issue:
It is often necessary to replace delimiters in a file with a form the receiving party expects. sed is my favorite method of meeting the need.
Solution:
Replace double pipes with tabs
Note: If you find the tab key simply does not work, try CTRL+V+I from your terminal.
Reference: http://forums.devshed.com/unix-help-35/replacing-tabs-with-spaces-372623.html
Caveats:
Special consideration (and a regex) will be required for data where delimiters are present in the data itself. Consider your use case and apply appropriately.
Create a pidof command to find PID numbers easily
Most UNIX environments include the pidof command which is put to use whenever one needs to quickly determine, by name, the pid of a running program. Apple's Mac OS X lacks the pidof command, however, one may approximate its function with the following shell script:
ps axc|awk "{if (\$5==\"$1\") print \$1}";
Save the script as /bin/pidof and be sure to set its executable bit:
Strip empty (null) lines from a file
sed meets the need; the recipe follows:
Disable Spotlight in Mac OS X 10.6 Snow Leopard
On occasion I need to process large volumes of text locally. Spotlight dutifully attempts to index this data, bringing my system to a crawl.
Proactively disabling Spotlight is a sure way to avoid such issues and here is how to do it:
Disabling Spotlight in Snow Leopard is pretty easy, launch the Terminal and type the following command:
This tells the Spotlight manager to disable all indexing on all volumes, the command will require your administrative password to execute.
Re-enabling Spotlight in Mac OS X 10.6 Snow Leopard is just as easy, just reverse the command to:
Now Spotlight indexing will be back on and work as usual.
NOTE: mds and mdsworker will persist in the process table; this is normal.
Clear Spotlight Index
From a terminal window:
That is, sudo (because you have to have admin rights to run this), mdutil (the program that does the work for you) -a for “work on all volumes”, -v for “be verbose in telling me what you’re doing”, and -E for “erase the data store and rebuild it”.
Convert a .dmg to a .iso
Issue:
Mac formatted disk image (.dmg) cannot be directly burned on Windows or Linux systems.
Resolution:
One can convert a .dmg to a CD master via the Disk Utility application embedded in OS X, or by opening a terminal window and issuing the following command:
The output file will be named savefile.iso.cdr -- you may strip the .cdr and burn the .iso with any standard utility for doing so.
Create a Large File For Testing
Issue:
Often you need a set of variable sized files for testing a particular scenario. Generating test data is a painless endeavor.
Resolution:
The Unix dd command is perfectly suited to dispatch this need.
The above command will create a 5 megabyte file full of zeroes. Lovely. You may adjust the count (or blocksize) to achieve the results you desire. This data also achieves stellar compression ratios based on its content.
One could also create a test file full of pseudo random data by pointing if to /dev/urandom.
Remote Mirroring Using nc and dd
You can use the dd and nc commands for exact disk mirroring from one server to another. The following commands send data from Server1 to Server2:
1 2 | Server2# nc -l 12345 | dd of=/dev/sdb Server1# dd if=/dev/sda | nc server2 12345 |
Make sure that you issue Server2's command first so that it's listening on port 12345 when Server1 starts sending its data.
Unless you're sure that the disk is not being modified, it's better to boot Server1 from a RescueCD or LiveCD to do the copy.
Reference: http://www.linuxjournal.com/content/tech-tip-remote-mirroring-using-nc-and-dd
rsync failed to set times on [filename]
This error occurs because the version of rsync on the system cannot preserve modified times for directories.
Run rsync with the following arguments to suppress this warning:
rsync -avrPO ./source/* ./destination
Explanation of switches:
1 2 3 4 5 | a -> Archive mode (do not preserve hard links, ACLs, or extended attributes) v -> Verbose (I like to know what is happening) r -> Copy directories recursively P -> Equivalent to --partial --progress (for long transfers that may be interrupted) O -> Omit directories from times |
Create thumbnails en-masse from a bash prompt
A simple one-liner and ffmpeg, basename, and cut gets this done.
1 | for i in *.f4v; do ffmpeg -i `basename $i` -s 320x240 `basename $i | cut -d'.' -f1`.jpg; done |
Thumbnail output size is configurable with the -s switch.