Replacing text in multiple files using the replace utility

First of all, a Happy New Year to you all!

A new year has begun, so it's time to update the copyright headers in the openATTIC source tree!

While there are countless options for resolving this task, I have become quite fond of a little gem hidden in the MySQL Server distribution called replace. It does exactly what you would expect from its name - from the manual page:

The replace utility program changes strings in place in files or on the standard input.

I first learned about replace when I started my first job as a systems administrator after finishing my computer science studies. Back then, my company was using David Hughes' mSQL database with Rasmus Lerdorf's PHP/FI 2.0, the predecessor of PHP.

One of my tasks was to port some existing web applications from mSQL to this new database called "MySQL" which just started to get some traction.

To my surprise, MySQL actually shipped with a script called msql2mysql, which I could just run over some existing code after importing the mSQL dump into MySQL. It basically just replaced all mSQL support function calls (msql_*()) with their MySQL equivalents (mysql_*()), but this process worked so flawlessly that I was immediately convinced of this new database and their developers. msql2mysql was removed in MySQL 5.7, but the underlying replace utility is still shipped with MySQL nowadays and is a very handy tool for performing string replacements in multiple files.

The usage is very simple:

$ replace "old string" "new string" -- <file(s)>

In this example, any occurrence of "old string" will be replaced with "new string" for any file provided after the double dashes. Keep in mind that replace actually overwrites the original files, so make sure you either have a backup or the files are under revision control!

replace can also perform multiple string replacement operations in the same run, which can also be used to swap strings! In the following example, any ocurrences of "old string" will be replaced with "new string", and vice versa:

$ replace "old string" "new string" "new string" "old string" -- <file(s)>

In order to update the copyright information of all files in the openATTIC source code repository, I used the following shell command:

$ replace "Copyright (C) 2011-2015, it-novum GmbH <>" \
"Copyright (C) 2011-2016, it-novum GmbH <>" -- \
`grep -rl "Copyright (C) 2011-2015, it-novum GmbH <>" . \
| grep -v .hg`

I basically run a recursive grep to find all files that contain the old copyright header, exclude the files in Mercurial's .hg directory and then hand over the resulting file list to replace for performing the string replacement. Of course, I could have simply used the find command and submit the complete list of files in the repository to replace (as it only performs the replacement, if there's a match).

And sure, I could have written a custom Perl/Python script or using some magic Shell script invoking sed and find, but in my opinion this command is hard to beat when it comes to simplicity...


Comments powered by Disqus