Problem with sed command


As you know sed is the one of the widely used stream editor to perform basic text transformations on an input stream (a file or input from a pipeline).

Most of the times I will use sed to process text file.

Recently I struggled couple of hours with sed when trying to  replace a pattern with regular expression.

At last I got to know there is some issue with regex in sed.

Sed won’t support recent versions of regular expressions, so it not performed as I expected.

Then I tried with perl, since I got to know perl supports recent version of regular expressions.

I will show couple of example how to write perl one-liners.

  • $ echo ab | perl -p -e "s/a//g"
    • Output: b
  • perl -p -e "s/a//g" <inputFile>
    • you can use any input file with perl

Perl will support most of the functionality’s  that are supported by sed, especially it support recent version of regular expression.

So try to avoid sed when you are using regular expressions instead use perl.


9 comments on “Problem with sed command

  1. One of the issue I faced while using any pattern after question mark symbol.
    The regex a.*?c is to match any thing between a to c including a and c.
    $ echo abcd | sed “s/a.*?c//g” and output for this example if we use sed is abcd, instead if we use perl we get the output d.
    $ echo abcd | perl -p -e “s/a.*?c//g” this will give correct output.


  2. Now I am confused. I am not that good with regular expressions but
    I think this echo abcd | sed “s/a.*?c//g” looks for a string that has a question mark.

    If I changed your example to echo ab?cd | sed “s/a.*?c//g” the result is d.

    If I want to remove any a string that starts with a followed by zero or more characters followed by c then I would change it to echo abcd | sed “s/a.*c//g”.
    This is greedy so echo abdddcssddcd | sed “s/a.*c//g” would also give you d as a result.

    If I wanted to match a string that starts with a followed by a single character followed by c then I would use echo abcd | sed “s/a.\?c//g”

    My version of sed is
    $ sed –version
    GNU sed version 4.2.1

      • Yes(I had it wrong). But you need to escape it in sed for it to behave like that.
        echo abcd | sed “s/a.*?c//g” expects the string to have a question mark ie ab?cd.

        echo abcd | sed “s/a.*?c//g” does not behave as you want since it does not find a literal question mark in the string. It returns abcd.

        echo abcd | sed “s/a.*\?c//g” behaves as you have stated. And the result of this is d.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s