Forgetting That Pipelines Make Subshells

Problem

You have a script that works just fine, reading input in a while loop:

# This works as expected
COUNT=0
while read PREFIX GUTS
do
    # ...
    if [[ $PREFIX == "abc" ]]
    then
        let COUNT++
    fi
    # ...
done
echo $COUNT

and then you change it to read from a file:

#Don't use; this does NOT work as expected!
COUNT=0
cat $1 | while read PREFIX GUTS
do
    # ... 
    if [[ $PREFIX == "abc" ]]
        then
            let COUNT++
        fi
        # ...
    done
    echo $COUNT   # $COUNT is always '0' which is broken

only now it no longer works…$COUNT keeps coming out as zero.

Solution

Pipelines create subshells. Changes in the while loop do not effect the variables in the outer part of the script, as the while loop is run in a subshell.

One solution: don’t do that (if you can help it). In this example, instead of using cat to pipe the file’s content into the while statement, you could use I/O redirection to have the input come from a redirected input rather than setting up a pipeline:

# Avoid the | and sub-shell; use "done < $1" instead
# It now works as expected
COUNT=0
while read PREFIX GUTS
do
    # ...
  if [[ $PREFIX == "abc" ]]
   then

            let COUNT++

        fi

        # ...

done < $1 # <<<< This is the key line

echo "$COUNT now lives in the main script"

Such a rearrangement might not be appropriate for your problem, in which case you’ll have to find other techniques.

Discussion

If you add an echo statement inside the while loop, you can see $COUNT increasing, but once you exit the loop, $COUNT will be back to zero. The way that bash sets up the pipeline of commands means that each command in the pipeline will execute in its own subshell. So the while loop is in a subshell, not in the main shell. If you have exported $COUNT, then the while loop will begin with the same value that the main shell script was using for $COUNT, but since the while loop is executing in a subshell there is no way to get the value back up to the parent shell.

Depending on how much information you need to get back to the parent shell and how much more work the outer level needs to do after the pipeline, there are different techniques you could use. One technique is to take the additional work and make it part of a subshell that includes the while loop. For example:

COUNT=0
cat $1 | ( while read PREFIX GUTS
do
  # ...
done
echo $COUNT )

The placement of the parentheses is crucial here. What we’ve done is explicitly delineated a section of the script to be run in a subshell. It includes both the while loop and the other work that we want to do after the while loop completes (here all we’re doing is echoing $COUNT). Since the while and the echo statements are not a pipeline, they will both run in the same subshell created by virtue of the parentheses. The $COUNT that was accumulated during the while loop will remain until the end of the subshell—that is, until the end-parenthesis is reached.

If you do use this technique it might be good to format the statements a bit differently, to make the use of the parenthesized subshell stand out more. Here’s the whole script reformatted:

COUNT=0
cat $1 |
(
    while read PREFIX GUTS
    do
        # ...
        if [[ $PREFIX == "abc"
        then
            let COUNT++
        fi
       # ...
    done
    echo $COUNT
)

We can extend this technique if there is much more work to be done after the while loop. The remaining work could be put in a function call or two, again keeping them in the subshell. Otherwise, the results of the while loop can be echoed (as is done here) and then piped into the next phase of work (which will also execute in its own subshell), which can read the results from the while loop:

COUNT=0
cat $1 |
(
    while read PREFIX GUTS
    do
        # ...
        if [[ $PREFIX == "abc" ]]
        then
            let COUNT++
        fi
        # ...
    done
    echo $COUNT
) | read COUNT
# continue on...
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.15.214.155