6 Masks and Channels

Lesson overview

In this lesson, you’ll learn how to do the following:

• Create a mask to remove a subject from a background.

• Refine a mask to include complex edges.

• Create a quick mask to make changes to a selected area.

• Edit a mask using the Properties panel.

• Manipulate an image using Puppet Warp.

• Save a selection as an alpha channel.

• View a mask using the Channels panel.

• Load a channel as a selection.

This lesson will take about an hour to complete. To get the lesson files used in this chapter, download them from the web page for this book at www.adobepress.com/PhotoshopCIB2020. For more information, see “Accessing the lesson files and Web Edition” in the Getting Started section at the beginning of this book.

As you work on this lesson, you’ll preserve the start files. If you need to restore the start files, download them from your Account page.

Working with masks and channels

When you want to edit a specific area of a layer, it can be difficult to keep edits from going past the precise edge you want. Or you might want to apply an adjustment layer or a filter to only part of a layer. Both are easier using a mask, a type of image overlay that controls which layer areas will be affected by edits.

You can also use a layer mask to mark transparent areas of a layer. Layer masks are essential for removing backgrounds, combining multiple images into a composite image, and controlling which areas are affected by adjustment layers. A layer mask is more useful than deleting unwanted areas of a layer, because it’s reversible: By painting in a layer mask, you can make hidden areas visible again.

In addition to color channels, an image can contain alpha channels, useful for saving and reusing selections and marking transparent areas for an entire document.

It can be challenging to paint or draw a mask or channel edge that precisely follows a non-sharp border of a subject, such as fuzzy hair. Photoshop provides tools that use automation to help you quickly create complicated selections and masks.

Getting started

First, you’ll view the image that you’ll create using masks and channels.

1. Start Photoshop, and then immediately hold down Ctrl+Alt+Shift (Windows) or Command+Option+Shift (Mac) to restore the default preferences. (See “Restoring Default Preferences” on page 5.)

2. When prompted, click Yes to delete the Adobe Photoshop Settings file.

3. Choose File > Browse In Bridge to open Adobe Bridge.

4. Click the Favorites tab on the left side of the Bridge window. Select the Lessons folder, and then double-click the Lesson06 folder in the Content panel.

5. Study the 06End.psd file. To enlarge the thumbnail so that you can see it more clearly, move the thumbnail slider at the bottom of the Bridge window to the right.

In this lesson, you’ll create a magazine cover. The model for the cover was photographed in front of a different background, and the background isn’t a single color, so it may be a challenge to mask. You’ll use Select and Mask to quickly isolate the model over the final background.

6. Double-click the 06Start.psd thumbnail to open it in Photoshop. Click OK if you see an Embedded Profile Mismatch dialog box.

7. Choose File > Save As, rename the file to 06Working.psd, and click Save. Click OK if the Photoshop Format Options dialog box appears.

Saving a working version of the file lets you return to the original if you need it.

Using Select and Mask and Select Subject

Photoshop provides a set of tools focused on creating and refining masks, collected in a task space called Select and Mask. Inside Select and Mask, you’ll use the Select Subject tool to get a fast head start on the mask that will separate the model from the background. Then you’ll refine the mask using other Select and Mask tools, such as the Quick Selection tool.

1. In the Layers panel, make sure both layers are visible and the Model layer is selected.

2. Choose Select > Select and Mask.

”Select and Mask opens with the image. A semitransparent “onion skin” overlay indicates masked areas, which are represented by a checkerboard pattern. For now, the checkerboard pattern covers the entire image, because you haven’t yet identified the areas to unmask.

3. In the Options bar, click the Select Subject button (or choose Select > Subject).

   

Using advanced machine learning technology, the Select Subject feature is trained to identify typical subjects of a photo, including people, animals, and objects, and then create a selection for them. The selection may not be perfect, but it’s often close enough for you to refine easily and rapidly with other selection tools such as the Quick Selection tool. First, get a clear idea of the precision of the selection by using a different way to see the selection.

4. Click the View menu in the View Mode section of the Properties panel, and choose Overlay. The masked area is now shown as a semitransparent red color instead of the onion skin checkerboard.

   

The different View Modes are provided so that you can see the mask more easily over various backgrounds. In this case, the red overlay makes it easier to see missed areas and edges where loose hair isn’t properly masked.

Notice that there are a few areas over the chest that were missed by Select Subject. You can easily add them to the selection using the Quick Selection tool.

5. Make sure the Quick Selection tool () is selected. In the options bar, set up a brush with a size of 15 px.

6. Drag the Quick Selection tool over the missed areas (without extending into the background) to add the missed areas to the selection. Notice that the Quick Selection tool fills in the selection as it detects content edges, so you don’t have to be exact. It’s OK if you release the mouse button and drag more than once.

   

Where you drag teaches the Quick Selection tool which areas should be revealed and not part of the mask. Do not drag the Quick Selection tool over or past the model’s edge to the background, because this teaches the Quick Selection tool to include part of the background, and you don’t want that. If you accidentally add unwanted areas to the mask, either choose Edit > Undo, or reverse the edit by painting over it with the Quick Selection tool in Subtract mode. To enable Subtract Mode for the Quick Selection tool, click the Subtract From Selection icon () in the options bar.

As you drag the Quick Selection tool over the model, the overlay disappears from the areas that you are marking to be revealed. Don’t worry about total perfection at this stage.

7. Click the View menu in the View Mode section again, and choose On Layers. This shows you how the current Select and Mask settings look over any layers that are behind this layer. In this case, you’re previewing how the current settings will mask the Model layer over the Magazine Background layer.

   

Inspect the edges around the model at a high magnification, such as 400%. Some of the original light background may still show between the model’s edge and the magazine background, but overall the Select Subject and Quick Selection tools should have created clean edges for the shirt and face. Don’t be concerned about edge gaps or imperfect hair edges, because you’ll take care of those next.

Refining a mask

The mask is pretty good so far, but the Quick Selection tool couldn’t quite capture all of the model’s hair, such as the strands coming off of the bun on the back of the model’s head. In Select and Mask, the Refine Edge Brush tool is designed to mask edges with challenging details.

1. At a magnification of 300% or higher, inspect the hair edges at the back of the model’s head.

   

2. Select the Refine Edge Brush tool (). In the options bar, set up a brush with a size of 20 px and Hardness of 100%.

3. Choose Overlay from the View menu in the View Mode section, so that the missing hair is visible.

4. Drag the Refine Edge Brush tool between the hair bun and the ends of the hair, where the selection needs to be improved. As you drag the Refine Edge Brush tool over the complex hair edge, you should see that the hair strands falling from the bun are now included in the visible areas.

   

5. Scroll down to the loop of hair that falls over the back of the shirt.

The original image background shows inside the loop. That background needs to be replaced with the magazine background, so you’ll use the Refine Edge Brush to add that loop interior to the mask.

6. In the options bar, set up a Refine Edge brush with a size of 15 px, with Hardness of 100%.

Julieanne Kost is an official Adobe Photoshop evangelist.

Tool tips from the Photoshop evangelist

Zoom tool shortcuts

Often when editing an image, you’ll need to zoom in to work on a detail and then zoom out again to see the changes in context. Here are several keyboard shortcuts that make zooming faster and easier.

• With any tool selected, press Ctrl (Windows) or Command (Mac) with the plus sign (+) to zoom in, or with the minus sign (−) to zoom out.

• Double-click the Zoom tool in the Tools panel to return the image to 100% view.

• When Scrubby Zoom is selected in the options bar, just drag the Zoom tool to the left to zoom out or to the right to zoom in.

• Press Alt (Windows) or Option (Mac) to change the Zoom In tool to the Zoom Out tool, and click the image. Each Alt/Option-click reduces the image by the next preset increment, centering the image where you clicked.

7. Drag the Refine Edge Brush tool over any obscured hairs to reveal them. You should see two things happen: Gaps in the mask within the loop of hair should become masked, while fine hairs are added to the visible areas.

   

8. Click the View option, and choose Black & White. This is another good way to check your mask. Inspect the mask at different magnifications, and when you’re done, choose View > Fit on Screen. Black areas indicate transparency.

   

If you see hairs or other details that are masked but should be revealed, drag the Refine Edge brush over them. The finer the details you want to mask, the smaller you should set the Refine Edge brush size. It’s OK if the brush size is slightly larger than the details you want to mask, and you don’t have to drag the Refine Edge Brush tool precisely.

If you see Refine Edge Brush tool mistakes that need to be erased, such as inner areas incorrectly added to the mask, drag the Refine Edge Brush tool over the mistakes while holding down the Alt (Windows) or Option (Mac) key.

”If you see individual spots or discrete areas that need to be fully visible or fully transparent, you can paint them out using the Brush tool, the third one down in the toolbox. To make areas visible, paint them with white; to make areas hidden, paint them with black.

Adjusting Global Refinements

At this point the mask is in good shape, but needs to be tightened up a little. You can tune the overall appearance of the mask edge by adjusting the Global Refinements settings.

1. Click the View menu in the View Mode section of the Properties panel, and choose On Layers. This lets you preview adjustments over the Magazine Background layer, which is behind the Model layer.

2. In the Global Refinements section, move the sliders to create a smooth, unfeathered edge along the face. The optimal settings depend on the selection you created, but they’ll probably be similar to ours. We moved the Smooth slider to 5 to reduce the roughness of the outline, Contrast to 20% to sharpen the transitions along the selection border, and Shift Edge to −15% to move the selection border inward and help remove unwanted background colors from selection edges. (Adjusting Shift Edge to a positive number would move the border outward.)

   

3. Take one more look at this preview of the current mask over the Magazine Background layer, and make any remaining corrections as needed.

Completing the mask

When you’re satisfied with the mask preview, you can create its final output as a selection, a layer with transparency, a layer mask, or a new document. For this project, we want to use this as a layer mask on the Model layer, which was selected when you entered Select and Mask.

1. If the Output Settings are hidden, click the disclosure icon () to reveal them.

2. Zoom in to 200% or more so that you can more easily see the light fringing around the face edge that’s due to the Model layer’s original background color seeping in behind the mask.

3. Select Decontaminate Colors to suppress those color fringes. If Decontaminate Colors creates unwanted artifacts, reduce the Amount until the effect looks the way you want. We set Amount to 25%.

   

4. Choose New Layer With Layer Mask from the Output To menu. Then click OK.

   

Clicking OK exits Select and Mask. In the Layers panel, the Model Copy layer now has a layer mask (pixel mask) that was generated by Select and Mask.

The layer was copied because using the Decontaminate Colors option requires generating new pixels. The original Model layer is preserved, and automatically hidden. If you wanted to start over, you could delete the Model Copy layer, make the original Model layer visible again and selected, and open Select and Mask again.

If the mask isn’t perfect, you can continue to edit it at any time. When a layer mask thumbnail is selected in the Layers panel, you can click the Select and Mask button in the Properties panel, in the options bar (if a selection tool is active), or choose Select > Select and Mask.

5. Save your work.

Creating a quick mask

To change the color of the glasses frames you’ll use a quick mask, because you won’t need the mask after you’re done with this one task, and so that you can create the mask by painting. First, you’ll clean up the Layers panel.

1. Hide the Magazine Background layer so you can concentrate on the model. Make sure the Model copy layer is selected.

   

2. Click the Edit In Quick Mask Mode button in the Tools panel. (By default, you have been working in Standard mode.)

   

In Quick Mask mode, a red overlay appears as you make a selection, masking the area outside the selection the way a rubylith, or red acetate, was used to mask images in traditional print shops. (This idea is similar to the Overlay view mode you saw in Select and Mask.) You can apply changes only to the unprotected area that is visible and selected. Notice that the highlight color for the selected layer in the Layers panel is red, indicating you’re in Quick Mask mode.

3. In the Tools panel, select the Brush tool ().

4. In the options bar, make sure that the mode is Normal. Open the Brush pop-up panel, and select a small brush with a diameter of 13 px and a Hardness of 100%. Click outside the panel to close it.

5. Paint the earpiece of the glasses frames. The area you paint will appear red, creating a mask.

6. Continue painting with the Brush tool to mask the earpiece of the frames and the frame around the lenses. Reduce the brush size to paint around the lenses. You can stop where the earpiece goes under the hair.

   

In Quick Mask mode, Photoshop operates in grayscale mask mode, where shades of gray correspond to degrees of mask transparency. When using a painting or editing tool in Quick Mask mode, keep these principles in mind:

• Painting with black adds to the mask (the red overlay), subtracting from the selected area.

• Painting with white erases the mask, adding to the selected area.

• Painting with gray adds semitransparent areas to the mask, where darker shades are more transparent (more masked).

7. Click the Edit In Standard Mode button to exit Quick Mask Mode.

The unmasked area is selected. Exiting Quick Mask mode converts the Quick Mask into a selection.

8. Choose Select > Inverse to select the area you originally masked.

9. Choose Image > Adjustments > Hue/Saturation. The selection is converted to a layer mask that restricts the Hue/Saturation adjustment to the unmasked area.

10. In the Hue/Saturation dialog box, change the Hue to +70. The new green color fills the glasses frame. Click OK.

    

11. Choose Select > Deselect.

Manipulating an image with Puppet Warp

The Puppet Warp feature gives you flexibility in manipulating an image. You can reposition areas, such as hair or an arm, just as you might pull the strings on a puppet. Place pins wherever you want to control movement. You’ll use Puppet Warp to tilt the model’s head back, so she appears to be looking up.

1. Zoom out so you can see the entire model.

   

2. With the Model Copy layer selected in the Layers panel, choose Edit > Puppet Warp.

A mesh appears over the visible areas in the layer—in this case, the mesh appears over the model. You’ll use the mesh to place pins where you want to control movement (or to ensure there is no movement).

3. Click to add pins around the body and along the base of the head. Each time you click, Puppet Warp adds a pin. Around ten to twelve pins should work.

The pins you’ve added around the shirt will keep it in place as you tilt the head.

4. Select the pin at the nape of the neck. A blue dot appears in the center of the pin to indicate that it’s selected.

   

5. Press Alt (Windows) or Option (Mac). A larger circle appears around the pin and a curved double arrow appears next to it. Continue pressing Alt or Option as you drag the pointer to rotate the head backwards. You can see the angle of rotation in the options bar; you can enter 170 there to rotate the head back.

   

6. When you’re satisfied with the rotation, click the Commit Puppet Warp button () in the options bar, or press Enter or Return.

   

7. Save your work so far.

Using an alpha channel to create a shadow

Just as different information in an image is stored on different layers, channels also let you access specific kinds of information. Alpha channels store selections as grayscale images. Color information channels store information about each color in an image; for example, an RGB image automatically has red, green, blue, and composite channels.

To avoid confusing channels and layers, think of channels as containing an image’s color and selection information, and think of layers as containing painting, shapes, text, and other content.

You’ll first convert the transparent areas of the Model copy layer to a selection, then fill it with black on another layer to create the shadow. Because the selection will be altered to make the shadow, you’ll now save the current form of the selection as an alpha channel so that you can load it again later if needed.

1. In the Layers panel, Ctrl-click (Windows) or Command-click (Mac) the layer thumbnail icon for the Model Copy layer. The masked area is selected.

2. Choose Select > Save Selection. In the Save Selection dialog box, make sure New is chosen in the Channel menu. Then name the channel Model Outline, and click OK.

   

Nothing changes in the Layers panel or in the image window. However, a new channel named Model Outline has been added to the Channels panel. The selection is still active.

3. Click the Create A New Layer icon () at the bottom of the Layers panel. Drag the new layer below the Model Copy layer, so that the shadow will be below the image of the model. Double-click the new layer’s name, and rename it Shadow.

4. With the Shadow layer selected, choose Select > Select and Mask.

5. Click the View menu in the View Mode section of the Properties panel, and choose On Black.

6. In Select and Mask, move the Shift Edge slider to +36%.

7. In the Output Settings section, make sure Selection is selected in the Output To menu, and then click OK.

   

8. Choose Edit > Fill. In the Fill dialog box, choose Black from the Contents menu, and then click OK.

   

The Shadow layer displays a filled-in black outline of the model. Shadows aren’t usually as dark as the person that casts them, so you’ll reduce the layer opacity.

9. In the Layers panel, change the layer opacity to 30%.

The shadow is in exactly the same position as the model, where it can’t be seen. You’ll shift it.

10. Choose Select > Deselect to remove the selection.

11. Choose Edit > Transform > Rotate. Rotate the shadow by hand, or enter −15° in the Rotate field in the options bar. Then drag the shadow to the left, or enter 545 in the X field in the options bar. Click the Commit Transform button () in the options bar, or press Enter or Return, to accept the transformation.

    

12. Click the eye icon for the Magazine Background layer to make it visible, and delete the Model layer (the one without the mask).

13. Choose File > Save to save your work so far.

Your magazine cover is ready to go!