Working with Text and Files
By now you’re pretty comfortable creating and manipulating files and folders from the CLI. But with some additional tools, you can solve more complex problems while building on the skills you’ve already learned.
Managing Files with ranger
ranger is a terminal-based graphical file manager. While this book is all about working on the command line, ranger makes it easier to work with files in batches, and it also makes previewing files a breeze. To use it, you first need to install it. On Ubuntu, use apt:
| | $ sudo apt install ranger |
On macOS, use brew:
| | $ brew install ranger |
Then execute the following command to generate ranger’s default configuration, which will be stored in ~/.config/ranger:
| | $ ranger --copy-config=all |
After that, start ranger:
| | $ ranger |
You will be greeted with a three-pane interface that looks like the first image.
The middle pane shows the current directory’s files and directories. The left pane shows the contents of the parent directory, with the current directory highlighted. The right pane shows the contents of the item selected in the middle pane. If the item in the middle pane is a directory, the right pane shows the directory’s contents. If it’s a file, the right pane shows the file’s contents.
Use the j key to move down one item in the middle pane. Use the k key to move up one folder. As you navigate through each directory or file, its contents automatically appear in the right pane. This makes ranger great for reviewing a directory full of code or documentation.
To navigate into a directory, press l or your right arrow key. To navigate to the parent directory, use h or your left arrow key. Each time you navigate, the panes update to reflect where you are and what you see.
You already know how to navigate around your filesystem, but ranger makes things a little easier if you have more complex operations you need to perform. For example, pressing the g key displays a list of predefined destinations:
If you type gv, ranger switches to the /var directory. If you type gh, you’ll end up back in your home directory.
You can use ranger to create new directories and files. Try it out. Use gh to go to your home directory. Then, type : to enter ranger’s command mode. Type mkdir testing, press Enter and ranger displays the new directory.
Use the j key to navigate to the testing directory. Then, press Enter to switch to the directory. Create a new file by typing : to enter Command mode, and enter the command touch test.txt. Press Enter to create the file.
The new file appears in ranger. Press Enter on the file. If you have the VISUAL environment variable set to nano, the file opens in the nano text editor so you can modify it. If you don’t have that variable set, ranger will ask you what text editor you’d like to use.
With the file open in nano, enter these lines into the file:
| | # This is a simple document. |
| | |
| | There's nothing too special about it. |
Save the file by pressing Ctrl+x, followed by y to confirm you want to save the file. nano displays the current filename, so press Enter to save the file and exit nano.
When you return to ranger, it displays the contents of the file you just saved in the right-hand pane:
Now let’s take a look at how you can use ranger to copy and move files. In the testing directory, create a new directory named backup. Again, type : followed by mkdir backup.
Now, use ranger to copy the test.txt file into the backup folder. First, ensure the test.txt file is highlighted. Then, type yy which copies the text.txt file. Navigate into the backup folder and type pp. The text.txt file appears in the backup.txt folder.
Duplicate this file a few times by pressing pp. ranger will create copies of the file.
You should now have these four files:
| | test.txt |
| | test.txt_ |
| | test.txt_0 |
| | test.txt_1 |
ranger lets you perform actions on multiple files if you mark them. Select the first file and press the Spacebar. Do the same for the other three files.
Rename these files. Press :, followed by the command bulkrename.
This launches nano again with the filenames in the editor window:
Modify the file so it looks like this:
| | one.txt |
| | two.txt |
| | three.txt |
| | four.txt |
Press Ctrl+x, followed by y to save the file. Accept the default filename by pressing Enter.
ranger determines the commands needed to rename the files and displays a script in nano for you to review:
| | # This file will be executed when you close the editor. |
| | # Please double-check everything, clear the file to abort. |
| | mv -vi -- test.txt one.txt |
| | mv -vi -- test.txt_0 two.txt |
| | mv -vi -- test.txt_ three.txt |
| | mv -vi -- test.txt_1 four.txt |
Press Ctrl+x to close nano. ranger renames the files and displays the new filenames.
Move these files back to the testing directory. Mark each of these files again by selecting each file and pressing the Spacebar key. Now, press dd. Then, navigate back to the testing directory and press pp. The files move to the new location.
You can delete these files by marking them and using the :delete command.
ranger is a great way to work with files and directories in a more graphical way while still integrating with the command-line interface.
Managing Documents with pandoc
If you work with text-based documents, you’ll find pandoc to be a lifesaver. pandoc can convert documents from one format to another. Let’s use it to convert a Markdown document to HTML and back.
Install pandoc with your package manager:
| | $ sudo apt install pandoc |
Now, create a new Markdown document called hello.md in your home directory with this content:
| | # Welcome |
| | |
| | This is a simple Markdown document. |
| | |
| | It has a [link to Google](http://google.com) in it. |
Use pandoc to convert this file to HTML:
| | $ pandoc -f markdown -t html -o hello.html hello.md |
The -f and -t arguments specify the source and destination formats. You can think of this as “from Markdown to HTML.” The -o argument specifies the output file.
pandoc converts the file. View it with the cat command:
| | $ cat hello.html |
| | <h1 id="welcome">Welcome</h1> |
| | <p>This is a simple Markdown document.</p> |
| | <p>It has a <a href="http://google.com">link to Google</a> in it.</p> |
You can go the other direction too. Create an HTML file called hello2.html with this content:
| | <h1>Testing</h1> |
| | <p>This is an HTML document.</p> |
| | <ul> |
| | <li>It has a list.</li> |
| | <li>With <a href="http://google.com">a link</a>.</li> |
| | <li>And some <em>emphasized</em> text!</li> |
| | </ul> |
| | <h2>And a subheading.</h2> |
| | <p>And there's even some extra <span class="keyword">markup.</span></p> |
Now, use pandoc to convert it. Use the -t markdown_mmd option this time to tell pandoc to use MultiMarkdown in the output:
| | $ pandoc -f html -t markdown_mmd -o hello2.md hello2.html |
Look at the new hello2.md file:
| | $ cat hello2.md |
| | Testing |
| | ======= |
| | |
| | This is an HTML document. |
| | |
| | - It has a list. |
| | - With [a link](http://google.com). |
| | - And some *emphasized* text! |
| | |
| | And a subheading. |
| | ----------------- |
| | |
| | And there's even some extra <span class="keyword">markup.</span> |
pandoc converted most of the HTML elements, but it left the inline markup alone. pandoc tries to make intelligent choices on how to handle the document. It is not always perfect, but it does keep showing improvement with each new version.
If you leave off the output filename, pandoc sends its output to STDOUT, which means you can pipe it to another program to transform it further. It also will take input from STDIN, meaning you can use it in the middle of a pipeline:
| | $ echo "# hello" | pandoc -t html | sed s/hello/Goodbye/g |
| | <h1 id="Goodbye">Goodbye</h1> |
Try it out. Using ls, awk, and pandoc, transform a directory listing into an HTML list:
| | $ ls -1 | awk '{print "* " $0};' | pandoc -t html |
In this example, you use ls -1 to get a listing of files, one per line. You then use awk to print out the line with an asterisk in front of the filename, which creates a bulleted list in Markdown.
Then, you send the output to pandoc, transforming it into an HTML list.
pandoc can also convert documents to PDFs if you install the necessary components. That’s something for you to look into on your own, though.
Running Tasks When Files Change
The entr command[28] can watch files for changes and execute any command you need. There are plenty of other utilities available, but entr is one of the most flexible options, and it requires no configuration to use.
Install entr using the package manager. On Ubuntu, run this:
| | $ sudo apt install entr |
On macOS, run this:
| | $ brew install entr |
entr takes a list of files from STDIN and executes the given command when the files change. Let’s use entr to watch a Markdown document for changes and use pandoc to convert it to an HTML file.
First, create a Markdown document to watch called watchme.md:
| | # Hello world |
| | |
| | This is a paragraph |
| | |
| | ## This is a a heading |
| | |
| | This is [a link](http://google.com). |
| | |
| | ### This is a third heading |
entr accepts a list of files as input. The easiest way to provide that input is to pipe the results of an ls command to entr. After that, you can tell entr what to do with those files. Execute the following command to tell entr to create the file watchme.html from the watchme.md file any time watchme.md changes:
| | $ ls watchme.md | entr pandoc -t html -f markdown -o watchme.html /_ |
The /_ at the end is a special entr argument that refers to the first file that changed. This way, you don’t have to specify the watchme.md filename to pass it to pandoc.
The command runs the pandoc command immediately, creating the watchme.html output file. It then watches watchme.md for any changes.
This command takes over your terminal session until you stop it with Ctrl+c, which means you can’t do anything else right now. Go ahead and stop it now so you can verify it did create the output file.
Back at the prompt, use the ls command to see if the watchme.html file exists:
| | $ ls -l watchme.html |
| | -rw-r--r-- 1 brian brian 236 Mar 3 22:02 watchme.html |
It’s there, so view it with cat:
| | $ cat watchme.html |
| | <h1 id="hello-world">Hello world</h1> |
| | <p>This is a paragraph</p> |
| | <h2 id="this-is-a-a-heading">This is a a heading</h2> |
| | <p>This is <a href="http://google.com">a link</a>.</p> |
| | <h3 id="this-is-a-third-heading">This is a third heading</h3> |
Now, start up entr again so it’s watching the file for changes. To keep it from taking over your terminal, run it as a background task in a subshell:
| | $ (ls watchme.md | entr pandoc -t html -f markdown -o watchme.html /_ &) |
With entr running in the background, add a blank line and a new paragraph to the watchme.md with echo. Use the -e flag so echo interprets newline characters:
| | $ echo -e " This is a new paragraph." >> watchme.md |
This change triggers entr to regenerate the watchme.html file. Use cat to view the watchme.html file. It now has a new paragraph at the end:
| | $ cat watchme.html |
| | <h1 id="hello-world">Hello world</h1> |
| | <p>This is a paragraph</p> |
| | <h2 id="this-is-a-a-heading">This is a a heading</h2> |
| | <p>This is <a href="http://google.com">a link</a>.</p> |
| | <h3 id="this-is-a-third-heading">This is a third heading</h3> |
| » | <p>This is a new paragraph.</p> |
entr will continue to watch this file for changes until you stop it. Locate the entr process and send it a SIGINT signal, which is equivalent to pressing Ctrl+c. You can use ps -ef | grep entr to find the process, and kill -SIGINT followed by the process ID, or you can use pkill if your system supports it:
| | $ pkill -SIGINT entr |
If managing entr as a background process like this seems more complicated, you can always use multiple terminal sessions, or leverage screen like you did in Keeping Commands Running with GNU Screen.
You can use entr to transform documents, reload your web browser when files change, watch entire directories, or even relaunch a process. Use the man page for entr to learn more.
Next, you’ll explore tools you can use to manage environment variables, manage processes, and move around your environment faster.