Tools and utilities

Throughout the book we've been using DMD to compile examples and DUB to manage the MovieMan project. Now it's time to look at some additional tools that can be part of a productive D workflow. The first tool on the list, though, is actually DMD. We're going to take a look at some of the compiler options that can be helpful during the development of D programs.

DMD

Thus far, we haven't used many options when compiling D programs, but there are quite a few of them. As demonstrated in Chapter 1, How to Get a D in Programming, the list of compiler options can be displayed by invoking DMD with no command-line options. Each option is accompanied by a brief description of what it does. Here are a few of those that you may find most useful.

Optimized and debug builds

There are a few DMD options that control optimizations and debugging. The -g switch adds debugging information in a D-specific format. For debuggers that don't support D debug symbols, -gc can be used to make the compiler use C-style debug output. The -O switch turns on optimizations; -inline will activate function inlining; -release will turn off contracts and asserts, cause assertion failures to be classified as undefined behavior, and disable array bounds checking in functions not annotated with @safe (more on that in Chapter 11, Taking D to the Next Level). Additionally, the –boundscheck= switch can be used to have a little more control over array bounds checking. It takes one of three possible arguments: on, safeonly, and off. The first is the default in the absence of –release, the second the default with it.

Changing the default output

The default behavior of DMD is to name the binary output the same as the first input source or object file it is given, and to write it in the current working directory, using the appropriate platform-specific file extension. The output directory for object files and library files can be changed with the -od command-line switch. The name of the target library or executable file can be changed with the –of switch. The argument here can be a file name or a path. For example, the following line writes main.obj to the out/obj subdirectory and creates an executable named app (or app.exe on Windows) in a subdirectory named bin. The directories out, out/obj and bin will be created if they do not exist.

dmd –odout/obj –ofbin/app main.d

Compiling libraries

Creating a library requires compiling several source files into object files, then handing the object files off to a tool that packs them all into the target library. DMD allows you to condense these separate steps into a single command line with the –lib switch. With this option, any source files, object files, or library files fed to the compiler will all be combined into a single library. Let's experiment.

In $LEARNINGD/Chapter08/libs, create two modules, hello.d and goodbye.d. The former should look like this:

module hello;
void sayHello() {
	import std.stdio : writeln;
	writeln("Hello");
}

And the latter should look this:

module goodbye;
void sayGoodbye() {
	import std.stdio : writeln("Goodbye!");
}

First, let's create two separate libraries, named hello and goodbye with the following two command lines:

dmd -lib -odseparate hello.d
dmd -lib -odseparate goodbye.d

This will create two libraries (hello.lib and goodbye.lib on Windows, libhello.a and libgoodbye.a elsewhere) in a subdirectory named separate. Let's say we want to combine them into a single library. There are multiple ways to go about this. If all we have is the source code, we would do this:

dmd -lib -odcombined -ofgreets hello.d goodbye.d

Now we have greets.lib or libgreets.a in the combined subdirectory. What if we have the two libraries and no source?

dmd -lib -odcombined -ofgreets separate/hello.lib separate/goodbye.lib

Same result. You could also pass, for example, hello.d and goodbye.lib, or hello.lib and goodbye.d, or compile the source modules into object files and pass those to the compiler instead. Each case will have the same result.

Tip

Dropping extensions

When passing files to DMD, pay attention to the file extensions. If no extension is specified, then the compiler will treat it as a source file. Forgetting to add the proper extension for a library or object file will not generate an error if a source module exists with the same path and filename.

Using libraries

Using a library with DMD can be as easy as making one, but there are some potential pain points. For the simplest cases, you can pass a library directly on the command line along with one or more source files, object files, or other libraries. As long as a main function exists somewhere in the mix, the compiler will make sure everything is passed to the linker to generate the executable. For example, given a module main.d in $LEARNINGD/Chapter08/libs, which looks like this:

void main() {
	import hello, goodbye;
	sayHello();
	sayGoodbye();
}

As we saw in Chapter 4, Running Code at Compile Time, we can add a lib pragma to the top of main.d and forgo the need to pass the library on the command line at all, but if we do decide to use the command line, there are some differences between Windows and other platforms.

On Windows, we can compile the preceding code and link with combined/greets.lib like so:

dmd main.d combined/greets.lib

This command line hides the linker flags from the user. Whether we're using the Digital Mars linker or the Microsoft linker, the compiler will do what needs to be done to link the binary. DMD allows flags to be passed directly to the linker with the –L switch. On Linux, Mac, and the BSDs, this is used to specify any libraries that should be linked with. On those platforms, the GNU linker expects libraries to be fed to it with the –l (lowercase L) switch. For example, given libgreets.a in the same directory as main.d, we would link with it like so:

dmd –L-lgreets main.d

The –L tells the compiler that what immediately follows is not for the compiler itself, but for the linker. So, now we have two different command lines for linking with libraries, one for Windows and one for other platforms. If we want to pass a common directory in which to find libraries, it gets a bit messier.

Whether we're passing libraries on the command line or using lib pragmas, when we are using multiple libraries from the same path, it can help keep things neat and concise if we tell the linker to look for the libraries in a specific directory. Moreover, due to the nature of the linker on Linux and other platforms, you typically don't want to be passing a full library name (such as libhello.a) to the linker. This is where the pain starts to set in if you are using a make file or custom build script to compile on multiple platforms, or on Windows with multiple compilers or architectures.

To specify a common directory, we have to explicitly pass the appropriate flag to the linker via –L, but the DM linker and the MS linker take different options. Not only that, if you throw in other platforms, you've got a third option to deal with. In short, telling the linker to search for libraries in the separate subdirectory and link with both the hello and goodbye libraries looks like the following with the different linkers:

  • With the Digital Mars linker:
    dmd -L+separate main.d hello.lib goodbye.lib
    
  • With the Microsoft linker:
    dmd -m64 -L/LIBPATH:separate main.d hello.lib goodbye.lib
    
  • With the GNU linker:
    dmd –L-Lseparate –L-lgoodbye –L-lhello main.d
    

This is where something like DUB really comes in handy. No matter which compiler, which platform, or which linker, it abstracts away all these differences so that you never have to worry about them.

Finally, it's worth noting that DMD looks for the LIB environment variable when it's time to link. If you have any special paths where you like to store libraries, you can add that path to LIB, either on the command line, via a shell script, or in the DMD configuration file.

Warnings

Warnings in DMD are not enabled by default. There are two ways to turn them on. With the –w switch, warnings will be treated as errors and cause compilation to halt. The –wi switch will allow compilation to continue, printing each warning to stderr as they are encountered. Whichever approach you prefer, it's a good idea to compile your project with these once in a while, if not all the time, to try and squelch any warnings that may arise.

Profiling

There are a number of profiling tools out there that may be usable with D when working with the GCC or Microsoft toolchains, but DMD ships with an easy-to-use option that allows you to get an idea of where your D program is spending its time. It's a simple command-line switch, -profile, which results in a binary that generates profiling information on execution.

In the $LEARNINGD/Chapter08/profile directory, you'll find a source file, main.d:

uint calc(uint x, uint y) {
    return x + y;
}
uint select() {
    import std.random : uniform;
    return uniform(1u, 1001u);
}
void main() {
    import std.stdio : writeln;

    for(size_t i=0; i<100; ++i) {
        writeln("Starting calculations...");
        uint result;
        for(size_t j=0; j<20; ++j) 
            result += calc(select(), select());
        writeln("The accumulated result: ", result);
    }
}

Compile this file with the following command:

dmd -profile main.d

Executing the resulting binary will produce two new files, trace.def and trace.log. The former is intended to be passed to the linker, telling it the optimal order in which to link the program's functions. The latter has two sections that display information about the program's call tree and the function timings.

The first section appears to be a jumbled mess of mangled function names and numbers, but there is a method to the madness: it's a call tree. Each entry is separated by dashed lines, with the first function called in the program (the main function) listed at the very bottom of the tree. Scroll down until you see the last few sections that look like this:

------------------
	 4000	_Dmain
_D4main6selectFZk	4000	2736	471
	 4000	_D3std6random27__T7uniformVAyaa2_5b29TkTkZ7uniformFNfkkZk
------------------
	    1	main
_Dmain	1	119380	464
	  100	_D3std5stdio16__T7writelnTAyaZ7writelnFNfAyaZv
	 4000	_D4main6selectFZk
	 2000	_D4main4calcFiiZk
	  100	_D3std5stdio18__T7writelnTAyaTkZ7writelnFNfAyakZv
------------------
main	0	0	0
	    1	_Dmain

The entry at the bottom tells us that the main function (which is actually internal to DRuntime) calls _Dmain exactly one time (_Dmain is the name by which DRuntime knows our main function in main.d). Moving up to the next entry, we see 1 main in the first line. This tells us this entry is for a function that's called one time from main. Next we have the name of that function (_Dmain), followed by all of the functions that it calls and how many times it calls each of them. Next to _Dmain we see three numbers, 1, 119380 and 464. The first is the number of times it is called, in this case by main, the next two are timings in ticks , or the number of times the timer has incremented. The first timing is the total number of ticks taken by _Dmain, including the tick count of the functions it calls (call tree time).

The second is the tick count of _Dmain minus that of the functions it calls (function call time). Moving up, we see an entry for one of the functions that _Dmain calls 4000 times, select, along with the functions it calls. Select has tick counts of 2736 and 471. Every function call in the program will have an entry in the same format as those shown here.

The second section, just below the bottom entry in the call tree, starts off like this:

======== Timer Is 3320439 Ticks/Sec, Times are in Microsecs ========

This is arguably the part of the file you'll usually be most interested in. It is a table listing the number of calls and timings for each function. The headers of the table are, as shown:

Num          Tree        Func        Per
Calls        Time        Time        Call

Tree Time is the total amount of time spent inside the function, including any function calls it makes. Func Time is the total amount of time spent in the function minus any function calls it makes. Per Call is Func Time / Num Calls. Following the headers is an entry for every function, most of which are listed in their demangled forms. The entry for the select function looks like this:

4000         823         141           0     uint main.select()

A total of 4000 calls, a total time of 823 microseconds, a total function time of 141 microseconds, and a per call time of 0 (meaning less than one microsecond per call).

Code coverage analysis

Code coverage analysis is useful to find dead code in a program, to make sure that all code paths intended to be taken actually are and, when used in conjunction with unit testing, to ensure all of the code is tested. Essentially, it means analyzing the run time of a program to determine what percentage of the total number of lines of code is actually touched. DMD has built-in support for code coverage analysis via the –cov switch.

Save the following code snippet as $LEARNINGD/Chapter08/cov/main.d:

import std.stdio;
void dontCallMe() {
    writeln("Not covered.");
    writeln("Me neither");
}
void callMe() {
    writeln("Covered.");
}
void main() {
    callMe();
    callMe();
}

Compile it with the following command line:

dmd -cov main.d

Execute the resulting binary and a new file, main.lst, will be created in the same directory. It contains the same source code found in main.d, but annotated with the number of times each line of code is executed. At the end of the file, it lists the total percentage of code coverage. In our case, with five total lines of code two of which are never executed, we have achieved only 60% coverage.

       |import std.stdio;
       |
       |void dontCallMe() {
0000000|    writeln("Not covered.");
0000000|    writeln("Me neither");
       |}
       |
       |void callMe() {
      2|    writeln("Covered.");
       |}
       |
       |void main() {
      1|    callMe();
      1|    callMe();
       |}
main.d is 60% covered

The –cov switch takes an optional argument specifying a target coverage percentage. If the target is not met, an error will be thrown at the end of execution, letting you know before you ever look at the output file if your target is met. For example, execute the following in the same directory:

dmd -cov=100 main.d

This yields a binary, whose output is this:

Covered.
Covered.
Error: main.d is 60% covered, less than required 100%

Compile and run

When compiling an executable with the –run switch, DMD will launch the binary as soon as it has been compiled and linked. Any files generated during the process are temporary and will be cleaned up once execution has completed. The syntax is as follows:

dmd <options> -run <srcfile> <args>

Here, <options> is any number of DMD options, including source files. However, one source file must follow the –run switch in <srcfile>. Any arguments passed in <args> will be treated as command-line arguments for the executed program.

GDC and LDC

The GDC and LDC D compilers were mentioned briefly in Chapter 1, How to Get a D in Programming. Here, you'll learn more about what they are and where to get them.

GDC

The GNU D Compiler is a community-driven, GPL implementation that integrates the D front end with the GNU Compiler Collection. This integration opens the door to compiling for platforms and targets not officially supported by DMD through the use of the existing GCC toolchain to generate output. It's available through the package managers of several systems, but the latest can always be found at http://gdcproject.org/downloads.

At the time of writing, the Linux versions of GDC are well-tested and suitable for production code. They are often used to produce the final release version of software due to GDC's ability to better optimize than DMD. The Windows versions require installation of the w64-mingw32 compiler, but as I write this, they are considered alpha-quality and not quite ready for production.

Most, if not all, of the supported DMD command-line options have GCC-style equivalents supported by GDC (the common GCC options are also supported). Code generated by GDC can be debugged with GDB. Iain Buclaw, the primary GDC maintainer, is also the official maintainer of D support in GDB. It is expected that GDC will one day become part of the GCC distribution. The source for GDC can be found at https://github.com/D-Programming-GDC/GDC. There is also a GDC-specific forum at http://forum.dlang.org/group/gdc.

LDC

LDC is an open-source, community-driven compiler that integrates the D frontend with the LLVM core libraries. It is available for several different platforms supported by LLVM. Where it can't be obtained through a package manager, binaries of the latest version can always be downloaded at https://github.com/ldc-developers/ldc/releases and the source is available at https://github.com/ldc-developers/ldc. At the time of writing, Windows binaries require the Visual Studio runtime, though past releases have also supported w64-mingw32.

In addition to LLVM-related command-line options, LDC has equivalents for many of the options supported by DMD. Output on most platforms can be debugged with GDB and with Visual Studio on Windows. Like GDC, LDC is also better at optimizing D code than DMD, so it's another alternative used for release versions of software. The LDC forum at http://forum.dlang.org/group/ldc is a good source of help.

RDMD

When compiling with DMD or any other D compiler, every module intended to be part of the final binary must be passed on the command line, either in the form of source code for compilation or as object or library files for linking. DUB eliminates that requirement by managing everything itself. DUB will compile all of the modules in a project's source directory into the output binary, though a directive can be added to the project configuration to exclude one or more files.

RDMD is a build tool that works a bit differently. First off, it has no concept of a project. All it cares about are source files, no matter where they may be located. Given a single source file, RDMD will determine all of the modules it imports, all of the modules imported by those modules, and so on, and pass all imported modules on to the compiler. In other words, if a module is not imported anywhere in a program, it is not compiled into the final binary. Once compilation is complete, RDMD will launch the newly created executable. All generated files are cached in a temporary directory so that each invocation of rdmd will only compile new or changed source files.

RDMD recognizes a handful of command-line options. Options not recognized by RDMD will be passed on to the compiler. For a list of supported command line-options, execute rdmd with no arguments. An important one is --build-only, which saves the compiled executable and does not run it. It also accepts a switch, --eval, which takes a string of code as an argument to compile and execute. Try this:

rdmd --eval="import std.stdio; writeln(`Hello World`);"

Another very useful option on systems that understand shebang lines (#!) in text files is the --shebang switch. This allows RDMD to be used to execute a D source file as if it were a script file, for example, ./myscript.d. Such scripts require the very first line in the file to be a shebang line that contains the command line for the program that should be invoked to execute the script. RDMD requires --shebang to be the first option in the command-line string. For example:

#!/usr/bin/rdmd --shebang -release -O
import std.stdio;
void main() {
    writeln("I'm a D script!");
}

The same can be done with DMD using the –run switch, but since RDMD will automatically compile all imported modules and only those that need compiling, it's the preferable choice. RDMD ships with DMD.

DustMite

You're hacking away at your keyboard one afternoon when you decide to take a quick break and check the D forums. You see an announcement for a new release of DMD, so you decide to upgrade. A short while later, you're neck-deep in D code again and it's time to compile. You pull up a command prompt, invoke DMD, and find yourself staring at an ICE (Internal Compiler Error). Congratulations! You've just discovered a compiler bug, potentially a regression introduced by the new release.

The thing to do in this situation is to head over to https://issues.dlang.org/ and file a bug report, but in order to do that you need to be able to provide a minimal test case that someone else can compile to reproduce the problem. Your program consists of dozens of modules and thousands of lines of code; you obviously can't upload it all. You need to reduce it somehow, but this ICE is the sort of error that makes it almost impossible to whittle things down manually. This is where DustMite comes in.

DustMite, which ships with DMD, takes two arguments: a path to a source directory and a string that serves as a test for a specific error. The string should contain one or more shell commands that have a return code of 0 to indicate that an error persists. The source directory should be cleaned of all files except D source modules and DustMite should be executed in the parent directory. With this information, DustMite will enter a cycle of removing portions of the source code and recompiling, looking for the minimal set of code that matches the command string it was given.

As an example, given a source file main.d in a clean directory named projects/myproject, we can navigate to the projects directory and execute the following command to get a minimal test case for an internal compiler error. The reduced source tree will be output to the projects/myproject.reduced directory.

dustmite myproject 'dmd main.d 2>&1 | grep -qF "Internal error"'

Linux users will be instantly familiar with the test string. First, it compiles main.d with DMD and redirects the output of stderr to stdout. The pipe then executes grep, using the output of stdout as its input. grep will search for "Internal error" and return 0 if it is found. The q in –qF tells grep not to print anything to stdout and the F tells it that its input is a newline-separated list of strings.

Tip

DustMite and Windows

grep is a utility that is not installed on Windows by default. There is a similar tool, called findstr, but I've been unable to get it to work with DustMite. Windows users might choose to use a custom script (perhaps written in D) or batch file, but it's probably more convenient to install a Windows version of grep and either execute DustMite in an MSYS or Cygwin command shell or, when using DUB to manage a project, simply invoke DustMite through DUB.

When you are using DUB to manage your project, you can also use it to call DustMite for you. Pass it the dustmite command, an output directory, and a string to use as a regex against which to test the compiler output. Here's an example:

dub dustmite ./output --compiler-regex="Internal error"

You can read more about DustMite and learn other ways to use it at https://github.com/CyberShadow/DustMite/wiki. For more about using DustMite through DUB, execute the following command:

dub dustmite –help

DCD

The D Completion Daemon is a client-server program that provides an auto-completion service for text editors and IDEs that support D. Many of the editors and IDEs listed earlier in the chapter are either configured to use DCD out of the box or have plugins available that support it. Essentially, the DCD server runs continuously in the background. The editor uses the client to communicate with the server, sending it all the information it needs to determine auto-completion possibilities, or suggestions for function calls, which the server then sends back to the client for display to the user. You can read more about DCD and download the source of the latest release at https://github.com/Hackerpilot/DCD.

DVM

When installing DMD with the Windows installer or one of the Linux packages, the compiler is automatically added to the system path for convenience (it's optional with the Windows installer). However, it's sometimes necessary to install multiple versions of DMD. Perhaps you want to test the latest beta, or you want to compile an older project with an older version of DMD and still use a newer version for your new projects. There are different ways to handle this on different systems, but they require varying degrees of effort in getting set up every time you want to add a different version of the compiler to your system. This is the problem DVM was created to solve.

When you download a DVM binary or build one yourself, you must first tell DVM to install itself like so:

dvm install dvm

This will cause DVM to do whatever it needs to ensure that your system is ready to start handling multiple versions of DMD. From that point, getting a new version of DMD is as simple as this:

dvm install 2.068.2

This will install version 2.068.2 of DMD. Nothing is added to the path at this point. You can install more versions of DMD as needed. Then, when you're ready to use version 2.068.2, open up a command prompt and type the following:

dvm use 2.067.1

You can find DVM at https://github.com/jacob-carlborg/dvm.

Tip

Downloading the DVM binary on Windows

The link to the DVM GitHub page will, if you choose not to build from source, ultimately lead you to the page where all DVM releases are listed. There, you will find a few binaries and a couple of source archives available for download. The Windows binary is named in a format similar to this: dvm-0.4.3-win.exe. This looks like it could be an installer, but it is not. When running the install command, be sure to use the actual name of the binary in place of dvm. Alternatively, save the downloaded file as dvm.exe.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.189.189.67