One must differentiate between "updating" a system and "upgrading" a system. In general, most of the NAV development work we will do is modifying individual NAV systems to provide tailored functions for end-user firms. Some of these modifications will be created by developers as part of an initial system configuration and implementation before the NAV system is in production use. Other such modifications will be targeted at a system that is in day to day production, to bring the system up to date with changes in business process or external requirements. We'll refer to these system changes as "Updating".
"Upgrading" is when we implement a new version of the base C/AL application code distributed by Microsoft and port all the previously existing modifications into that new version. First we'll discuss updating, and then we'll discuss upgrading.
Any time we are updating a production system by applying modifications to it, a considerable amount of care is required. Many of the disciplines that should be followed in such an instance are the same for a NAV system as for any other production application system. But some disciplines are specific to NAV and the C/SIDE environment.
Increasing the importance of designing for ease of updating is Microsoft's process of providing NAV updates on a frequent basis so that systems can be kept more up to date with fixes and minor feature enhancements. Keeping up with these Microsoft provided updates is especially important for multi-tenant systems running in the cloud (that is, systems serving multiple unrelated customers with the software and databases being resident on Internet based server systems). Fortunately, in support of the pressure to apply updates more frequently, Microsoft has also provided a set of tools to help us. Many of these tools are based on Windows Powershell scripts, also referred to as cmdlets. For additional information, refer to the Help topics on Deployment and Upgrading.
Even though there are new tools to help us update our NAV systems, we should still follow good practices in our modification designs and the processes of applying updates. Some of these recommendations may seem obvious. That would be a measure of our personal store of experience and our own common sense. Even so, it is surprising the number of projects that go sour because one (or many) of the following are not considered in the process of developing modifications:
- One modification at a time
- Design thoroughly before coding
- Design the testing in parallel with the modification
- Use the C/AL Testability feature extensively
- Multi-stage testing:
- Cronus for individual objects
- Special test database for functional tests
- Copy of production database for final testing as appropriate
- Setups and implementation
- Testing full features:
- User interface tests
- System load tests
- User Training
- Document and deliver in a predefined, organized manner
- Follow up, wrap up, and move on
It is important to make changes to objects in a very well organized and tightly controlled manner. In most situations, only one developer at a time will make changes to an object. If an object needs to be changed for multiple purposes, the first set of changes should be fully tested (at least through development testing stage) before the object is released to be modified for a second purpose.
If the project is so large and complex or deadlines are so tight that this "one modification at a time" approach is not feasible, we should consider using a software development version control system. Because most version control systems don't interface smoothly with C/SIDE, some significant effort will be required. One notable exception is the iFacto ReVision Source Code Control system, specifically designed to work with Dynamics NAV (see http://www.ifacto.be/en/solutions/revision).
Similarly, we should only be working on one functional change at a time. As developers, we might be working on changes in two different systems in parallel, but we shouldn't be working on multiple changes in a single system simultaneously. It's challenging enough to keep all the aspects of a single modification to a system under control without having incomplete pieces of several tasks all floating around in the same system.
If multiple changes need to be made simultaneously to a single system, one approach is to assign multiple developers, each with their own components to address. Another approach is for each developer to work on their own copy of the development database, with a project librarian assigned to resolve overlapping updates. We should learn from the past. In mainframe development environments, having multiple developers working on the same system at the same time was common. Coordination issues were addressed as a standard part of the project management process. Applicable techniques are well-documented in professional literature. Similar solutions still apply.
As we all know, there is no substitute for complete and thorough testing. Fortunately, NAV provides some very useful tools, such as those previously discussed earlier in this chapter in the sections C/SIDE Test-driven development and Debugging in NAV 2015, to help us to be more efficient than we might be in some other environment. In addition to the built-in testing tools, there are also some testing techniques that are NAV-specific.
If modifications are not tied to previous modifications and specific customer data, then we may be able to use the Cronus database as a test platform. This works well when our target is a database that is not heavily modified in the area on which we are currently working. As the Cronus database is small, we will not get lost in large data volumes. Most of the master tables in Cronus are populated, so we don't have to create and populate this information. Setups are done and generally contain reasonably generic information.
If we are operating with an unmodified version of Cronus, we have the advantage that our test is not affected by other pre-existing modifications. The disadvantage, of course, is that we are not testing in a wholly realistic situation. Because the data volume in Cronus is so small, we are not likely to detect a potential performance problem.
Even when our modification is targeted at a highly modified system where those other modifications will affect what we are doing, it's often useful to test a version of our modification initially in Cronus. This may allow us to determine if our change has internal integrity before we move on to testing in the context of the fully modified copy of the production system.
If the target database for our modifications is an active customer database, then there is no substitute for doing complete and final testing in a copy of the production database using a copy of the customer's license. This way, we will be testing the compatibility of our work with the production setup, the full set of existing modifications, and of course, live data content and volumes. The only way to get a good feeling for possible performance issues is to test in a recent copy of the production database.
While it is always a good idea to thoroughly test before adding our changes to the production system, sometimes we can safely do our testing inside the production environment. If the modifications consist of functions that do not change any data and can be tested without affecting any ongoing production activity, it may be feasible to test within the production system.
Examples of modifications that could be tested in the live production system can range from a simple inquiry page or a new analysis report or export of data that is to be processed outside the system to a completely new subsystem that does not change any existing data. There are also situations where the only changes to the existing system are the addition of fields to existing tables. In such a case, we may be able to test just a part of the modification outside the production, and then implement the table changes to complete the rest of the testing in the context of the production system.
Finally, we can use the Testing functions to control tests so that any changes to the database are rolled back at the conclusion of the testing. This approach allows for testing inside a production database with less fear of corrupting live data.
From a testing point of view, the most realistic testing environment is a current copy of the actual production database. There are often apparently good excuses about why it is just too difficult to test using a copy of the actual production database.
Remember, when we implement our modifications, they are going to receive the "test by fire" in the environment of production. We need to do everything within reason to assure success. Let's review some of the potential problems involved in testing with a copy of the production database and how to cope with them:
- It's too big—this is not a good argument relative to disk space. Disk space is so inexpensive that we can almost always afford plenty of disk space for testing. We should also make every possible useful intermediate stage backup. Staying organized and making lots of backups may be time consuming, but done well and done correctly, it is less expensive to restore from a backup than recovering from being disorganized or having to redo a major testing process. This is one of the many places where appropriate use of the C/AL Testability tools can be very helpful by allowing various approaches to repetitive testing.
- It's too big—this is a meaningful argument if we are doing file processing of some of the larger files (for example, Item Ledger, Value Entry, and so on). But NAV's filtering capabilities are so strong that we should relatively easy to carve out manageable size test data groups with which to work.
- There's no data that's useful—this might be true. But it would be just as true for a test database unless it were created expressly for this set of tests. By definition, whatever data is in a copy of the production database is what we will encounter when we eventually implement the enhancements on which we are working. If we build useful test data within the context of a copy of the production database, our tests will be much more realistic and, therefore, of better quality. In addition, the act of building workable test data will help to define what will be needed to set up the production system to utilize the new enhancements.
- Production data will get in the way—may be true. If this is especially true, then perhaps the database must be preprocessed in some way to begin testing, or testing must begin with some other database such as Cronus or a special testing-only mockup. As stated earlier, all the issues that exist in the production database must be dealt with when we put the enhancements into production. Therefore, we should test in that environment. Overcoming such challenges will prepare us to do a better job at the critical time of going live with the newly modified objects.
- We need to test repeatedly from the same baseline. or We must do regression testing—both are good points, but don't have much to do with the type of database we're using for the testing. Both the cases are addressed by properly managing the setup of our test data and keeping incremental backups of our pre-test and post-test data at every step of the way. SQL Server tools can assist in this effort. In addition, the C/AL Testability Tools are explicitly designed to support regression testing.
Remember, doing the testing job well is much less expensive than implementing a buggy modification and repairing the problems during production.
As experienced developers, we will already be familiar with good testing practices. Even so, it never hurts to be reminded about some of the more critical habits to maintain.
Any modification greater than trivial should be tested in one way or another by at least two people. The people assigned should not be a part of the team who created the design or code of the modification. It would be best if one of the testers is an experienced user because users seem to have a knack (for obvious reasons) of understanding how the modification operates compared to how the rest of the system acts in the course of day-to-day work. This helps us to obtain meaningful feedback on the user interface before going into production.
One of the testing goals is to supply unexpected data and make sure that the modification can deal with it properly. Unfortunately, those who were involved in creating the design will have a very difficult time being creative in supplying the unexpected. Users often enter data that the designer or programmer didn't expect. For that reason, testing by experienced users is beneficial.
The C/AL Testability Tools provide features to support testing how system functions deal with problem data. If possible, it would be good to have the users' help to define test data, and then use the Testability Tools to ensure that the modifications properly handle the data.
After we cover the mainstream issues (whatever it is that the modification is intended to accomplish), we need to make sure that our testing covers all boundary conditions. Boundary conditions are the data items that are exactly equal to the maximum, minimum, or other range limit. More specifically, boundaries are the points at which input data values change from valid to invalid. Boundary condition checking in the code is where programmer logic often goes astray. Testing at these points is very effective for uncovering data-related errors.
Create useful documentation and keep good records of testing processes and results. Testing scripts, both human-oriented and C/AL Testability Tool-based, should be retained for future reference. Document the purpose of the modifications from a business point of view. Add a brief, but complete, technical explanation of what must be done from a functional design and coding point of view to accomplish the business purpose. Record briefly the testing that was done. The scope of the record keeping should be directly proportional to the business value of the modification being made and the potential cost of not having good records. All such investments are a form of insurance and preventative medicine. We hope they won't be needed but we have to allow for the possibility that they may be needed.
More complex modifications will be delivered and installed by experienced implementers, maybe even by the developers themselves. Small NAV modifications may be transmitted electronically to the customer site for installation by a skilled super-user. Whenever this is done, all the proper and normal actions must occur, including those actions regarding backup before importing changes, user instruction (preferably written) on what to expect from the change, and written instruction on how to correctly apply the change. There must also be a plan and a clearly defined process for restoring the system to its state prior to the change, in case the modification doesn't work correctly. As responsible developers, whenever we supply objects for installation by others, we must make sure that we always supply .fob format files (compiled objects), not text objects. This is because the import process for text objects simply does not have the same safeguards as the import process for compiled objects.
Bring projects to conclusion, don't let them drag on through inaction and inattention—open issues get forgotten and then don't get addressed. Get it done, wrap it up, and then review what went well and what didn't go well, both for remediation and for application to future projects.
Set up ongoing support services as appropriate, and then move on to the next project. With the flexibility of the Role Tailored Client allowing page layout changes by both super users (configuration) and users (personalization), the challenge of user support has increased. The person offering support can no longer expect to know what display the user is viewing today.
Consequently, support services will almost certainly require the capability for the support person to view the user's display. Without that, it will be much more difficult, time consuming, and frustrating for the two way support personnel – user communication to take place. If it doesn't already exist, this capability will have to be added to the Partner's support organization tool set and practices. There may be communications and security issues that need to be addressed at both the support service and the user site.
The ability to upgrade a customized system is a very important feature of NAV. Most other complex business application systems are very difficult to customize at the database-structure and process-flow levels. NAV readily offers this capability. This is a significant difference between NAV and the competitive products in the market.
Complementing the ability to be customized is the ability to upgrade a customized NAV system. While not a trivial task, at least it is possible with NAV. In many other systems, the only reasonable path to an upgrade is often to discard the old version and re-implement with the new version, recreating all customizations. Not only is NAV unusually accommodating to being upgraded, but with each new version of the system, Microsoft has enhanced the power and flexibility of the tools it provides to help us do upgrades. In the Microsoft Dynamics NAV 2015 Development Shell, among other useful cmdlets, there is Merge-NAVApplicationObject cmdlet. Refer to the Developer and IT Pro Help files for details on starting and using a Development Shell session (hint: use the Search box to find information on the Development Shell).
We may say, Why should a developer care about upgrades? There are at least two good reasons we should care about upgrades. First, because our design and coding of our modifications can have a considerable impact on the amount of effort required to upgrade a system. Second, because as skilled developers doing NAV customizations, we might well be asked to be involved in an upgrade project. Since the ability to upgrade is important and because we are likely to be involved in one way or another, we will review a number of factors that relate to upgrades.
Just so we are on common ground about why upgrading is important to both the client and the NAV Partner, following is a brief list of some of the benefits that an upgrade can give:
- Easier support of a more current version
- Access to new features and capabilities
- Continued access to fixes and regulatory updates
- Improvements in speed, security, reliability, and user interface
- Assured continuation of support availability
- Compatibility with necessary infrastructure changes, such as new operating system versions
- An opportunity to do any necessary training, data cleaning, and process improvement
- An opportunity to resolve old problems, to do postponed "housekeeping", and create a known system reference point
This list is not complete and not every benefit will be realized in any one situation.
The most challenging and most important part of an upgrade is porting the code and data modifications from the older version of a system to the new version. When the new version has major design or data structure changes in an area that we have customized, it is quite possible that our modification structure will have to be re-designed and perhaps even be recoded from scratch.
On the other hand, often the changes in the new product version of NAV don't affect much existing code, at least in terms of the base logic. If our modifications are done properly, it's often not difficult to port custom code from the older version into the new version. By applying what some refer to as "low-impact coding" techniques, we can make the upgrade job easier and thereby less costly.
In the earlier chapters, we discussed some documentation practices that are good to follow when making C/AL modifications. Here is a brief list of practices that should be followed:
- Identify every project with its own unique project tag
- Use the project tag in all documentation relating to the modification
- Include a brief but complete description of the functional purpose of the modification in a related
Documentation()trigger - Include a description of the modifications to each object in the
Documentation()trigger of that object, including changes to properties, Global and Local variables, functions, and so on - Add the project tag to the version code of all modified objects
- As much as possible, make all code self-documenting, using meaningful names for all data elements and functions, and breaking code segments into logical functions so that process flow is self-evident
- Bracket all C/AL code changes with inline comments so that they can be easily identified
- Retain all replaced code within comments, using
//or{} - Identify all new table fields with the project tag
We have already discussed most of these practices in other chapters, but we will review them here in the context of coding to make it easier to upgrade. We won't be able to follow each of the following listed options, but we will have to choose the degree to which we can implement low-impact code and which options to choose:
- Separate and isolate new code
- Create functions for significant amounts of new code that can be accessed using single code line function calls
- Either add independent Codeunits as repositories of modification functions or, if that is overkill, place the modification functions within the modified objects
- Add new data fields; don't change the usage of existing fields
- When the functionality is new, add new tables rather than modifying existing tables
- For minor changes, modify the existing pages, otherwise copy and change the clone pages
- Copy, then modify the copies of reports and XMLports, rather than modifying the original versions in place
- Don't change field names in objects, just change captions and labels as necessary
In any modification, we will have conflicting priorities regarding doing today's job in the easiest and least expensive way versus doing the best we can do to plan for future maintenance, enhancements, updates, and upgrades. The right decision is never a black and white choice, but must be guided by subjective guidelines as to which choice is really in the customer's best interest.