A typical DevOps pipeline finishes as soon as the package has been deployed to a secure environment. DevSecOps introduces extra steps to your pipeline to verify and support the compliance of your product in international markets. The topic of compliance takes more than just a license into account. International markets introduce their own set of legal requirements for a solution provider. European countries, for example, have GDPR (the General Data Protection Regulation). This requires the solution vendors and ISVs to apply a set of rules across their organization (changes such as recruiting a Data Protection Officer) as well as the solution (such as user “consent” for data collection and applying a data removal policy). This compliance rule not only applies to solutions being used from within Europe, but also to the solutions that provide services to Europeans, even from outside Europe.
A similar U.S. compliance regulation was formed and applied on January 1, 2020, called the California Consumer Privacy Act (CCPA). This shows that the world is now moving toward an open and more transparent cyberspace. You are responsible not only for protecting your solutions, but for protecting the data being generated in your system by your customers. They should have access to the “delete” buttons for their data or be able to download their archive completely if they ever wish to.
For software applications, these licensing requirements introduce several hurdles. Such as when an enterprise can have access to the software’s source code. Open source licenses1 have different flavors and permissions that can be granted. I personally chose to use the MIT License for most of my projects. It grants freedom to the users upon giving credit to you, and it grants freedom to use it for commercial use without bringing any liability to you. If you are building software that depends on open source packages and frameworks, you are required to credit the authors or packages in your software. Most licenses require you to do this, and legal actions can be taken against your organization or product if you do not meet these requirements.
To provide a better user experience, organizations also integrate customer feedback forms and icons into their products. This makes it easier for customers to communicate their ideas, suggestions, and problems with the development teams. You protect your software from licensing related issues, data privacy and protection rules, and intellectual property and user experience related concerns. Enterprises tend to avoid these legal problems to protect their brand name and maintain trust with their user base.
Topics mentioned and discussed in this chapter are for a regular customer with no legal advice. You must consult a legal entity to discuss how your application should behave in the certain areas to comply with international rules and policies. The points I mention are just observations for a more general audience. To get personalized suggestions, speak to a lawyer about your product and its development process.
You might often need to monitor the application usage and trends in your organization. You need to understand how an application is being consumed.2 This is where your audit structures come in to play. You will typically receive this feature as a part of your hosting environment. For example, Microsoft Azure offers a complete set of audit reports for actions taken by your team members. Other cloud vendors offer a similar feature, Alibaba Cloud, GCP, AWS, and so on.
Auditing
A good audit can help you understand how your application is working and how your engineers are controlling different elements of your solution. Remember that this audit does not normally include what your users are doing. Some software applications also expose the information generated by the customers, such as when a name is updated by a customer in the database. This helps customers understand how their information is being used in the system. It will also help you provide a detailed report to customers if you want to respond to an incident.
The Microsoft Azure website showing the activity logs for the subscriptions and services that are in the subscription. The activity logs are the audit reports for every individual service. These reports only contain actions that are taken in the Azure and not in the code or user-generated data
Microsoft portal showing two actions that took place in the product and the data related to them
Figure 7-2 shows an app that has changes in it.
I just made a few changes to the application’s state and Azure shows the complete log for these changes. It shows who initiated the change and when it happened. This can help your team perform a good postmortem analysis and understand what went wrong. Without these audit reports, anyone can change the state of your applications without leaving a trace. These traces can help you refute legal arguments against your product or your company profile.
You must be thinking, why would this be applied to a .NET Core based solution? Well, it applies to all domains where .NET Core can publish code. Your mobile applications require a strict checkup against any key modification.3 If you can detect any unwanted changes early on, it is easier to find a remedy. This can protect you from any legal actions while also protecting your revenue streams.
You can add any number of custom fields4 that you need to help you with the audit reports. Entity Framework Core does not provide a built-in solution for auditing fields, which is a downside in my opinion. I have worked with Sequelize ORM for Node.js, and that provides a native experience of these columns. The framework automatically updates the records based on the changes. You can add more fields to this type, such as AddedBy, ChangedBy, and so on.
Data Privacy and Control
You should give all the control needed by your customers to perform all sorts of actions on their data. There is no “.NET Core” side to this, so just make sure that you are transparent in your approach. This requires that your delete operations delete the necessary data of a customer from the database when requested to do so. Your application must be storing the data in a backup as needed for availability reasons. You can declare these settings in your policies and terms of service to make the customer aware of any backups. Also ensure that you declare the data processing units and operators that you are using. If you are using a cloud storage provider, clearly explain that to your customers. I have experience with customers who feel secure in storing their data with one cloud vendor and not with another. You should let customers select where they want to have their data stored by providing UI controls on the interface. One example can be to ask for consent from the customers before storing any of their data. An improvement to this can be that you guide them through the privacy consents and help them understand how the data is stored, processed, and deleted if required. Your users should be able to request a complete copy of their data, or to delete the complete archive. Generally, you should keep the data in the region where the customers live. This helps you decrease the network latency as well as helps your users trust your application for being transparent.
Luckily, .NET Core is not like Blockchain technologies. All information that is stored can be downloaded, updated, and deleted. You can perform custom5 tests against your application to check if it complies properly with the regulations.
It is beyond the scope of this book to discuss scenarios of GDPR and other regulations. But I recommend that you study how to design your UX and permissions that your app demands. These are the links that I highly recommend:
https://www.privacypolicies.com/blog/gdpr-compliance-apps/
https://iapp.org/news/a/this-tech-scans-apps-for-privacy-and-security-compliance/ (This web page also discusses the Kryptowire software that tests your Android apps for compliance and regulation checks.)
Your users will tend to use your products over your competitors if they trust your applications.
DevOps Audit Defense Toolkit
DevOps Audit Defense Toolkit is a comprehensive guide6 that you can use to educate your IT and development teams to prepare them for auditing and other compliance issues. This book is trusted7 by industry experts to contain useful learning material for beginners.
You should help your team better understand what an audit means for your organization. This helps your organization build better products, because your engineers understand what they need to do. Your engineers can design better software that exposes more information, making it transparent to customers, auditors, and legal bodies. This goes from recording every action that takes place in the system, to generating a report about incidents taking place, to sending an email to your customers. Keeping your customers updated with the latest security and privacy issues on time can help you avoid heavy fines and charges against your company.
You should include third-party auditors and penetration testers to check your products. This can give you a good amount of time to plan your next steps when you are in hot water. Your legal department can prepare documented reports and your engineers can fix the problems before they harm anyone.
Automated Issue Tracking
Open source products tend to use open source DevOps solutions that are provided free of cost. GitLab, for example, allows you to use their service for private projects. If you invite external contributors, you need to provide a clear policy of how you use their data.
Never share your email address on the public platforms. If a user is interested in contacting you, they can ping you on the same thread. This prevents spammers from harvesting your emails off public websites.
Never share passwords online. Even if your solution depends on the username/password combination. A person interested in providing a solution to you will try it with his own username/password. This is a common error that new developers make. They share their own username/passwords in the code they write.
This applies to user-generated content. You can also tune your software to avoid publishing information about user’s system that can identify the user or the location. You must always get consent from the users before collecting information that can identify their system, location, or personalized data. Customers that use a Git-based issue tracking and management system should also avoid requesting information about the customer’s account details. The reason is that Git keeps a history of all the actions that have taken place. This poses a similar problem as Blockchain. You cannot8 remove or modify the information that has been checked in. When a user does not need to add their personal information, they begin to trust the platform and can more easily share their problems. For application developers, this can be important, because your customers will likely recommend your application to others if they trust it.
DevOps tools offer an API endpoint that can be used to generate the content. GitLab, BitBucket, Jira, and other products are well integrated with each other. A customer can provide feedback to your product, which then creates a new issue (in GitLab) or a new ticket (in Jira). These can then provide you with insights on how your engineering teams adopt and apply changes to the software. You can also study how your operations and security teams analyze the product before releasing an update.
GitLab gives you an option to preview the complete DevOps velocity for your team and every department
This is a new addition to GitLab (since GitLab Premium 12.3) and it shows you the average time that it takes to move an issue to production. It starts by calculating the time it takes to create an issue, to develop the solution, and to test and perform all the verifications. For DevSecOps, this also includes the compliance and security tests.
If you have followed the steps mentioned in this book, then your DevOps cycle will also include the build environment verification and production scanning. You can include the platform problems in the issue tracking to keep a history of the updates that are applied.
History maintenance can help you include the GitOps in your projects and improve the overall development experience for your teams.
IaC and other DevOps concepts improve the compliance and regulations for your software. An IaC document is easier to review from a compliance and regulation standpoint. Since your infrastructure is deployed using text files that are processed by the software, you can be certain that the infrastructure was not modified beyond control. GitOps ensures that every change that you make in the IaC files is visible across departments and can maintain a history. A Git tree is not a good legal report to be submitted to a legal body, but it can help during verification and auditing (whether internal or external).
Summary
DevOps enforces automation to reduce the friction between your departments. These departments can be from different workforces, such as development and operations. Everyone comes into a single room9 and discuss the project’s plan. The goal of DevSecOps is to introduce the rest of the teams to this process. Teams like customer support, legal, and marketing all come to the same project and collaborate. As a developer, your job is to develop the solutions. Regulations and compliance are managed by other teams that DevSecOps helps to introduce in the project room. You, as a developer, incorporate the changes they recommend.
Compliance is a trending topic in computer science, not because of its fanciness, rather because of the fines that are being imposed by countries. You will find every website showing a cookie banner on their website (some even preventing the users from accessing the website at all). As a DevOps engineer, you are responsible for running compliance tests on your solutions. DevOps tools provide you with all the tools that you require to run the tests. Remember, just because the code compiles does not mean it meets quality tests. Similarly, a QA-approved product does not always comply with all the regulations imposed in the global market. It also does not mean that your product is free from bugs, especially new ones that come up during production.
This brings me to the end of this book. We did not discuss every aspect of the development and deployment of .NET Core projects with DevSecOps, as this is not a hands-on guide or a cookbook for .NET Core or DevOps. I provided the references and off-book reading material to help you understand the concepts and get hands-on tutorials when needed.
I tried to give you the tools and practices that will help you improve your automation and CI pipelines. For a .NET Core to utilize the practices of DevOps, it is important to start from the beginning. A good DevOps pipeline integrates every stage of the software development lifecycle in the automation. Throughout the book, I tried to give you an idea of DevOps (and DevSecOps) concepts without recommending a specific tool. The DevOps tools you use depend on your enterprise size and its requirements. You can study a use-case for the customers either by the product vendor or by the customers themselves. Regardless of the DevOps tool you use, you will be able to perform good automation for software compilation and delivery if you follow abstract DevOps principles in a concrete fashion.
You should explore the appropriate books in the Apress library to dive deeper in the DevOps concepts, or to get hands-on expertise with the DevOps tools and software mentioned in this book. I am not recommending specific books here, because you can decide the software after a discussion with your team. You should also join the online websites, such as https://www.devsecops.org/, and join the events that take place around the globe.
You should join online communities of experts, such as Stack Overflow, CodeProject, C# Corner, and so on, where experts are available to give you personalized (but volunteer!) support for your projects and problems. In open source, GitHub has a good place as a community leader. You should always consult the repositories available online. On GitHub, users have a habit of creating conceptual repositories with a leading “awesome.” Such as https://github.com/devsecops/awesome-devsecops. Try this repository for a comprehensive list of learning material for DevSecOps: https://github.com/maksimyugai/awesome-devsecops. You can visit these repositories and get learning material or hands-on labs to explore the concepts. You should also search for the “awesome-concept,” where “concept” is replaced with the concept you want to learn.
You should also follow the hash tags for DevOps and DevSecOps on Twitter and other social channels to learn new trends. Remember that DevOps and DevSecOps are used for the same event due to SEO reasons. Most of these channels are friendly, and you might get support for your queries in design and implementation of DevOps. We might cross threads someday.
You can also follow me on Twitter @afzaalvirgoboy and connect with me directly if you have any queries regarding the book material or about what’s going on in the DevOps for .NET Core.