Hour 23. Getting an App Certified

To sell our apps in the Windows Store, those apps need to pass certification. During this hour, we look at why certification is important. We detail the Windows App Certification Kit (Windows ACK) and how we can use it to test our apps to make sure they pass certification. We look at each test the Windows ACK runs and cover different items we can check if any of those tests fail.

In addition, we explore the certification requirements our apps must meet to be available in the Windows Store. We don’t list the requirements, but we look at some that might not be extremely obvious or haven’t been called out in the book beforehand, such as making calls over metered networks. Fortunately, most of the requirements have already been called out as we discussed all components that make up a Windows Store app throughout the book.

Understanding the Need for Certification

Probably the biggest problem for consumers is security regards what is installed on the PCs. Malware and spyware are widespread. Users purchase branded and packaged software from trusted sales outlets because they trust the integrity of the software. The same cannot be said about downloading software from the Internet. When folks download files from the Internet, even if the actual source is free of spyware/malware, they put themselves at risk of infecting their machines. Web browsers typically show a warning message with a blurb about the dangers of downloading data from the Internet, but these browsers don’t actually check to see if the data being downloaded is what it claims to be. The Internet is a dangerous place.

To make the Internet a more reliable source for software, Microsoft has created Microsoft Authenticode. This ensures authenticity and assures users that Microsoft knows where the code that was downloaded came from. It also ensures integrity by verifying that the code was not tampered with since it was published. What this doesn’t address is the fact that the code is actually safe to run.

To accommodate this, digital signatures are in place. A digital signature enables the producer of the program to sign the program so that the receiver knows it came from the sender. This makes the assumption that the lack of anonymity is a deterrent to the purposeful distribution of harmful code.

The previous two paragraphs greatly oversimplify this issue, but the point is valid: Although the Internet itself is a dangerous place to obtain code, the issues are solvable with some controls in place. This is where the Windows Store comes into play. Any developer or company that wants to submit an app to the Windows Store must be authenticated. A third-party company makes sure that the person or company is a real company that can be held accountable if it purposefully or neglectfully transmits harmful code to the store. Then when Microsoft has the app package, it signs it with the certificate presented in Hour 22, “Understanding the Windows Store.” The OS can determine whether the package was altered in any way and, if so, can refuse the app from running. Again, this is an overly simplified description, but it shows how having a central store and a certification process helps end users (and companies) trust that the products being deployed were intended to be deployed and were not altered to be malicious.

Beyond the security benefits of certification, having a certification process helps with the quality of software. The certification process is not a QA process, in the sense that all testing is done before the app package is uploaded, but if errors occur during testing, the app will fail certification—after all, having an app crash is a horrible user experience.

The last point deals less with the certification process than with the overall benefit of having the apps run in a sandbox mode. Even if some malicious code is introduced, the app does not have access to the full system. This extra security keeps an app from accessing off-limit files or creating permanent damage on the machine.

Using the Windows App Certification Kit

We have seen how creating an app package prompts us to run the Windows App Certification Kit (Windows ACK) against the package just created. We can also launch the Windows ACK directly to validate Windows Store apps or desktop apps (see Figure 23.1).

Figure 23.1. The Windows App Certification Kit can be run manually, and a Windows Store app can be selected.

Clicking Validate Windows Store App brings up a screen that lists all installed apps (not just our apps), and we can select the one to run the test on. Typically, running this after creating the package is beneficial, but the app also can be run manually. It can even be run from the command line and invoked for a continuous build scenario, but that is beyond the scope of this book.

The actual validation process goes through the same process when the app is selected, either manually or after creating an app package. It prepares to validate the app, actually validates the app, and finally generates the report. Figure 23.2 illustrates this validation process.

Figure 23.2. The Windows App Certification Kit validates an app.

After the app has completed validation through the Windows ACK tool, the validation results display with an overall score of passed or failed, as in Figure 23.3. The tool provides a link for us to submit the app to the Windows Store, if it passed. We also see a link where we can view the full report, which is stored as an HTML document on our machine.

Figure 23.3. The Windows App Certification Kit displays validation results after attempting to validate the app.

The HTML document provides information on which areas passed and which ones failed. The Windows ACK report in Figure 23.4 has been slightly modified, in that all the content is present, but instead of taking up three screens, additional columns have been created to show all the results in a single image. We can see that, just as the tool reported, the app passed validation. This validation tool is the same one that Microsoft runs, and it cuts down on the feedback loop for what we can correct right away. Eleven categories are tested. Each can have a value of Pass or Fail, except for the Performance test, which can have the additional value of Warning.

Figure 23.4. The Windows App Certification Kit creates a report that shows which tests passed and which tests failed.

These categories are checked:

Crashes and Hangs Test

We certainly don’t want our app to crash or hang. This would cause the user to lose data, in most cases, and is always a poor experience. This test checks the resilience and stability of the app throughout the certification testing. The test will fail if the machine running Windows ACK does not have UAC turned on. It will also fail if the resolution is less than 1,024×768 (or 768×1,024). If we run the test only on Windows 8, this won’t be an issue because Windows 8 requires that resolution as a minimum. This tool can also certify Windows 7 desktop apps, where the resolution can practically be any resolution.

App Manifest Compliance Test

The app manifest compliance test checks the contents of the app manifest to confirm that the contents of the manifest are valid. The manifest file must be correctly formatted. More information can be found at http://msdn.microsoft.com/library/windows/apps/hh694075.

Windows Security Features Test

The Windows security features test makes sure the app is using the Windows security features and strong access control lists (ACLs). If an app changed the default Windows security protections, it would put the users at risk. The Windows ACT runs the BinScope Binary Analyzer and the Attack Surface Analyzer to confirm that the app is compliant.

More information on these tests can be found at http://msdn.microsoft.com/library/windows/apps/hh750314.

Supported API Test

The supported API test makes sure that the only system APIs being used are the Windows Store APIs. Running the certification tests with a debug build of our app causes this test to fail. If this test fails and the app was a nondebug build, then the error message can be examined to find the offending API.

Performance Test

The performance test checks to see how fast the app launches, how fast it suspends and resumes, and the peak memory usage the app uses. For x86/x64 processors, if the launch test completes in less than 3 seconds, the app passes. For ARM processors, the launch test must complete in less than 5 seconds. On both architectures, if the launch test completes in more than 5 seconds, the app fails. The warning comes on x86/x64 machines when the launch test completes in more than 3 seconds but less than 5. The reason for the warning is that the app may fail this test on a low-power ARM device, so the app should be tested on such a device.

The peak memory usage must be less than 500MB, or the test will fail. To test this, we open the Task Manager and go to the Processes tab. We must check the value in the Memory column to make sure that it doesn’t exceed 500MB. We should run our app through different scenarios, especially anywhere we think it may be consuming a lot of memory.

App Manifest Resources Test

The app manifest compliance test opens the app package manifest and confirms that all the resources defined are actually present in the package. It also makes sure they are valid. If the images or strings are not correct, the tile might not get displayed correctly and also might cause a crash. If a problem arises, information is identified in the error message. For example, one such message is “The image {imageName} failed the size restrictions....” If we encounter that, we need to confirm that our app images conform to the appropriate size restrictions. A list of error messages can be found at http://msdn.microsoft.com/library/windows/apps/hh920286.

Debug Configuration Test

The debug configuration test simply makes sure that the app is not a debug build. This can occur if we are using other frameworks that were compiled in debug mode.

File Encoding Test

The file encoding test makes sure that the files included with the app package are encoded properly. JavaScript files must be encoded as Unicode (UTF-8) with a byte-order mark (BOM). This allows them to utilize bytecode caching and supports the performance of our JavaScript apps.

Direct3D Feature Level Support Test

When we submit our app to the Windows Store, we need to specify the Direct3D feature level our app supports. The Direct3D feature level support test makes sure that what our app uses agrees with what was set in the app submission process. Locally, where the Windows ACK does not have access to that information, it makes sure that the app at least supports the minimum Direct3D feature level of 9_1. As JavaScript developers, we get the benefit of using Direct3D, but we do not explicitly set any values. This mainly applies to developers who talk directly to the GPU using HLSL.

App Capabilities Test

The app capabilities test makes sure the app isn’t using any capabilities it hasn’t declared. We ensure that the capabilities we declare are related to the core functions and value proposition of the app.

Windows Runtime Metadata Validation Test

The last test is the Windows Runtime metadata validation test. This test makes sure that we are using the right metadata files. The metadata files are used for language projection, as discussed during Hour 3, “Exploring the Windows Runtime (WinRT) and Windows Library for JavaScript (WinJS).” This test makes sure that any code we are referencing is talking to the correct version of the API.

Understanding the Certification Requirements

We can save some trees (and/or bytes) by not listing the entire list of certification requirements. All the requirements can be found at http://msdn.microsoft.com/en-us/library/windows/apps/hh694083.

Fortunately, we have covered most items throughout the course of the book. Still, reading the document is advantageous, in case details have changed since the book was published. If the certification requirements do change, Microsoft lists the revisions at the bottom of the document so that we can easily stay in compliance.

Instead of listing individual points, we list the main points and discuss in general concerns to watch for.

1. Windows Store Apps Provide Value to the Customer

All subpoints under 1 describe ways in which our app needs to provide value to the customer. Most of these are common sense but need to be laid out to avoid confusion. However, we can take note of point 1.4, which states, “Each app must display only one tile after it is installed.” When the app is installed, it cannot automatically create secondary tiles. Secondary tiles can be generated only explicitly by the user. No user wants an app running the first time and spamming the Start screen with a bunch of secondary tiles.

2. Windows Store Apps Can Display Ads but Are More Than Just Ads or Websites

Certification point 2 is all about ads. We discuss ads during Hour 24, “Making Money with Windows Store Apps.” Two subpoints deserve special attention. The first is point 2.3, which states, “Your app must not use its tiles, notifications, app bar, or the swipe-from-edge interactions to display ads.” The second is point 2.5, which states, “Ads must not execute program code that didn’t come from the ad provider.” When displaying ads in our apps, we need to take care that we don’t allow code to be executed. We can do this by creating an iframe in a sandbox mode, as discussed during Hour 7, “Debugging, Securing, and Measuring Our App’s Performance.”

3. Windows Store Apps Behave Predictably

To start, having our apps support touch, mouse, and keyboard is extremely important. As discussed during Hour 8, “Working with Multitouch and Other Input,” we need to develop our apps for touch first, and we get mouse and pen for free. If an app doesn’t support touch, it won’t pass certification. Subpoint 3.5 states, “Your app must fully support touch input, and fully support keyboard and mouse input.”

Subpoint 3.3 borders on common sense, but it is important to note that we need to test apps on different available hardware. That subpoint states, “Your app must provide the same user experience on all processor types that it supports.” We can’t change functionality based on the processor type in the same app. Instead, if we need to handle the different hardware differently, we need to create separate apps and, during submission, describe the differences of each app.

Subpoint 3.9 states, “All app logic must originate from, and reside in, your app package.” We cannot execute any code that isn’t included in our app that changes how the app interacts with the Windows Runtime. For example, we can’t bring in an external script and then run it in our local context of our app package. However, we can download the external script and then include it in our package so that it can be used in the local context. Only trustworthy third-party libraries should be brought into an app package.

4. Windows Store Apps Put the Customer in Control

We need to make sure that the app behaves correctly regarding user privacy. Subpoint 4.1.1 states, “Your app must have a privacy statement if it collects personal information.” If we collect user information, we must provide a privacy policy and explain how it is used and also, if it is stored, how it is secured and disclosed. The policy needs to let the users know how to access their information and what controls they have over the use and sharing of their information. Similarly, subpoint 4.1.2 states, “Your app must obtain opt-in or equivalent consent to share personal information.” We cannot simply collect and publish the user’s personal information to a service or other person unless the user has explicitly opted in. The user must be made aware of the how the information will be used or shared, and a mechanism must be provided to let the user opt out as well. As discussed during Hour 22, “Understanding the Windows Store,” we want to make sure that we do not collect any information from children under 12.

Subpoint 4.5 states, “Your app must protect customers from unintentional large data transfers over metered networks.” If our app is sending data larger than 1MB, we need to use the ConnectionCost class. This class has the networkCostType property, which tells us whether the current network cost for the app is unknown, unrestricted, fixed, or variable. Variable means that the connection price is determined on a per-byte basis; it is a metered connection. Fixed means the connection is unrestricted, up to a certain point. We can also use the approachingDataLimit property to see if the connection is approaching the data usage allowance.

5. Windows Store Apps Are Appropriate for a Global Audience

All the subpoints seem to be common sense, so we can list none or list all of them. Because only eight subpoints exist, with no real need for embellishment, we list them as follows:

5.1      Your app must not contain adult content, and metadata must be appropriate for everyone.

5.2      Your app must not contain content that advocates discrimination, hatred, or violence based on membership in a particular racial, ethnic, national, linguistic, religious, or other social group, or based on a person’s gender, age, or sexual orientation.

5.3      Your app must not contain content or functionality that encourages, facilitates, or glamorizes illegal activity.

5.4      Your app must not contain or display content that a reasonable person would consider to be obscene.

5.5      Your app must not contain content that is defamatory, libelous or slanderous, or threatening.

5.6      Your app must not contain content that encourages, facilitates, or glamorizes excessive or irresponsible use of alcohol or tobacco products, drugs, or weapons.

5.7      Your app must not contain content that encourages, facilitates, or glamorizes extreme or gratuitous violence, human rights violations, or the creation or use of weapons against a person or animal in the real world.

5.8      Your app must not contain excessive or gratuitous profanity.

6. Windows Store Apps Are Easily Identified and Understood

Some of the subpoints of this certification requirement were discussed earlier in this hour when looking at the Windows App Certification Kit. Many more were discussed during Hour 22. One of these merits mention again. Subpoint 6.3 states, “You must provide technical support for your app.” Microsoft doesn’t provide customer support or technical support for apps. We must provide the support and make sure the user can easily find the technical support info. We need to put the technical support info in the Support Contact Info field of the Description page when we submit the app. Also, if our app requires a username and password, we must provide a way for the user to reset the password. Responding to customers within 24 hours is required.

Further Exploration

As we discussed the Microsoft Authenticode and digital signatures, we left out many details. If this is of interest, the article “Introduction to Code Signing” has excellent information on the app certificate signing process. See http://msdn.microsoft.com/en-us/library/ms537361.

In making sure our app performs well, we need to take advantage of bytecode caching. The following documentation about reducing an app’s loading time has excellent information on bytecode caching: http://msdn.microsoft.com/en-us/library/windows/apps/hh849088.

An excellent quickstart to walk through is “Quickstart: Managing Connections on Metered Networks.” If any major network traffic will move through the app, this is a must-read: http://msdn.microsoft.com/en-us/library/windows/apps/hh750310.

Similarly, we can walk through “Quickstart: Managing Connection State Changes” to see how we can tie into the connection state change events: http://msdn.microsoft.com/en-us/library/windows/apps/hh700376.

A lot of helpful people, both Microsoft employees and the community-at-large, can provide help in the Windows Store forums. Many questions already have answers, which can be found by searching the forums. The forums are located at http://social.msdn.microsoft.com/Forums/en-US/category/windowsapps.

The JavaScript- and HTML5-specific threads are located at http://social.msdn.microsoft.com/Forums/en-US/winappswithhtml5/threads.

Summary

This hour, we looked at why certification is important and explored some of the ways Microsoft protects against apps with malicious intent and apps that were potentially tampered with during transmission. We looked at the Windows App Certification Kit and how we can use it to test our app to make sure it passes certification. We looked at the different tests the Windows ACK runs, along with the items we can check if any of those tests fail.

We looked at the actual certification requirements our apps must abide by to be put into the Windows Store. We covered some requirements that might not be obvious. We described how to work with metered networks because that wasn’t shown in the source of the book. Overall, the certification requirements are pretty straightforward and expected. Knowing about them up front can definitely help with development efforts.

Q&A

Q. Can Windows Store apps contain viruses?

A. Anything is possible, but only a very sneaky virus is likely to get past the certification process. The whole point of the Windows Store is to provide users with the confidence that the apps they are downloading are free of malware, spyware, and viruses. An app containing malicious code will not likely get through the certification process and into the store.

Q. Why do I get a “Passed with Warnings” from the Windows App Certification Kit?

A. The performance test checks to see how fast the app launches and how fast it suspends. When running the Windows ACK on x86/x64 processors, the test passes with warning if the launch test completes in more than 3 seconds but less than 5. The warning appears because the app might fail this test on a low-power ARM device and should be tested on such a device.

Q. What object do we need to interrogate to determine whether the app’s connection is on a metered network?

A. We need to check the ConnectionCost object to determine whether the app is on a metered network. It is important to not send large data (more than 1MB) over a metered network without the user’s consent.

Workshop

Quiz

1. How many crashes or hangs can occur in an app during the certification process before it fails certification?

2. The peak memory usage must be less than 500MB, or the app will fail certification. True or false?

3. To debug issues with an app, we need to upload a debug version of our application to the Windows Store after running it through the Windows App Certification Kit. True or false?

4. Does Microsoft provide technical support for apps?

Answers

1. No crashes or hangs can occur in an app during the certification process, or it will not be approved for the Windows Store.

2. True. If peak memory usage is greater than 500MB, the app will fail certification.

3. False. Apps must be submitted to the Windows Store in a nondebug build (that is, release build). The Windows App Certification Kit fails apps in debug mode. The Windows Store dashboard provides data if we turn on telemetry data when we submit the app or through the dashboard after it has been submitted.

4. No. Technical support and customer support are the responsibility of the app creator.

Activities

1. Take time to read the entire list of certification requirements, located at http://msdn.microsoft.com/en-us/library/windows/apps/hh694083.