AWS CloudFormation provides sample templates that you can use as a starting point for defining your specific requirements. You will need to create one or more templates to translate your design into stacks. For example, if you have designed a services-oriented application then your application contains units of functionality and contracts that define its interfaces. You might also have dependencies between your services. Hence, your stacks will need to reflect these services' characteristics in terms of parameters, output, and so on.

Creating CloudFormation templates is very similar to software development practices. For example, you will need to develop and conduct code reviews, maintain repositories and version control, and test, run, and maintain them. In addition, when you hit errors you will need to debug and fix your code.

In order to minimize errors and the time taken to develop production quality CloudFormation templates, ensure that you:  

• Validate the template (check for structure and API usage, JSON syntax, the presence of circular dependencies, and so on).
• Use parameter types to avoid bad input parameters and specify appropriate parameter-related constraints and regex patterns. These parameters are validated at the beginning of the stack creation process, so if there are any errors you will know almost immediately.
• Grant IAM permissions for creating the full stack and all the resources specified in the template. In addition, ensure that permissions are given to create/update the stack as well as rollback the changes.
• Ensure sufficient quotas for all the resource types in your stack, for example, the number of EC2 instances, RDS storage limits, and so on.

It is good practice to leverage CloudFormation::Init to declaratively specify the packages to be installed and users to be created, as well as executing configuration scripts and so on. CloudFormation::Init makes your application updatable; for example, you can update the running stack with a new version of your application. If you prefer to use tools such as Chef, then you can also use Chef recipes to install and/or update your application in the stack.

As a security best practice, never include secret keys and access keys in your CloudFormation templates. You can leverage IAM roles to achieve the same result. Even if you need to include parameters such as database passwords, you should mark them with a no echo option. That will ensure that the parameter is not revealed in the logs or stack events.

Leverage CloudFormation's integration with other AWS services and features to get a better handle on managing your stack. For example, use CloudFormation’s integration with CloudTrail to log CloudFormation API calls. Furthermore, you can query these logs and set alerts. These features can enable you to troubleshoot or debug any issues.

There are other AWS tools that can help you with creating and updating your CloudFormation templates. For example, you can use AWS Config for detecting changes in the stack made from outside of CloudFormation. You can also use CloudFormer to create CloudFormation templates from existing resources in an active stack.