.NET Memory Allocation Profiling

Visual Studio 2012 has a wealth of valuable functionality, so much so that I periodically hear developers that already use Visual Studio asking for a feature the IDE already has and that they’ve just never discovered. Other times, I hear developers asking about a specific feature, thinking it’s meant for one purpose, not realizing it’s really intended for another.

Both of these cases apply to Visual Studio’s .NET memory allocation profiler. Many developers that could benefit from it don’t know it exists, and other developers have an incorrect expectation for its purpose. This is unfortunate, as the feature can provide a lot of value for particular scenarios; many developers will benefit from understanding first that it exists, and second the intended scenarios for its use.

Why memory profiling?

When it comes to .NET and memory analysis, there are two primary reasons one would want to use a diagnostics tool:

1. To discover memory leaks. Leaks on a garbage-collecting runtime like the CLR manifest differently than do leaks in a non-garbage-collected environment, such as in code written in C/C++. A leak in the latter typically occurs due to the developer not manually freeing some memory that was previously allocated. In a garbage collected environment, however, manually freeing memory isn’t required, as that’s the duty of the garbage collector (GC). However, the GC can only release memory that is provably no longer being used, meaning as long as there are no rooted references to the memory. Leaks in .NET code manifest then when some memory that should have been collected is incorrectly still rooted, e.g. a reference to the object occurs in an event handler registered with a static event. A good memory analysis tool might help you to find such leaks, such as by allowing you to take snapshots of the process at two different points and then comparing those snapshots to see which objects stuck around for the second point, and more importantly, why.
2. To discover unnecessary allocations. In .NET, allocation is often quite cheap. This cost is deceptive, however, as there are more costs later when the GC needs to clean up. The more memory that gets allocated, the more frequently the GC will need to run, and typically the more objects that survive collections, the more work the GC needs to do when it runs to determine which objects are no longer reachable. Thus, the more allocations a program does, the higher the GC costs will be. These GC costs are often negligible to the program’s performance profile, but for certain kinds of apps, especially those on servers that require high-throughput operation, these costs can add up quickly and make a noticeable impact to the performance of the app. As such, a good memory analysis tool might help you to understand all of the allocation being done by the program, in order to help spot allocations you can potentially avoid.

The .NET memory profiler included in Visual Studio 2012 (Professional and higher versions) was designed primarily to address the latter case of helping to discover unnecessary allocations, and it’s quite useful towards that goal, as the rest of this post will explore. The tool is not tuned for the former case of finding and fixing memory leaks, though this is an area the Visual Studio diagnostics team is looking to address in depth for the future (you can see such an experience for JavaScript that was added to Visual Studio as part of VS2012.1). While the tool today does have an advanced option to track when objects are collected, it doesn’t help you to understand why objects weren’t collected or why they were held onto longer than was expected.

There are also other useful tools in this space. The downloadable PerfView tool doesn’t provide as user-friendly an interface as does the .NET memory profiler in Visual Studio 2012, but it is a very powerful tool that supports both tasks of finding memory leaks and discovering unnecessary allocations. It also supports profiling Windows Store apps, which the .NET memory allocation profiler in Visual Studio 2012 does not currently support as of the writing of this post.

Example to be optimized

To better understand the memory profiler’s role and how it can help, let’s walk through an example. We’ll start with the core method that we’ll be looking to optimize (in a real-world case, you’d likely be analyzing your whole application and narrowing in on the particular offending areas, but for the purpose of this example, we’ll keep this constrained):

public static async Task<T> WithCancellation1<T>(this Task<T> task, CancellationToken cancellationToken)
{
    var tcs = new TaskCompletionSource<bool>();
    using (cancellationToken.Register(() => tcs.TrySetResult(true)))
    if (task != await Task.WhenAny(task, tcs.Task))
        throw new OperationCanceledException(cancellationToken);
    return await task;
}

The purpose of this small method is to enable code to await a task in a cancelable manner, meaning that regardless of whether the task has completed, the developer wants to be able to stop waiting for it. Instead of writing code like:

T result = await someTask;

the developer would write:

T result = await someTask.WithCancellation1(token);

and if cancellation is requested on the relevant CancellationToken before the task completes, an OperationCanceledException will be thrown. This is achieved in WithCancellation1 by wrapping the original task in an async method. The method creates a second task that will complete when cancellation is requested (by Registering a call to TrySetResult with the CancellationToken), and then uses Task.WhenAny to wait for either the original task or the cancellation task to complete. As soon as either does, the async method completes, either by throwing a cancellation exception if the cancellation task completed first, or by propagating the outcome of the original task by awaiting it. (For more details, see the blog post “How do I cancel non-cancelable async operations?”)

To understand the allocations involved in this method, we’ll use a small harness method:

using System;
using System.Threading;
using System.Threading.Tasks;
 
class Harness
{
    static void Main() 
     { 
         Console.ReadLine(); // wait until profiler attaches
         TestAsync().Wait(); 
     }
    static async Task TestAsync()
    {
        var token = CancellationToken.None;
        for(int i=0; i<100000; i++)
            await Task.FromResult(42).WithCancellation1(token);
    }
}
 

static class Extensions
{
    public static async Task<T> WithCancellation1<T>(
    this Task<T> task, CancellationToken cancellationToken)
    {
        var tcs = new TaskCompletionSource<bool>();
        using (cancellationToken.Register(() => tcs.TrySetResult(true)))
            if (task != await Task.WhenAny(task, tcs.Task))
                throw new OperationCanceledException(cancellationToken);
        return await task;
    }
}

The TestAsync method will iterate 100,000 times. Each time, it creates a new task, invokes the WithCancellation1 on it, and awaits the result of that WithCancellation1 call. This await will complete synchronously, as the task created by Task.FromResult is returned in an already completed state, and the WithCancellation1 method itself doesn’t introduce any additional asynchrony such that the task it returns will complete synchronously as well.

Running the .NET memory allocation profiler

To start the memory allocation profiler, in Visual Studio go to the Analyze menu and select “Launch Performance Wizard…”. This will open a dialog like the following:

Choose “.NET memory allocation (sampling)”, click Next twice, followed by Finish (if this is the first time you’ve used the profiler since you logged into Windows, you’ll need to accept the elevation prompt so the profiler can start). At that point, the application will be launched and the profiler will start monitoring it for allocations (the harness code above also requires that you press ‘Enter’, in order to ensure the profiler has attached by the time the program starts the real test). When the app completes, or when you manually choose to stop profiling, the profiler will load symbols and will start analyzing the trace. That’s a good time to go and get yourself a cup of coffee, or lunch, as depending on how many allocations occurred, the tool can take a while to do this analysis.

When the analysis completes, we’re presented with a summary of the allocations that occurred, including highlighting the functions that allocated the most memory, the types that resulted in the most memory allocated, and the types with the most instances allocated:

From there, we can drill in further, by looking at the allocations summary (choose “Allocation” from the “Current View” dropdown):

Here, we get to see a row for each type that was allocated, with the columns showing information about how many allocations were tracked, how much space was associated with those allocations, and what percentage of allocations mapped back to that type. We can also expand an entry to see the stack of method calls that resulted in these allocations:

By selecting the “Functions” view, we can get a different pivot on this data, highlighting which functions allocated the most objects and bytes:

Interpreting and acting on the profiling results

With this capability, we can analyze our example’s results. First, we can see that there’s a substantial number of allocations here, which might be surprising. After all, in our example we were using WithCancellation1 with a task that was already completed, which means there should have been very little work to do (with the task already done, there is nothing to cancel), and yet from the above trace we can see that each iteration of our example is resulting in:

• Three allocations of Task`1 (we ran the harness 100K times and can see there were ~300K allocations)

• Two allocations of Task[]

• One allocation each of TaskCompletionSource`1, Action, a compiler-generated type called <>c_DisplayClass2`1, and some type called CompleteOnInvokePromise

That’s nine allocations for a case where we might expect only one (the task allocation we explicitly asked for in the harness by calling Task.FromResult), with our WithCancellation1 method incurring eight allocations.

For helper operations on tasks, it’s actually fairly common to deal with already completed tasks, as often times operations implemented to be asynchronous will actually complete synchronously (e.g. one read operation on a network stream may buffer into memory enough additional data to fulfill a subsequent read operation). As such, optimizing for the already completed case can be really beneficial for performance. Let’s try. Here’s a second attempt at WithCancellation, one that optimizes for several “already completed” cases:

    public static Task<T> WithCancellation2<T>(this Task<T> task, 
                             CancellationToken cancellationToken)
    {
        if (task.IsCompleted || !cancellationToken.CanBeCanceled)
            return task;
        else if (cancellationToken.IsCancellationRequested)
            return new Task<T>(() => default(T), cancellationToken);
        else
            return task.WithCancellation1(cancellationToken);
    }

This implementation checks:

• First, whether the task is already completed or whether the supplied CancellationToken can’t be canceled; in both of those cases, there’s no additional work needed, as cancellation can’t be applied, and as such we can just return the original task immediately rather than spending any time or memory creating a new one.

• Then whether cancellation has already been requested; if it has, we can allocate a single already-canceled task to be returned, rather than spending the eight allocations we previously paid to invoke our original implementation.

• Finally, if none of these fast paths apply, we fall through to calling the original implementation.

Re-profiling our micro-benchmark while using WithCancellation2 instead of WithCancellation1 provides a much improved outlook (you’ll likely notice that the analysis completes much more quickly than it did before, already a sign that we’ve significantly decreased memory allocation). Now we have just have the primary allocation we expected, the one from Task.FromResult called from our TestAsync method in the harness:

So, we’ve now successfully optimized the case where the task is already completed, where cancellation can’t be requested, or where cancellation has already been requested. What about the case where we do actually need to invoke the more complicated logic? Are there any improvements that can be made there?

Let’s change our benchmark to use a task that’s not already completed by the time we invoke WithCancellation2, and also to use a token that can have cancellation requested. This will ensure we make it to the “slow” path:

    static async Task TestAsync()
    {
        var token = new CancellationTokenSource().Token;
        for (int i = 0; i < 100000; i++)
        {
            var tcs = new TaskCompletionSource<int>();
            var t = tcs.Task.WithCancellation2(token);
            tcs.SetResult(42);
            await t;
        }
    }

Profiling again provides more insight:

On this slow path, there are now 14 total allocations per iteration, including the 2 from our TestAsync harness (the TaskCompletionSource<int> we explicitly create, and the Task<int> it creates). At this point, we can use all of the information provided by the profiling results to understand where the remaining 12 allocations are coming from and to then address them as is relevant and possible. For example, let’s look at two allocations specifically: the <>c__DisplayClass2`1 instance and one of the two Action instances. These two allocations will likely be logical to anyone familiar with how the C# compiler handles closures. Why do we have a closure? Because of this line:

using(cancellationToken.Register(() => tcs.TrySetResult(true)))

The call to Register is closing over the ‘tcs’ variable. But this isn’t strictly necessary: the Register method has another overload which instead of taking an Action takes an Action<object> and the object state to be passed to it. If we instead rewrite this line to use that state-based overload, along with a manually cached delegate, we can avoid the closure and those two allocations:

private static readonly Action<object> s_cancellationRegistration =
    s => ((TaskCompletionSource<bool>)s).TrySetResult(true);
…
using(cancellationToken.Register(s_cancellationRegistration, tcs))
  

Rerunning the profiler confirms those two allocations are no longer occurring:

Code Metrics

Code Metrics

As mentioned previously, Code Metrics is a new tool window that helps users find and act upon complex and unmaintainable areas within an application.
The following shows the results of running Code Metrics over a fictional business application:

As you can see from above, for Visual Studio 2008, we're providing five metrics. These are detailed below.
Class Coupling
At each level, this indicates the total number of dependencies that the item has on other types. This number excludes primitive and built-in types such as Int32, String and Object. The higher this number, the more likely changes in other types will ripple though this item. A lower value at the type level can indicate candidates for possible reuse.
The following shows how coupling is calculated:

For example, as you can see above, Account is coupled to two other types, Address and Order, whereas Country is not dependent on any other type.

Depth of Inheritance
At the type level, depth of inheritance indicates the number of types that are above the type in the inheritance tree. For example, a type that derives directly from Object would have a depth of inheritance of 1. At the namespace and project level, this indicates the highest depth of inheritance of all the types contained within it. This number does not take into consideration the depth of any implemented interfaces. Deep inheritance trees can indicate an over-engineering of a problem and can increase the complexity of testing and maintaining an application.
The following shows how depth is calculated:

 

For example, in the above inheritance hierarchy, ListControl and Label have a depth of inheritance of 3, whereas Component has a depth of inheritance of 1.
Cyclomatic Complexity
At each level, this measures the total number of individual paths through the code. This is basically calculated by counting the number of decision points (such as if blocks, switch cases, and do, while, foreach and for loops) and adding 1. This number is also a good indication on the number of unit tests it will take to achieve full line coverage. Lower is typically better.
The following shows how complexity is calculated:

 

Lines of Code
At each level, this is a measure of the total number of executable lines of code. This excludes white space, comments, braces and the declarations of members, types and namespaces themselves. Lower is typically better.
The following shows how the lines are calculated:

 

Maintainability Index
At the member and type level, this is an index from 0 to 100 indicating the overall maintainability of the member or type. At the namespace and assembly level, this is an average of the maintainability index of all types contained within it. This index is based on several other metrics, including Halstead Volume (which factors in the number and use of operands and operators), Cyclomatic Complexity and Lines of Code. A low number indicates code that is complex and hard to maintain.
The Maintainability Index column also includes a icon that gives a quick indication as to the overall maintainability and complexity of a particular item. The following table shows the range at which an icon is shown:

Icon	Level	Range
	High Maintainability	Between 20 and 100 inclusive
	Moderate Maintainability	Between 10 and 19 inclusive
	Low Maintainability	Between 0 and 9 inclusive

These icons allow you to see at a glance any trouble spots that you should start to focusing on or filling bugs against.

Conclusion

Now that you know what these metrics measure, and how they are calculated, the next thing you'll want to know is what you should do when you start to see some red in your application. In a future post, I will cover why should should care about each metric, and how to go about fixing some of the common issues that cause particular metrics to high (or low in the case of Maintainability Index).

Code Quality–Different ways to measure it

In software industry the word quality holds different meaning to different people. For a developer it means different, for a tester it means different, for the management it means different, for the client it means different. The perspective of each person is different. The tools used by each person to measure quality are different. The methods used to measure the matrix are also different. In todays post I am going to illustrate the quality metrics for software developers.

Reasons for quality check

There is lot of material written on the net for the need to do quality checks in software. The oblivious need it to reduce the number of bugs and improve the reliability and efficiency of the system. Most of the times when we say quality check, the first thing that comes to our mind is the test team doing some manual or regression tests. That is the functional quality of the product or the system. But what I am interested in is the quality of the code that is being written by the developers. How do we measure the quality of the code? Here are some of the ways used for measuring code quality.

Method 1 : Code reviews / Peer Reviews

One of the oldest form of quality check is the Peer Review. Once the functionality is developed by one developer, it is reviewed by someone else in the team. Typically this is done by a senior developer who is aware of the functional requirement as well as technically sound to find out any deficiencies in the code. In some cases it is done by the technical lead or project lead. The idea here is to get the second opinion and try and improve the code.
If your team follows Agile or Extreme Programming (XP) methodology for development, peer reviews can be reduced a lot by using a technique called Pair Programming. Most of the code is already getting reviewed by the person you are pairing with during the development phase itself.
There are benefits of this method as well as some disadvantages. One of the advantage is that it can help reduce bugs because some of the things can be caught during the review which could have led to bugs. It can also help different people approach the same problem in different perspectives and choose the best solution. It also helps in evaluating a developers performance in annual appraisal 
Lets look at some of the disadvantages. The method is rendered ineffective if the person doing the review is biased and is not open to new ideas or latest trends in the technology. Let me take an example of the biased approach. Assume an individual is used to particular style of coding. While reviewing some other developers code he might expect the other developers to follow the same coding style. As an example consider the case of using a ternary operator in a simple if else statement or using the if else in its expanded state. Some people have their own preferences with the way they define the code structure. They want all the variables, constants, methods to be organized into regions. While others don’t prefer this style of coding.
I remember once one of my ex-colleague told me his experience with his technical lead. My ex-colleague had implemented a piece of functionality in C# using LINQ and Lambdas. During the code reviews his tech lead asked him to rewrite the code without using LINQ and Lambdas because the Tech Lead did not know anything about LINQ and Lambdas. I am sure most of us would have gone through similar situation at some point of time in our developer lifecycle. I know lot of instances where the code was rejected because a variable was not named to the liking of the reviewer.
The point I am trying to make here is that manual code reviews are left to the discretion of the individual. What feels right for one person might not be the case for someone else. They are time consuming and in reality can delay the delivery of the software if there is lot of rework needed to fix the reviewers comments. If these manual code reviews are used purely to improve the quality of the code they are fine. But I know in many organizations they are used as a measure to rate the developers during annual reviews. To me it sounds absurd.
Long ago I had a personal experience when I was working as a Technical Lead on one of the project. The project lead asked me to review each line of code and create an excel sheet with the name of the developer who was responsible for writing the piece of code which was supposed to be refactored. I thought it was an utter waste of time. First of all, reviewing each line of code written by your team mate, means you don’t show trust in their ability. And even if I had so much time in the world to do this nonsense thing, I knew where it would have ended. It would have come up during the annual reviews filtered by the name of the developer and with some pivot table applied to it to prove how useless the developer was. Another thing that stopped me from doing it was that the developers on the team would have been prompted to fix only the issues found in their code and ignore other piece of code which is totally against the principle of Collective Code Ownership.
My preferred way of dealing with such issues is to come up with a well defined coding standard that is agreed by all the team members.And even after that if someone violates the coding standards I would prefer to talk to the individual to try and understand the reasons behind doing so instead of creating an Excel sheet which doesn’t hold any relevance after a short period of time. Having a coding standard might solve majority of the issues but there will always be the edge cases which can go either way. How can we overcome such issues?

Method 2 : Use Standard tools for code analysis

One way I have used previously to address these issues is to use the industry standards and best practices. That way you are at least following the footsteps of people who are bound to influence the majority of people. If your team is using Microsoft platform and tools for development then you can take advantage of some of the tools provided by Microsoft. Some tools are built into the Visual Studio itself while others are available as additional add ins or plugins. There are tools which can be used for checking the consistency of the code structure. Two such tools which come instantly to my mind are FxCop and StyleCop. FxCop is used for managed code analysis and StyleCop for static code analysis.
There are some rules which are common to both and some which are unique to each of them. One of the example where FxCop can be helpful is to ensure the Framework guidelines like Abstract class should not have a constructor. Similarly StyleCop can be used to ensure that there are no multiple lines between method definitions in a file. Both these tools can be integrated with theContinuous Integration (CI) build.
These tools are good for bringing in the consistency in the code structure. We can ensure that someone is not coming up with their own naming style when rest of the team is following a particular set of rules. Is it good enough? Lets assume you have integrated both FxCop as well as StyleCop in your build process, there are no errors reported after the build is completed. Does that mean the code is quality code. Definitely not. Just having a code that passes the FxCop and StyleCop rules does not ensure that it is good code. There has to be additional measures to ensure the quality of code.

Method 3 : Use Code Coverage

I assume most of the teams are using Test Driven Development (TDD) or at least plan to do so in future. If not then you can skip this section. When we are developing software using TDD, we can make use of Code Coverage feature to measure the percentage of code covered as part of unit tests. Again this feature can be integrated into the build process. Generally minimum 80% code coverage is considered as a standard. Now a days I find teams striving to achieve 100% code coverage. This link here demonstrates how to get started with Code Coverage in Visual Studio.
The Code Coverage feature is available only for Ultimate and Premium editions of Visual Studio. If you are using other editions of Visual Studio, you can use other free tools which are counterparts of the Code Coverage. I have used TestDriven.Net which used to integrate even with VS2003 IDE. If I am not wrong then Code Coverage was introduced with Visual Studio since 2005 release. If you are still one of those people using VS 2003, I would recommend using TestDriven.Net. We had integrated it with our build in couple of projects that I had worked in my previous organization.
There is also an Open Source alternative which I haven’t used personally but have heard from few people called PartCover. I can’t comment much on this as I haven’t had any experience in using it. But if you are interested you can give it a try.
There are some critical issues related to code coverage. Using code coverage as a measure of quality can be quite misleading. Let me share my own experience here. In one of our project we were using Code Coverage to decide the build quality. If the coverage was below a certain percentage the CI server would fail the build. This was to ensure that all the newly built software was properly unit tested. But as always there are people who know how to get around in certain situations. The project lead of that particular project did not understand the importance of unit tests and the value add that we got from them. He assigned a junior programmer to write unit test for all the classes which were not covered. The junior programmer worked alone on writing those unit tests due to tight deadlines. He ensured most of the classes were covered to ensure the build was green. But the tests that he had written were completely useless to say the least. Most of the tests did not have any asserts to verify the expected functionality. In essence all he did was to make sure the unit tests executed the production code without really verifying anything.
This is an abuse of TDD. If your team does not believe in the principle of TDD there is no point writing unit tests just to pass the build. Its a waste of time and effort and most annoyingly a maintenance headache. Other developers in the team had to spend hell lot of time to fix those tests when the issue was identified. By the time it was identified the person who wrote those tests was already out of the organization.
My suggestion is that if you are using TDD make sure you write test which are meaningful and help you validate the code. What's the point if you have 100s of test and none of them verify anything in the end. Now assume you have one of the best code coverage statistics for all your projects, does that mean you have good quality code? I am not entirely convinced. Let me illustrate why. You can have the best coverage, but the developer might have written code which is 1000 lines of executable code in a single class file. There could be methods which are 500 lines of executable code. This scenario was quite common with developers who started doing Object Oriented development after spending years in developing procedural code. Believe me those are not the only creatures writing such code, I have seen such methods and classes myself written by people who have worked with only Object oriented languages right from day one of their career. Its bit sad that people don’t really make use of Object Oriented features to come up with code which is structured correctly.
You might ask me what’s bad with such code? The answer is not so difficult. Its a maintenance nightmare. You might know what is happening inside the class or method while developing it, imagine what will happen when you have to revisit the code after 6 months, may be 1 year later or even 2 years later. Spare a thought for the developer who is going to refactor that piece of code after you have moved onto a different project or a different company. So how do we take care of this situation?

Method 4 : Use industry standards such as Cyclomatic Complexity

Long ago I had written couple of post related to DIfferent Metrics available in Visual Studio and how to use Code Metrics. You might want to read them before continuing further.
One of the commonly used matrix to gauge the quality is Cyclomatic Complexity. It is the measure of the decision points a function has to evaluate. Decision points are measured by the number of logical statements such as an if statements, switch statements, for loops, while statements etc. At a class level its the sum of Cyclomatic Complexity of all the methods and similarly at the Module or Assembly level it is the aggregate of the Cyclomatic Complexity of all its child elements. Cyclomatic Complexity can tell us approximately how many unit tests will be required to fully cover all the code paths within a function. Different teams have different acceptance levels for Cyclomatic Complexity. In general the recommended value is 40. If this value is higher, we should try to refactor the method to make it more readable and maintainable. It is far more easier to understand a method which is 10 to 15 lines then go through a page full of code and figure out what is going on.
There are no hard and fast rules related to Lines of Code (LoC) per method, class etc. But a general guidelines is to make the methods smaller and self explanatory. If we follow that, most of our methods should have between 8 to 15 lines of code. At a class level this should generally be around 100 to 150 lines. You can argue that there can be classes with more than 200 lines none of which have methods more than 15 lines. This does happen in real life. You’ll find that the Cyclomatic Complexity for such classes will be higher compared to the smaller classes.
Can we rely entirely on Cyclomatic Complexity as our preferred measure of code quality? Not entirely. Because we’ll come across situation where we cannot split methods and classes beyond certain extent and their complexity is on the higher side. So how can we arrive at a reliable measure. In Visual Studio Code Matrix report we can also find Maintainability Index which indicates how easier or difficult it would be to maintain a particular method or a class or an Assembly. Along with maintainability index we can also use another metrics called Depth of Inheritance. It is a measure of how many levels of inheritance are involved. All these matrix can be helpful in deciding the quality of the code. I remember once refactoring a very lengthy method with Cyclomatic Complexity of more than 300 into a smaller one with less than 50 Complexity points. These metrics are available from within the Visual Studio development environment. Using these metrics should be a good starting point to build quality code.
Once the whole team becomes comfortable with using these tools and techniques we can go a step forward and automate this using some dedicated tool like NDepend which can give us numerous other quality metrics including Cyclomatic Complexity. NDepend supports a language called Code Query Language (CQL) which can be used to query runtime code attributes and define custom rules. For example we can define a rule in NDepend which says a warning or an error should be reported if a method has more than 50 lines of code and has Cyclomatic Complexity is more than 50. As of today NDepend can measure up to 82 different metrics related to the code quality. All these comes with a cost. NDepend is a commercial product and you need to pay for the licences. Having used the trial version I think its worth investing in NDepend for long term benefits.

Conclusion

As we saw, the code quality can be measured in different ways. Each measure can be used to validate different aspect of the code. We can never rely on one single tool or technique to measure code quality. In my opinion we should use a combination of two or more metrics so that the short comings of one technique might be covered by the other. The biggest advantage of using a standard metrics like Cyclomatic Complexity is that it is not left to the discretion and interpretation of one or two individuals. You’ll get the same result if we measure the metrics on the same piece of code 100 times or even 1000 times.
We cannot rely 100 percentage on the tools like we saw in the example I mentioned above in case of unit test code coverage. We need to get the balance right between the manual and the automated tools and techniques. There also needs to be right mix between the tools that we use. These tools and techniques are there to help you write better code. If there are 10 tools available in the market we shouldn’t be using each and every one of them. If a particular tool slows you and your team down instead of helping you its better to look for the alternatives. I am sure there are more tools and techniques out there to improve the quality of code code and I would like to learn more about them.

Singleton Design Pattern

One of the simplest and easiest to understand among design patterns is the Singleton Design Pattern. It is a creational pattern which centralizes the creation of an instance of a class. This also helps to ensures that the class has only one instance across application. Singleton provides a global access to an instance of a class.

Problem statement

Imagine we want to ensure that only one instance of a class is created within an application. This class can be a application host class like App in a WPF / Silverlight application or some of the operating system services like the logon service or the service which listens to the mouse or keyboard events. We need to ensure that at any point of time there is only one instance of this services.

Classic Singleton

In order to ensure that there is only one instance of a class across an application boundary, we can restrict the mechanism through which new instances of a class are created. In C#, since we use constructors to create new instances, we can restrict the constructor itself. In general the constructors are mostly publicly accessible. But in case of Singleton implementation we make the constructor as private. This means that we cannot create a new instance of a class using the constructor from any other class. By making the constructor private, we force the class to take the responsibility of creating its own instance. Here is a classic implementation of the Singleton pattern as described in the Gang of Four book.

    public class SimpleSingleton
    {
        private static SimpleSingleton _instance;

        private SimpleSingleton()
        {
        }

        public static SimpleSingleton Instance
        {
            get
            {
                if (_instance == null)
                {
                    _instance = new SimpleSingleton();
                }

                return _instance;
            }
        }
    }

In order to create an instance we create a static property or a method. In this case I have used a property. Note that this is a ready only property. In order to control the number of instances, we have declared a private static member variable. As a convention this private member as well as the public property are both named as _instance and Instance respectively. We can name it anything we would like. The getter check if the _instance is null and creates a new one if that is the case. Otherwise it simply returns the already created instance. This way we have ensured that there is only one instance of this class and also it can be created only using the same class.
This solution works fine in a single threaded model. In todays world where most of the CPU’s are multi core and most applications support multi threading in native ways, this code is bound to create problems. If multiple threads are accessing the Singleton and trying to create an instance at the same time this code is bound to fail in the step where we are trying to check if the _instance is null. One thread might find that the _instance is null and call the private constructor. Before the object is initialized, another thread might try to execute the same code and since the _instance is uninitialized another new instance might be created. This scenario clearly defeats the purpose of creating a Singleton instance.

Thread safe Singleton

To overcome this problem, we can use the synchronization mechanism and control the access to the constructor as shown below.

    public class ThreadSafeSingleton
    {
        private static ThreadSafeSingleton _instance;
        private static object _synchronizingObject = new object();

        private ThreadSafeSingleton()
        {
        }

        public static ThreadSafeSingleton Instance
        {
            get
            {
                if (_instance == null)
                {
                    lock (_synchronizingObject)
                    {
                        if (_instance == null)
                        {
                            _instance = new ThreadSafeSingleton();
                        }
                    }
                }

                return _instance;
            }
        }
    }

This approach makes the Singleton implementation Thread safe. We acquire a lock before creating a new instance. And to be double sure that the instance is not yet initialized, we check for the null value. This double checking prevents two threads from simultaneously accessing the private constructor.

Optimized Singleton

All this seems bit too much just to control instancing of an object. Dotnet framework provides an optimized way of implementing a singleton using readonly variables.

    public sealed class OptimizedSingleton
    {
        private static readonly OptimizedSingleton _instance = newOptimizedSingleton();

        private OptimizedSingleton()
        {
        }

        public static OptimizedSingleton Instance
        {
            get
            {
                return _instance;
            }
        }
    }

By marking the private variable as readonly, we can take care of the thread safety. The readonly variables can only be assigned during static initialization like I have done here or through a constructor. This approach is the simplest among all.

Conclusion

We saw 3 ways of implementing singleton pattern in this post. Since all the 3 approaches are already available here as inline code, this time for a change I have not uploaded the complete working solution. There is only one class involved in singleton apart from the client who uses the singleton instance. In most modern day applications we are using some for of Inversion of Control (IoC) container. These containers have built in support for converting a normal class into a singleton without writing any additional lines of code. You can use this approach demonstrated here to implement singleton pattern if you are one of those persons who do not use any IoC container for object lifecycle management. I would recommend using any of the commonly available IoC container like Spring.Net, StructureMap, Castle Windsor, NInject etc to manage the lifetime of objects as well as to resolve the dependencies among objects. It can save a lot of boilerplate code like the one shown here.

Improve code quality using Visual Studio Code Metrics

Background

In my previous post, I had blogged about improving code quality using software code metrics. In this post I’ll try to demonstrate how to use Visual Studio Team System edition and the code metric feature to improve code quality.

Visual Studio Team System Code Metrics

You can open the solution or project for which the code metrics has to be analyzed. This can be done at the complete solution level or at individual project level.

As can be seen from above screenshot, you can select the Analyse menu option from the menu bar. Towards the bottom of the pulled down menu you can see Calculate Code Metrics for the selected project or Calculate Code Metrics for Solution options.
I have selected the project option and the output is shown below

This shows the various metrics at the project level as well as the individual namespace levels within the project. Except for Depth of Inheritance all other metric are aggregates of the values at each namespace level. This is just the beginning. What I like the most is the drilldown feature where in you can drill down to the method level by navigating through the hierarchy in the following order project –>Namespace –> Class –> Method.

In the above screenshot I have drilled down to the method level. If you look carefully, in the Maintainability Index column you can find a green box next to the value of Maintainability Index. VSTS gives you a visual indicator based on the value of the Maintainability Index at each level. The indicator values based on the range in the following order

• 0-10 : Red which means that the class is very hard to maintain
• 10-20 : Orange / Amber which means that the class is somewhat maintainable
• 20-100 : Green which indicates that the class is maintainable and easier to change

If you observe carefully, the depth of inheritance matrix is not applicable at the method or function level. As mentioned in my earlier blog post, Depth of Inheritance tells you how many classes are inherited by the current class or how many levels of inheritance is involved in implementing a particular class. Hence this metrics is not applicable at the function level.
Lines of code is pretty straightforward metric which tells how many executable lines of code is there in a method or class or namespace level.
Two of the most interesting metrics are the Class Coupling and Cyclomatic Complexity. Class Coupling tells how many unique classes are used to implement a method or function. It also indicates the aggregate of unique classes used within a class implementation at the class level. At the namespace level its the aggregate of unique classes used in all the classes and same applies at the project and solution level. This metrics can be used to split the responsibilities among  classes. If some of the classes are overloaded trying to do too many thing you might get to know using this metric.
Cyclomatic Complexity is one of the most commonly used metric to measure code quality. It indicates the number of decision points within a function or class. To be very frank this requires a dedicated blog post in itself and I might do that sometime in future. For the time being I’ll demonstrate it using a small code snippet.
I have a function which verifies if a string is a valid EAN13 type barcode based on the length of the string which is passed in as a parameter.

private bool ValidateBarcodeLength(string barCode)
        {
            if (barCode.Length == 13)
            {
                return true;
            }

            return false;
        }

I’ll run the code metric and monitor Cyclomatic Complexity for this function.

If you look at the highlighted line above it shows the Cyclomatic Complexity for this function as 2. I’ll do a small refactoring and lets see if it impacts the Cyclomatic Complexity.

 private bool ValidateBarcodeLength(string barCode)
        {
            return barCode.Length == 13;
        }

Again I run the Code Metrics once again.

Immediately the Cyclomatic Complexity drops down by 1. This is a very trivial example but good enough to demonstrate the feature I wanted to show.

Conclusion

One of the best thing with code metrics in Visual Studio Team System is that it integrates nicely into the IDE. You need not open separate windows.






Using these metrics can help us to improve code quality to a good extent. Its very recent that I have started using these metrics. I might soon do a  full fledged post on Cyclomatic Complexity.
Till then Happy Programming :)

Signs that you're a bad programmer

1. Inability to reason about code

Reasoning about code means being able to follow the execution path ("running the program in your head") while knowing what the goal of the code is.

Symptoms

The presence of "voodoo code", or code that has no effect on the goal of the program but is diligently maintained anyway (such as initializing variables that are never used, calling functions that are irrelevant to the goal, producing output that is not used, etc.) Executing idempotent functions multiple times (eg: calling the save() function multiple times "just to be sure") Fixing bugs by writing code that overwrites the result of the faulty code "Yo-Yo code" that converts a value into a different representation, then converts it back to where it started (eg: converting a decimal into a string and then back into a decimal, or padding a string and then trimming it) "Bulldozer code" that gives the appearance of refactoring by breaking out chunks into subroutines, but that are impossible to reuse in another context (very high cohesion)

Remedies

To get over this deficiency a programmer can practice by using the IDE's own debugger as an aide, if it has the ability to step through the code one line at a time. In Visual Studio, for example, this means setting a breakpoint at the beginning of the problem area and stepping through with the 'F11' key, inspecting the value of variables--before and after they change--until you understand what the code is doing. If the target environment doesn't have such a feature, then do your practice-work in one that does. The goal is to reach a point where you no longer need the debugger to be able to follow the flow of code in your head, and where you are patient enough to think about what the code is doing to the state of the program. The reward is the ability to identify redundant and unnecessary code, as well as how to find bugs in existing code without having to re-implement the whole routine from scratch.

2. Poor understanding of the language's programming model

Object Oriented Programming is an example of a language model, as is Functional or Declarative programming. They're each significantly different from procedural or imperative programming, just as procedural programming is significantly different from assembly or GOTO-based programming. Then there are languages which follow a major programming model (such as OOP) but introduce their own improvements such as list comprehensions, generics, duck-typing, etc.

Symptoms

Using whatever syntax is necessary to break out of the model, then writing the remainder of the program in their familiar language's style (OOP) Attempting to call non-static functions or variables in uninstantiated classes, and having difficulty understanding why it won't compile (OOP) Writing lots of "xxxxxManager" classes that contain all of the methods for manipulating the fields of objects that have little or no methods of their own (Relational) Treating a relational database as an object store and performing all joins and relation enforcement in client code (Functional) Creating multiple versions of the same algorithm to handle different types or operators, rather than passing high-level functions to a generic implementation (Functional) Manually caching the results of a deterministic function on platforms that do it automatically (such as SQL and Haskell) Using cut-n-paste code from someone else's program to deal with I/O and Monads (Declarative) Setting individual values in imperative code rather than using data-binding

Remedies

If your skills deficiency is a product of ineffective teaching or studying, then an alternative teacher is the compiler itself. There is no more effective way of learning a new programming model than starting a new project and committing yourself to use whatever the new constructs are, intelligently or not. You also need to practice explaining the model's features in crude terms of whatever you are familiar with, then recursively building on your new vocabulary until you understand the subtleties as well. For example: Phase 1: "OOP is just records with methods" Phase 2: "OOP methods are just functions running in a mini-program with   its own global variables" Phase 3: "The global variables are called fields, some of which are private and invisible from outside the mini-program" Phase 4: "The idea of having private and public elements is to hide implementation details and expose a clean interface, and this is called Encapsulation" Phase 5: "Encapsulation means my business logic doesn't need to be polluted with implementation details" Phase 5 looks the same for all languages, since they are all really trying to get the programmer to the point where he can express the intent of the program without burying it in the specifics of how. Take functional programming as another example: Phase 1: "Functional programming is just doing everything by chaining deterministic functions together" Phase 2: "When the functions are deterministic the compiler can predict when it can cache results or skip evaluation, and even when it's safe to prematurely stop evaluation" Phase 3: "In order to support Lazy and Partial Evaluation, the compiler requires that functions are defined in terms of how to transform a single parameter, sometimes into another function. This is called Currying" Phase 4: "Sometimes the compiler can do the Currying for me" Phase 5: "By letting the compiler figure out the mundane details, I can write programs by describing what I want, rather than how to give it to me"

3. Deficient research skills / Chronically poor knowledge of the platform's features

Modern languages and frameworks now come with an awesome breadth and depth of built-in commands and features, with some leading frameworks (Java, .Net, Cocoa) being too large to expect any programmer, even a good one, to learn in anything less than a few years. But a good programmer will search for a built-in function that does what they need before they begin to roll their own, and excellent programmers have the skill to break-down and identify the abstract problems in their task, then search for existing frameworks, patterns, models and languages that can be adapted before they even begin to design the program.

Symptoms

These are only indicative of the problem if they continue to appear in the   programmer's work long after he should have mastered the new platform. Re-inventing or laboring without basic mechanisms that are built-into the language, such as events-and-handlers or regular expressions Re-inventing classes and functions that are built-into the framework (eg: timers, collections, sorting and searching algorithms) * "Email me teh code, plz" messages posted to help forums "Roundabout code" that accomplishes in many instructions what could be done with far fewer (eg: rounding a number by converting a decimal into a formatted string, then converting the string back into a decimal) Persistently using old-fashioned techniques even when new techniques are better in those situations (eg: still writes named delegate functions instead of using lambda expressions) Having a stark "comfort zone", and going to extreme lengths to solve complex problems with primitives * - Accidental duplication will also happen, proportionate to the size of the framework, so judge by degree. Someone who hand-rolls a linked list might Know What They Are Doing, but someone who hand-rolls their own StrCpy() probably does not.

Remedies

A programmer can't acquire this kind of knowledge without slowing down, and it's likely that he's been in a rush to get each function working by whatever means necessary. He needs to have the platform's technical reference handy and be able to look through it with minimal effort, which can mean either having a hard copy of it on the desk right next to the keyboard, or having a second monitor dedicated to a browser. To get into the habit initially, he should refactor his old code with the goal of reducing its instruction count by 10:1 or more.

4. Inability to comprehend pointers

If you don't understand pointers then there is a very shallow ceiling on the types of programs you can write, as the concept of pointers enables the creation of complex data structures and efficient APIs. Managed languages use references instead of pointers, which are similar but add automatic dereferencing and prohibit pointer arithmetic to eliminate certain classes of bugs. They are still similar enough, however, that a failure to grasp the concept will be reflected in poor data-structure design and bugs that trace back to the difference between pass-by-value and pass-by-reference in method calls.

Symptoms

Failure to implement a linked list, or write code that inserts/deletes nodes from linked list or tree without losing data Allocating arbitrarily big arrays for variable-length collections and maintaining a separate collection-size counter, rather than using a dynamic data structure Inability to find or fix bugs caused by mistakenly performing arithmetic on pointers Modifying the dereferenced values from pointers passed as the parameters to a function, and not expecting it to change the values in the scope outside the function Making a copy of a pointer, changing the dereferenced value via the copy, then assuming the original pointer still points to the old value Serializing a pointer to the disk or network when it should have been the dereferenced value Sorting an array of pointers by performing the comparison on the pointers themselves

Remedies

"A friend of mine named Joe was staying somewhere else in the hotel and I didn't know his room number. But I did know which room his acquaintance, Frank, was staying in. So I went up there and knocked on his door and asked him, 'Where's Joe staying?' Frank didn't know, but he did know which room Joe's co-worker, Theodore, was staying in, and gave me that room number instead. So I went to Theodore's room and asked him where Joe was staying, and Theodore told me that Joe was in Room 414. And that, in fact, is where Joe was." Pointers can be described with many different metaphors, and data structures into many analogies. The above is a simple analogy for a linked list, and anybody can invent their own, even if they aren't programmers. The comprehension failure doesn't occur when pointers are described, so you can't describe them any more thoroughly than they already have been. It fails when the programmer then tries to visualize what's going on in the computer's memory and gets it conflated with their understanding of regular variables, which are very similar. It may help to translate the code into a simple story to help reason about what's going on, until the distinction clicks and the programmer can visualize pointers and the data structures they enable as intuitively as scalar values and arrays.

5. Difficulty seeing through recursion

The idea of recursion is easy enough to understand, but programmers often have problems imagining the result of a recursive operation in their minds, or how a complex result can be computed with a simple function. This makes it harder to design a recursive function because you have trouble picturing "where you are" when you come to writing the test for the base condition or the parameters for the recursive call.

Symptoms

Hideously complex iterative algorithms for problems that can be solved recursively (eg: traversing a filesystem tree), especially where memory and performance is not a premium Recursive functions that check the same base condition both before and after the recursive call Recursive functions that don't test for a base condition Recursive subroutines that concatenate/sum to a global variable or a carry-along output variable Apparent confusion about what to pass as the parameter in the recursive call, or recursive calls that pass the parameter unmodified Thinking that the number of iterations is going to be passed as a parameter

Remedies

Get your feet wet and be prepared for some stack overflows. Begin by writing code with only one base-condition check and one recursive call that uses the same, unmodified parameter that was passed. Stop coding even if you have the feeling that it's not enough, and run it anyway. It throws a stack-overflow exception, so now go back and pass a modified copy of the parameter in the recursive call. More stack overflows? Excessive output? Then do more code-and-run iterations, switching from tweaking your base-condition test to tweaking your recursive call until you start to intuit how the function is transforming its input. Resist the urge to use more than one base-condition test or recursive call unless you really Know What You're Doing. Your goal is to have the confidence to jump in, even if you don't have a complete sense of "where you are" in the imaginary recursive path. Then when you need to write a function for a real project you'd begin by writing a unit test first, and proceeding with the same technique above.

6. Distrust of code

Symptoms

Writing IsNull() and IsNotNull(), or IsTrue(bool) and IsFalse(bool) functions Checking to see if a boolean-typed variable is something other than true or false

Remedies

Are you being paid by the line? Are you carrying over old habits from a language with a weak type system? If neither, then this condition is similar to the inability to reason about code, but it seems that it isn't reasoning that's impaired, but trust and comfort with the language. Some of the symptoms are more like "comfort code" that doesn't survive logical analysis, but that the programmer felt compelled to write anyway. The only remedy may be more time to build up familiarity.

Signs that you are a mediocre programmer

1. Inability to think in sets

Transitioning from imperative programming to functional and declarative programming will immediately require you to think about operating on sets of data as your primitive, not scalar values. The transition is required whenever you use SQL with a relational database (and not as an object store), whenever you design programs that will scale linearly with multiple processors, and whenever you write code that has to execute on a SIMD-capable chip (such as modern graphics cards and video game consoles).

Symptoms

The following count only when they're seen on a platform with Declarative or Functional programming features that the programmer should be aware of. Performing atomic operations on the elements of a collection within a for or foreach loop Writing Map or Reduce functions that contain their own loop for iterating through the dataset Fetching large datasets from the server and computing sums on the client, instead of using aggregate functions in the query Functions acting on elements in a collection that begin by performing a new database query to fetch a related record Writing business-logic functions with tragically compromising side-effects, such as updating a user interface or performing file I/O Entity classes that open their own database connections or file handles and keep them open for the lifespan of each object

Remedies

Funny enough, visualizing a card dealer cutting a deck of cards and interleaving the two stacks together by flipping through them with his thumbs can jolt the mind into thinking about sets and how you can operate on them in bulk. Other stimulating visualizations are: freeway traffic passing through an array of toll booths (parallel processing) springs joining to form streams joining to form creeks joining to form rivers (parallel reduce/aggregate functions) a newspaper printing press (coroutines, pipelines) the zipper tag on a jacket pulling the zipper teeth together (simple joins) transfer RNA picking up amino acids and joining messenger RNA within a ribosome to become a protein (multi-stage function-driven joins, see animation) the above happening simultaneously in billions of cells in an orange tree to convert air, water and sunlight into orange juice (Map/Reduce on large distributed clusters) If you are writing a program that works with collections, think about all the supplemental data and records that your functions need to work on each element and use Map functions to join them together in pairs before you have your Reduce function applied to each pair.

2. Lack of critical thinking

Unless you criticize your own ideas and look for flaws in your own thinking, you will miss problems that can be fixed before you even start coding. If you also fail to criticize your own code once written, you will only learn at the vastly slower pace of trial and error. This problem originates in both lazy thinking and egocentric thinking, so its symptoms seem to come from two different directions.

Symptoms

Homebrew "Business Rule Engines" Fat static utility classes, or multi-disciplinary libraries with only one namespace Conglomerate applications, or attaching unrelated features to an existing application to avoid the overhead of starting a new project Architectures that have begun to require epicycles Adding columns to tables for tangential data (eg: putting a "# cars owned" column on your address-book table) Inconsistent naming conventions "Man with a hammer" mentality, or changing the definitions of problems so they can all be solved with one particular technology Programs that dwarf the complexity of the problem they solve Pathologically and redundantly defensive programming ("Enterprisey code") Re-inventing LISP in XML

Remedies

Start with a book like Critical Thinking by Paul and Elder, work on controlling your ego, and practice resisting the urge to defend yourself as you submit your ideas to friends and colleagues for criticism. Once you get used to other people examining your ideas, start examining your own ideas yourself and practice imagining the consequences of them. In addition, you also need to develop a sense of proportion (to have a feel for how much design is appropriate for the size of the problem), a habit of fact-checking assumptions (so you don't overestimate the size of the problem), and a healthy attitude towards failure (even Isaac Newton was wrong about gravity, but we still love him and needed him to try anyway). Finally, you must have discipline. Being aware of flaws in your plan will not make you more productive unless you can muster the willpower to correct and rebuild what you're working on.

3. Pinball Programming

When you tilt the board just right, pull back the pin to just the right distance, and hit the flipper buttons in the right sequence, then the program runs flawlessly with the flow of execution bouncing off conditionals and careening unchecked toward the next state transition.

Symptoms

One Try-Catch block wrapping the entire body of Main() and resetting the program in the Catch clause (the pinball gutter) Using strings/integers for values that have (or could be given) more appropriate wrapper types in a strongly-typed language Packing complex data into delimited strings and parsing it out in every function that uses it Failing to use assertions or method contracts on functions that take ambiguous input The use of Sleep() to wait for another thread to finish its task Switch statements on non-enumerated values that don't have an "Otherwise" clause Using Automethods or Reflection to invoke methods that are named in unqualified user input Setting global variables in functions as a way to return multiple values Classes with one method and a couple of fields, where you have to set the fields as the way of passing parameters to the method Multi-row database updates without a transaction Hail-Mary passes (eg: trying to restore the state of a database without a transaction and ROLLBACK)

Remedies

Imagine your program's input is water. It's going to fall through every crack and fill every pocket, so you need to think about what the consequences are when it flows somewhere other than where you've explicitly built something to catch it. You will need to make yourself familiar with the mechanisms on your platform that help make programs robust and ductile. There are three basic kinds: those which stop the program before any damage is done when something unexpected happens, then helps you identify what went wrong (type systems, assertions, exceptions, etc.), those which direct program flow to whatever code best handles the contingency (try-catch blocks, multiple dispatch, event driven programming, etc.), those which pause the thread until all your ducks are in a row (WaitUntil commands, mutexes and semaphores, SyncLocks, etc.) There is also a fourth, Unit Testing, which you use at design time. Using these ought to become second nature to you, like putting commas and periods in sentences. To get there, go through the above mechanisms (the ones in parenthesis) one at a time and refactor an old program to use them wherever you can cram them, even if it doesn't turn out to be appropriate (especially when they don't seem appropriate, so you also begin to understand why).

4. Unfamiliar with the principles of security

If the following symptoms weren't so dangerous they'd be little more than an issue of fit-n-finish for most programs, meaning they don't make you a bad programmer, just a programmer who shouldn't work on network programs or secure systems until he's done a bit of homework.

Symptoms

Storing exploitable information (names, card numbers, passwords, etc.) in plaintext Storing exploitable information with ineffective encryption (symmetric ciphers with the password compiled into the program; trivial passwords; any "decoder-ring", homebrew, proprietary or unproven ciphers) Programs or installations that don't limit their privileges before accepting network connections or interpreting input from untrusted sources Not performing bounds checking or input validation, especially when using unmanaged languages Constructing SQL queries by string concatenation with unvalidated or unescaped input Invoking programs named by user input Code that tries to prevent an exploit from working by searching for the exploit's signature Credit card numbers or passwords that are stored in an unsalted hash

Remedies

The following only covers basic principles, but they'll avoid most of the egregious errors that can compromise an entire system. For any system that handles or stores information of value to you or its users, or that controls a valuable resource, always have a security professional review the design and implementation. Begin by auditing your programs for code that stores input in an array or other kind of allocated memory and make sure it checks that the size of the input doesn't exceed the memory allocated for storing it. No other class of bug has caused more exploitable security holes than the buffer overflow, and to such an extent that you should seriously consider a memory-managed language when writing network programs, or anywhere security is a priority. Next, audit for database queries that concatenate unmodified input into the body of a SQL query and switch to using parameterized queries if the platform supports it, or filter/escape all input if not. This is to prevent SQL-injection attacks. After you've de-fanged the two most infamous classes of security bug you should continue thinking about all program input as completely untrustworthy and potentially malicious. It's important to define your program's acceptable input in the form of working validation code, and your program should reject input unless it passes validation so that you can fix exploitable holes by fixing the validation and making it more specific, rather than scanning for the signatures of known exploits. Going further, you should always think about what operations your program needs to perform and the privileges it'll need from the host to do them before you even begin designing it, because this is the best opportunity to figure out how to write the program to use the fewest privileges possible. The principle behind this is to limit the damage that could be caused to the rest of the system if an exploitable bug was found in your code. In other words: after you've learned not to trust your input you should also learn not to trust your own programs. The last you should learn are the basics of encryption, beginning with Kerckhoff's principle. It can be expressed as "the security should be in the key", and there are a couple of interesting points to derive from it. The first is that you should never trust a cipher or other crypto primitive unless it is published openly and has been analyzed and tested extensively by the greater security community. There is no security in obscurity, proprietary, or newness, as far as cryptography goes. Even implementations of trusted crypto primitives can have flaws, so avoid implementations you aren't sure have been thoroughly reviewed (including your own). All new cryptosystems enter a pipeline of scrutiny that can be a decade long or more, and you want to limit yourself to the ones that come out of the end with all their known faults fixed. The second is that if the key is weak, or stored improperly, then it's as bad as having no encryption at all. If your program needs to encrypt data, but not decrypt it, or decrypt only on rare occasions, then consider giving it only the public key of an asymmetric cipher key pair and making the decryption stage run separately with the private key secured with a good passphrase that the user must enter each time. The more is at stake, then the more homework you need to do and the more thought you must put into the design phase of the program, all because security is the one feature that dozens, sometimes millions of uninvited people will try to break after your program has been deployed. The vast majority of security failures traceable to code have been due to silly mistakes, most of which can be avoided by screening input, using resources conservatively, using common sense, and writing code no faster than you can think and reason about it.

5. Code is a mess

Symptoms Doesn't follow a consistent naming convention Doesn't use indentation, or uses inconsistent indentation Doesn't make use of whitespace elsewhere, such as between methods (or expressions, see "ANDY=NO") Large chunks of code are left commented-out Remedies Programmers in a hurry (or The Zone) commit all these crimes and come back to clean it up later, but a bad programmer is just sloppy. Sometimes it helps to use an IDE that can fix indentation and whitespace ("pretty print") with a shortcut key, but I've seen programmers who can even bludgeon Visual Studio's insistence on proper   indentation by messing around with the code too much.

Signs that you shouldn't be a programmer

The following may not have any remedies if you still suffer from them after taking a programming course in school, so you will stand a better chance of advancing your career by choosing another profession.

1. Inability to determine the order of program execution

Symptoms

a = 5 b = 10 a = b   print a You look at the code above and aren't sure what number gets printed out at the end

Alternative careers

Electrician Plumber Architect Civil engineer Artist

2. Insufficient ability to think abstractly

Symptoms

Difficulty comprehending the difference between objects and classes Difficulty implementing design patterns for your program Difficulty writing functions with low cohesion Incompetence with Regular Expressions Lisp is opaque to you Cannot fathom the Church-Turing Thesis

Alternative careers

Contract negotiator Method actor

3. Collyer Brothers syndrome

Symptoms

Unwilling to throw away anything, including garbage Unwilling to delete anything, be it code or comments The urge to build booby-traps for defense against trespassers Unwilling to communicate with other people Poor organization skills

Alternative careers

Antique dealer Bag lady

4. Dysfunctional sense of causality

Symptoms

You seriously consider malice to be a reason why the compiler rejects your program When called on to fix a bug in a deployed program, you try prayer You take hidden variables for granted and don't think twice about blaming them for a program's misbehavior You think the presence of code in a program will affect its runtime behavior, even if it is never invoked * Your debugging repertoire includes rituals like shining your lucky golf ball, twisting your wedding ring, and tapping the nodding-dog toy on your monitor. And when the debugging doesn't work, you think it might be because you missed one or didn't do them in the right order * - Memory constraints, shifted offsets, and compiler peculiarities notwithstanding. See discussion on Reddit. Judge accordingly.

Alternative careers

Playing the slot machines in Vegas

Contrapositives

What Makes a Good Programmer by Cam Riely

5. Indifference to outcomes

Programming could still be a hobby for you, but it would be in society's best interests to defend itself against your entry into the world of professional software development.

Symptoms

You aren't interested in fixing a bug that can be worked around by rebooting the computer Your installation program silently deploys unsolicited third party programs that are unrelated to the function of yours * You don't use any ergonomic model when designing user interfaces, nor do you have any interest in usability studies Your program exhibits pretension and grandeur beyond its utility, eg: displaying splash screens over active programs while loading in the background, or placing multiple launch icons in premium desktop        locations * Your program produces output to be read by another (eg: a browser), or implements a network protocol, and relies on the other party's software to be significantly tolerant to spec violations You write busy-wait loops even when the platform offers event-driven programming You don't use managed languages and can't be bothered to do bounds checking or input validation Your user interfaces do not make the difficulty of accidentally invoking a function proportionate to its destructiveness (eg: the "Delete Database" button is next to "Save", just as big, has no confirmation step and no undo) You don't use whitespace, indentation or comments * - These are actually imposed by management more often than by the programmer, who only implements them. We'd still group them together for the sake of this self-test, though, and at the most suggest that one seek employment at a better firm, while the other goes back to business school to learn less destructive ways of making a profit.

Alternative careers

Debt collection Telemarketing