Imprudence Begets Impetus - No Go For GoDaddy

It's funny how unrelated things can finally give you the motivation to check off an item that's been on the TODO list for a while. In this case it was the online movement to boycott GoDaddy, due to their stance on SOPA.

One of the items I've been meaning to get to Real Soon Now, was to register a domain for Mutability Detector. If for no other reason than making the org.mutabilitydetector package name in the source code actually match Java conventions.

Since I had no existing GoDaddy accounts, I couldn't lend my support to the boycott by moving my domains away. But I could vote with my wallet and go with a competitor. Enter Namecheap, another domain name registrar like GoDaddy, but with a much more amenable outlook on SOPA. Since this was the first domain name purchase I've made, and I'm currently only using it as a redirect, I can't comment on the service in too much detail. But the process was cheap and convenient and their interface was quite friendly and easy to use.

I might do more with www.mutabilitydetector.org in the future, but for now it just redirects to the project homepage, and gives me an ever-so-slight warm and fuzzy for doing a teensy-tiny bit in the efforts against SOPA.

Version 0.7 Released

Mutability Detector v0.7 has been released. It is available for download at the project homepage, or from Maven Central with the details:

Group ID	Artifact ID	Version
org.mutabilitydetector	MutabilityDetector	0.7

This release included improvements in two areas: improve support allowing reasons for mutability in unit tests,  and the introduction of analysis on the 'this' reference escaping. It also includes more documentation to help get started with unit testing for immutability (view online or as JavaDoc embedded in the sources jar).

Other more minor improvements include:

• improving the semantics of the overall result for a class
• adding an 'allowing for subclassing' reason

Any and all feedback welcome.

Also, if you do use Mutability Detector, and your source is accessible, please get in touch. When releasing new versions I want to have as little inconvenience for users as possible, and currently I only judge this using a pretend client codebase. Potentially I could add your project to my continuous integration setup, and test my trunk code against yours to give more realistic feedback.

There's already a couple of issues I want to include in the next release (issues 17, 18, 19 at least) and of course, there's always "The String Problem", but if there's anything that's affecting you, please do get in touch.

New EscapedThisReferenceChecker due for next release

In a previous post I discussed a new check that Mutability Detector should make to provide a more accurate result. The implementation of that checker is now complete, and will be included in an upcoming 0.7 release.

As with any other checks the tool does, the analysis is unsound, that is, it doesn't guarantee correctness. It makes as best a guess as it can, and I hope that empirical study of its use would show that it's good enough to rely on, but it is possible that the tool can be fooled into emitting false positives or false negatives. I thought it would be useful to talk about the checker does and doesn't do.

The checker analyses every constructor of your class, and checks for the following problems.


Assigning 'this' to a field
The tool will render your class mutable if it does any of the following:

1. assigns 'this' to static field 
• SomeOtherClass.staticField = this;
2. assigns 'this' to instance of other object
• someOtherObject.instanceField = this;
3. assigns 'this' to field of this instance
• this.myField = this;
4. assigns 'this' to static field of this class
• ThisClass.staticField = this;

Some of these will be a false positive, e.g. with #2, if you construct an object in the constructor, and assign 'this' to one of its fields, before it's published, it'll be safe.  Also assigning 'this' to an instance field is safe as well (or at least, I can't think why it is unsafe), but it's weird enough that I don't want to lose too much sleep trying to be correct about it.


Passing 'this' as a parameter

The tool will render your class mutable if it does any of the following:

1. passes 'this' as parameter to another method
• new SomeOtherClass(this);
• SomeOtherClass.staticMethod(this);
• someOtherObject.instanceMethod(this);
2. constructs an inner class (which has an implicit reference to 'this')
   
• new InnerClass()

A false positive in this case could be, with #1, passing 'this' to somewhere that only saves the reference, without trying to read any values from it, or with #2, the instance of the inner class might not escape, meaning the 'this' reference hasn't escaped.

Unexpected false positives
The last examples of false positives are ones that can occur with a correct implementation. However, there's one false positive that I didn't expect to create. While the tool is trying to check for 'this' being passed to other methods, it also accidentally raises an issue when instance fields are passed as well.


Here's an example from java.util.Vector:

public Vector(Collection c) {
    elementData = c.toArray();
    elementCount = elementData.length;

    if (elementData.getClass() != Object[].class)
        elementData = Arrays.copyOf(elementData, elementCount, Object[].class);
    }

}



The issue being the call to Arrays.copyOf() with two instance fields. This piece of code will cause Vector to be reported as mutable (among many other reasons), and quite nicely demonstrates the issues involved. Here, the method is single threaded, but it hypothetically could change at any point to publish the references it receives to multiple threads. Without further (expensive) analysis into what Arrays.copyOf() does, I can't conclude whether Vector is immutable or not. I'm choosing to err on the side of caution. There is the scope for a potential whitelist of safe methods, but that's not on the cards just yet.


Another interesting point is that one field is a primitive int, the other is an array. Since the value of the int will be copied to the method, it's probably safe anyway, no matter whether threads are involved or not. However, for the array, the reference to the field will be passed, not the contents. If the array is not completely populated and made visible when it's passed, the receiving method could observe it to change, breaking immutability. This leads on to the wider question of any mutable fields escaping at any point in the object's life cycle, not just in the constructor. For now, I'm embracing the happy accident that the checker will warn you of some potentially dodgy code, and will deliberately leave that particular false positive active.


I don't expect any of these false positives to really cause an issue If you disagree, get in touch, email me, leave a comment, or raise a bug (that may require sign in).



I expect to release a new version within the next week, stay tuned for more news!

Introducing Google Group

A Google Group is now available for questions, discussions or abuse, relating to Mutability Detector. The pertinent details of the group are:

• Group name: mutability-detector
• Group home page: http://groups.google.com/group/mutability-detector
• Group email address mutability-detector@googlegroups.com

Don't be shy!

A better way of categorising immutability?

Currently Mutability Detector categorises a class' immutability as one of the following:


public static enum IsImmutable {
COULD_NOT_ANALYSE,
DEFINITELY,
MAYBE,
DEFINITELY_NOT;
}

For example:

 java.util.Date is DEFINITELY_NOT immutable.    Reasons:
        (... some reasons ommitted for brevity ...)
        Field [fastTime] can be reassigned within method [setTime]

Putting aside the general error case of COULD_NOT_ANALYSE, the three main categories are kind of... unsatisfying.

For example, if your class has a primitive array field, it will be classed as MAYBE immutable. In this case it's due to the tool being naive and not doing the kind of analysis it could, and there were so many false positives being generated. At the time I thought it was a pragmatic (read "weasel") way to have my cake and eat it too - I got to reduce false positives, without the effort of making the analysis more powerful. The problem is that it's not particularly helpful. When Joe Hypothetical runs the analysis, what is he supposed to concur from seeing that his class is "maybe immutable"?

Another example is the DEFINITELY category. In the discussion following a previous blog post it was pointed out that where I had used a class I considered DEFINITELY immutable, a commenter pointed out that under certain threading conditions, it could be seen to mutate. These categories were not being used to communicate certain, specific, well defined, and ultimately useful information.

So I've been thinking about having different categories, that will hopefully be more useful. I've borrowed a couple, which, because the book is so darn good, are based on the ideas and semantics from Java Concurrency in Practice.

• IMMUTABLE 
• EFFECTIVELY_IMMUTABLE
• NOT_IMMUTABLE
• COULD_NOT_ANALYSE

Immutable is the strictest category of immutability. All fields are final and the class is final. All fields are immutable (a 'turtles all the way down' kind of arrangement). Instances of this class can be published in any way, under any threading conditions, as the Java Memory Model guarantees that writes to the final field Happens-Before any reads from that field. Implicitly thread safe. Issues include: that old favourite - lazily loading fields that are expensive to compute. Detecting benign data races is Hard. I may have to hard code some common cases, e.g. java.lang.String/java.math.BigDecimal.

Instances of effectively immutable classes can be safely shared across threads, as long as they are safely published (JCIP covers what 'safely published' means in more detail). The class doesn't need to be final. Fields don't need to be final, but they can only be, and must be, assigned in the constructor or private methods called only by the constructor. Fields should all be immutable, effectively immutable, or mutable-but-never-mutated, meaning for example, a field can be of type java.util.List, or an array, but as long as it isn't mutated after construction, it's fine. This includes allowing references to mutable instances to escape, e.g. returning the array field from a method call without copying first. Issues include: the 'method called only from constructor' clause allows for serialisation... but should it? Also, confidently identifying fields which are mutable but never mutated is non-trivial.

Not immutable is everything else: interfaces and abstract classes, classes whose fields can be reassigned after construction, whose mutable fields are published, which assign a mutable type (e.g. List or an array) to a field. The analysis will still need to get better at detecting valid and common patterns, but the more esoteric the code gets, the more likely the analysis is just going to throw it's hands up in the air in desperation and admit mutability.

This scheme, would represent a slight change in direction for the analysis. I can remember when the code was thrown together as part of a uni assignment, that the requirements for being immutable were very strict, and since then I've tried to improve the analysis to allow for more leniency. Now I'm starting to think that strictness may be more of a strength - particularly since it would be possible to manually 'override' the result using a flexible and fluent API available in unit test assertions.

So not a particularly ground breaking or earth shattering change suggestion, but hopefully something that could potentially be more useful, communicative and more broadly understood.  As usual comments and suggestions welcome, thanks for reading.

Announcing *cough* Better Maven Support

In a previous post I gave details of how to retrieve Mutability Detector in a build system such as Maven. It included details of the snapshot repository that had to be added to your pom.xml (or equivalent file), and the specifier for the latest SNAPSHOT version - just for extra complicatedness.

While trying to use my own project at work, where we use Gradle as a build system, I found that retrieving a snapshot dependency from a Maven repository was not particularly well supported. Since it at best required some non-default behaviour (if it's at all supported, I never found a solution), I leapt into action, much like a ninja mongoose, determined to rectify the situation immediately. So, just a short(!) 8 days later, I'm proud to announce that Mutability Detector is available from Maven Central, as a bonafide release version.

The relevant details now are:

Group ID	Artifact ID	Version
org.mutabilitydetector	MutabilityDetector	0.6

Mutability Detector is packaged as a single JAR, which can be used for runtime analysis, as an assertion mechanism in unit testing, or as a standalone application launched from the command line.

The 0.6 release which made it to Maven Central also fixed two issues I mentioned in the previous post (discussed here, and here), which are fairly significant.

As always, if you're a user with any questions or issues, please let me know. Well, if you're a user with no issues, it would be nice to hear that you actually exist...

Announcing Maven support for the latest version of Mutability Detector

Mutability Detector is now available for Maven users*. If you want to try it out on your own code, do the following:

Make the Sonatype Snapshot repository available to your build system. In Maven, this means adding the following snippet within the <project> section of your pom.xml.

<repositories>
  <repository>
    <id>Sonatype Snapshots</id>  
    <url>http://oss.sonatype.org/content/repositories/snapshots</url>
  </repository>
</repositories>

Next add the Mutability Detector dependency to your pom.xml with this snippet:

<dependency>
  <groupId>org.mutabilitydetector</groupId>
  <artifactId>MutabilityDetector</artifactId>
  <version>0.6-SNAPSHOT</version>
  <scope>test</scope>
</dependency>

For other build systems (Gradle, Ivy, etc.) the pertinent bits of information for the dependency are:

Group ID	Artifact ID	Version
org.mutabilitydetector	MutabilityDetector	0.6-SNAPSHOT

Some points:

• Source code is not available through Maven, working on fixing this. In the meantime, you can checkout or browse the source, details available at the Googlecode project page.
• You may get surprising results with the current snapshot, as described in this issue
• The packaged jar file includes dependencies (e.g. ASM, Classpath Explorer), so there may be some issues putting this jar on your classpath. Working on fixing this also.

As always, any questions, please ask.

Lightning Talk Introducing Mutability Detector

Here's a video of a lightning talk I gave at work, introducing Mutability Detector to my colleagues. I had to cut it short so as not to break the rules of a maximum 5 minute duration, but I think it gives a good overview. If you find my Scots accent tough to understand, you might prefer to check out a couple of the previous blogs, particularly the introduction to the first version released, and an overview of a more recent release.



The Mutability Detector from youdevisetech on Vimeo.