Mutability Detector Blog

Discussing the design, decisions and discovery involved in developing Mutability Detector, an open-source analysis tool for Java.

Saturday, 22 March 2014

New release with JDK 8 support

A new version of Mutability Detector, which supports JDK 8 bytecode,  has been released to Maven Central. Simply bump the version number from 0.9 to 0.9.1 in your build system, or hop over to the project's downloads page to grab the latest artifacts.

Sunday, 15 September 2013

Supporting FindBugs v2.0.0+

Over the weekend, a whopper of a bug was reported against MutabilityDetector4FindBugs - the custom detector used to check your classes annotated with @Immutable. Turns out that it only worked on FindBugs v1.3.9, throwing exceptions on v2.0.0+. FindBugs v2.0.0 was released in December 2011, so it's not exactly recent. This is what happens when you don't dogfood (I generally use Mutability Detector in unit tests, not through FindBugs) anyhoo...

The bug has been fixed, but due to source incompatibilities, I am unable to support multiple versions of FindBugs with a single jar. As such, if you want to use MutabilityDetector4FindBugs, you'll have to grab the jar which is appropriate to your installation of FindBugs. Hop over to the project's README for details on how to get the right version.

Wednesday, 28 August 2013

Meet A Project: An Opportunity to Join An Open Source Project

Are you interested in getting involved in open source development? Maybe you want to improve your skills, raise your chance of landing a job, or just get a taste of what it's like to collaborate in the open source model. However, there's always that problem: "how to get involved"? Scratching your own itch is how many projects are started, but that doesn't guarantee that you'll get to work with others. Finding an existing, active project will let you work with others, but taking that first step can be daunting. You need a place where:

  • You can ask questions without feeling intimidated, because you need to be able to learn and make mistakes.
  • There are some people who are eager to help you get started, because thrashing around on your own in an alien codebase can be such a drag.
  • You can work on real issues and features, not toy ones, because working on something that actually helps the project is a vital part of the experience.
What you need is a project that ticks those boxes. Mutability Detector is one of those projects. Based on the concept of the London Java Community's "Meet-A-Project", I want to invite you, potential contributor, to get involved.

What is Mutability Detector?

A Java library that analyses classes to find out if they are immutable. Using immutability is a popular technique for improving readability and robustness of code. Immutability is built into newer JVM languages like Scala and Clojure, but takes a little more effort to get it right in Java. Mutability Detector helps Java developers implement immutability correctly.

It uses git for version control, hosted on GitHub. It uses Maven for a build tool and CloudBees for continuous integration. The code has been written using test driven development and uses such libraries as ASM and Guava. Mutability Detector has various ways of being executed, but it's mostly standalone; there's no databases or web pages involved. It's intended to work from the command line, work within any and all unit testing frameworks like JUnit and TestNG, and work as a plugin within FindBugs.

If none of that means anything to you, Don't Panic, I can help explain all this stuff. If you're interested in coding in Java, and eager to learn, we can work on picking up that knowledge. Read on.

What is there to work on?

There are three issues in Mutability Detector's issue tracker on GitHub that I intend to get fixed, one way or the other. Two issues are new features that I think would help more users, thus improving popularity of the library. The third is an issue raised by an end user, that, although it has a workaround, would make using the library easier. Alongside the primary goal of getting these features implemented, and that bug fixed, I also want to increase contribution from others. Each of these issues are small enough that we can make progress without giving up our day jobs or taking a sabbatical, but not so small as to be trivial. If a potential employer asks in an interview what your involvement in this project was, you will be able to say "I had a major part in implementing Feature X", rather than, "I fixed a typo in the documentation"[0].

New features:
Again, unless you're a user, those descriptions may mean little to you. Don't Panic.

What's in it for me, as project owner?

To be blunt, I could implement these features myself. As the main developer, I understand the problems, and how to solve them. Even if other people write the code, I'm still likely to spend more time helping others write the code than I would just writing it myself. So why bother? Well, the most selfish reason is that I want experience of technical leadership. I have not had a technical lead role in my career yet (in my whopping 3 years of experience), and want to experience the challenges of mentoring, coaching and directing others. I also expect to learn something from everyone who gets involved, no matter their experience level. And I think it will be fun.

Also, what's the worst that could happen? Nobody responds and I implement the features myself? Big whoop, I'm no worse off. I spend some time helping people contribute and they lose interest? Ah, so what. I accept a contribution and a bug slips into the code? Not the end of the world, we're not developing heart monitors or air traffic control systems. I want open source development to be fun and interesting.

What's in it for you, as a contributor?

Like you, I have wanted to get involved in open source projects and either felt intimidated, didn't know where to begin, or didn't know how to be useful. Here I'm listing several features, all of which I want implemented. Hopefully I'm also making it clear that I am opening myself up to lots of questions, so you shouldn't feel shy about asking. There will be no Linus Torvalds-style flaming here. We may disagree, and I may criticise your code, or reject it until I'm happy to incorporate it, but I think that's fine -- that's collaboration.

If you are struggling to make the move into Java, you can learn by studying how someone else writes it. Not that I'm saying my code is perfect, it's most definitely not, but it can give you a frame of reference, and I'm happy to explain why I wrote code a certain way, or made a particular design choice.

Since I'm strict about it, any code which is eventually accepted will:

So if you don't know what those things are, you'll learn about them in relation to actual code.

If you are looking for something to put on a CV, contributing to open source can help distinguish you from other candidates. It's not just about coding, it's about the collaboration aspect of it. An employer being able to see not just how you code, but how you communicate with people, helps take the risk out of the hiring decision. Fun fact: I wrote and released the first version of Mutability Detector before leaving university; and it was one of the factors in securing a job to go to after I graduated. An interviewer at TIM Group told me he looked at my code and was impressed by the quality.

So how to get started?

Post to the mailing list and announce your interest in contributing, tell me a little about yourself. Read this blog, and try to use Mutability Detector. Try to get a feel for its purpose; run it on some of your own code to see the results. That will help you understand the features. Even just understanding what this project is for is beyond beginner-level Java.

Follow the project README on GitHub, fork, clone and build the project, and submit a super-tedious pull request (like fixing a typo, or renaming a variable). That will get you familiar with building the project, importing it into your development environment, and going through the code submission process.

From there, pick one of the features you are interested in contributing to, and begin working on it. If, at any stage of the process, you hit a bump, ask for help on the project mailing list. I don't mind how 'basic' you think the question is, if it prevents you getting involved, I'm happy to give you pointers. Stuff like "how to install Java?", or "what does this error mean?" are fine. I can't promise I will be a great teacher, but I'll try.

I look forward to hearing from you.




[0] patches which fix typos are very much welcome, they are an important contribution -- they just don't say much about you as a software developer.



Sunday, 27 January 2013

v0.9 released, java.lang.String now considered immutable!

In the past hour I just released a new version of Mutability Detector, v0.9. This release is quite significant. If you currently use Mutability Detector, I'd highly recommend upgrading. If you've never used it before, there's never been a better time to try*.

* I reserve the right to make this statement on all future releases ;-)


If I've already convinced you, here's the snippet you'll need for your pom.xml:


    <dependency>
      <groupid>org.mutabilitydetector</groupid>
      <artifactid>MutabilityDetector</artifactid>
      <version>0.9</version>
      <scope>test</scope>
    </dependency>


If I haven't convinced you yet, here's some of the new features that might get you there:

  • you can now use java.lang.String in your immutable classes, without your test failing
  • analysis recognises safe usage of Collections.unmodifiableXXX methods
  • new out-of-the-box "allowed reasons" to help you suppress false positives, in a readable and safe way
  • using the Effective Java-style builder pattern to create immutable objects should now not fail tests
  • we migrated from Google Code to GitHub, so we're in with the cool kids now, right?
  • the usual host of bug fixes and improvements

The String Problem

Finding java.lang.String to be mutable has long been a problem for Mutability Detector, recently rearing it's ugly head during a case study of ThreeTen, the new Date&Time API for Java 8. Although the general case of benign data races has still not been solved, I have taken the pragmatic choice to hard code it as immutable. Similarly for other JDK types, such as the primitive wrappers (Integer, Double, Character, etc), BigDecimal, BigInteger, and Class.

The way String et al have been hardcoded has been exposed to users, so that if you have false positives "tainting" your codebase, you too can tell Mutability Detector to regard certain types as immutable. For further information on this configuration, refer to the JavaDoc.

Unmodifiable Collections

Classes that follow this pattern:

public final class HasUnmodifiableCollection {
    private final List<String>myStrings;
 
    public HasUnmodifiableCollection(List<String> original) {
        this.myStrings = Collections.unmodifiableList(new ArrayList<String>(original));
    }
    // ...
}

... are now considered immutable. Provided you use the methods from the standard JDK to copy (the new ArrayList call) and wrap in an unmodifiable collection (the unmodifiableList call) an error won't be raised. That condition is rather strict, for example, I personally use Guava's Lists.newArrayList method to copy, and that is still flagged as an issue. Let me know how useful it is for you.


Those two additions should hopefully make a major difference in the usability of Mutability Detector. Feedback, as always, is welcome.

What's Next?

That kinda depends on the feedback I get, bug fixes that are getting in your way will likely be top priority. More ambitious plans include: allowing mutable fields, provided they are not mutated and don't escape; studying popular projects like Guava to do more case studies, and/or bundle more useful configuration.

Once the release hits Maven Central, I'll also push out a new version of the FindBugs plugin.

Let me know what you'd like to see being worked on!


Sunday, 9 December 2012

Mutability Detector has relocated.

With thanks to some fine work from Mani (@neomatrix369) Mutability Detector has been successfully moved to GitHub. The main motivation for this was to make collaboration easier, so hopefully those pull requests can start flooding in :-)

The GitHub project is available at:

    https://github.com/MutabilityDetector/MutabilityDetector

mutabilitydetector.org now points to the GitHub Pages project at the same repository.

If you have considered fixing or adding something to Mutability Detector, it has never been easier, so get involved!

Sunday, 25 November 2012

Mutability Detector In Action

Quick video demonstrating Mutability Detector In Action


Thanks to Jon Dickinson for the video editing.

Thursday, 19 July 2012

ThreeTen Case Study: A Time (And Date) To Reflect

We started this series of blogs with 55 test cases, each using Mutability Detector to assert immutability of classes within ThreeTen. 6 of those test cases were failing, and worth investigating. We saw how Mutability Detector can prompt a developer to carefully analyse those classes, and if those classes are actually immutable, how those warnings can be easily and flexibly suppressed. 4/6 of those failures were false positives, and the tests were adapted to suppress those warnings. 2/6 were considered real problems, and have since been fixed in the master branch of ThreeTen.

From this case study, there are several things I have learned about Mutability Detector.

A high pass rate is possible
32 tests passed without suppressing warnings, 23 required suppressions. I am particularly pleased with this success rate, given the strictness of the tool.

The String Problem is both surprising and annoying
An issue that has consistently suprised users is that java.lang.String is found to be mutable. Users don't particularly care that String has a benign data race where its hash field is lazily calculated and assigned outside the constructor. They only care that String is so often held up as an example of an immutable Java class.

As well as being surprising, it causes problems even when you understand what causes it. For example, ThreeTen's ZoneOffset has a String field which requires a suppression like "provided(String.class).isAlsoImmutable()" in the test case. But more annoying than that, ZoneOffset's supposed mutability flows back to the other classes which have a ZoneOffset field. That causes an extra 5 suppressions in the tests for classes which have a ZoneOffset field.

I am now fairly convinced that automatically detecting that String is immutable, which involves correctly deciding if a data race is benign, is just too damn difficult. I have started work to allow hardcoding String as immutable in Mutability Detector. Since ThreeTen has shown that benign data races/internal caching is not limited to String, there should also be an out-of-the-box suppression for this particular use case. 

Panicking at the sight of a mutable type is simplistic
Mutability Detector does not yet allow the safe, encapsulated use of a mutable type. When your class has a field which is a mutable type (or an array) it will raise an error. Checking for non-mutating calls is non-trivial, but I believe, achievable. For arrays it would have to check for element mutation. For mutable objects, it would involve building up a dictionary of mutating methods, and ensuring you don't call them. So your class can have java.util.Date field, as long as it doesn't call, e.g. setYear. With this, I would have to be very wary of the performance implication of such an analysis.

Also, we saw that there is the potential for false negatives if your mutable field escapes. As in, the class never invokes setYear, but it passes the instance as a parameter to a method in another class which does.

Warnings about visibility in Java Memory Model are naive
We saw warnings about non-final fields not receiving the guarantee that they will be visible, even if assigned in the constructor. This ignores that it is possibly to obtain this guarantee in other ways: if your non-final field is volatile, or is of a particular type, e.g. AtomicReference; and if there are other memory barriers e.g. assignment to a volatile field coming last in a constructor where non-final fields are assigned.

The former should be an easy enough change to make. The latter might be more tricky, but still doable.

Analysis is implementation-focussed
The team behind ThreeTen's implementation previously discussed the possibility of using Mutability Detector or other tools for enforcing immutability. The idea being that the test could be part of JSR310's Technology Compatibility Kit (TCK). However, being on the strict side, it would be difficult to meaningfully test for immutability, while also allowing internal caching, in implementations other than the OpenJDK's. This limitation may also apply to anyone wishing to use the analysis at runtime. For now, Mutability Detector is best suited for testing specific implementations of your own classes.

Can help experts
Following the blog entries which exposed real mistakes in immutable implementations, the problems were deemed important enough to warrant fixing. ThreeTen is being developed by experts in their field, for inclusion in the reference implementation, lead by a developer who created one of the most popular examples of using immutability in Java. With Mutability Detector, I am aiming to serve both beginners learning how to write immutable classes, and experienced developers familiar with the technique who have made a "typo". This case demonstrated the latter does happen, and that the tool can help, with little cost in terms of setup and maintenance.

API for allowing reasons for mutability is flexible and powerful
In several of the examples, we saw how we could implement a custom Hamcrest matcher to suppress a false positive. I used Mutability Detector in the way that users are intended to, in that I did not make any changes to the source code. In each case, we could suppress false positives, in a custom way, without any modifications to Mutability Detector itself. Over time I would expect to incorporate some of the more common matchers into the out-of-the-box offerings.

In conclusion, I'm pleased with the results from analysing ThreeTen, but there's lots of room for improvement.

Have you tried Mutability Detector? Loved it? Hated it? I want to hear from you! Any kind of feedback is welcome, either here, on the discussion group, or in email to grundlefleck at gmail dot com.