Running with Scissors
This discussion could be extended by the discussion about the fact that Microsoft and other software developers make a lot of the functionality internal or private, protecting the developers who are using the software from making changes to parts of the framework that are subject to change. Stating that the public API should be backward compatible and by making parts internal or private they reduce the risk of breaking backward compatibility. Jeff Handley wrote a blog post about this in response to a blog discussion initiated by Jeremy Miller a while back.
So let’s talk about protecting developers from them selfs; I think this is the wrong approach, instead of protecting we should be teaching the developers to understand that they are working with sharp tools. Take a look at another profession where they work with sharp tools; carpenters work with sharp tools every single day. When they are an apprentice they don’t get a dull saw or dull wood chisel, no instead they are educated on the fact that these are sharp tools and how to properly use them, without loosing valuable body parts. This approach seems to be working for quit some time now.
So why doesn’t this happen in our profession? Why don’t we properly educate our apprentice developers so they know they can’t just do anything with the frameworks they use?
And why not define a specific API for your framework and warrant that this will be backward compatible, also mention that the other parts of the framework are not being considered backward compatible, and that no effort will be spend on trying to keep them so. Provide clear extension points to your framework, don’t mix behavior that should remain with behavior that can be altered. Perhaps a better way would be to split this up and put the behavior that should remain behind the public API? This way normal usage of the framework would be fine and would be backward compatible, but when needed the developers have the power to make changes inside your framework knowing very well that you may decide to change these classes at your own will, without having to explain yourself period.
You could also have unit tests specifying the behavior you rely on, when developers want to extend your system they can run these unit tests to verify that they have not broken some important behavior.
Now let’s go back to having all methods be virtual by default, lets assume this true and instead of the virtual accessor we would have a real or notvirtual accessor so you can specify that this is behavior that you don’t want to be replaced. Would that not be sufficient so serve both scenario’s?
Hi Ward,
Thanks you for the elaborate answer!
I read the 'Paternalism' part as; its ok if people run with scissors and fall, as long as these where not my scissors, perhaps a misinterpretation from my side :)
I may have mixed two concerns in my post by having the virtual by default and internal and private discussions together. They are closely related but different in how you can change the functionality.
Using virtual you can change the functionality of the class implementation itself. And yes this is to facilitate mocking and proxying the classes. And I can see your point there that you want to ensure that the behavior that you specify is always going to be the behavior of that class. But if the language would have supported the virtual by default and have a keyword to indicate that things should not be virtual, then especially in testing usage of a third party framework things would go a bit smoother. But you could still specify places where you want non virtual behavior.
As for marking entire classes as internal or private, I think this is a shame because it limits you from ever making a different implementation of one of these. And especially for a framework this could be important, as when I create an application I usually have enough freedom to make the changes I need, but when using a framework I have to use what is provided, and frameworks don't always behave in such a way as I need them to.
Now I am no framework builder, I only contribute a little bit to FubuMVC, but there the whole framework (granted that it is relative small) is open and nearly all behavior can be replaced by something custom. I think this offers great value, but yes imposes greater risk.
One thing I am looking forward to is the Design by Contract implementation in .Net so you can specify DBC on an interface and ensure it acts the way you need it to be but still providing complete freedom in the implementation. Really a shame that Spec# was not completely incorporated since there it was part of the language which makes for nicer syntax. I can see this having great benefits for framework designers.
Great posts I learn a lot from reading them!
Mark Nijhof, Sunday, August 16, 2009 at 6:54 PM
The argument that Ward is making against virtual-by-default it not primarily about protecting developers from themselves. As he has stated repeatedly, his argument is about protecting his own ability to evolve his framework to meet the needs of his customers. What many who argue for virtual-by-default, especially those who have never built frameworks themselves, don't understand is that virtual methods *limit* extensibility. Don't believe me? Check out http://blogs.msdn.com/brada/archive/2003/07/29/50202.aspx:
"For example, we found that ArrayList item enumeration calls several virtual methods per each MoveNext and Current. Fixing those performance problems could (but probably doesn't) break user-defined implementations of virtual members on the ArrayList class that are dependent upon virtual method call order/frequency."
Additional analysis of the problem referred to can be found at http://blogs.msdn.com/ricom/archive/2005/08/26/performance-quiz-7-generics-improvements-and-costs-solution.aspx.
The primary problem with making methods virtual is that it invites consumers to take dependencies, not only on the behavior of the methods, but on the *order* and *frequency* in which they are called. This severely limits your ability to refactor and improve the class in the future. *That* is why it is so important to only make methods virtual that have explicitly been designed to be extensible.
David Nelson, Wednesday, September 16, 2009 at 10:54 PM
Recent blog posts
- Follow me @ Elegant Code
- CQRS à la Greg Young
- CQRS à la Greg Young example code
- My Kindle DX
- My book: Are You Better Than Yesterday?
- Running with Scissors
- Applying Domain-Driven Design and Patterns
- Hey Developer, YAGNI I tell you
- Hey Developer, the product you create is your code
- NDC videos are published
Mark - Thanks for giving my post a serious reading.
I addressed your point directly in the section "Is This Paternalism?" where I tried to establish that protecting developers from themselves was _not_ my motivation. I think that's noble, but I wouldn't do so at the cost of inhibiting the expert developer.
I limit the surface area of my classes in order to deliver a high quality, thoroughly designed product that you can count on. Remember I am a framework provider. You are building on my classes. You trusted me to do a job. If you trust me, let me do it.
I'm making the case for encapsulation. If I have closed off some of my code, it is because that code is pure implementation. Reflection may reveal my implementation, but by announcing via limited accessibility that certain classes and methods are off-limits to you, I'm doing my best to draw a bright line between extensions points I support and implementation that is mine to change at will.
I could try to annotate my classes with comments to indicate the boundaries as you suggest. In fact, I have had to do so in those cases where peculiarities of .NET oblige me to make something public that should not be. I'll say "Internal IdeaBlade Use Only".
These comments have not been effective in practice. They are routinely ignored, largely because people don't read comments. They get angry when the signatures change, the classes disappear, or their side-effects are
Do you make your fields public? Of course not. Why does your reasoning not apply to fields? If I may answer for you in one word, it would be "encapsulation". Encapsulation frees you to make changes without altering consuming code.
YAGNI reminds us that we should not over-design. Many of these non-public classes and non-virtual methods are just good enough to satisfy as implementation. They often lack guard logic, for example, because I can be sure when and how they are called. They may rely on facilities that I _know_ will change, be superceded, or even disappear in a forthcoming release. Sometimes they are just half-assed. We can't perfect everything; we choose what to polish and what to leave rough.
I cannot ... and I should not ... expose these implementations to the wild. I don't have unlimited time or resources; no one does. It would be crazy to sacrifice valuable features just so I can supply you with documentation and certification tests for functionality I never wanted to support in the first place.
Let me return briefly to the original point of my post. I don't want my argument against virtual-by-default to hinge on the occasional necessity of sub-par implementation. When I seal a method it is because I want to make a guarantee that the method will do what I said it will do. Making it virtual can void that guarantee, obliging me to prop it up with costly, perhaps fruitless efforts, that I could easily avoid. I may choose to make it virtual anyway. That's a design decision, not something foisted upon me by an edict from on high.
It's important to get back to why you want virtual (or public) in the first place. Your desire to simplify stubbing, mocking, or proxying of my classes is absolutely on the money. The CLR should help you realize that desire. But it doesn't. Your wish, however worthy, is not a good enough reason to put at risk my product quality or my product roadmap.
I'm not trying to protect you from yourself. Call me selfish but I'm trying to protect the future of the product and give you the service you paid for.
Ward Bell, Sunday, August 16, 2009 at 5:56 PM