Software engineering and conceptual product design

The overall goal of software development is at least in 95% of all situations to retrieve the highest possible Return on Investment (ROI). It is important to note in this situation that it is not that important to retrieve the biggest possible revenue. It is always also a matter how much you had to invest to reach this goal.

Following this directive there are two major possibilities to reach a high ROI. Increase your revenue when keeping the costs constant or decreasing costs when holding revenue (or of course both of them which would also be nice). I am a software developer and architect and I am highly involved in this business. So of course I will not consult you to lower the costs because this would basically mean that you should pay less money to guys like me. So I suggest increasing revenue.

So far so good. Let us increase revenue. But how should we do that? In fact we have to sell as much units of our product to the highest possible price. We can force our users to buy our products. Here marketing and comes into play and I am not a marketing specialist at all. You should contact M$ (or a marketing expert of your choice) to learn how to do this. The other possibility is to build a product the users want to buy because they like it and they are willing to pay for it. To archive this you need specialist in Software Engineering. As you can see I am defining Software Engineering in a very broad manner. I am also including product planning and conceptual design, usability design and graphics, domain modelling, user acceptance testing and so on within Software Engineering. So for my definition Software Engineering is much more than just programming or testing work. But this is nothing really new. Most approaches to software project engineering are including requirements and specification work and quite some other stuff.
But in all project engineering models I have seen so far I always miss a very important thing: product planning and conceptual design. This is not requirements engineering. This is not specification. This is not software architecture. This is another discipline. Requirements are describing the goal of software. So you can take a piece of software and check if it fulfils the requirements (even this is very hard with most of existing requirements). But you never can build software only based on requirements.

An example
Consider contact management software. You find following requirement in your collection: The user can search for contacts based on first or last name. The search parameters can include wildcards as used in a typical file search.
This is a nice requirement. Perhaps you have also a use case which describes the procedure in a stepwise approach and tells you where in the navigation to reach this function and it also tells you, which data to show in the result. This requirement given it is very easy to check if a product can do this. But try giving this requirement to 10 developers and you will get 10 very different implementations.

I want to show that a requirement or a use case is never feasible to build software from it. It is even not possible to make a specification out of that because when building software you always need something like the overall vision or the overall story. You cannot realize software feature by feature or use case by use case. It will get just a collection of use cases and not one piece of software. By the way – this is my biggest concern with eXtreme Programming (XP).

To realize a good piece of software you need an overall vision and an overall concept at the very beginning of implementation. You have to have something like your vision of how the users will work with the software. You also must not implement requirements and uses cases on a piece by piece approach. You will not get a highly integrated system by this. Your users will not tell you exactly how they want to do their things. Often they tell you just what the want to do and how they are doing it currently. The transformation of requirements, user stories and use cases to a vision and a planning of a product is what I call product planning and conceptual design. This is an extremely creative process – even more creative than software architecture. It is interdisciplinary. It needs experts from the system under construction´s domain area, it needs usability experts and software architects. From my point of view this transformation step is very poorly integrated in must software engineering process and there is much too less attendance in the community on how to realize this cooperation and how to combine this knowledge areas into a creative and high quality software product.

Conclusion
In this blog entry I just described that I miss well defined processes for product planning and conceptual design. I did not give any suggestions on how to solve it. So I will try this in one of my next entries. I will try to write down my ideas about something I call “User Manual Driven Design” and also I like the idea of “Interaction forms” very much and I will prepare something about this too. So stay tuned.