Robert Schmelzer is Thinking about IT

Often there is the situation where you serve multiple top level domains with one software / hosting package. Because a lot of providers map all subdomains to the same virtual host often the problem of duplicate content arises.

So I tried to find rewrite rules for mod_rewrite to solve this problem.

As a base I used the example provided the mod_rewrite documentation:
http://httpd.apache.org/docs/2.0/rewrite/rewrite_guide.html#canonicalhost

RewriteCond %{HTTP_HOST} !^fully\.qualified\.domain\.name [NC]
RewriteCond %{HTTP_HOST} !^$
RewriteRule ^/(.*) http://fully.qualified.domain.name/$1 [L,R]

But with this example it was only possible to redirect one domain. But I have several domains on my hosting and I want to redirect all of them to the correct www subdomain:

RewriteCond %{HTTP_HOST} !^www\.(.*)$
RewriteCond %{HTTP_HOST} ^([a-zA-Z0-9\-]+\.)?([a-zA-Z0-9\-]+)\.([a-zA-Z]+)$
RewriteRule ^(.*) http://www.%2.%3/$1 [NS,R=301,L]

Maybe you know the situation - you want to execute initialization code as soon as possible but not before the DOM is ready. With Prototype 1.6 a very handy custom event was introduced for this purpose. It is called "dom:loaded" so you can do your initialization as soon as the DOM has been created.

E.g.:

document.observe('dom:loaded', f)


Now you do some improvement of your code an you introduce some AJAX stuff and suddenly things not working anymore. After some debugging you notice, that the main problem is, that the initialization is not working anymore. This happens because after an AJAX call no window load event or the custom dom:loaded will be fired.

Therefore you need an event handler, which executes your initialization code as soon as possible - this means on dom:loaded when you page is newly created and immediately when you are in an AJAX call.

I extended the prototype Event class to realize this:

Object.extend(Event, {

  _isDomReady : false,

  

  asap: function(f) {

  	this._isDomReady ?

  		f() : document.observe('dom:loaded', f);

  }

});

document.observe('dom:loaded', function() {Event._isDomReady = true;});

I added a class member to the Event class which indicates, if the DOM is already loaded. A event handler is registered to set the class member, during the dom:loaded event. Now you can use the Event.asap(function handler) call to execute the function handler as soon as possible. But you must be aware, that you have to implement the function asynchronously because you will not know if your handler is executed immediately or deferred.

You can see this stuff in action here.

Today Michael Stal wrote about how the background of people influences their understanding of commonly used terms.

I often had exactly the same feeling. One of the most challenging tasks of a software engineer is to bridge the gap between people talking in business terms and the programmers talking in technical terms.

One very actual example of this is “Web 2.0”. Ask a programmer what “Web 2.0” is and you will get answers like AJAXing, DHTMLing, Partial Updates, Inline Editing. Ask a designer and you will get answers like One/Two column layouts, modern colours, cleaned up design, nice badges, big buttons and last but not least ask a management guy and he will answer things like User Generated Content, Revenue Sharing with the user, building communities. Is anyone of them right or wrong? Surly not the all mean the right things in their mindset and it’s about software architects and engineers to bring them together.

Michael Stal is also talking about the importance of a project glossary. I think such a project glossary can be the first step towards a domain specific language (DSL). I would like to make this clearer with an example:

A business guy tells you the following:
“If the order is still open after two weeks it must be routed to a manual clearing.”

Ideally you can implement this by:

if (order.isOpenSince(now() – 2 weeks)){
order.route(STATE.MANUAL_CLEARING);
}

To make such a thing possible it is important that the terms “Order”, “Is open”, “Route”, “Manual Clearing” means the same to both sides and that during software architecture the terms were already defined.

So as a conclusion from my point of view, a domain specific language is the technical implementation of a common language thought the various stakeholders within a project. A project glossary is an important step towards a DSL and it is also an important addition to it.

Additionally a reading recommendation about the topic: Domain-Driven Design

After quite a while I looked into my blog again and what did I see.... My last post was written about one year ago???? - The time goes by.... So now I am back again and I will use this blog to write about my technical thoughts. I was thinking about writing this stuff into our new project blog on blog.cusoon.com but I realized that all the technical stuff I want to write does not fit there. So I decided to restart my blog activities here.

Recently Andreas told me about a very nice PDF Creator tool. It is free even for Enterprise use very easy to use. I got it work within five minutes and had my first PDF file within 10 minutes (including download). That´s the way how tools should work...

... or why a search engine indexing bot makes 160 000 incorrect requests within 48 hours.

China has implemented their own answer to Google. The search engine is called: Qihoo.com. The are some major differences to Google. The first thing I noticed was, that I cannot read chinese websites..... that´s bad....

But there is major problem with Qihoo.com. I noticed in the statistics of our project www.fertighaus.biz and www.mba-studium.info that we had about 160.000 requests all bringing up a 404 error. After some research in the log files, resolving the ip and using the better search engine Google I found out that the requests came from the Qihoo Bot. After some research using again Google I found statements from various web masters that this bot is working like a Zombie. It does not react on 404 errors either it reacts on 403 errors and it also does not take robots.txt file into account. Furthermore it makes several requests per second which can cause serious performance problems on small hostings. The bot is officially declared as "beta". But if this is the chinese definition of beta I will never ever use their beta products.

In our particular situation we had the problem that our error reporting was a little bit to exhaustive. We are using a front controller answering all request to the domain. Because we are using mod_rewrite our front controller inspects every request. For the wrong requests in made log statements to our application log and during two days the log files filled up our whole web space and the application was not able to create new sessions. This caused our application to malfunction.

Now I just blocked the IP addresses of the Qihoo.com. I really have no interest to be listed on chinese search engines. I used .htaccess and mod_access for it:

order allow,deny
deny from 220.181
allow from all

The most interesting references on the web are:
Qihoo - oder wehe wenn China Google ablösen will
Wenn ein chinesischer Crawler selbständig wird

It has been a while since I made my lost blog entry. In the last two month I had a very intensive private and working life. My contract at Siemens Enterprise is very near to run out and so I already take care for my time after Siemens. The first official and very important step to this will be Friday next week. My brother Manfred Schmelzer, Peter Krammer and I will found a new company. We are going into the Web 2.0 market and are about creating a new very exiting project. I do not want to tell very much about it right now, but check out my blog in the future and I will keep you up to date. It will get especially interesting, when we are looking for Alpha testers...

Another update has to be made for my post about User Manual Driven Design. I really like the idea and I am still thinking a lot about it. I recently found a very interesting blog article of Amazons´s CTO Werner Vogels. He is talking about Working Backwards were he proposes a very similar approach to software development as I do in my post about User Manual Driven Design.

In his last blog entry Michael Stal mentioned the importance of defining a new technology stack for Web 2.0 so that development can fulfil the requirements of upcoming (and already established) usability and interactivity trends on the web. Currently we are using a very chaotic technology stack, which consists of HTTP as a transport protocol, SOAP, REST or JSON for service integration, (D)HTML, CSS for mark up and JS to put logic into our static pages. By the way you have to use JS frameworks, CSS hacks, HTML hacks, HTTP hacks to make all the stuff working on various browsers and then you still have left the full bandwidth of server technologies.
What you see based on this listing that a full fledged end to end web developer must be really multi talented. So it is not really wondering that salaries for web developers are steadily increasing. So this is quite fine for guys like me, which are some sort of end to end web developers but it is very dangerous for the business. This in fact means that development of Web 2.0 like web apps is getting even more expensive. Not to mention the starting nightmare with IE 7 so that we will face the problem to be compatible to at least three major browsers within the next years (FF2.0, IE6, IE7).
So I agree to Michael Stal that we need new technology stacks. But the internet is too big for the revolution approach. This means we can only evolve the technologies using an evolutional approach. And evolution on something outdated like HTTP and HTML will get really hard.

Conclusion
Be careful when deciding to build modern web apps. It will get very expensive. Desktop applications are easier to develop and there must be a very well evaluated decision which technology brings more ROI.

Today I want to follow up on my recent blog entry about conceptual product design.

I have worked in various projects starting from very small ones with just 3 people involved up to rather big ones with more than 100 developers. A very interesting thing I noticed was that as bigger the project was the desired outcome was more and more unspecified. The requirements got more and more abstract and less people knew what problems the system should solve, how they should be solved and how the solution should look like.

But in fact in every project only one thing is really important: the outcome. Software engineering is NOT about defining good architecture or writing robust code or something like that. Software Engineering is about making the customer happy. And the customer is happy if the software fulfils his need in a comfortable manner. By the way you should take a look that you are not getting into troubles when you have to change something later on. Therefore good architecture is quite nice. In big projects I often had the feeling that people are paying good architecture and code which fulfil documentation standards and fit to checkstyle definitions more attention as they are giving to the output and functionality they produce. This is very dangerous. It can happen that you have a piece of wonderful software at the end which does nothing useful to the user.

This brought me to the very expressive idea of User Manual Driven Design (UMDD). I would define UMDD as a software engineering process to organize project or product development in an extremely user oriented way. It concentrates on the principle of “Getting the RIGHT things done”!

UMDD is simple. In fact you can express it in one sentence. The basic rule of UMDD is:

Start your project with writing the User Manual together with your users.

Examples
In fact I tried UMDD basically two times by now. Both times it have been small projects were I had quite much influence on the project organisation. One project was done during my time at the polytechnical university of Hagenberg. We built a web system in the medical area. I was responsible for the conceptual product design and requirements engineering. I was thinking how I can communicate to the medical stuff at the customer site and to my team members in development using the same artefacts. I started with drawing nice Visio drawings containing big sequence diagrams and a simple form of class diagrams. But it turned out that the medical stuff did not understand anything with it and it was even to abstract for the rest of the team because they kept asking me how it should look like. So I just started writing the handbook and added screenshots from an html prototype to it. I wrote chapter by chapter and the medical stuff had a change to take a look. I also showed the html prototype to them and they were able to do a few clicks at it. We started development after the technical evaluation and at that time the user manual already had about 50 pages describing quite fine what to develop.

In other projects together with one of my most important customer plus Media GmbH we are working in very similar way. We do not write actual user manuals because we do not have them at all and I strongly recommend not writing any documents which no one uses afterwards. Therefore my customer is providing me screenshots with some informal documentation what should happen within the screens. Additionally we have defined some cross cutting issues which have to apply to all parts of the system. It is also an extremely productive cooperation. In addition we know each other well in the meantime. So we have a common base of communication which makes teamwork easier. I strongly suggest for all parties involved in a project to maintain a long, cooperative and fair partnership. It is much easier to work successfully then.

Conclusion
Some rules for a successful project:
1. First write the user manual and then start development.
2. Tell the developers WHAT to develop.
3. Use the same communication artefact for users and developers.
4. Maintain long, cooperative and fair partnerships with your customers or your contractors.

Outlook
Because the entry already got rather long I postponed my argumentation why UMDD is better than collecting requirements and use cases and using them as the base for development. I know that this idea is not complete by now so that it would be possible to pray it as a religion. I will also follow up on how this can fit together with other techniques like test driven development or prototyping. I still have to think about it in more detail and how to integrate it well established processes and so comments on it a very, very welcome. I will also try to integrate this idea more and more in my current projects.

Special Thanks
Special thanks also goes to Alexander Buchmann working at Siemens for his input to this idea.

I just read on Ajaxian that Symfony 1.0 beta is out. This is a great MVC framework for PHP. It is based on the ideas of ROR and combines it with the big benefits of PHP and its ease of development. I think I have to plan some time to upgrade my projects to the new version! Great work! Go on!

I have just registered at Technorati and so the asked me to make a blog containing a link to my Technorati Profile. Of course I am doing this. I want to attract some more users to my blog.

The overall goal of software development is at least in 95% of all situations to retrieve the highest possible Return on Investment (ROI). It is important to note in this situation that it is not that important to retrieve the biggest possible revenue. It is always also a matter how much you had to invest to reach this goal.

Following this directive there are two major possibilities to reach a high ROI. Increase your revenue when keeping the costs constant or decreasing costs when holding revenue (or of course both of them which would also be nice). I am a software developer and architect and I am highly involved in this business. So of course I will not consult you to lower the costs because this would basically mean that you should pay less money to guys like me. So I suggest increasing revenue.

So far so good. Let us increase revenue. But how should we do that? In fact we have to sell as much units of our product to the highest possible price. We can force our users to buy our products. Here marketing and comes into play and I am not a marketing specialist at all. You should contact M$ (or a marketing expert of your choice) to learn how to do this. The other possibility is to build a product the users want to buy because they like it and they are willing to pay for it. To archive this you need specialist in Software Engineering. As you can see I am defining Software Engineering in a very broad manner. I am also including product planning and conceptual design, usability design and graphics, domain modelling, user acceptance testing and so on within Software Engineering. So for my definition Software Engineering is much more than just programming or testing work. But this is nothing really new. Most approaches to software project engineering are including requirements and specification work and quite some other stuff.
But in all project engineering models I have seen so far I always miss a very important thing: product planning and conceptual design. This is not requirements engineering. This is not specification. This is not software architecture. This is another discipline. Requirements are describing the goal of software. So you can take a piece of software and check if it fulfils the requirements (even this is very hard with most of existing requirements). But you never can build software only based on requirements.

An example
Consider contact management software. You find following requirement in your collection: The user can search for contacts based on first or last name. The search parameters can include wildcards as used in a typical file search.
This is a nice requirement. Perhaps you have also a use case which describes the procedure in a stepwise approach and tells you where in the navigation to reach this function and it also tells you, which data to show in the result. This requirement given it is very easy to check if a product can do this. But try giving this requirement to 10 developers and you will get 10 very different implementations.

I want to show that a requirement or a use case is never feasible to build software from it. It is even not possible to make a specification out of that because when building software you always need something like the overall vision or the overall story. You cannot realize software feature by feature or use case by use case. It will get just a collection of use cases and not one piece of software. By the way – this is my biggest concern with eXtreme Programming (XP).

To realize a good piece of software you need an overall vision and an overall concept at the very beginning of implementation. You have to have something like your vision of how the users will work with the software. You also must not implement requirements and uses cases on a piece by piece approach. You will not get a highly integrated system by this. Your users will not tell you exactly how they want to do their things. Often they tell you just what the want to do and how they are doing it currently. The transformation of requirements, user stories and use cases to a vision and a planning of a product is what I call product planning and conceptual design. This is an extremely creative process – even more creative than software architecture. It is interdisciplinary. It needs experts from the system under construction´s domain area, it needs usability experts and software architects. From my point of view this transformation step is very poorly integrated in must software engineering process and there is much too less attendance in the community on how to realize this cooperation and how to combine this knowledge areas into a creative and high quality software product.

Conclusion
In this blog entry I just described that I miss well defined processes for product planning and conceptual design. I did not give any suggestions on how to solve it. So I will try this in one of my next entries. I will try to write down my ideas about something I call “User Manual Driven Design” and also I like the idea of “Interaction forms” very much and I will prepare something about this too. So stay tuned.

Based on the suggestion from Wallace in his recent blog entry I also tried out Google Analytics. It is really an amazing tool and again for free up to 5.000.000 page impressions a day. I especially like the easy handling of profile to track special user navigation behaviour and the extreme easy way to integrate it into your own page. It is just 3 lines of js you have to include. I did not tried it out by now but using the given js function it should also be very easy to track AJAX requests or clicks to external links.

Currently I am planning for an upcoming project and I am about to choose the technology stack to use. I know that there are a lot of possibilities out there. Luckily I already decided not to learn completely new things. So at least I cannot exclude things like Perl, Ruby on Rails, .NET + Atlas, Zope and other stuff from my evaluation.

I am quite familiar with the worlds of JAVA and PHP. So I consider these two as possible candidates. But it also makes a big difference, which frameworks on top of these environments to use. This is the point where it gets really complicated.

Of course there is already a huge amount of information and comparison about these two technologies available. But you have to be very carefully when reading this material because in a lot of situations it is some sort of a religious war between both of them. But I do not want to make religious decisions. I want to be pragmatic. That’s why I will try to make a short competition for myself to find out which one is more productive for me.

The candidates:

Option 1) PHP5 with Symfony for MVC without a template engine. Propel as ORM. Not sure which AJAX toolkit to use. Research must be done here.

Option 2) JAVA with Hibernate and Spring and SpringMVC + WebFlow. Plain JSP with JSP 2.0 Taglib as View Helper. SiteMesh for page decoration. DWR for AJAX remoting.

Option 3) Grails (Groovy + JAVA) with Grails View Technology

It was quite easy to choose the PHP stuff because Symfony is the most exciting and productive MVC framework I ever saw for PHP. It solves a lot of day-to-day real style problems in a comfortable and well designed way. Allthough it still has its issues I am looking forward that they will disappear when Symfony gets more mature and its heading to 1.0 version.

To choose a framework in Java is much more difficult because there are a lot of possiblites and a lot of mature frameworks. The frameworks which come to mind first are typically Struts and JSF. Both are inadequate from my point of view. Struts is too much work in terms of configuration and XML wiring. Additionally I absolutely do not like the form bean stuff. Java Server Faces (JSF) is well suited for big back office solutions where you want to make reuse of widgets and where you have massive reuse of widgets. But JSF has to less flexibility. It does too much of the web staff for you. It makes it extremely difficult two build SEO optimized pages. It has big troubles with a GET requests. Components development is awful. So all JSF based frameworks like JBoss SEAM, Oracle ADF, … are out of scope too.

I choose Spring and Hibernate because they are some sort of well-known best practices. I like SpringMVC because it tightly integrates the IOC approach and is very well designed. WebFlow might help us with out wizards a lot. I want to use JSP because I like well introduced technologies with a lot of resources on the net (for example compared to Velocity). The tag libraries as view helper are a very clean and useful approach. Especially with JSP2.0 they get much less painful to implement, because you can use JSP to implement tags which make work much more comfortable.

The third option was chosen because it is the most similar based development approach on the JAVA platform compared to Ruby on Rails and Symfony at PHP. Ruby on Rails always gets extremely good feedback from the developer community. So I would like to see, if this approach is useful on the JAVA platform two. Grails itself is built on top of my personal Option 2 technology stack. So it will also be interesting which value the added compared to using the underlying technology.

The Procedure

I will set up a test use case, which has database connection and must do some emailing. The test case will consist of a multi step wizard and there must be various input including date / time done. The use case must be realized for two locales. Automated tests must be written on the service and action layer. The db schema must be the same for all. I will invest 2 days of development on each option. Afterwards I will collect results in a criteria catalogue. I am really looking forward to the results. Of course I will post them.

So there is only one big question left:
When do I have time to spend 6 days for doing the comparison? Perhaps at X-mas or New Year time? Arrrggg…. a day should have more than 24 hours.

I have very good news today. For two days now I did not had to struggle around with new IE bugs and other boring stuff. So beside travelling and meeting friends I had the chance to do a little more work on the new project I am preparing. Currently I am looking for an interesting name for the project. It should not be a generic word, it should not be English or borrowed from English and the main domain names should be free. Of course it should sound nice and easy to speak and write. Overall it should be pronounceable and writable in almost every European language. Not an easy task at all. So I am very happy if you have a good suggestion for me.

Furthermore I am looking for a nice online collaboration tool, which we can use for communication and structuring our ideas and concepts and it should also support later development. But it must not introduce organizational overhead. The nicest solution I found is Trac. I already saw Trac in action in the Symfony project and I really liked it. But I found the installation guide rather difficult and so I looked for a hosted solution. I found the interesting project Assembla hosting Trac and development collaboration for free. But I am still not sure, if I really want to host my project data and source in an online collaboration platform.

So far you will think: “What’s about the post title?” – Okay now the answer:
Last week I bought the book “Fish”, which was some sort of a bestseller some time ago. It is about a highly frustrated department in an office work situation and the department head – a very motivated lady. She really wanted to bring new motivation into the group and found the solution in a nearby fish market. The main message is that your work environment is as exciting as you desire to make it exiting. What I liked very much was that they are introducing “playing” at work. I really like this idea. Because everyone – at least every man – wants to play. Children like every minute playing. And if we feel us a little more playing at work it will be much more fun.

I think an exciting work environment is very important. Software Engineering is creative work. Creative work cannot be forced. Creative work cannot be done just by spending time and doing stuff. Software which is developed without creativity is bad software because there will be nothing new in it. So I think good software engineers a very creative people which like solving problems and challenges in new and unlikely ways.