Pressure on product development has risen hugely over recent years. Global competition is steadily increasing, intervals between innovations are getting shorter and shorter and companies are more and more in a hurry to put products on the market without delay. At the same time, customers place increasingly high demands on product performance.

Consequently, R&D departments need more feedback from customers - feedback which has to be more detailed and available sooner than usual. Traditional feedback channels, e.g., service departments, are not able to fulfil this need. In many instances, the information which they provide is only available when the third-generation product has been put on the market. Besides, the quality which is required to expose mismatches between technical specifications and consumer requirements is frequently lacking.

The Internet may provide a solution. Product comparison websites and social media, including Twitter and Facebook, are making consumer feedback about products increasingly simple. It is a hopeless task for companies, though, to scan and analyse all feedback manually and convert it into useful information, as it is often only available as plain text. That is why there is a huge demand in the business community for smart and swift tools which can convert large quantities of texts into applicable knowledge.

Unstructured texts with emotion

In cooperation with five manufacturers of end products, the University of Groningen (COPE) and Eindhoven University of Technology have conducted a five-year research into consumer feedback and the ways in which this feedback can be analysed and translated into information. This research was co-financed by the Ministry of Economic Affairs in the framework of the IOP-ICPR research programme.

One of the first conclusions achieved was that much of the feedback not only refers to the product, but also includes service provision. Just think of companies which promise customers a money back guarantee and don't live up to that promise - at least, in the view of the customer who expresses his grievances online. Gradually, therefore, the scope of the research project has been widened beyond feedback about products.

Much feedback appears to consist of brief, unstructured texts without any punctuation, often drenched with emotion, and with many specific terms. Consequently, existing text analysis tools experience great difficulty in unravelling and ranking these texts.

COPE research has mainly focused on the latter subject. It has culminated in the development of a number of techniques which allow us to withdraw and cluster texts from different sources using algorithms from the natural language processing (NLP) domain. Tests with real texts have shown that the algorithms which were developed perform well.

New techniques and algorithms have been made available to the participating companies and are now applied in practice. Besides that, we continued our partnership with some of these companies when the research project had formally been completed, with the aim to further develop and implement the tools which we developed.

Substudy: analysis of texts with errors and without dots

Techniques for automatically scanning, analysing and ranking of texts have existed for a long time. Over the years, scientists have developed various algorithms for that purpose. 'The problem, though, is that these algorithms were developed for well-written, grammatically correct texts. A lot of consumer feedback, however, is only available in the form of informal texts, which are frequently brief and unstructured, without punctuation, and maybe with spelling mistakes. There were no algorithms yet for these kinds of texts', said Ashwin Ittoo. He obtained his doctorate at the University of Groningen with a thesis on this subject and is now an assistant professor at the University of Liège.

Mr Ittoo's research focused on existing texts acquired from service centres, repair centres and call centres and on product reviews posted on Amazon.com. It involved data from structured databases and random notes made by mechanics and product developers. The algorithms developed by Ittoo are embedded in tools which have been used by the participating parties. 'They have proved to work properly, not only in respect of unstructured texts, but in structured ones as well. That is why we will make another step in cooperation with Philips Consumer Lifestyle', said Mr Ittoo, who sees a great deal of potential in his tools. Social media such as Twitter and Facebook are constantly used for comments about products. 'We have not tested my algorithms for texts on social media yet. That will be our next step.'