The dark side of social networking

We spend a lot of time talking about the benefits of different social networking systems. We talk about the conversations that are enabled and the shortcuts to meaningful relationships, whether business or social. What we rarely, if ever, talk about is the dark side. The big brotherish side that can, if it wants to, track our activities in minute detail.

If big business is involved, and it is, you can be certain that this information is like gold. Of course it wants to track you. It pays very good money for the privilege of learning as much about you as possible. And a terrific way to do this is to know who you are then watch your behaviour: what websites you visit, how much time you spend on various activities, where you're connecting from, who you communicate with, whether you're a man or a woman and so on.

The instant you log in to a service - Facebook, Yahoo!, Microsoft, whatever, you're no longer anonymous. At a recent session with a company in this space, the first couple of hours were dedicated to how users could be exploited rather than served. I won't name names because, whatever the public face of these companies, the conversations behind closed doors are likely to be very similar.

But we're willing participants. These organisations provide a platform for communities to form and, because of our desire to connect, we share all manner of personal information. Not to the host, but to our online chums. Sadly, every time we contribute something or click the mouse, we freely, and unwittingly enable the host to refine our profiles and to deliver the advertisements most likely to appeal.

If we want to participate in online, public, social networking communities then it's best to assume we're regarded by many as victims rather than beneficiaries.

Mind the Language Gap

One of this morning's IRFS Language Gap sessions asks the question of why we need a cross lingual patent retrieval system in an Asian language. Well for a start, consider that three of the top five patent filing countries hail from the region; Japan is first, followed by China in third and the Republic of Korea in fourth. The US and Europe take second and third places respectively. 

If you operate in the patent world then sooner or later you will probably have to work with filings from Asian origins. The question is how, seeing as there are some fundamental structural differences between Western Latin-based languages and their Asian counterparts. Among numerous examples, there are for instance six varieties of expression for the colour red in Korean, how a word is spaced in Chinese can also heavily affect its meaning and translation to English considerably. When applying this to the ambiguous nature of legally constructed patent documents, the challenge is considerable.

Relying on just human translation is not an option, in part due to the sheer volume of documents constantly being filed as well as existing material.

Minah Kim from the Korean Institute of Patent Information has been explaining how their cross lingual retrieval system copes with the issues but also what still needs to be done.

She called for efforts to improve quality, such as a semantically based query expansion, whereby each word in an original search is expanded to a related search term such as boat to ship to vessel to water. Time spent on a query is also an issue that needs improvement, with the average amount spent on one document retrieval being 10 seconds; that can cause problems with a 200 page document, never mind the rest.

For the short term, the plan is to consistently upgrade the systems dictionary and establish the query expansion by the end of this year. In the longer term, Kim says there must be the development of a cross lingual retrieval system for Chinese, Japanese and Korean patent documents. Addressing the language translation issues with similar models is something that Western organisations can also only benefit from.

Information Retrieval Symposium opens

For the next two days, I will be covering the latest developments at Vienna’s Information Retrieval Facility Symposium (IRFS). It is a meeting of patent experts and business leaders and is designed to address the challenges that face those who operate in the emerging industry of patent information retrieval.

The overarching theme for the convention is one of patent experts meeting information scientists and opening up a meaningful dialogue. Hopefully there will be a merging of minds, finding common ground; you get the idea.

According to joint IRF Chairman Francisco Webber, there are many complex issues of transferring knowledge between science and business, what the IRFS hopes to identify is the shortcomings in the Intellectual Property (IP) world as well as the methods and potential solutions from the Information Retrieval (IR) sector.

Keith Van Rijsbergen, Webber’s co-Chairman and professor of Computing Science at the University of Glasgow, also talked about how tools for IR specialists are somewhat lacking. There are also issues that include how users of the future will utilise the next generation of patent searching technology. This could apply to organisations gaining a competitive advantage by monitoring others patent search audit trails. In parallel, the industry will see a rise of ‘naïve’ rather than ‘expert’ searchers over the next decade.

Of the many issues still to be put to the floor, it was discussed how government and business alike are not utilising available patent information to assist with their commercial interests. Open Access is also considered to be a central aspect of the innovation cycle of IP, IR and IRF

Of the other big topics to come will be the difficulties overcoming the language gap around the globe as well as patent search technology. More to follow…

A full analysis on the future of patent search technology will feature in December’s issue of IWR

Microsoft drops the search bomb

Some football fans critisicse David Beckham because "all he can do" is cross the ball. All Microsoft can do, in some pundits' books, is take market sectors that were once the domain of specialists charging pricey tariffs, and make them available to all. Today, Microsoft began to do that to enterprise search.

This might be looked back on as a rather momentous day in search as Microsoft's Search Server announcement is likely to change the rules of engagement in the field. The most notable points of the announcement that make it such a disruptive move are:

One, neither the freebie product nor the free product, Search Server Express, has a ceiling on number of searchable documents.

Two, neither the paid-for nor the freebie SKU requires a dedicated server.

Three, there are out-of-the-box connectors to FileNet, Documentum and Lotus Notes.

Four, this is a pure search move that does not lean on SharePoint, Microsoft's smash-hit entry into ECM.

Five, and this might well be the biggest factor, this is Microsoft, so the brand and ability to integrate with other key infrastructure will be huge plus factors for many buyers.

The net effect of the move will be to put pressure on Google's search appliances and IBM's OmniFind but Microsoft will not stop there and it is already talking about Office '14', the next major release, packing in high-end features. Autonomy made a very smart move in buying Zantaz, investing in video search and in other moves that have taken it away from basic enterprise search, which is showing every sign of being commoditised.

Google has done a sound job and IBM made a bold move in introducing its free version of OmniFind last year but the Microsoft manoeuvre is a major land grab. We still need to see the product and pricing details but if rivals are feeling afraid, we can understand their fear.

IWR at the Information Retrieval Facility Symposium 2007

Towards the end of the week I will be blogging live from the Information Retrieval Facility Symposium (IRFS). This year the meeting will be held in (what I am told) is the very fair city of Vienna.

Billed as a convention where “Science meets Business” the experts in attendance will attempt to hammer out how best to handle the vital retrieval of digital patent information as well as its storage.

The organisers say that as the field of Information Retrieval is at such an experimental phase, this will be the driving force behind the conference sessions.

The two-day event will be split into five sessions and a series of working groups. Kicking off on opening day will be the Data Quality session with speakers from GlaxoSmithKline and the Royal School of Library and Information Science.

Session 2 will focus on Language gaps in the Information Retrieval world. Particularly in relation to original Chinese and Korean documents

Session 3 is entitled Corpus Enrichment, or rather how technology can be utilised to recognise and extract the implicit information in a document. This can include the difference between preamble, a detailed description, claims made in a patent document or alternate examples. 

Sessions 4 and 5 will examine the related tools available for information professionals and management and research respectively.

I will be covering the significant areas, developments and intellectual jousting that will be going on as soon as doors open on Thursday morning and close Friday evening.

* A full analysis on the future of patent search technology will feature in December’s issue of IWR.

Free concept search from Yahoo!

In the relentless game of public search engine leapfrog, Yahoo! may have just leapt into the lead. It has added some fresh intelligence to help the hesitant user.

No doubt other search companies will already be unpicking the Yahoo! offering to see whether they can improve on it. And it's highly likely they can. But, for the moment, Yahoo! is the benchmark with its 'Explore concepts' extension.

Yahoo! senses when a user hesitates while typing a search query and pops up auto-complete suggestions. This, of course, is not a new idea. The new bit comes when you get to the results page. If the results aren't what you expected, you can click on a little arrow to drop down a panel containing the autocomplete suggestions and an 'Explore concepts' section to the right:

In the above example 'information world' produced, not surprisingly, over a billion results. The autocomplete would have done the job for you and me, but the concepts on the right are designed to help the user who's thrashing around a bit.

As you click on each concept, new search results appear along with the concepts which relate to the new search expression.

I tried 'electron spin resonance', about which I know little. After clicking through 'free radicals' (thought I might hit a political pressure group), I was intrigued by 'magnetic moments'. Here's the first result.

Bear in mind that this functionality is not part of some hugely expensive enterprise search system, it's freely available to the general public.

If you're like me, you've regarded Yahoo! largely as an organisations that wants to push ads at you and keep you within its semi-walled garden. A bit of a turn off, especially in Europe apparently. But developments like this and, in a totally different context, Pipes suggest that Yahoo! has realised that life is not all about take, it's about give as well.

Wiley goes on Safari

Global publisher John Wiley & Sons is not afraid of new technology and ventures, as I recently discovered in a meeting with them at the Frankfurt Book Fair. The Bookseller reports today that Wiley has now inked a deal with Safari Books Online, an increasingly important on demand reference platform.

Wiley will add its business and technology reference books to the Safari platform and see its content aligned with other leaders like Pearson, O'Reilly and the publishing arm of software giants Microsoft. Wiley will add its For Dummies books, which it acquired from web and magazine publishers IDG, and the Bible range of computer books.

This is an important deal. Reference books are still an amazing resource for users, and a method of information delivery and publishing that still has plenty of legs in it. Like all information resources though, it is a sector that has been threatened by amateur services like Wikipedia. Reference is clearly an information set very well suited to the web. Safari is a platform that offers a genuine alternative to Wikipedia. Because the content on Safari is from credible publishing companies that check the veracity of information, use knowledgeable experts and put a great deal of effort into the writing, editing and presentation of the information, it is more credible than Wikipedia. Wiley has increased the desirability of Safari and improved reference information on the web.