Getting paid to visit and use websites?

http://www.businessweek.com/the_thread/techbeat/archives/2005/10/should_flickr_p.html#trackback

Cheap Robots (PCs on wheels)

http://www.businessweek.com/the_thread/techbeat/archives/2005/05/brainy_pcs_on_w.html

http://www.engadget.com/entry/1234000907021772

The Long Tail (Wired Magazine article)

http://www.wired.com/wired/archive/12.10/tail.html

The Power Of Us
Mass collaboration on the Internet is shaking up business

The 35 employees at Meiosys Inc., a software firm in Palo Alto, Calif., didn't know they were joining a gang of telecom-industry marauders. They just wanted to save a few bucks. Last year they began using Skype, a program that lets them make free calls over the Internet, with better sound quality than regular phones, using headsets connected to their PCs. Callers simply click on a name in their Skype contact lists, and if the person is there, they connect and talk just like on a regular phone call. "Better quality at no cost," exults Meiosys Chief Executive Jason Donahue. Poof! Almost 90% of his firm's $2,000 monthly long-distance phone bill has vanished. With 41 million people now using Skype, plus 150,000 more each day, it's no wonder AT&T (T ) and MCI Inc. (MCIP ) are hanging it up.
Advertisement

How can a tiny European upstart like Skype Technologies S.A. do a number on a trillion-dollar industry? By dialing up a vast, hidden resource: its own users. Skype, the newest creation from the same folks whose popular file-sharing software Kazaa freaked out record execs, also lets people share their resources -- legally. When users fire up Skype, they automatically allow their spare computing power and Net connections to be borrowed by the Skype network, which uses that collective resource to route others' calls. The result: a self-sustaining phone system that requires no central capital investment -- just the willingness of its users to share. Says Skype CEO Niklas Zennström: "It's almost like an organism."

A big, hairy, monstrous organism, that is. The nearly 1 billion people online worldwide -- along with their shared knowledge, social contacts, online reputations, computing power, and more -- are rapidly becoming a collective force of unprecedented power. For the first time in human history, mass cooperation across time and space is suddenly economical. "There's a fundamental shift in power happening," says Pierre M. Omidyar, founder and chairman of the online marketplace eBay Inc. (EBAY ) "Everywhere, people are getting together and, using the Internet, disrupting whatever activities they're involved in."

Collective Clamor
Behold the power of us. It's the force behind the collective clamor of Weblogs that felled CBS (VIA ) anchorman Dan Rather and rocked the media establishment. Global crowds of open-source Linux programmers are giving even mighty Microsoft Corp. (MSFT ) fits. Virtual supercomputers, stitched together from millions of volunteers' PCs, are helping predict global climate change, analyze genetic diseases, and find new planets and stars. One investment-management firm, Marketocracy Inc., even runs a sort of stock market rotisserie league for 70,000 virtual traders. It skims the cream of the best-performing portfolios to buy and sell real stocks for its $60 million mutual fund.

Although tech companies may be leading the way, their efforts are shaking up other industries, including entertainment, publishing, and advertising. Hollywood is under full-scale assault by 100 million people sharing songs and movies online via programs such as Kazaa and BitTorrent. The situation is the same with ad-supported media: Google Inc.'s (GOOG ) ace search engine essentially polls the collective judgments of millions of Web page creators to determine the most relevant search results. In the process, it has created a multibillion-dollar market for supertargeted ads that's drawing money from magazine display ads and newspaper classifieds.

Most telling, traditional companies, from Procter & Gamble Co. (PG ) to Dow Chemical Co., are beginning to flock to the virtual commons, too. The potential benefits are enormous. If companies can open themselves up to contributions from enthusiastic customers and partners, that should help them create products and services faster, with fewer duds -- and at far lower cost, with far less risk. LEGO Group uses the Net to identify and rally its most enthusiastic customers to help it design and market more effectively. Eli Lilly & Co. (LLY ), Hewlett-Packard Co. (HP ), and others are running "prediction markets" that extract collective wisdom from online crowds, which help gauge whether the government will approve a drug or how well a product will sell.

At the same time, peer power presents difficult challenges for anyone invested in the status quo. Corporations, those citadels of command-and-control, may be in for the biggest jolt. Increasingly, they will have to contend with ad hoc groups of customers who have the power to join forces online to get what they want. Indeed, customers are creating what they want themselves -- designing their own software with colleagues, for instance, and declaring their opinions via blogs instead of waiting for newspapers to print their letters. "It's the democratization of industry," says C.K. Prahalad, a University of Michigan Stephen M. Ross School of Business professor and co-author of the 2004 book The Future of Competition: Co-Creating Unique Value with Customers. "We are seeing the emergence of an economy of the people, by the people, for the people."

Peer Production
That suggests even more sweeping changes to come. Howard Rheingold, author of Smart Mobs: The Next Social Revolution, sees a common thread in such disparate innovations as the Internet, mobile devices, and the feedback system on eBay, where buyers and sellers rate each other on each transaction. He thinks they're the underpinnings of a new economic order. "These are like the stock companies and liability insurance that made capitalism possible," suggests Rheingold, who's also helping lead the Cooperation Project, a network of academics and businesses trying to map the new landscape. "They may make some new economic system possible."

Perhaps they already are. Yochai Benkler, a Yale Law School professor who studies the economics of networks, thinks such online cooperation is spurring a new mode of production beyond the two classic pillars of economics, the firm and the market. "Peer production," as he calls work such as open-source software, file-sharing, and Amazon.com Inc.'s (AMZN ) millions of customer product reviews, creates value with neither conventional corporate oversight nor market incentives such as payment. "The economic role of social behavior is increasing," he says. "Things that would normally just dissipate in the air as social gestures become economic products."

Indeed, peer production represents a sea change in the economy -- at least when it comes to the information products, services, and content that increasingly drive economic growth. More than two centuries ago, James Watt's steam engine ushered in the Industrial Revolution, centralizing the means of production in huge, powerful corporations that had the capital to achieve economies of scale. Now cheap computers and new social software and services -- along with the Internet's ubiquitous communications that make it easy to pool those capital investments -- are starting to give production power back to the people. Says Benkler: "This departs radically from everything we've seen since the Industrial Revolution."

Sound pretty threatening to anyone invested in the status quo? You bet. Indeed, as the title of Rheingold's book implies, there could be a dark side to this new cooperative force, especially if it results in mob rule. Quite often, the best solution to a problem comes from the sudden flash of insight from a solitary genius such as Charles Darwin or Albert Einstein. It would be a tragedy if these folks, sometimes unpopular in their times, got lost in the cooperative crowds. Clearly, peer production has its limits. Almost certainly, it will never build railroads, grow wheat, run nuclear power plants, or write great novels.

Yet this cooperative force may spread beyond such easily shared commodities as information, knowledge, and media. People are starting to use the Net to pool tangible goods as well. In a sense, Skype enables people to share computer hardware. Thanks to the Web's ability to serve as a meeting ground and scheduling coordinator, it's becoming economical to share cars, for example. Services such as Zipcar Inc. and Flexcar let members use the Net to reserve one of a fleet of autos in crowded cities, almost on demand, for an hourly fee.

What's driving all this togetherness? More than anything, an emerging generation of Net technologies. They include file-sharing, blogs, group-edited sites called wikis, and social networking services such as MySpace and Meetup Inc., which has helped everyone from Howard Deaniacs to English bulldog owners in New York form local groups. Those technologies are finally teasing out the Net's unique potential in a way that neither e-mail nor traditional Web sites did. The Net can, like no other medium, connect many people with many others at the same time.

What sets these new technologies apart from those of the Internet's first generation is their canny way of turning self-interest into social benefit -- and real economic value. They have what tech-book publisher Tim O'Reilly calls an "architecture of participation," so it's easy for people to do their own thing: create a link on their Web site to another Web site they like; rate a song; or just show off their knowledge with an online product review. Then, those actions can be pooled into something useful to many: the 3 billion song ratings that help people create personalized Net radio stations on Yahoo (YHOO )! Inc. or Amazon's millions of customer-generated product reviews, which help decide hits and duds. Exclaims Amazon CEO Jeffrey P. Bezos: "You invite the community in, and you get all this help."

It's surprisingly good help, too. New research indicates that cooperation, often organized from the bottom up, plays a much greater role than we thought in everything from natural phenomena like ant colonies to human institutions such as markets and cities. It's what New Yorker writer James Surowiecki, in his illuminating 2004 book of the same name, calls "the wisdom of crowds." Crowds can go mad, of course, but by and large, it turns out, they're smarter at solving many problems than even the brightest individuals.

The Internet's supreme group-forming capability suggests the rise of an almost spooky group intelligence. Within minutes of Pope John Paul II's death, hundreds of eBay sellers had posted related products for sale. Whether it is responding to world events or new products such as Sony Corp.'s (SNE ) PSP game machine, eBay's hive mind reacts to shifts in demand much faster than traditional companies with layers of management approval. Although eBay recently has seen some mature markets in the U.S. and Germany slow, the group smarts have helped keep growth more than respectable, with gross merchandise sales this year expected to rise 32%, to $45 billion. As eBay CEO Margaret C. Whitman has noted: "It is far better to have an army of a million than a command-and-control system."

More companies are starting to understand the logic. If they can get others to help them design and create products, they end up with ready-made customers -- and that means far less risk in the tricky business of creating new goods and markets. So businesses are accessing the cyberswarm to improve everything from research and development to marketing. Says Alpheus Bingham, vice-president for Eli Lilly's e.Lilly research unit: "If I can tap into a million minds simultaneously, I may run into one that's uniquely prepared."

Procter & Gamble's $1.7 billion-a-year R&D operation, for instance, is taking advantage of collective online brain trusts such as Lilly company InnoCentive Inc. in Andover, Mass. It's a network of 80,000 independent, self-selected "solvers" in 173 countries who gang-tackle research problems for the likes of Boeing Co. (BA ), DuPont (DD ), and 30 other large companies. One solver, Drew Buschhorn, is a 21-year-old chemistry grad student at the University of Indiana at Bloomington. He came up with an art-restoration chemical for an unnamed company -- a compound he identified while helping his mother dye cloth when he was a kid. Says InnoCentive CEO Darren J. Carroll: "We're trying for the democratization of science."

And apparently succeeding. More than a third of the two dozen requests P&G has submitted to InnoCentive's network have yielded solutions, for which the company paid upwards of $5,000 apiece. By using InnoCentive and other ways of reaching independent talent, P&G has boosted the number of new products derived from outside to 35%, from 20% three years ago. As a result, sales per R&D person are ahead some 40%.

The online masses aren't just offering up ideas: Sometimes they all but become the entire production staff. In game designer Linden Lab's Second Life, a virtual online world, participants themselves create just about everything, from characters to buildings to games that are played inside the world. The 45-person company, which grossed less than $5 million last year, makes money by charging players for virtual land on which they build their creations. Second Life's 25,000 players collectively spend 6,000 hours a day actively creating things. Even if you assume only 10% of their work is any good, that's still equal to a 100-person team at a traditional game company. "We've built a market-based, far more efficient system for creating digital content," says Linden CEO Philip Rosedale.

Likewise, groups online are starting to turn marketing from megaphone to conversation. LEGO Group, for instance, brought adult LEGO train-set enthusiasts to its New York office to check out new designs. "We pooh-poohed them all," says Steve Barile, an Intel Corp. (INTC ) engineer and LEGO fan in Portland, Ore., who attended. As a result, says Jake McKee, LEGO's global community-development manager, "we literally produced what they told us to produce." The new locomotive, the "Santa Fe Super Chief" set, was shown to 250 enthusiasts in 2002, and their word-of-mouse helped the first 10,000 units sell out in less than two weeks with no other marketing.

Corporate planners are even starting to use the wisdom of online crowds to predict the future, forecasting profits and sales more precisely. Prediction markets let people essentially buy shares in various forecasts, often with real money. Most famously, they've been employed in the University of Iowa's experimental Iowa Electronic Markets to determine, with remarkable accuracy, the most likely winner of the Presidential election. The ease of organizing groups on the Net has caused an explosion in their use, says Emile Servan-Schreiber, CEO of NewsFutures Inc., a consultant that has run 40,000 prediction markets for companies and publications.

Mob Mentality
Hewlett-Packard Co.'s (HPQ ) services division was having trouble a few years ago with forecasts in the first month of a quarter. So Bernardo A. Huberman, director of HP Labs' Information Dynamics Lab, set up a market with 15 finance people not normally involved in such planning. They bought and sold virtual stock that represented a range of forecasts at, above, and below the official company forecast. Their collective bets yielded a 50% improvement in operating-profit predictability over conventional forecasts by individual managers.

For all the benefits, Net-based cooperation holds plenty of peril for the unwary. Obviously, not all crowds are wise. Even The Wisdom of Crowds author Surowiecki wonders if the Net connects like-minded people so well that it can amplify groupthink. "The more we talk to each other, the dumber we can get," he notes. Groups that discourage independent thought potentially could put a damper on out-of-the-box ideas from brilliant individuals. They can also become herds that buy or dump stocks on momentum alone. For that matter, they can devolve into lynch mobs and terrorist groups.

As companies have learned, the online hordes can quickly turn against them. Last September bike-lock manufacturer Kryptonite tried to downplay a blogger video that showed how to open its bike locks with a BIC pen. But the video instantly spread across the Net, forcing the company to spend more than $10 million on lock replacements.

To contend with this rising people power, corporations will have to craft new roles for themselves and learn new ways to operate in order to stay relevant. They'll be unable to keep secrets for long amid the chorus of online voices, as Apple Computer Inc. (AAPL ) learned when fan sites spilled the beans on unreleased products. Managers and employees will have to learn how to take orders from customers more than from bosses. "Networks are becoming the locus for innovation," says Stanford University professor Walter W. Powell. "Firms are becoming much more porous and decentralized."

The challenges, though, go to show that we're not talking about merely a new capitalist tool -- at least not one that's dominated by big capitalists. Upstarts, both ad hoc groups and new companies, are seizing the initiative far more than are established businesses. They're transforming industry after industry faster than individual companies can cope with.

Nowhere has that phenomenon happened faster than in software. Collaborative open-source development is rapidly moving beyond basic utility software like Linux to mainstream applications as well. An especially eye-opening example is SugarCRM Inc., which provides an open-source version of customer-relationship management software now dominated by Siebel Systems (SEBL ) and salesforce.com Inc. (CRM ) The 10-person outfit's software, which CEO John Roberts calls "the collective work of bright CRM engineers around the world," has been downloaded more than 235,000 times for free.

The company makes money from services such as technical support and a $40-a-month Web-based service, as well as more fully featured corporate software for which it charges $239 per user per year. Scarcely a year old, SugarCRM won't reveal its finances, but its business model suggests a big change in how the software industry works. "The fact that everyone can participate [in open-source] is creating a new market ecology," says Kim Polese, CEO of SpikeSource Inc., a startup providing bundles of open-source products. Or, as Roberts adds brightly: "We're turning a $10 billion market space into a $1 billion market space."

The same scary prospect lies ahead for other information-based industries, such as entertainment, media, and publishing, that are rapidly going digital. People are not only sharing songs and movies -- legally or not -- but also creating content themselves and building sizable audiences. The threat comes from more than the 10 million-plus blogs. Overall, 53 million Americans have contributed material to the Net, from product reviews to eBay ratings, according to the Pew Internet & American Life Project.

The most breathtaking example: Wikipedia. Some 5 million people a month visit the free online encyclopedia, whose more than 1.5 million entries in 200 languages by volunteer experts around the globe outnumber Encyclopedia Britannica's 120,000, with surprisingly high quality. "Our work shows how quickly a traditional proprietary product can be overtaken by an open alternative," says co-founder Jimmy Wales. Unlike Britannica, Wales is not aiming to generate much, if any, revenue. But "that doesn't mean that we won't destroy their business," he notes. Britannica spokesman Tom Panelas says sheer volume of articles isn't a measure of quality and may be overload for most readers and researchers.

Then again, the cooperative crowds offer a lifeline to beleaguered media such as newspapers. The five-year-old online paper OhmyNews in South Korea has marshaled 36,000 "citizen journalists" to write up to 200 stories a day on everything from political protests to movies. Its popularity with 1 million daily visitors has made it the sixth-most influential media outlet in Korea, according to a national magazine poll -- topping one of the three television networks. "It's participatory journalism," explains founder Oh Yeon Ho, who says OhmyNews turned a profit last year. The idea is starting to catch fire in the U.S., too, via independent citizen-media efforts such as Backfence Inc. and Bayosphere and budding initiatives by E.W. Scripps Co. (SSP ) and others. The New York Times Co. is also testing the waters: In March, it bought About.com, which has 475 citizen experts on consumer electronics, personal finance, and other topics.

Even industries that traffic in physical goods are being turned upside down by Net-driven sharing. In retail, for instance, "consumers" are becoming active participants in the merchants they buy from, transforming the venerable suggestion box into something more influential. At Amazon.com, thousands of volunteers write buyer's guides and lists of favorite products. Amazon also lets thousands of merchants, from Target Stores (TGT ) to individuals, sell on Amazon pages.

What's more, Amazon is opening up the technology behind product databases, payment services, and more to 65,000 software developers. They're creating new services, such as the ability to compare brick-and-mortar store prices with Amazon's by scanning a bar code into a cell phone. Thanks in part to such moves, the company is solidly profitable on $6.9 billion in sales last year. "We're all building this thing together -- Amazon itself, outside developers, associates, and customers," says Jeff Barr, Amazon's Web services evangelist.

That raises a key point: All of us will have to take on more responsibility. And to get the most out of the new cooperative tools and services, we'll have to contribute our time and talent in new ways -- such as rating a seller on eBay or penning a short essay in Wikipedia. But the rewards will be more personalized products and services that we don't merely consume, but help create.

Ultimately, all this could point the way to a fundamental change in the way people work together. In 1968, ecologist Garrett Hardin popularized the notion of the tragedy of the commons. He noted that public resources, from pastures and national parks to air and water, inevitably get overused as people act in their own self-interest. It's a different story in the Information Age, contends Dan Bricklin, co-creator of the pioneering PC software VisiCalc and president of consultant Software Garden Inc. in Newton Highlands, Mass.

Instead, he says, there's a cornucopia of the commons. That rich reward may be worth all the disruption we've seen and all the more still to come.

(Source: http://www.businessweek.com/@@D1zKxYYQFbYvyQMA/magazine/content/05_25/b3938601.htm)

Information Visualisation

http://www.k-praxis.com/archives/information_visualization/000062.html

Map Hacks on Crack

The internet's two largest search engines are begging to get hacked.

On Wednesday, Google opened a set of programming interfaces for its popular service, in the hope hackers will overlay the maps with data from outside sources -- such as wireless cafes.

Just one day later, Yahoo announced its own set of programming tools for its map service, Yahoo Maps.

Both companies are hoping the new mapping APIs, or application programming interfaces, will excite developers, help the companies find new employees and, perhaps most importantly, result in free product prototyping.

But the search giants were also reacting in no small part to web hackers who had already started to reverse-engineer the two mapping services in ways neither had expected.
"We are doing this because they were already doing it," said Google Maps product manager Bret Taylor, half jokingly, at Where 2.0, a new conference focusing on innovations in mapping and location-based services.

Outlining Google's new offerings, Taylor stressed that Google is a company of engineers that wants to foster a network of developers and provide a formal way to work with outside programmers.

"Frankly we like new and innovative solutions," Taylor said. "We expect new and creative ideas to come out of this that we haven't thought of yet."

Before Google opened the Maps API, several "mashups" had been created, including Paul Rademacher's housing map, which layers craigslist housing ads onto Google maps, and an anti-gridlock site that marries Yahoo traffic data with Google maps. There's also Adrian Holovaty's ingenious Chicago crime map, which lets users create custom views of crimes from auto theft to bribery on maps as tiny as individual police beats.

Other hacks include a small-town walking tour with annotations, a map with clickable London traffic cameras and a map for finding cheap gasoline that's no longer online.

Rademacher's Google-craigslist hack earned him a star role at Where 2.0. Tim O'Reilly, the web-trend predictor at the helm of the O'Reilly publishing company, sees the future of the internet in these innovative, unpaid hacks.

"Google Maps with craigslist is the first Web 2.0 application," O'Reilly said, referring to his belief that the future of the internet will be created from interlocking connections of open data services that will be constantly improved and tweaked.

The API announcements illustrate that both companies are serious about devoting resources to outside projects that the companies have little control over.

The APIs, which specify the rules for what kinds of data can be passed to the mapping server and how the maps can be used, require user documentation, updates and support forums.

Not surprisingly, the two companies have different rules.

Yahoo is a bit more flexible in the kinds of data that can be passed and uses several open data standards, including RSS. The company also hosts the resultant map on its own servers, which could save hackers from having to pay for expensive bandwidth if their application becomes popular. It also allows Yahoo to serve advertising, if it chooses. However, the hosting offer is not negotiable, even for geeks with deep pockets who want the map featured on their own website.
Google, on the other hand, expects developers to host their own hacks by running Google's innovative JavaScript to power the map's smooth rendering, but reserves the right to place ads next to the mashup map in the future.

Yahoo sees the legitimized hacking as another extension of its effort to turn its customers into participants. Its local service, Yahoo Local, for example, attempts to merge blogging, restaurant reviews, social software and portable electronic devices.

Hackers, who had to constantly keep an eye on Google Maps for code changes to keep their jury-rigged applications running, welcomed the news. But they also had detailed questions about data specifications and gripes about the licenses.

"With this level of excitement around (Google Maps), it really makes sense for them to really foster development, instead of having people worry about their sites being taken down," Rademacher said.

They also started to build some Yahoo mapping hacks in less than 24 hours.
One such hacker married Yahoo's San Francisco Bay Area traffic data with live feeds from government traffic cameras to a map of the Bay Area. Another created a map-based tribute to the wine-tasting buddy film Sideways.

What's new in search: Prospective Search?

February 28, 2005

The ‘search’ functionality forms the base of most internet operations today.K-praxis has, in previous posts, speculated on the future of search. With the advent of ‘prospective search’ this functionality is now branching off in a different direction. What is the scope and potential of prospective search? How is it different from traditional search methods? What kind of a future does prospective search have? In this article k-praxis examines the definition and uses of prospective search as opposed to retrospective search.

Defining Prospective and Retrospective Search

Most of the search based on the Internet has been reference based. Google, the forerunner of all searches tends to give out results that are relevance oriented. Thats is, you type in a keyword and Google performs the search based on these keywords alone. Instead of using HTML, the delivery protocol for web pages is a desire for a new, feed-centric protocol, that is RSS. To search chronologically-ordered content, a relevance-based search like Google destroys the chronology and is inappropriate. More innovative in the search technology are sites such as PubSub, Technorati and Newsgator.

There are two ways in which a user can search the Internet for information:

Retrospective Search: This is the traditional form of search as performed by search engines such as Google, Yahoo! and other search engines. This method is called ‘searching the past’ where information is gathered by spiders, net crawlers and other forms of data entry. This information is sorted into a historical searchable selection. In such a system of search, queries are constantly changing and can be considered to be static. The results of these queries may change over a period of time but they are considered to be static because the results can be gathered only against the background of informationthat is already available on the web.

Prospective Search: This new method of search was introduced by PubSub. It is termed as ‘searching the future’. Information is gathered by a variety of methods against newly updated documents. Every time a prospective search is performed the queries are stored in the database while the results are discarded. When a new document matching the query is made available, the user receives an update. In this system of search, it is the queries that are static while the results are dynamic.

With information on the Internet constantly expanding, searching for information can become a cumbersome, tiring and confusing process which takes up a lot of time as the users sift though pages and pages of information. One could argue that retrospective search is more comprehensive and heterogeneous. While this may be true, prospective search is useful in that it reduces the time taken by users to search or repeat search for a particular bit of information.

Exploring Keyword Search

Merely using keywords to locate data isn't comprehensive enough. More often than not, you receive results that you do not care for. It is advisable therefore, to use key phrases instead of keywords alone. Phrases are less generic and more descriptive. This is the kind of language that a search engine understands.

Prospective Vs Retrospective

Prospective search is not meant to be viewed as a replacement for traditional methods of search. It is useful only when it is used to complement retrospective search. While retrospective search allows users to source information that is already known and documented, prospective search allows the user to be updated whenever a new piece of information pertaining to the query is added to the wealth of the Internet database.

Prospective search throws many doors open for those in the business of content creation and management. It means that newly generated content is immediately forwarded to those who have expressed an interest in the subject matter.

Prospective search is also changing the way the Internet works by making the web more personalized. Users can receive only the information they are interested in rather than having to go through large amounts of data that they may or maynot be interested in at all. Using prospective search their query is stored and the user receives a feed or an update every time something new (pertaining to the query of course) pops up on the Internet.

Instead of going through a lot of irrelevant 'junk' information it is simpler to use prospective search offered by PubSub, Technorati, Feedster, Blogpulse and Bloglines. These services are currently offered free. In order to make this kind of search more innovative, prospective search engines could team up with library databases such as Proquest whereby a user will receive a feed once any new information is available.

Conclusion

Prospective search will make the job of information hunting and gathering less time consuming and more accurate and updated. In fact the idea that users will be able to access the latest and updated information is its greatest strength. Prospective searches will be most useful for business purposes where the availability of new information is key to the progress of an enterprise.

However, it is still important to access information that is already known and this is where retrospective search holds sway. Prospective and retrospective search are complementary to one another. One cannot be used in place of the other. The future of prospective search is largely merged with the continued usage of retrospective search. The user will most probably use a combination of retrospective and prospective search to capture data.

Google Gets Earthy

By Daniel Terdiman
Story location: http://www.wired.com/news/technology/0,1282,68042,00.html
02:00 AM Jun. 30, 2005 PT

Google mappers can now tag and share their favorite locations with other users, who can then spot those choice Yosemite camp sites or find the most intriguing Polish medieval buildings.
On Tuesday, Google launched Google Earth, a free software package that gives detailed, 3-D views of cities across the globe replete with thousands of restaurants, schools, hotels and other establishments. It also provides amusing motion graphics of the route between locations or 3D cities.

But as useful as that kind of information can be, Google Earth's true special sauce is the way it allows users to create markers for just about any venue or location, write a note describing it and then share it with the application's entire user base.

"The goal with Google Earth is to create a kind of browser for the planet," said John Hanke, general manager of the Google Keyhole group, which developed the application. "(Our mission) was to take it in the most extreme direction in terms of making this a virtual model of the planet."

Clearly, Google believes that the best way to achieve that goal is to let users populate its databases with countless markers for places and things its own programmers would never think of.

For months, users have been gaga over Google Maps, which quickly and easily conjures up maps of nearly anywhere in the United States. Particularly popular is the service's satellite views, which show detailed images of neighborhoods or particular streets. And because of its open architecture, others can create imaginative hacks that combine Google Maps with third-party services like craigslist.

Google Earth is Google Maps on steroids -- and the company has incorporated the hacking element into the product by allowing anyone to add their own details.
The system already includes dozens of hacks and thousands of tags acquired by Google when the company bought Keyhole, a satellite imagery provider, last year. Keyhole has encouraged its community of users to post their own data about places and things throughout the world.
One is a hack that mixes mapping information with data from the bus system in Boulder, Colorado, showing in real-time the location of the city's buses on the 3-D map.

There are plenty of others: a lighthouse tour of the United States, good hikes in Marin County, California, the best camping spots in Yosemite National Park and lists of medieval buildings in Gdansk, Poland.

"I'm reminded of all the railroad nuts and how they could create an annotated map of the (United States), showing all the best bridges, tunnels and places to watch trains go by," said Matt Haughey, creator of the popular MetaFilter group blog and an evangelist of user-created content.

To Haughey, the open service means users can develop and share far more information than any one company could produce alone.

Indeed, according to Hanke, Keyhole users -- and now those of Google Earth – have been doing just that, and the service's bulletin board system is full of countless such examples, many of which can be found under discrete, searchable categories -- like Beijing, U.S. warships, theme parks and many more -- on the Keyhole BBS.

But Haughey cautioned that Google Earth could be open to exploitation.

"Looking at Portland (Oregon), I see someone made an entry for some storage place," he said. "Probably someone that works there or owns it. Kind of like map spam."

Still, Hanke feels the upside of an open-ended database outweighs the problems. He noted that the service could be expanded by users in almost infinite ways -- an open-endedness that mirrors the infinite extensibility of the web.

"You see the promise of a view that lets you understand the web and graphical information on the web," Hanke said, "with the map as kind of the underpinning of that."

The Dawn of the E-Lance Economy

In this eye-opening article, Thomas W. Malone and Robert J. Laubacher of the Massachusetts Institute of Technology look at how a new kind of organization could form the basis of a new kind of economy--an e-lance economy--where all the old rules of business are overturned and big companies are rendered obsolete.

Drawing on their research at MIT's Initiative on Inventing the Organizations of the 21st Century, the authors postulate a world in which business is not controlled through a stable chain of management in a large, permanent company. Rather, it is carried out autonomously by independent contractors connected through personal computers and electronic networks. These electronically connected freelancers--e-lancers--would join together into fluid and temporary networks to produce and sell goods and services. When the job is done--after a day, a month, a year--the network would dissolve and its members would again become independent agents.

Far from being a wild hypothesis, the e-lance economy is, in many ways, already upon us. We see it in the rise of outsourcing and telecommuting, in the increasing importance within corporations of ad-hoc project teams, and in the evolution of the Internet. Most of the necessary building blocks of this type of business organization--efficient networks, data interchange standards, groupware, electronic currency, venture capital micromarkets--are either in place or under development. What is lagging behind is our imagination.

But, the authors contend, it is important to consider sooner rather than later the profound implications of how such an e-lance economy might work. They examine the opportunities, and the problems, that may arise and anticipate how the role of managers may change fundamentally--or possibly even disappear altogether.

A good paper on e-lancing

http://www.janus-eu.org/Documents/EBEW/eLancing.pdf

The Future of Knowledge Management

by Ross Dawson

August 2004

In the much vaunted "hype cycle" of business trends and fads, knowledge management has already plumbed the depths of disillusionment. However as it edges towards maturity as a business discipline, it is spawning successors that are more relevant to our times, and that offer more direct business traction. The original premise of knowledge management was that if the most valuable resource of organisations is knowledge, then it should be leveraged and made more productive. This absolutely still holds. However the hype around knowledge management over the years has made what was always an amorphous and slippery concept even harder to grapple with and convert to business results. Today, managers need more focused frames, first to think about these issues, and then to take pointed action.

The rapid evolution of our intensely connected global economy means developing knowledge capabilities is a business imperative. The pioneers of knowledge management developed valuable tools and approaches. What they learned is now being applied in a range of emerging business disciplines. In the course of my travels and speaking and consulting engagements around the world, I have found that there are five key frames for leveraging knowledge in organisations that are emerging as the successors to knowledge management, and that executives find relevant, compelling, and actionable.

Social networks. Traditional organisational charts and business process maps tell you very little about how work is actually performed in an organisation. The reality is that work and knowledge flow in often highly informal patterns, based on who people actually communicate with in doing their work. Social network analysis is being applied by many leading companies around the world to gain insights into this "invisible organisation," and to design interventions that enhance the productivity and effectiveness of knowledge work.

Collaboration. In an economy based on highly specialised knowledge, collaboration is essential. Many of the approaches pioneered in knowledge management, such as communities of practice, are extremely relevant and useful. However what is critical now is a focus on fostering collaboration between individuals, teams, divisions, and organisations. Collaboration tools such as video conferencing and web conferencing are becoming standard. Now companies are working as a top priority on developing the skills and culture that enable high-value collaboration. However implementing a whole new set of businesses processes is also required to unlock the full potential of collaboration.

Relevance. In a world of massive information overload, we want to see only information that is highly relevant to our work and interests. Among the many evolving technologies that support this, there are two key practices that will be central to enhancing information relevance. Implicit profiling learns from what we search for and look at, when, and for how long, to improve over time at understanding what we find useful. Collaborative filtering allows us to draw on the insights and discoveries of people who have similar profiles and interests to us. Amazon.com uses similar approaches in a basic form to point us to books and CDs we might like. The future lies in finding relevance for individuals from vast oceans of information.

Workflow. Knowledge work literally flows through an organisation. The next decade will see companies shifting their business processes to platforms that enable smooth and efficient workflow. Once this shift is made, you can reconfigure at will how work is done, and even allow clients and suppliers to participate in your processes, creating powerful lock-in. The emerging discipline of "workflow learning" integrates access to every type of learning-whether it is information, elearning modules, or human experts-into the everyday flow of work, so these are available as and when they are needed.

Knowledge-based relationships. In our global hyper-connected economy, the drive to commoditisation is relentless. What this means is that an increasing proportion of business value resides in trusting, knowledge-based relationships, that allow companies to create value with clients, suppliers, and alliance partners in ways they could not do otherwise. Organisations are realising that outsourcing and offshoring only work if there are effective flows of knowledge between companies. Professional firms are finding not only that clients are increasingly demanding knowledge transfer, but also that engaging in knowledge-based relationships increases customer loyalty and profitability. Relationships are the future of society and business, and rich knowledge exchange will be at their heart.

Workplace Predictions

What's Next? Work Auctions? Resume Implants?

by James E. Challenger

Imagine one day employers bidding for workers in online auctions, work schedules set by the year vs. the week, and colleges offering a degree program in generational mediation.
These are just some of the scenarios that could result from a society where the number of retirement-age Americans - 65 and older - will grow at a much faster rate than the age groups that will be needed to replace them.

A Future Filled with Retirees

Between 2005 and 2015, the number of people 65 and older is expected to increase 26 percent. Meanwhile, the population of 40- to 54-year-olds will shrink by five percent. Furthermore, there is not much relief behind them - the number of Americans 25 to 39 will grow by only six percent between 2005 and 2015.

The retirement surge could result in a human capital crisis for employers across the country. For some, the crisis is a matter of months away. The federal government alone will see 31 percent of its workforce - nearly 500,000 employees - reach retirement eligibility by 2006.
This will present challenges for employers, workers and society in general. However, it could also present some intriguing opportunities for enterprising individuals and companies. The response to these challenges and opportunities will drive workplace trends, many of which are already beginning to take shape.

It may be difficult to envision amid downsizing and the highest unemployment rate in a decade, but within ten years jobseekers will again be in extremely high demand. This could lead to bidding wars over the most talented workers, just as it did in the late 1990s.

In addition to the aging population, the trends shaping the workplace of the future will be heavily influenced by rapidly advancing technology, such as nanotechnology; record downsizing and the push toward a just-in-time workforce; and women's ascent up the education and corporate ladders.

Here are my predictions of the top trends:

Mediators will be used to create generational harmony. With baby boomers vowing to stay in the workforce and a growing number of seniors over 65 coming back to work, it will not be long before companies see the age range of their workers span four generations. To help companies manage the delicate balance between these diverse generations, a new breed of consultant will be created: the corporate age advisor. He or she will mediate conflicts between age groups, ensure that each department has the right mix of generations to achieve its goals, and develop programs to increase the harmony between groups.

Online auctions will replace recruiting. Online auction sites have been used to sell everything from used record albums to multi-million-dollar homes, so why not temporary workers? Temps will become increasingly important as companies try to maintain a just-in-time workforce. However, labor shortages could drive up salaries as well as the fees paid to firms that provide temporary workers. To eliminate the middleman, companies will go directly to the workers through online sites. A precursor to this can already be found at reEmploy.com, which places retired workers with their former employers on a temporary basis.

Consultants will be hired to teach retirement. Where there is a dilemma, a consulting business cannot be far behind. As the nation's baby boomers begin reaching early retirement age in less than five years, a new business opportunity will arise: couples retirement consulting. Such a service will be in high demand, especially in dual-income households where both husband and wife retire around the same time. The adjustment of going from a situation where both are working to one where both are retired will almost certainly be a difficult one to manage.
Flextime will be measured by the calendar, not the clock. The typical 40- to 50-hour workweek will one day be a footnote in the history books, as households with two full-time earners push for more flexible scheduling alternatives in order to balance work and personal lives. One plan that may eventually take hold is setting an annual work requirement. In this option, workers would be required to dedicate 2000 to 2500 hours to work-related activities. This would be particularly well suited for industries and businesses that experience seasonal slowdowns which might otherwise compel companies to initiate layoffs.

Women's gains will help shrink wage gap. The gap between men's and women's wages will continue to shrink as women flock to labor-short industries for jobs. Women are branching quickly into sectors of the economy that are currently suffering or are expected to suffer from shortages of skilled labor. They also represent a growing portion of graduates in the high-paying fields of dentistry, medicine and law. More companies will launch programs designed to clear the obstacles hindering women's ascent to the upper ranks of the corporate ladder, such as mentoring programs, audits of compensation issues, and continuing education in countering harassment and discrimination.

Dot.coms will make a comeback. A rebirth of the high-tech industry, which has announced more than 1.3 million job cuts in the last three years, may occur just as rapidly as its downfall. Some of the growth will be driven by the Internet. Security is one area well-suited for the Internet and already there is a growing number of companies offering online background checks and identity confirmation. Another area of technology that will take off and require more workers in the years to come is nanotechnology, digital processing on a microscopic scale.

Corporate recruiters will follow the NBA in recruiting from high schools. Since the NBA has had success hiring raw basketball talent right out of high school, why not corporate America? The share of workers with post-high-school education will grow just four percent between now and 2020. As a result, corporations will be compelled to take over the education of their future workers. High school graduates will go directly to employer universities, where they will be educated in math, science and technology, the areas where skills seem to be weakest. Corporate educators will continue the tradition of teaching literature, philosophy, psychology, etc, to develop the reasoning and problem-solving skills which are also essential in today's workplace.

Entrepreneurs will get younger and employers will suffer. Undaunted by the downfall of the dot.coms, the generation in college during the recession that began in March 2001 may initiate a new wave of entrepreneurial ventures as the idea of working for someone else loses appeal. The number of colleges and universities offering programs and degrees in entrepreneurship has grown to more than 550, up from just a handful in the 1980s. Making room for entrepreneurs in the corporate structure is one challenge employers will have to consider, lest they lose some of the best and the brightest to self-employment. Companies that can offer positions allowing candidates to utilize their entrepreneurial skills will have an edge over their competitors.

Resumes will go high tech. Nanotechnology that will soon beused for tracking consumer products may one day be used in humans to store a tamper-proof record of one's personal, professional and educational background. A microchip about the size of a grain of sand could be imbedded in workers who, simply by passing through a radio frequency field (the doorway into the human resources department?), would send the information to the hiring manager's computer. Such technology will not only eliminate the need for traditional paper resumes, but it will save employers from hiring firms to conduct expensive background checks.

There will be civil war between the public and private sectors for workers. The federal government will battle with the private sector for the best workers, as retirements sweep away the most knowledgeable employees in both sectors over the next seven years. Labor shortages threaten to impact agencies critical in the fight against terrorism, including the departments of defense, transportation, and state. The Federal Emergency Management Agency could lose up to 45 percent of its employees. The deciding factor in the war for workers may come down to healthcare. While all federal employees are eligible for health insurance, the number receiving such coverage in the private sector is shrinking. Private-sector employees who do have coverage are being asked to pay a larger share of rising health-insurance premiums. If these trends continue, the hiring advantage will most assuredly go to the government.

e-Lancing

Temp freelancers mine the web for work

by Robert Celaschi
--------------------------------------------------------------------------------

After Bruce Bockie got caught in his third layoff in 1994, he knew he needed to take a different approach to his career. He didn't change his line of work - he's still a data process engineer - but he did change the way he offered his services.

Back then he was director of information and strategic planning at a large corporation in Sacramento. Now he works as an independent contractor, getting his clients from Internet job boards such as guru.com, dice.com and monster.com.

"I just signed up with Guru. I've used Dice for over five years [and] Monster for about three," he notes. Bockie browses the postings to find companies that need his expertise, then contacts prospects and offers his services. Once he finishes a project, he starts looking for the next one. It usually doesn't take long to find a new gig. "The longest break I've had is about three weeks."

No More Middleman
Bockie is an example of how the World Wide Web has blurred the old lines of employment. To get the same kind of short-term jobs a decade ago, he would have had to sign up with a technology-savvy temp agency or spend a lot of time knocking on doors. Today he's letting the Internet serve as middleman.

"I do something called business process re-engineering," he explains. "I go into companies and have them tell me how they would like to improve, and I facilitate that improvement through automation." Bockie has helped Nike Corp. streamline order processing so shoes can be shipped a few days after they are ordered, instead of the 30 days which had been the norm previously. He also helped the State of Nevada improve its welfare system.

Internet job-hunting is not limited to only the technically inclined. It works for freelancers in a wide variety of fields, as well as people looking for traditional permanent work. Even the old-line employment agencies have integrated the Internet into the way they do business.

Carol Larsen says it has completely changed the way she does her job as district recruitment coordinator for Kelly Services in Sacramento. The nationwide company posts jobs at www.kellyservices.com and participates in other job boards such as Monster.

"For us, the last two years have seen a huge use of the web," she confides. "I use the web on a daily basis. It has helped to speed up finding people."

In the old days, if a client company wanted Larsen to find new talent, she would first place a classified ad in the newspaper. Typically, the deadline was Friday afternoon for the Sunday edition. It would be Monday morning before the first applicants called, a gap of several days from the time the client company had asked for help. That lag time is gone now. "I can put an ad on the web today and get responses today," Larsen explains. "And in today's tight employment market, that's critical."

In some ways, temp agencies like Kelly end up competing with the same companies they are trying to serve, since those employers also post openings on Internet job sites. "But it's who does the better job of recruiting that counts, according to Larsen. For her, it's a full-time job.

For someone at a client company, recruiting new talent may be one of many jobs, and might not always have top priority. Employment agencies with well-known names like Kelly, Accountemps or Snelling also have the advantage of a lot of walk-in clients in addition to Internet "hits."

Another aspect is expense. Most job boards and agency sites are free to jobseekers. But the sites are there to make money, and they do that by charging the companies to post their job openings. Kelly spends $1600 a year to be on one Sacramento-area job board. Unless a company is large, it may not have the budget to post jobs on a lot of websites, she notes.

Jobs posted on the Internet run the full range from warehouse workers to executives. Even people with no Internet access or experience with the web can tap into its job-finding potential, thanks to statewide taxpayer- supported resources.

California Connection
The State of California has its own job service on the Internet at www.caljobs.ca.gov. Part of the Employment Development Department, CalJOBS has been on the web for about four years. "The majority of people who access CalJOBS come into the EDD office. We have staff who help walk people through it," says Michael Bernick, director of EDD. "We have about 200 offices and over 1000 people statewide involved in our job placement efforts. There's no substitute for the person-to-person connection."

Being state sponsored gives CalJOBS a couple of advantages. "First of all, CalJOBS is accessible at no cost: It doesn't cost employers and it doesn't cost jobseekers," Bernick points out. It's also connected to America's Job Bank, a nationwide employment database operated by the Department of Labor.

New Sites Every Second
It's hard to keep track of all the new sites popping up. Guru.com, based in San Francisco, has been around only eight months, and has signed up 100,000 freelance professionals, or "gurus." It also registered more than 9000 companies wanting to hire.

While Guru, Monster and others are generalist sites, there are also sites catering more to specific types of work. Dice, for example, concentrates on high-tech jobs, while staffwriters.com deals solely with writers, editors, researchers and graphic artists.

The "e-lance" route isn't for everyone. Being independent usually means having to track your own Social Security and employment tax payments, since there is no automatic withholding from paychecks. Health benefits, 401k plans and vacation days don't automatically drop in your lap. It can also mean extra hours tracking down late payments. And, of course, nobody pays you for the time you spend looking for the clients, discussing potential projects and figuring out how much to charge.

But even that extra work has its compensations. Bruce Bockie recalls one client that offered a job involving a lot of travel. The work was appealing, but the travel wasn't. "So I added on $20 an hour because I didn't really want to do it," he recalls.

The client still wanted him at that price. Suddenly the travel didn't seem so bad.

(Source: https://hestia.ntsecure.net/jobjournal/article_full_text.asp?artid=9)

How Marvellous Minds Make Money

This article was originally published in Vol 16 No 12 December 1999 of Executive Excellence - Australia Edition

The application of technology is rapidly and dramatically changing the face and form of business. We've read it, we've seen it, we know it. The business models of Internet companies—and increasingly all companies—are constantly changing in response to unpredictable shifts in the way business is done. Currently few seers in business attempt to look beyond a few years out, and even then they risk being proved wrong within a far shorter timeframe.

What this amounts to is an eternal game of catch-up, with extremely ephemeral phases of leadership for a few. The ubiquity of technological developments, as well as the shortening of the product development and marketing cycle, means that leadership in the implementation of technology can differentiate you from your competitors only fleetingly. The investment in technology which is necessary to gain leadership can be immense, yet provide advantage only briefly. Substantial investment and carefully considered implementation of technology will be fundamental to the survival and success of all businesses from now on, but this alone will by no means ensure competitiveness or success.

So what will make the difference into the 21st century? The answer lies in the distinction between information and knowledge. Information is what is or can be digitised, and thus easily stored, copied, communicated and accessed through databases and systems. Knowledge is the capacity to act effectively in complex situations, which for a long time hence will be only a capability of humans. In the realm of information, by its very nature, it is almost impossible to gain lasting advantage. Knowledge, however, whether it be held by individuals or in the capabilities of effective teams, is extremely difficult to replicate, and is itself the source of innovation in technology, marketing, product development and strategy that creates superior value for the organisation and its clients.

So while technology will provide the backdrop for business into the next millenium, the only lasting source of differentiation will be in the knowledge of the people in the organisation, and how effectively those people can work together to create results. The so-called field of knowledge management is beginning to address the issues of providing the technological links and storage which allows information to be shared between people. This is an important but limited part of how to develop knowledge capabilities. Real improvement of individual's and teams' knowledge capabilities is based on understanding how people acquire, use and create knowledge. In this, business has an immense amount to learn from the field of cognitive psychology, which has very practical application in presenting information and ideas more effectively, assisting effective high-level thinking, and making better decisions as individuals and groups.

The productivity and effectiveness of knowledge workers—whatever the field or industry in which they work—are almost by definition based on their skills at dealing with information and knowledge. When adding value to information, generating knowledge, and applying knowledge to business processes are the primary source of value creation, then people's capabilities at information and knowledge processes such as filtering information, analysis, decision-making and conceptual communication will determine success. Human skills are increasingly become the primary source of differentiation, and while many knowledge workers are already excellent at knowledge tasks, their skills can and must be developed further on an ongoing basis in order to to maintain and develop competitive advantage.

In many ways the most valuable knowledge skill in the next millenium will be facilitation, in the sense of assisting groups to pool their expertise and knowledge constructively in making effective decisions and creating new knowledge. Most organisations hold immense knowledge in their staff and executives, however in many teams and committees the diversity of views and opinions represented can result in decisions and actions which reflect power and politics rather than what will be best for the organisation. Good facilitation uses frameworks that allow a wide variety of perspectives to be synthesised and integrated into a richer view of the whole to result in better decisions, rather than resulting in destructive arguments and conflict.

The complexity of the business environment in the 21st century will undoubtedly continue increasing ever further; in this world effective strategic thinking and action will require richer ways of thinking. While computers will take more and more low-level work functions from people, the domain of high-level strategic decision-making will remain the work of humans. Thinking, creating knowledge and making decisions more effectively as individuals and groups will emerge as the only true source of sustainable differentiation and competitiveness, despite the popular focus on the ever-changing context of technology, which will remain a tool and not an end in itself.

While arguably the primary field of business competition is already in attracting and retaining the best talent and knowledge, it is exciting to envisage a world in which developing the dynamic knowledge capabilities of individuals and organisations is recognised as the key source of advantage. We stand on the threshold of that world, and those that leap into it before others will have a great headstart in what will be an incredibly challenging and exciting race.

Information farms in developing countries
© Copyright 1994-2002, Rishab Aiyer Ghosh. All rights reserved.
Electric Dreams #3424/October/1994

The greatest effect of the information revolution will be felt in developing countries. While in much of the world, the growth of connectivity means a choice of 500 channels on television instead of 50, this is hardly revolutionary. Allowing people to do more of what they do already is an incremental change over the past. Change is revolutionary when it enables large sections of society to do what they could never have done before - what they could never have even dreamed of, and what could totally change their way of life.

Rural communities throughout the developing world remain largely agricultural. They have remained poor through the industrial revolution. Although their standard of living has improved, their position in society has not. They have remained a source of cheap labour, while wealth, capital and power accumulate in urban population centres.

It is the nature of industry to centralize. Industrial societies prosper when the sources of wealth are collected together in physical proximity, which is why cities are formed. Agriculture, on the other hand, is decentralized - the more sprawling the countryside, the better. Rural communities are intrinsically suited to such a decentralized economy, dispersed as they are, away from any single concentration of power. But there are alternatives to farming wheat and sugarcane.

The information revolution threatens to reverse the industrial economy's magnetic pull to the cities. In the information age knowledge, not the diesel turbine, is power. Luckily knowledge does not have to be carted around from place to place in expensive trucks and cargo trains. Knowledge floats rapidly through far cheaper modes of transport, from radio waves between satellites to flashes of light reflected within strands of fibre optic cable. Knowledge does not need to be geographically concentrated. In fact, an economy based upon knowledge grows especially when it is possible to take advantage of the fluidity of data, selecting at an instant from the vast array of choices available only because they originate from the whole world. If access to information were limited to the physically near, it would lose much of its value.

The creation of wealth in an information economy is not dependent on the fertility of the soil, nor on huge investment in monsters of cogs and wheels in conventional industry's monuments to centralization, but only on the human brain - and the means to get brains together, over geographical distances.

Electronic mail is already available in most countries of the world, even in many poor regions of Asia and Africa. Building an information infrastructure is not as expensive as building an industrial one, and its enormous catalytic benefits for overall growth and prosperity are being recognized in many developing nations. If they act, then people still stuck in Toffler's First Wave of agriculture might leap finally into the Third Wave.

The information revolution could do for rural and poor communities what the industrial revolution could not - distribute control and capital among those who currently rely on their exploitable labour for survival, not possessing large banks of land or money. Capital and knowledge are increasingly becoming equivalent, and nothing necessarily prevents knowledge being created in villages.

Knowledge brokers and the decline of corporations

© Copyright 1994-2002, Rishab Aiyer Ghosh. All rights reserved.
Electric Dreams #367/November/1994

An information economy will have little use for traditional corporations with their hierarchy of managers and employees. Decentralization will be taken to its extreme limit, with the individual as the smallest economic unit of production. Greater and freer networking between individuals will lead to more efficient channels of communication, to flexible organizations of information capital and expertise.

Corporations are formed for a number of reasons. Their primary function is to collect - to bring together in a legal and often geographical entity a group of people, capital and expertise; and to present a unified interface to the rest of the world. Improvements in technology and resulting changes in society will make much of this unnecessary. The spread of anonymous digital commerce and multiple privacy-protecting identities may, in fact, make rigid corporate structures impossible to sustain. As more of the present advantages of a corporation become available to individuals, companies will have to yield greater control to them through flexible structures in order to exist at all. Of course all this is primarily applicable to information age industries - oil refineries will be less affected.

Many knowledge workers already telecommute. Working from home (or laptop) offices, telecommuters don't need a corporation as a physical place of work. The corporation needs knowledge workers because of their information capital. Knowledge workers themselves usually require little capital other than information, their own or that of other workers. What the traditional corporation offers are sophisticated ways of getting all that knowledge together and presenting it to end users, the customers.

However, it is becoming easier and cheaper for those who want services to communicate directly with those who provide them, and for knowledge workers to share information without intermediaries. By limiting individuals' resources to others within an organization, rather than the pick of the world's best, intermediaries may even have a negative effect on productivity. In the past, such hoarding of trade secrets provided an edge over the competition. In the future it will be very difficult to keep information secret - a fallout of advances in cryptographic technology has been the appearance of spontaneous anonymous markets for knowledge, allowing individuals to bypass their employers to benefit from their expertise.

The model for future information corporations already exists. Stock exchanges (and investment banks to some extent) are very loosely held organizations. There is some amount of common infrastructure and interface to customers, but there are relatively few restrictions on flows of information and capital. The more participants depend on their own capital or expertise, the more they directly gain. A choice of the best alternatives is always available.

A knowledge exchange can be even more open. Customers will always have access to the best resources; and providers of services themselves will be able to source additional expertise from anywhere in a global information market.

The information economy will work when the fluidity of knowledge is at its peak. This will be achieved by removing restrictions on the distribution of capital and expertise, by decentralizing the economy to the maximum where individuals join forces depending on their changing needs and abilities, by letting knowledge workers become knowledge brokers.

Knowledge, revolution, and cotton-mill capitalism

© Copyright 1994-2002, Rishab Aiyer Ghosh. All rights reserved.
Electric Dreams #8027/November/1995

Knowledge revolution is a catchy term, that's why it's been overused into being almost an epithet. Actually, what is happening in the world today is hardly a revolution at all. Although the shape of the world may be changing, it will not happen all at once, but over time. In the near future, there may be confusion and excitement, and even a few big changes, but anything remotely revolutionary will come much later. We should not, however, be misled by the similarities between the present and the apparent future, into examining the knowledge economy through the lens of today's industrial one.

This is especially true in our understanding to the relationship between the knowledge economy and industry, and of the impact of social, legal and political structures favouring knowledge upon society in general.

Knowledge is often treated as an extension of industry. It doesn't have any identity of its own - yes, there is all that stuff about bits rather than atoms, but then there are also industry's more familiar distinctions: between the services provided at a check-out counter and the goods being bought. Knowledge will be just another ingredient in the industrial soup, of corporations, cities, fixed jurisdiction, hierarchical government, the nation-state and, of course, money.

If all this were to change, well, then it would really be revolutionary - but not a revolution.

The period of the industrial "revolution" in England stretched over some fifty years, well into the 19th century. For an enormous length of time for a revolution, the new industrial economy was based, eventually, upon agriculture. This meant not just food, the pre-agricultural commodity that all societies must eventually depend upon. No: the huge factories and mills that were the emblem of progress and the new age, before, during, and after the introduction of mechanisation, were all dependent on the cotton crop. Without cotton, flax, wool, silk and other such raw, agricultural, materials, all the automated looms in the world would do no good.
Arguments for the continuation of industrial socio-economic organisation in a knowledge economy invariably rest upon the supposition that knowledge is useful only when applied - to industry. So an oil company networks its offices, organises its expertise, and becomes a "knowledge company". A software firm is wealthy, according to this line of reasoning, only because its products support drilling rigs. In fact, knowledge will be traded for knowledge and grow as apart from industry as the latter has grown, today, from the cotton crop.

By the time industry had become anything like what it is at present, built several layers over agriculture and apparently quite separate the social infrastructure of capitalism was already in place. Labour and capital swarmed to the cities; informal liberties were replaced by formalised rights and new forms of punishment; the distributed power of local councils or parishes was subverted by the strengthened central authority of the nation-state. It was only then that the railway, not the mill, became was the emblem of the times. It took even longer for the arrival of the car, symbol of purely industrial production where the train was one of transport.

The impact of this social infrastructure was much wider than the domain of industry itself, extending into everyone's lives and even to whatever agricultural economy remained. The impact of the knowledge "revolution" could be just as universal.

The social infrastructure brought by knowledge, of decentralised power and informal systems of justice and trade, cannot be expected to materialise immediately. But that change is slow cannot be taken to mean it is not happening. Knowledge is at least as revolutionary a socio- economic force as was industry. And it will result in great changes equally gradual.

Information Overload - Problem or Opportunity?

This article was originally published in Company Director, October 1997

"We have for the first time an economy based on a key resource [information] that is not only renewable, but self-generating. Running out of it is not a problem, but drowning in it is." John Naisbitt, author of Megatrends.

Information overload is a fact of life for company directors, senior managers, and all professionals. Information is coming in from all sides in the form of reports, memos, newspapers, journals, and letters, and now the advent of e-mail and Internet has turned the torrent into a flood.

How can directors cope with the onslaught? Or rather, since we are businesspeople, it seems the question should be how can we turn the reality of the new business environment into an opportunity and a competitive advantage?

"The competitiveness of firms will reflect the way their businesses receive and process information to create intelligence." John Prescott, Chief Executive, BHP.

Sustainable competitive advantage is ultimately about making better strategic and management decisions than your competitors. And those decisions need to be based on having a better insight than your competitors into how your business environment is developing.

The increasing globalisation and diversification of business means that you can no longer look only at your own industry in your own town. Developments in other countries and industries and in technology could all suddenly and dramatically change the dynamics of the business your companies operate in.

And this is one of the main reasons information overload is a vital issue - not only has there been a dramatic increase in the availability of information, but you now need to be informed on far broader range of issues in order to guide your companies forward in an increasingly rapidly changing environment.

"In the period ahead of us, more important than advances in computer design will be the advances we can make in our understanding of human information processing - of thinking, problem solving, and decision making..." Herbert Simon, Economics Nobel-prize winner

Technology is perhaps at the root of the advent of information overload, though it will also be part of the solution, by providing increasingly refined ways of filtering data to present the most valuable information to the right people.

In the end, however, people are the ultimate processors of information, and - until computers replace company directors - people are the only ones who can effectively integrate and synthesise what they know to make effective management decisions in the light of their business environment.

So, what can we as managers and directors do to improve the way we deal with a world of massive information input, and in fact turn our ability to do so into one of our core competences, and a source of competitive advantage?

There is no quick fix; enhancing our skills requires effort and is an ongoing process. Implementing the following principles can make a real difference to your effectiveness in dealing with the new reality of information overload, and in your role as a company director.

1. Set information objectives. You can't begin to sort through the 'infoglut' unless you know what is most important to you. To set effective information objectives you need to start from your own objectives and those of the company boards you sit on. In order to achieve these, what do you need to know about what is happening in your company, your industry, technology, the economy, and around the world? What are the forces at play which might impact your business and industry? Putting thought into identifying the key areas you need to be informed on, and prioritising these by importance and timeliness will let you know what you should be focusing on.

2. Select your information sources. Once you have a clear idea of your information objectives, you can make a deliberate decision about what sources you will use. Which reports, newspapers, magazines, journals, news services and television programmes do you need to look at regularly? Once you know your priorities you can often rationalise the sources you need, rather than being swamped by a mountain of paper you can never get through. Can you use media summaries or industry updates? Getting the information you need more efficiently is well worth the cost of a subscription. Aim to read things your competitors aren't likely to see - get a broad perspective.

3. Set time aside for reading. Because reading is not urgent, it often doesn't get done. Reading and learning are what Stephen Covey refers to as Quadrant II activities - important but not urgent. You will make better decisions and be more effective for the rest of your working career if you keep abreast of developments in your industry, the business environment and technology. How long you should spend reading each day will vary tremendously for each individual, though most directors would benefit enormously from spending at least two hours each day learning more about developments in their industry and the broad social, economic and business environment. Block out periods for reading each day at times which work for you - whether it's in the early morning, after lunch, the evening, or while commuting.

4. Filter aggressively. You have to be ruthless in getting rid of surplus information. Get yourself off mailing lists, use assistants to filter your messages, and use e-mail filtering software. From there, filtering is in its essence a process of scanning everything to judge according to your information objectives whether it's worth reading or dealing with in more detail. Those things that pass are read or put aside for your reading time, the rest go in the bin. If you're still getting too much to read, you need to go back to your objectives and refine them until you only get all the most important information that you have time for. As knowledge management guru Karl Erik Sveiby points out, most information has negative value: if you read something and it's not useful, you've wasted your valuable time.

5. Be open to useful information. This seems to contradict the last dictum, but both must coexist. Nicholas Negroponte, director of MIT's Media Lab and author of "Being Digital", talks of the 'serendipity factor' in information. If you're too blinkered in what you look at, you may never stumble on the most valuable information and insights. Companies and industries don't exist in isolation, they are all part of a broader system. Some executives skim through a trade journal from a different industry each month, for insights into their own situation. Much may be irrelevant, but you can also find valuable information and insights, and you can be sure your competition aren't seeing it!

6. People are your best resource. In all the hype over Internet, intranets, groupware and push technology, many seem to have forgotten that often the most effective and efficient way of finding information is through people. How well-informed you are depends largely on the quality of your personal network, and how effectively you can gather and trade information through the people you know. If you need to learn about recent developments in a field or find out something - ask the right person!

7. Develop your reading and note-taking skills. There are many courses and books available on developing reading speed and comprehension. If you're going to spend a couple of hours a day reading, then it's well worth investing time and money if it can bring even a small improvement in your effectiveness. And everyone, however fast and effective they already are, can improve their reading skills. Taking effective notes can be one of the best ways of assimilating new ideas and perspectives into a framework which can help you make better decisions. Mind-mapping, which allows you to represent the relationships between concepts, can be a very valuable tool for managers and directors.

8. Sleep on it! The human mind learns by connecting new ideas and information with your existing memories and experience. The most useful insights and perspectives often come after your unconscious mind has had the opportunity to sort through and link ideas together. Some of the ways you can assist this process and tap your intuition are by finding ways to represent your ideas visually, regularly putting yourself in relaxed, meditative states, and playing games which require spontaneity.

Meeting Government's Need for Enterprise Search

Sameer Kalbag of Convera

The government market for enterprise search products has witnessed substantial growth over the past several years, a trend that is projected to continue well into 2005 and beyond. This increased demand for search solutions can be attributed to several major factors:
Information overload: The ubiquity of digital communications, personal computers, cheap storage and Web publishing has created an ever-growing glut of digital information. To deal with the huge influx of this digital content, government agencies are looking for innovative solutions that help filter, find, organize and route information to those persons who need it, when they need it.

Legislation: The E-Government Act of 2002 is fundamentally reshaping the way government does business. Subsection (207.f.1), for instance, requires all federal agencies by the end of 2004 to have committed a strategy and timetable for making pertinent public information available in electronic form. Effective search can improve the accessibility of published information by reducing the time spent by users to find relevant information.

Evolving threats: The events of 9/11 and subsequent war on terror has forced the intelligence, law enforcement and defense departments to rethink the ways they gather, re-use and share information both within and across organizations. Effective search technologies play a critical role in helping analysts connect the dots.

Technology-savvy users: A Web-savvy population, coupled with the popularity of Web search engines, has turned “search” from a skill used primarily by librarians and researchers to an everyday household activity. Both government employees and the general public have come to rely on search as a reliable tool to find the information they need quickly.

For government organizations starting to deal with these issues, choosing the right search solution can be challenging. From search appliances to question-answer systems, the choices seem endless. To complicate matters, many vendors claim to support similar features but differ greatly in their approach and level of coverage. To ensure successful deployment, organizations need to move past the product literature and high-level feature checklists to gain understanding of the fundamentals of enterprise search and how it applies to their environment.

Enterprise Search vs. Web Search

For many people, the term “search” has become synonymous with services such as Google, Yahoo and MSN. A user can enter a few keywords into a search box, press a button, and usually find some relevant Web pages within a few clicks. The buzz generated around these businesses has led some IT decision-makers to assume that the same approaches and technologies for Web search can be directly applied within an enterprise. Closer examination shows that the spaces are quite distinct. To see why, let’s look at few of the underlying assumptions of Web-search.
One significant difference between enterprise and Web search involves the nature of the content being searched over. Web search companies have invested heavily in technologies to address filtering of spam, promotion of paid sponsored links and development of algorithms that depend primarily on the social popularity of Web sites. While effective over the entire World Wide Web, these techniques provide little or no advantage when applied to enterprise assets such as office documents, database tables, XML records or internal Web sites. Second, simple keyword searching works well in most Web search engines because chances are extremely high that someone on the WWW has already constructed a Web page that uses the exact keywords specified by the user. The scale of the Web and social network effect contributes to the impression that the highly relevant results are always returned. A more accurate description of what is going on is that users usually get an acceptable answer, not necessarily the best or complete answer. Users rarely realize this is because of the scale and patchy nature of the Web. For enterprise search, different approaches beyond keyword search and popularity-based ranking are required to get the best documents back and to ensure higher recall of all the relevant information to a query.

Data Aggregation

In most organizations, critical business information is spread across file-servers, Web-servers, databases, desktops, content management systems, e-mail, collaboration servers and other business applications. Most of these systems limit searching to content managed locally on the server and provide very basic search capabilities. An effective enterprise search solution can often solve this problem by providing a single access point that spans across multiple heterogeneous repositories. The decision of which content repositories/data to index is highly dependent on the business problem being solved. For example, let’s say a regulatory agency wants to reduce the time it takes their agents to research a complaint. By indexing the e-mail server mailbox where complaints are received, an e-mail archive of historical complaints and a database where investigative evidence is stored, agents would be able to quickly piece together all relevant information about a case. When evaluating vendor’s capabilities in this area, data-integration, issues to consider are:

Does the vendor support the content repositories you plan to index?
Does the solution deal well with indexing databases?
Does the solution use native APIs to connect to the repository?
How quickly are changes in the underlying repository reflected in index?
Does the vendor provide a robust, scalable spider to crawl internal Web sites?
Does the solution allow you to configure which custom metadata is retrieved from each repository?
Does the solution provide a flexible API to allow integration into legacy IT applications?
Does the solution support hierarchical objects: embedded files, zips or multi-page scanned documents?

Note that indexing solutions that rely primarily on Web crawling instead of native APIs will be limited to finding assets accessible through a Web page and have very limited flexibility in dealing with metadata.

Security

While the power of search stems from its ability to aggregate data, along with its benefits come some risks. The highly publicized incidents of personal data theft from companies such as Seisint and Choicepoint illustrate the problems that occur if appropriate policies, procedures and technologies are not in place to protect access to information. Clearly, strong security capabilities must be a cornerstone of every enterprise search solution. Issues to consider are:

Does the solution use industry standard authentication and access control protocols?
Does the solution protect server-to-server communication?
Does the solution provide flexible policies for choosing what is indexed, what is not indexed and where indexes are stored?
Can the system overlay additional access control policies for logical groups of aggregated data?
Does the solution provide access control down to the user and document level?
Does the solution integrate with IT directory infrastructures used for managing users and group membership (e.g. LDAP, Active Directory, Custom)?
Can the solution use document access control attributes stored in the repository?
Can the solution deal with multiple authentication and access control systems available for each repository?
Can the security system be extended to support custom access control schemes?
Does the system provide sufficient audit and logging capabilities?

Carefully evaluate solutions to determine if vendors have added security as an after-thought. These add-on approaches can leave security holes or result in significant performance impacts when security is turned on.

Metadata Search

While keyword searching represents a base capability available in all search engines, more sophisticated search strategies are often called for. In many circumstances, end-users can more quickly locate relevant information assets by using attributes of the content rather than simple keywords applied directly to the document contents. To enable powerful metadata search, solutions should be evaluated on their ability to support:

Indexing of multiple metadata schema in the same repository;
Mapping of metadata schema across repositories
Powerful parser scripting language to easily extract metadata from unstructured documents;
Indexing, extraction and search of metadata stored in XML records;
Natural language, integer, floating point and date searching over metadata fields;
Exact match as well as range searches for numbers and dates;
The ability to aggregate metadata stored in separate repositories at runtime (e.g., a system should be able to index documents in a file system while using metadata attributes stored in separate RDBMSs);
Query operators such as AND, NOT, OR and field weighting across the document body and across fields; and
Incremental indexing for a document whose useful lifetime is short or metadata changes often.

Text Mining and Advanced Discovery

Aside from the modes of searching described so far, many civilian regulatory, law enforcement and intelligence organizations need sophisticated discovery capabilities around the area of text mining. Let’s say, for example, an intelligence analyst is given the task to scour large amounts of raw intelligence data to determine the most likely location of a wanted terrorist. Obviously, using a simple keyword query such as “Where is terrorist XYZ?” will be fruitless. The clues to the answer may be spread across multiple nuggets of information that only a human analyst can reasonably piece together. The purpose of the search application, then, is to provide a set of tools to enable analysts to efficiently explore the information space, filter out noise, connect the dots and see a pattern. Many different questions will be asked, which over time will help the analyst piece together the puzzle. For these types of applications, the analysts require a wide range of capabilities including:

High-performance and scalable system that supports complex queries (thousands of terms and operators);
Ability to work with extremely large sets of results;
Storing of results and queries for reuse and refinement;
Filtering huge amounts of information, signal detection and notifying users of new relevant information;
A powerful query language with, for instance, Boolean, proximity and adjacency, wildcard and pattern-matching operators;
Support for multiple languages, diverse document formats and noisy data;
Easy integration with other analysis tools and repositories;
Ability to slice and dice data to see the big picture and quickly drill down to relevant information;
Support for searching over time-based multimedia assets;
Support for multiple languages and cross-lingual search;
Customizable domain-specific knowledge resources;
Entity extraction; and
Co-occurrence detection.

The Change to E-government

The push to e-government and the ubiquity of digital information is forcing agencies all across the government to re-evaluate the way they get business done on a daily basis. Enterprise search solutions can introduce significant efficiencies and cost savings by helping agencies filter, find, organize and route critical information to those persons who need it, when they need it. When selecting search solutions, it is important to understand that different business problems require fundamentally different approaches to search.

The Future of Work
RAND corporation
http://www.rand.org/pubs/monographs/2004/RAND_MG164.sum.pdf

Optimizing sales territories

http://www.empower.com/territory_management.htm

Managing Silos of Knowledge Within Government Agencies

Mark L. Moerdler, Ph.D., of MDY Inc.

Virtually every government agency has invested substantial capital and resources to implement specific systems for different departments to create and manage documents. This has historically been done for numerous reasons:

Different departments have different needs, therefore best served with different document, imaging, content or e-mail management systems;

Some departments have additional applications (e.g. case management, accounting) that manage documents; and

Government re-organization has led to the merger of agencies with pre-existing systems.
The effect has been the creation of different repositories, or “silos of knowledge,” such as:
Physical records in one or more records centers, or those stored off-site;
E-mails stored in an e-mail and/or e-mail archiving system;
Documents in a document management, content management or collaboration system;
Electronic documents in specialized systems; and
Images in an imaging system.

These silos, although coveted for departmental efficiencies, create challenges as well. Agencies are mandated to manage records in accordance with federal, state, local and departmental policies and procedures. These include:
Retention schedules that define how long records, independent of media or form, should be retained;
Proof the record is an unedited original; and
Detailed audit trails.

The challenge is how to apply these rules across independent silos of knowledge:
The U.S. DoD 5015.2 Standard—mandated to most federal agencies and adopted by state and local levels—defines requirements for the management of records.
Document discovery becomes more complex as silos to search increase—especially important given the Freedom of Information Act.

Managing Silos of Knowledge

Agencies are presented with three alternatives to managing silos of knowledge:
Separate silos with separate records, where each silo is managed separately;
Existing silos are replaced by a single-vendor solution; and
Federated RM, in which all silos are managed by a single enterprise-wide records management system.
Separate silos with separate records systems: While managing silos separately can be efficient for departmental needs, meeting records requirements with them, or adding records capabilities directly into each system, can fall short for several reasons:
Many of these applications have limited, if any, retention management capabilities. Their capabilities are often inconsistent or incomplete, resulting in failure to meet records requirements;

Most applications do not meet DoD 5015.2. Even if not mandated to deploy a certified application, by using a certified product an agency ensures functionality and compliance to requirements;
Records capabilities of e-mail archiving systems are limited; and
Discovery and FOIA searches must be performed separately in each application, leading to an arduous and inconsistent document-hold process based on differing functionality across products.

With this approach, riddled with inconsistencies in process, the agency can expend valued resources while still not meeting records management requirements.

Single vendor solutions:

Under this approach, the agency would select a single vendor’s solution to manage all documents, e-mails, images and physical documents together, replacing existing systems. This approach, while idealistic to some, has drawbacks as well:

Hard costs (e.g. license fees, hardware, consultants, document/data migration, training) and soft costs (e.g. productivity loss) of replacing existing systems are often substantial;
This solution may not offer specific functionality required by the agency and custom development would increase ongoing costs, failing to meet requirements; and
These solutions typically do not have sufficient functionality to manage the physical records of the agency, resulting in lost records and/or costly attempts to image assets.
Implementation of a single agency-wide solution is expensive, and often leads to decreased efficiency due to limited functionality. In many cases, this results in a “check box” rather than an actual working solution that will haunt the agency when legal or regulatory discovery occurs.

Federated RM: Under the Federated RM methodology, records knowledge and functionality exist outside the silos in much the same way that an agency’s network infrastructure exists separate from the agency’s servers and workstations. This approach has the following advantages:
Assures that all records are managed consistently while simplifying administration, allowing records to remain in their source repository while controlled by the records management system;
Agency-wide discovery and FOIA compliance become simple, while disposition holds can be quickly applied;
A natural outgrowth is the ability to run agency-wide searches, within the limitations of the security paradigm, effectively gaining a substantial knowledge management advantage;
Agency-wide document security becomes available; and
Substantial cost advantage with one records system to maintain—compared to the management of many with separate silos, or requiring the replacement of existing systems.
This best-of-breed approach is the most cost-effective, feature-rich and scalable approach offered for agencies with disparate repositories that must be managed.

Summary

As agencies attempt to meet the growing requirements to manage their records, they have three basic options:
1. Manage records separately in each silo;
2. Convert to a single vendor for management of all content; or
3. Implement an agency-wide Federated RM solution that manages records across existing silos of knowledge.

Each solution has some advantages, however the best solution for most agencies, based on ROI and flexibility required to meet diverse departmental and business requirements, is to implement a Federated RM approach.

Challenges Facing the Public Sector An Infrastructure for 21st Century Government

Cheryl McKinnon of Hummingbird Ltd.

Governmental organizations worldwide are facing several challenges as administrative, executive and judicial bodies continue to evolve into an electronic work environment. Pushed by paperwork-reduction mandates, requirements to handle increased workloads with fewer personnel and the rapid adoption of electronic communication channels by taxpayers and citizens, governments are often on the forefront of adopting new approaches to electronic information management. Many agencies worldwide are stepping back and looking at the way in which electronic transactions, documents and records are being captured, managed, secured and preserved.

Three Big Trends

Three significant trends are converging and are causing public-sector organizations to look carefully at their information management infrastructure: a widely anticipated knowledge worker turnover due to retirements; an acceleration of e-government initiatives; and a move toward an enterprise architecture are the key issues facing government at all levels and across jurisdictions. These three pressures are causing program owners, as well as IT and IM management, to assess how agencies need to be equipped to manage risk, achieve operational efficiencies and build an infrastructure that can handle the accelerating shift into an electronic workplace.

Several studies have been done over the last few years that look at the demographics of government knowledge workers, particularly at the United States federal and state levels. A recurring theme of a generational shift is common among these analyses: At the highest risk, as the wave of retirements hits over the next decade, is the loss of senior project and technical managers. The Clinger-Cohen Study of Federal IT Staff, done in 2003, noted that 76% of the surveyed managers were over the age of 40, with the majority aged 45-50. Only 5% were under 30 years old. Similar studies looking at California and Texas state employees identified similar patterns, with the number of retirement-eligible knowledge workers rising exponentially in the last couple of years. Organizations that have recognized this impending turnover are asking questions about the preservation of institutional memory and the recordkeeping practices which must form the foundation of a knowledge retention, management and dissemination program. Understanding of how things get done...the informal chains of command... an understanding of how a process has evolved...the recognition of the human sources of knowledge on particular subject matters: this is corporate memory that is often ignored and is rarely communicated through formal programs.

The electronic knowledge assets that reside in e-mail repositories, on users’ desktops and on network servers are often poorly captured during periods of staff turnover or agency reorganization. Many government departments still have not established consistent programs for the capture, management and disposition of e-mail and other forms of electronic communication. The ability to separate junk, spam and duplicates from items having long-term business, legal or historical value to the organization is critical in order to preserve the knowledge contained in the data beyond the tenure of its creator. Agencies that will successfully ride out the impending retirement wave are those who have recognized the risk of not capturing the processes and information. Once the risk has been identified and articulated, technology can be leveraged to build structured workflows and implement electronic document/record/e-mail capture processes in order to protect the ongoing business of government.

Managing the Electronic Age

E-government, or government on-line initiatives, have been underway for several years. Agencies at federal, state/provincial and municipal levels have been offering an increasing range of services available to businesses and residents through Web sites and citizen portals. Electronic communication between citizen and government is also increasing: the ability to submit applications, forms or to ask routine questions through e-mail. The rapid adoption of e-mail for both personal and business use by constituents is pushing government departments to review their approaches to the collection, management and preservation of this new form of communication and correspondence.

Government departments must ensure that internal processes that were developed in order to manage the flow of paper-based correspondence, contracts or application submissions can be applied to the management of the same information when it arrives in electronic format. Management of e-mail information requests needs to be viewed in the context of its subject matter and business and historical importance, not managed based on arbitrary storage capacity rules imposed by IT administrators. A shift in how electronic correspondence and other forms of public communication are perceived needs to occur in many agencies, as the electronic work environment overtakes the traditional paper processes.

Legislation is also pushing the public sector to move into an electronic information dissemination model. The US Federal Government 1996 amendments to the Freedom of Information Act obligates agencies to proactively publish frequently requested records to an electronic reading room on the agency Web site in order to reduce the number of requests needing to be handled, as well as to conform to Paperwork Elimination targets. Governments have been facing transparency and openness issues for decades. While private sector has only recently been obligated to manage disclosures under new legislation such as Sarbanes-Oxley, most Western jurisdictions have had some form of Freedom of Information legislation for years. Electronic records, including e-mail, are subject to these laws within the parameters of specific exemption or exclusion clauses. As more information is created, stored and transmitted electronically, management and indexing systems are essential to be able to conform to these access laws. The establishment of an information management infrastructure in which these electronic records can be captured, managed and screened for personal or sensitive information is critical to ensuring appropriate compliance. The UK government is among the last jurisdictions to adopt a comprehensive FOI mandate, with their law only coming into full force in 2005. Like many other such laws, the UK government has established specific turnaround times in which a response to the information request must be given—in this case, within 20 days. Technology is a critical component of building an information management infrastructure to assist with the new compliance mandate. The volume of electronic record collection and creation makes manual processes and search impractical and a risk area if the response deadline is to be met.

Security, and Other Concerns

Another contributing factor to the expansion of e-government initiatives has been the heightened concern over national security and intelligence gathering. First-responders and field workers need to be able to communicate rapidly as information is gathered and observations are made. Many jurisdictions are now investing in a mobile framework for knowledge workers who are no longer considered deskbound. Hand-held electronic devices such as PDAs, smart phones and tablets are now replacing laptops and desktop PCs as the primary information conduit for inspectors, detectives, emergency services and case workers. Agencies moving into a more distributed model using more wireless and mobile technologies need to fully understand the implications of such a cultural and technical change. If the capture and control of electronic communication is a concern today, consider the issues when even more e-mail and messaging devices are in the hands of government workers. An assessment of the information management capacity of such technology solutions needs to be made, and the system must lend itself to effective record capture.

Conversely, the mobile field worker needs to have the same level of access to corporate records repositories and intranets/portals as the deskbound colleague. The ability to search for the correct versions of forms, templates and be able to instantly submit field reports and other objects (such as digital photographs) to a centralized, easily accessed repository is a fundamental requirement when building an effective mobile platform. Being notified of new workflow tasks, of updates to key documents, and the ability to view data collected by peers across geographic districts are all essential to making full and effective use of wireless systems and smart hand-held devices.

Several jurisdictions are also expanding forms of personal privacy legislation to cover electronic records held by government agencies. HIPAA (the Health Insurance Portability and Accountability Act, US) and PIPEDA (Personal Information Protection and Electronic Documents Act, Canada) are two of the more well-known privacy bills. HIPAA outlines specific rights for the protection of personal medical and health records, while PIPEDA is intended to encourage electronic commerce by granting rights to be able to protect and request personal information held by both public- and private-sector organizations. An increasing number of state and provincial governments have also passed or drafted privacy laws for their jurisdictions. This form of legislation places increased burden on government and private-sector organizations to ensure that electronic records, including e-mail, are not forwarded, published or disclosed inappropriately where specific forms of personal data are found in the record. The ability to restrict access to particular employees, and the ability to mark such records as containing sensitive data, are critical requirements of building an information management infrastructure to help streamline these new compliance mandates.

The third pressure facing governments is the constant battle to develop operational efficiencies in the face of budget and program cutbacks. At both federal and state/provincial levels, increasing attention is being paid to “enterprise architecture” or “shared services” models of technology infrastructure. Establishing common network platforms, operating systems, e-mail systems and information management platforms is an ongoing goal for many jurisdictions. The Canadian Federal RDIMS initiative was among the first large-scale programs, looking to establish a common document/records management platform to manage office format, image and e-mail items. Since the late 1990s, the Canadian government has led the way in establishing a set of shared best practices, deployment strategies and requirements definitions which are shared across the various ministries and departments. Several provincial jurisdictions have recently followed suit, as have state governments in Australia.

Government’s Advantages

Cost efficiencies can be easily realized when an agency can focus on supporting, maintaining and troubleshooting a smaller set of technology products. Having skilled resources with similar technical and business process knowledge in sister agencies allows the government staff to develop a deeper level of in-house expertise as well as reusable templates, customizations and training/change management techniques. An enterprise approach to technology platforms also allows a government body to negotiate more favorable support and licensing models with system vendors, leveraging license volume and allowing for greater input into product direction when speaking with a unified voice. Governments have long been on the forefront of standards development with respect to electronic information management, specifically with respect to records. DoD 5015.2, UK TNA and VERS are three of the more well-known and established government-sponsored standards. Each program has rigorous technical requirements and an established testing program that assures a purchasing agent that the product will meet a specific set of baseline functions for managing electronic data. Ongoing research continues to move these specifications forward as technology evolves. Updates to the TNA in recent years have moved into areas of metadata and interoperability requirements and the US standard is expected to follow suit. Enterprise content management vendors such as Hummingbird recognize that the compliance culture that permeates the commercial sector today has in fact been present in the public sector for years. Government represents the priorities and interests of the citizens within its borders. Transparency, privacy protection and information management mandates have been front-and-center in the government sector for many years. Agencies that move smoothly into an online model of citizen service are those who are now looking at building IM practices into their technology framework. Those organizations will be in the best position to meet the challenges of the retirement wave, the move to electronic services, and will achieve the highest value for the tax dollars spent.

Process Visibility: The Key to Optimizing Business Operations

Lee Roberts of FileNetCorporation

It’s no mystery that businesses need to react to changing market conditions faster than ever. But how do you improve your current business operations while adapting to constantly changing market and customer needs? One of the keys to this effort is gaining an understanding of your organization’s business processes, which can define how effectively the company is managed.
To address these needs, a new generation of tools is emerging to give executives visibility into business processes, providing essential information for understanding—and improving—their operations.

These tools are critical for businesses in a variety of industries—from manufacturing companies to financial services organizations to government agencies. For example, a mortgage lender with a goal to reduce approval cycle time by 30% could track progress and identify potential roadblocks to achieving the goal. By analyzing various metrics of its business processes, the organization could evaluate types of loans processed, processing time for each, customer response time, productivity per employee, and also take corrective action to realign any areas of the process.

This article describes the market drivers and customer requirements for business process analysis and reporting, and describes new reporting and analysis solutions that can add to the bottom line by helping companies measure and improve their business processes.

Demand for Analytic Applications on the Rise

With the exponential growth in corporate data, managers need faster, more granular and more flexible information processing. At the same time, sifting through the data to identify the organization’s strengths and weaknesses remains a daunting task.

Business visibility is the term that best describes the ability to sort through vast amounts of data to provide insight into the effectiveness of business processes. Using comprehensive metrics and reports, executives can identify business patterns, make more timely decisions and continuously improve the processes that drive their operations.

Industry experts agree that the ability to analyze process performance is a critical element of maximizing the returns enterprises realize from their Business Process Management (BPM) investments.

“Now more than ever, enterprises are looking for a rapid ROI from their process technology investments,” says Jim Sinur, vice president and research area director at Gartner Group.
“Understanding process performance is key to demonstrating the value of BPM implementations—and a critical component of the end-to-end BPM functionality that successful businesses require in today’s marketplace.”

Enhancing Business Value by Monitoring Process Performance

To the end user, business visibility provides interactive reports that allow them to monitor their processes, and if necessary, drill down into the detail of what they are monitoring. Visibility into these processes can provide critical information for understanding customer needs, assessing organizational strengths and weaknesses, and guiding executive decision making.

Monitoring is typically implemented in a Business Management Console (BMC), where high-level reports are constantly updated to reflect the changing business metrics. This approach is very useful in helping business users understand how current performance metrics compare to estimated standards of performance, otherwise known as Key Performance Indicators (KPIs).
If one of the KPIs falls out of the specified range, managers can click on the relevant report and drill down to a more granular view of the data. From here, the user may organize the data to determine a cause and effect relationship, and take corrective action to fix the problem.

For the mortgage lending organization mentioned in the previous example, a quick approval cycle time would be considered one of the company’s KPIs. In this case, the line-of-business manager might want to request detailed reports if approval cycle times went beyond the desired goal. These detailed reports would help identify information such as cycle time by day of the week, by time of day or by employee to help pinpoint potential causes of any approval delays.

New Reporting Mechanisms Improve Organizational Agility

Traditional reporting mechanisms interrogate data stored in a database. A programmer or business analyst, in consultation with the user, usually implements the report templates. As a result, these reports are typically static in nature and difficult to modify.
The data structures of a traditional database do not provide the required flexibility for producing interactive reports. This approach, therefore, limits an organization’s agility and its ability to react to changing market conditions because valuable time is lost in creating new reports.

To provide true business visibility, a new data structure is needed. This structure is an online analytical processing (OLAP) data cube. OLAP is a category of database software technology that enables analysts, managers and executives to gain insight into data through fast, consistent and interactive access to a variety of possible views of the same information.

Data cubes differ from traditional relational database representations in that they store the entire data set in a multi-dimensional format. This can be queried through a BMC or drag-and-drop spreadsheet tools like Microsoft Excel.

Two types of data cubes are required to address the needs of business visibility: real-time cubes for system monitoring and historical cubes for in-depth analysis. Referencing the mortgage lending organization mentioned earlier, management could track approval cycle time using a real-time cube. It could also use a historical data cube to measure cycle time by employee, region or month, or examine other more detailed metrics.

Moving BPM into the Front Office

Until recently, process-analysis solutions have been add-on packages that required costly and time-consuming integration. Today, analysis tools are available that make process analysis data accessible for line-of-business managers and executives. With an easy-to-use interface based on popular applications such as Microsoft Excel, these new analytics packages monitor data without the added time and expense of an outside analyst or consultant. They also offer sophisticated reporting and analytics capabilities, including the ability to create customized and detailed reports on the fly.

Simulation Capabilities Enable Continuous Process Improvement

Next-generation process analysis tools will incorporate simulation capabilities that allow business analysts to model business processes and then simulate them under real-world conditions. This will allow managers to uncover potential bottlenecks as well as resource or cycle time issues, and will enable processes to be tested and optimized before being put into a production environment.

Business analysts will be able to use their own simulation data, historical data or a combination of both as input into the simulation tool. To improve the original model, analysts will replicate a process, simulate it, deploy it in production, report on its performance and use historical data as input into the simulation tool.

Driving ROI Through Business Visibility

Numerous organizations have used BPM technology to enhance efficiency while using fewer resources. For example, Standard Bank of South Africa improved productivity by 140%, while decreasing financial decision cycle time from one week to seven minutes. In addition, Advance Bank, the first German banking institution to redesign and automate the complex process of opening an account via the Web, dramatically reduced bank processing time and costs, and enhanced its competitive edge with faster service and improved quality.

By understanding process performance and gaining greater business visibility, organizations can optimize their ROI from BPM initiatives. Through detailed data analysis on business processes, the business analyst and user can improve existing business processes, remove bottlenecks and repurpose underutilized resources. In this way, business visibility is essential for enterprises seeking to maximize the value from BPM implementations.

A Cure That May Cost Us Ourselves

One of the pioneers of human genetic engineering predicts that within 30 years, there will be a gene-based therapy for most diseases. But he fears the profound dangers of his own work.

by Dr. W. French Anderson (Anderson is professor of biochemistry and pediatrics at the USC Keck School of Medicine.) Jan 01 '00

A revolution is sweeping medicine--only the fourth one since Hippocrates argued, some 2,400 years ago, that the workings of the body can be explained by the laws of nature rather than the supernatural. The first revolution occurred soon after British surgeon John Snow discovered, in 1854, that cholera is spread by contaminated water: this led to sanitation systems that protected people from the devastating infections that had habitually plagued mankind. The second revolution, surgery with anesthesia, came at about the same time, allowing doctors to readily fix ailments such as appendicitis and bowel obstruction. The third revolution was the introduction of vaccines and antibiotics: many infectious diseases could finally be prevented or cured. But aside from remedying infectious diseases and some surgical problems, we physicians do not actually "cure" anything. Our medicines just help the body heal itself. Our treatments relieve symptoms but do not correct the underlying problems. Human genetic engineering--the fourth medical revolution--will profoundly change the practice of medicine over the next 30 to 40 years. But more than that, its effects will be felt far beyond medicine. It will influence every aspect of our culture. Used carefully, it will increase health and human happiness. But if used unwisely, the genetic engineering of human beings could endanger everything we value--including who and what we are.

Human genetic engineering, also known as gene therapy, is based on the premise that our genes are the defense and healing system of our body. It is our genes that protect our body from the assaults of nature; it is our genes that repair the damage caused by disease and restore us to health; it is our genes that, when they function abnormally, bring on not only such traditionally understood "genetic" diseases as sickle cell anemia and Huntington's disease, but also contribute to cancer, heart disease, Alzheimer's and mental illness. If we want to cure a disease, therefore, we must do it at the level of the genes.

There are two primary ways that genes can be used to treat disease. The first is gene therapy, in which one or more genes are injected into the patient to replace those that are absent or not working properly. This approach has been used to treat rare enzyme disorders, including one known as ADA deficiency, and clinical trials have employed gene therapy against a broad range of disorders: heart disease, many forms of cancer, arthritis, AIDS, hemophilia, cystic fibrosis and muscular dystrophy. The second way to exploit genes to treat disease is known as small-molecule therapy. In this approach, a small molecule (that is, a drug) is given to the patient to modify the function of one or more genes in the body. Pharmaceutical and biotech companies are investing heavily in both of these approaches.

As the Human Genome Project identifies all of the 70,000 to 130,000 human genes and, in time, teaches us what they do, we will rapidly develop the ability to screen for defects or weaknesses in all of our genes. By "weaknesses" I mean genes that do not function optimally for the environment in which the individual lives, which may be unusually stressful because of diet, toxins, radiation or some other factor and therefore will result in the patient's developing a disease. Once a defective or poorly functioning gene is discovered, we will be able to give the individual a more effective gene to replace the "weak" one. Or if the gene is making a normal product but just too much or too little of it, a small molecule (drug) can be given to regulate production. Thirty years from now, essentially every disease will have gene-based therapy as a treatment option.

Gene therapy is still too inefficient to be helpful in most cases. But progress is rapid, and the first treatments are expected to be available to the public over the next five years. The greatest success so far has been in stimulating new blood-vessel growth in the heart to treat heart failure or in the limbs to correct faulty circulation. Treatment of a number of genetic diseases, such as hemophilia, appears promising as well. There has also been significant progress in the use of gene therapy to deliver vaccines for protection against AIDS and several types of cancer. Most physicians expect that in the first 10 years of the new millennium we will see an explosion of gene-therapy treatments for many maladies that have been a scourge to human health. Genetic engineering should allow people to lead healthier, happier lives and add decades to our life span.
But there is also a more worrisome side to this story. For one thing, this technology is not risk-free. Although toxicity has been extremely low during 400 or so clinical trials in the past nine years, the recent unexpected and unexplained death of a gene-therapy patient in a University of Pennsylvania clinical trial underscores how little we still understand about human bodies and how our bodies respond to potent treatments.

There is also a broader danger. Unlike small-molecule therapy--which can be considered a "smart drug" strategy--gene therapy alters an individual's genetic blueprint. Once we have the ability to give a patient any gene we want in order to treat a disease, then we will also have the ability to give a human being genes for any purpose besides therapy. The downside of this powerful technology? Eugenics could be practiced on a scale far larger than any "selective breeding" policy could accomplish. Just a few weeks ago, a gene was discovered that seems to make mice more intelligent. Human genes have been identified that appear to influence behavior: an affinity for risk-taking, intelligence and even sexual preference. We've known for years which genes influence body size and muscle mass. The temptation to try to use genes such as these to "improve" ourselves is very strong--maybe even irresistible.

Already the first indications of potential abuse are surfacing. For example, one company is developing a treatment for the hair loss that occurs as a result of chemotherapy for cancer. It has already developed a salve that can transfer a functional gene into the hair follicles in human skin. Now the company is searching for a growth-factor gene that would stimulate hair growth. No one would object to preventing the psychologically traumatic side effect of hair loss caused by cancer therapy. But the real motivation is to sell the product to the millions of healthy men who are naturally going bald. Is this bad? Not necessarily, but it does start us down a slippery slope of using human genetic engineering for cosmetic purposes. Where does one draw the line? If hair growth, then hair color? If hair color, then skin color? If skin color, then other "racial" features? Where would the re-engineering of the human body end?

Society faces a real danger. In the name of minor "improvements" that we see as conveniences, we might start using human genetic engineering to attempt to change ourselves and then our children. Engineering the human germ line would result in permanent changes in the gene pool. We as a society have yet to end discrimination, including its most virulent expression, "ethnic cleansing." What would happen if we add intentional genetic enhancement to the mix? In the 1997 movie "GATTACA," only the genetically enhanced can hold good jobs. "Love children," who were produced by natural means and have a natural set of genes with all their weaknesses, are relegated to the bottom of the social and economic ladder.

Our only protection is to accept clear stopping points. And the only way to achieve those is to make sure that society is informed and can recognize the dangers and prevent misuses before it is too late. If such crucial decisions are left to the marketplace, might we ultimately engineer ourselves to the point where we are no longer human beings? We cannot dictate to the people of 100 years from now what they should do. They will care as little about our opinion as we care about the mandates of our 19th-century forebears. They might want to engineer their genes as routinely as we take vitamins. But what they do is not our responsibility. Our duty is to go into the era of human genetic engineering as respectfully as possible. That means that we should not use human genetic engineering for any other purpose than the treatment of serious disease, no matter how tempting it might be.

Far horizons: Our 2000 baby will live longer than his mother and father--the average boy will live to be 73 years old, and the average girl will reach 82.

The World on a String

Physicist Brian Greene explains what may turn out to be a unifying theory of everything in the cosmos

Brian Greene likes to think he's got it all figured out. Literally. And it all boils down to string. A Columbia University physics professor, Greene is one of the world's leading thinkers and writers on string theory, which purports to be the unifying theory of everything. String theory predicts that everything in the universe (from stars and suns to atoms and subatomic particles) can be broken down to incomprehensibly small loops of vibrating string. The building blocks of reality, as it appears to us, are merely a pattern of their vibrations, just as strings on a guitar vibrate at different rates to produce different notes. If true (and string theory has never been experimentally tested) the theory would be the unified, overarching explanation of how the universe works--the solution that physicists from Isaac Newton to Albert Einstein have been seeking for centuries.

Greene did not invent string theory. But in 1999, he published "The Elegant Universe" (Norton ), a popular presentation of string theory that became a major best seller and Pulitzer Prize finalist. Last fall he hosted a "Nova" television series based on that book. Now he's back with "The Fabric of the Cosmos: Space, Time, and the Texture of Reality" (Knopf ) published last month. Once again, assuming an audience of lay readers, Greene explains some of the more mind-melting features of today's cutting-edge physics in a language that is easy to understand. For example, one feature of string theory--also known as superstring theory--is that it suggests the universe has more than three, and possibly up to 11, spatial dimensions. Reality as we perceive it may in fact just be an approximation of the universe we inhabit. Time, which is relative to space, may not allow us to ever visit the past, but jumping into the future is possible within the laws of physics.

Greene recently spoke with NEWSWEEK's Brian Braiker, who is--under this theory--just a vibrating mass of tiny string, about the ideas he explores in the new book. Excerpts:
NEWSWEEK: Why is going from the concept that everything is made up of little tiny atoms and subatoms to the concept that it's all fundamentally squiggly, vibrating lines such a revolutionary idea?

Brian Greene: At first sight it wouldn't be. But when you study it in detail, you find that it is for a number of reasons. The first is that we believe that it gives a uniform description of all matter and radiation and all the forces of nature in one unified language. Prior to string theory, when you spoke about the elementary constituents of matter, you had to talk about electrons, you had to talk about protons and neutrons, you had to talk about the quarks that make them up. When string theory comes on the scene, everything simplifies because you have one entity: the string. And like any other string that you're more familiar with, like on a violin or a cello, the string in string theory can vibrate in different patterns. When a string on a violin vibrates differently, it produces different musical notes. Here when the little strings vibrate, they produce different particles.

So you and I are just piles of vibrations? What is it that's vibrating?

What is the string made of? We don't know for certain. One answer is that this may be the one time when that question fails to make sense. When you look at anything around you--the mug on your desk or the tabletop that you're working on--you can say, "What's it made of?" You can ask what the atoms that you hypothesize and prove by experiment make up the entity are themselves made of. You can ask what the nucleus of an atom is made of and get to the neutrons and protons. And you can ask what they're made of: quarks. But when you get down to strings, it may be that that's where the story ends. It may be that they are the fundamental entity.

You keep saying "may," which means none of this is certain at all.

Oh, string theory is definitely a work in progress. It's definitely a theory; it has yet to be experimentally confirmed. [But] if string theory is right--and again, I always emphasize the "if"--it is a unified theory. I think the important thing to bear in mind is that general relativity has been tested up the wazoo to incredible accuracy. Quantum mechanics has been tested; it works with fantastic precision. String theory simply puts them together in the first consistent framework.

General relativity basically being an explanation of the macro-scale universe and quantum mechanics at the micro level?

Exactly. General relativity is a theory of gravity. Gravity becomes most relevant when things are big--the earth's gravity, the sun's gravity--but rarely do we talk about the gravity of a coffee cup because it's too small. Quantum theory is most relevant on the opposite end of the spectrum; it comes into play when you are talking about atoms and subatomic particles. But we have come upon realms where you need both theories at the same time, such as black holes and the big bang--two examples where you have a lot of materials crushed to a very small size. Therefore you can't live your life keeping general relativity and quantum mechanics permanently separate. That's not how the world works, so you need a theory that can put them together in a consistent manner. String theory is the first theory to do that.

In your new book you talk about the possibility of there being up to 11 possible dimensions, where are they? What do they look like?

I think this is the most stunning and surprising element of this theory. One approach, which is one that I've actually worked on for maybe 15 years now, is that they're all around us, they're just tightly curled up. If you imagine a garden hose that's unfurled long and horizontal stretched out, but it's far in the distance, it'll look like a straight line because you won't be able to see the thickness. You'll think that it only has a left-right dimension along its horizontal extent and nothing else. But then you take a pair of binoculars and zoom in, you know that there's more to the surface than just left-right. It also has clockwise and counterclockwise, the circular girth of the hose, which is a curled-up circular dimension you don't see without binoculars. We think the same may be true of the universe, namely there may be big easy-to-see dimensions--the horizontal extent of the hose--but there might also be tiny, curled-up dimensions all around us--like the circular part of the hose--but so tiny that we don't have the magnifying equipment like binoculars adequate to reveal the existence of these extra dimensions.

What's another way of envisioning the possibility of other dimensions?

The idea is that the extra dimensions that we don't see might be big like the ones we know about, but we don't see them because we see with light. It might be that light is trapped in our three dimensions and it can't escape into the other dimensions and that's why they remain invisible to us. Imagine a universe as a big loaf of bread and what we've always thought to be the universe is merely one slice of bread in this huge cosmic loaf. Light in this picture will be trapped in our slice of bread; it can't travel to the other slices. The only force that wouldn't be trapped, it turns out, is gravity. So it might be possible one day to detect these extra dimensions through gravity. Experiments are actually underway today in an attempt to do that.

But we still wouldn't be able to see them.

We wouldn't see them with light. But we would see them indirectly with gravity. The experiment is so simple: at the Large Hadron Collider, which is an atom-smasher being built [at the particle-physics laboratory] CERN in Switzerland, they're going to take protons and send them circling around a huge tunnel in opposite directions right near the speed of light. Every so often they'll use magnets to direct the beams of protons to smash into each other in head-on collisions. The idea is that in the collision, a certain amount of gravity will be produced. It might be that some of that gravity can leak off our dimensions, off of our slice of bread, and disappear into the other dimensions. If that happens, the amount of energy before the collisions will be a little bit bigger than the amount of energy after the collision because some of it will have seeped away. So the scientists are going to look for missing energy. If it's missing in just the right pattern, it could be very strong evidence that the extra dimensions are real, that it has gone into other dimensions.

Why is time forward moving when everything else seems to tend toward randomness?

It's a real big puzzle as to why time seems to be different. You stand in space and can move at will--left or right, back or forth, up or down, no constraint--but with time we seem to be relentlessly dragged forward. Why is that? We think that the answer, surprisingly, is to be found in the big bang itself. The big bang started the universe off in an incredibly ordered state and things tend to become more disordered over time. For that disorder to happen, you've got to begin highly ordered. There are sequences of events that we only see happen in one order--eggs splatter, they never unsplatter; glasses shatter, they never unshatter. I think it's a wonderful idea--every time you drop an egg and it splatters, it's actually telling you something very deep about the big bang itself.

So time travel may never be possible?

I firmly believe that one day we will rule out the possibility of time travel to the past. On the same point, it's worth emphasizing that time travel to the future is a completely different ballgame. That is within the laws of physics as we understand them. Einstein himself showed us how to accomplish time travel to the future. If you want to see what the earth is like ten or 100 or a million years into the future we know in principle how to do it: You build a spaceship; you travel out into space at near the speed of light; you turn around and come back. Perhaps a year may elapse for you, but because time slows down when moving at high speeds, 1,000 or a million years may have elapsed when you return to earth. That means you have jumped into earth's future. We can't build such ships yet, but these are technological issues. But physics definitely shows that this kind of leapfrog into the future is within the laws of physics.

When you really think about this--that we're just made up of vibrating strings, that we fail to perceive everything that constitutes reality--is it hard not to feel a little despair?

When we recognize that the same laws that govern us are the laws that govern the molecules and atoms in interstellar space, the processes in the sun and the countless other stars in the heavens; when we learn, for instance, that the very atoms that make up our body were produced in stars and spewed out into space through supernova explosions, I think it makes us feel more connected to the cosmos. We may not be anything particularly special, but we're definitely part of the grand scheme. I think that can be very uplifting.

Google Me Not

by David Whelan Aug 16 '04

Keeping confidential company data, or last weekend's cocktail party photos, away from search engines has become a growth business.

Type the right words into Google and up comes a trove of files documenting an acrimonious divorce between two business executives in San Diego. Support payments are calculated based on a $450,000 income. The husband accuses the wife of being a "shop-a-holic." He lists all her possessions, including furs worth $15,000. He's eager to finalize the divorce, because, as he writes, he was to marry again in June. All this is personal, private information, no longer even up on the original Web site, yet stored by Google for everyone to see, including friends, family and business associates who enter in the divorced couple's names. When reached by phone, the husband says he is "stunned and shocked" that FORBES is interested in the matter at all.
As Web pages pile up like garbage in a landfill--1 billion will go up this year--sensitive, defamatory, confidential or embarrassing information is increasingly finding its way into search results. The search industry is raging, with $1.5 billion in revenue expected this year, up 150% from 2003. Google's hotly anticipated $3.8 billion public offering is just around the corner. These riches are fueling a technological arms race among Yahoo, Microsoft, Google and others whose software combs the Web constantly, indexing and storing all that you seek and ranking pages on their relevancy.

Search engines can store results in their "cache" for between a month and forever. As archiving improves, it will get harder to clean up what's been revealed. Rarely are leaks intentional: Somebody at work might post a file on a server to download at home, a wrongly configured server might make too much of a hard drive searchable or a Web site's password-protection might be flimsy enough to be accessible to search engines.

Google turned up detailed monthly expenses and employee salaries at the National Speleological Society's site, caves.org. Says the group's president Scott Fee, "That ain't supposed to be up there." On the site of the Naval Postgraduate School in Monterey, Calif. course materials are posted for a class on computer networking, including log-in files that could be of interest to the right hacker. All manner of personal correspondence, including transcripts of intimate instant-messaging exchanges, can be unearthed by search engines. Pamela Dixon, a privacy advocate at the World Privacy Forum, tells of an elementary school teacher whose contract was not renewed with a Solano County, Calif. school district. This item appears in the minutes of a school board meeting. The announcement still comes up second upon Googling her name. Dixon says this has been devastating to the teacher's job search and that attempts to have the minutes taken down have so far been fruitless.

The 15-person staff at Craig's List, the popular community message board, acts as a rapid response team to keep people from harassing each other by name before one of the 150 search engines that index the site grabs the offending page. "Google, because of the way it stores things, can perpetuate a problem," says founder Craig Newmark, who recently spent an afternoon chasing a user who wrote six nasty messages about a neighbor.

Search programs will be casting a wider net. Microsoft is tinkering with a technology called Stuff I've Seen that will pull results from the Web along with one's own computer and its network. (Microsoft claims this won't make personal files viewable by the public.) Google's Internet Explorer toolbar tracks the sites users surf and relays the addresses back to headquarters. That may be how Google results can turn up Web pages people naively consider private, since they're not linked to or from anywhere. Google lets users disable the feature: Just turn off the PageRank feature in the Options menu.

Google recognizes the Web's power to publicize and offers Web masters a simple interface to remove their own pages from its index. (Type "remove" into Google to get started.) Other preventive measures include putting sensitive info behind password-protected walls and attaching so-called robot files to Web sites that tell search engines not to index a particular page or site. All the search engines follow these robot directives, which, while not a perfect security solution, limit the entry points used by the bots and spiders that index the Web.

Foundstone, an Internet security company in Mission Viejo, Calif., has developed a tool called SiteDigger that piggybacks on Google to point up information leaks. Point it at a site or an entire domain, such as .edu, and it generates a list of e-mails, log-in screens, database errors and source code, all of which are classic ways to gain entry onto a server. The free tool is a nice way to attract business for Foundstone, and also a scary reminder of how much information is out there, says SiteDigger designer Mark Curphey. Combing through sites ending in .mil, the domain for the armed forces, a recent Googling uncovered 17,300 Excel spreadsheets, 56,000 PowerPoint presentations, 258,000 Word files and 681,000 Acrobat Reader files.

Search results can be rigged to gain a higher position; pranksters can manipulate your page so that it shows up whenever someone Googles the word "jerk," a practice known as Google bombing. So why not do the opposite to play down bad publicity? Public relations firm Weber Shandwick employs a search expert in Irving, Tex. named Jeffrey Martin, whose specialty is advising clients on how to bury bad news under rosier search results.

Systems integration company Science Applications International Corp. of San Diego offers a product called Open Source Monitoring that scans the Web, newsgroups, Listservs and any other public forums for names and trademarks. Companies use it as an early warning system for hackers, stock manipulators, disgruntled employees and bad-mouthers. The program has helped companies avoid violent protests and direct police to ex-employees who make threats or try to leak confidential information, says Timothy Appleby, the project's chief scientist.
Better archiving technologies are extending the shelf lives of data you don't want people to see. Google and its competitors save copies of Web pages in a database that makes the pages accessible even when links to the site have been taken down. These cached links are destroyed the next time Google re-indexes the Internet, anywhere from a day to a month. But an online service called the Wayback Machine at archive.org keeps records of old Web sites. You never know what's going to turn up there.

Like libraries, Google and its search rivals do not assume responsibility for the content that they catalog. But with lawsuits filed at the drop of a hat, search services should take note. In June Varian Medical Systems won a case against two former employees who defamed company executives online, winning $775,000 in damages in what's become a landmark Internet speech case. A judge ruled last year in New Hampshire that Docusearch, a purveyor of personal information, could be sued for the death of a stalking victim whose murderer used its services. (The case was settled before trial.)

Sun Microsystems Chairman Scott McNealy famously said, "You already have zero privacy. Get over it." That's not quite true. You can fight back, up to a point.

Copyright © 2004 Forbes.com

Google Alert Launches Personalized Auto-Filtering

Google Alert is an extremely effective and well designed online service that allows you to track any topic, search or reference by leveraging a sophisticated query engine that taps Google for answers and automates this process so that it generates results on a daily basis. You can also control how often your searches are run. Google Alert provides easy access to all of Google's advanced search features to enable you to personalize and refine your searches. This includes targeting searches by language, country, domain and file format. Duplicates can be easily dropped, domains excluded, and support for search across multiple document formats is also supported. Google Alert is one of the ideal instruments for the emergent newsmaster as it can automatically generate HTML pages and/or RSS feeds of any results it finds. Google Alert supports also trackbacks, allowing search results to be integrated into your blog. Google Alert newest feature, SightPoint, automatically learns which search results are most relevant to you based on results you have clicked on in the past. These relevance ratings, out of five stars, become more accurate as the time passes.

RSS NewsMastering - The Impact of Ten Years, Ten Trends

On September 23, 2004 The Annenberg School Center for the Digital Future released its study "Ten Years, Ten Trends" outlining a decade of trends it's identified since it started studying online behavior.

"After a decade of observing the evolution of the Internet, and four years of our formal studies of online technology, we are seeing clear trends in how the Internet has changed the United States,"said Jeffrey I. Cole, director of the Center for the Digital Future.

"Our annual projects explore more than 100 major issues concerning online behavior,” said Cole.
“Of these findings, ten broad trends have emerged that have particular relevance as we reach the 10-year anniversary of the opening of the Internet to the public: Ten Years, Ten Trends."
A summary of those ten trends is available here. Although all ten findings are fascinating in themselves, there are four that should be of particular interest to independent publishers, newsmasters and communications agents.

One of the observed trends demonstrates how events and rapid technology development can make such findings quickly obsolete, and three (more sustainable, long-term) trends reinforce what are the major forces driving information consumers away from the traditional media and towards authoritative, independent and ethically motivated sources of news, research and analysis.

1. Established Media And Government Sources

Year Four of the Digital Future Report shows that most users trust information on the websites they visit regularly, and on pages created by established media and the government. The information that users don't trust is on Web sites posted by individuals.

The enormous and ever-growing popularity of independently published commentary and analysis on expert blogsites, especially in relation to coverage of the war in Iraq, and the run up to the US Presidential election, would indicate a new appetite that is not being satiated by "established media and the government"

2. The Internet Has Become The Number One Source For Information For Internet Users

The Internet has become the most important source of current information for users – the primary place they go for research, general information, hobbies, entertainment listings, travel, health, and investments. The 'always-on' function of broadband has accelerated this importance.
"When one can turn to the computer in the kitchen to instantly access movie schedules from a bookmarked location, what does that say about the functionality – or even the need – for the entertainment sections of the local traditional media?" Cole said.

This Trend goes somewhat against that identified in 1. above.

As Internet experience increases, perceptions of the importance of the Internet as an information source also increase. Eventually, almost every American will be an experienced user. How will that change the perception of the Internet as an information source? And how will that ascendancy affect other sources of information?

3. Email Is Becoming An Irritation

"E-mail is certainly a great convenience, but it is also a great irritation," Cole said. "What user today doesn't voice some concern about e-mail – spam, inbox overload, time commitments required to respond – in addition to its benefits? No Internet user wants to get rid of their e-mail, but they are tired of e-mail defining their lives."

As email 'attentiveness' declines, the opportunities to provide information consumers with the ability to 'subscribe' to feeds that are of specific interest to them at any given time will increase.

4. Broadband Will Change Everything

Broadband is changing entirely our relationship with the Internet at home – how often we go online, how long we stay online, and what we do online. Simply, modem use is disruptive; broadband use is integrative.

Always-on broadband access, combined with wireless, is resulting in the creation of a vast range of new products, services and applications for information consumers - once again providing unlimited opportunities for authoritative and respected independent content creators who are readily adopting the new platform, channel and format agnostic publishing technologies.

Become A NewsGod - How To Be The First One To Know Everything About Anything.

Intelligent Information Agents Come of Age.

With the changing economics and dynamics of news publishing an avalanche of news content has started. Theoretically, without any technological support, you would need to spend more and more time and energy checking out all your news sources and searching for new content with search engines. The goal of this Robin Good Mini-Guide is to provide communication agents, professionals, academics, librarians, journalists and researchers with the means to search and collect highly selective information from the Internet without engaging themselves in repeated long visits to reference sites or in intense Google searching. Today, you can monitor, track and keep yourself updated with any online content resource that you identify as valuable to you. The tools that are reviewed in this guide allow you to easily capture, automatically gather and selectively filter only those news items and stories that are particularly relevant to you or that come from the specified sources you have selected.

Search For Tomorrow We Wanted Answers, And Google Really Clicked. What's Next?

By Joel AchenbachWashington Post Staff WriterSunday, February 15, 2004; Page D01

In the beginning -- before Google -- a darkness was upon the land.

We stumbled around in libraries. We lifted from the World Book Encyclopedia. We paged through the nearly microscopic listings in the heavy green volumes of the Readers' Guide to Periodical Literature. We latched onto hearsay and rumor and the thinly sourced mutterings of people alleged to be experts. We guessed. We conjectured. And then we gave up, consigning ourselves to ignorance.

Only now in the bright light of the Google Era do we see how dim and gloomy was our pregooglian world. In the distant future, historians will have a common term for the period prior to the appearance of Google: the Dark Ages.

There have been many fine Internet search engines over the years -- Yahoo!, AltaVista, Lycos, Infoseek, Ask Jeeves and so on -- but Google is the first to become a utility, a basic piece of societal infrastructure like the power grid, sewer lines and the Internet itself.

People keep finding new ways to use Google. It is now routine for the romantically savvy to Google a prospective date. "Google hackers" use the infiltrative powers of Google to pilfer bank records and Social Security numbers. The vain Google themselves.

It was about three years ago that the transitive verb "to Google" entered the lexicon, but it was only last year that Google passed all rival search engines in the number of queries handled -- now upwards of 200 million a day. So phenomenal is its success that some industry watchers think an initial public offering of Google stock could raise $20 billion and trigger a second dot-com boom.

"You build a better mousetrap and the world will beat a path to your door," Stewart Brand, computer guru and president of the Long Now Foundation, says of Google. "A wider path, I think, has never been beaten in the history of the world. It's an astonishing mousetrap story."
In the dot-com world, nothing stays the same for long, and it's not clear that Google will forever maintain its dominance over such ferocious rivals as Yahoo! and Microsoft. But the business story of Google is less interesting than the technological one: If information is power, then Google has helped change the world. Knowledge is measurably easier to obtain. Google works. Google knows.

The world used to be transformed by voyages of discovery, religious movements, epidemic globe-circling diseases, the whims of kings and the depredations of armies. But over the centuries, technology has emerged as the primary change agent, the thing that can shrink a planet, undermine dictators and turn 14-year-olds into publishers.

The question is, who's going to build the next mousetrap? What will it do? The laboratories of Internet companies are furiously trying to come up with the next generation of search engine. Whatever it is and whatever it's called, it will likely make the current Google searches seem as antiquated as cranking car engines by hand.Mom, What's a Library?

The transition into the Google Era has not occurred without some anguish. The stacks of a university library can be a rather lonely place these days. Library circulation dropped about 20 percent at major universities in the first five years after Internet search engines became popular. For most students, Google is where all research begins (and, for the frat boys, ends).
A generation ago, reference librarians -- flesh-and-blood creatures -- were the most powerful search engines on the planet. But the rise of robotic search engines in the mid-1990s has removed the human mediators between researchers and information. Librarians are not so sure they approve. Much of the material on the World Wide Web is wrong, or crazy, or of questionable provenance, or simply out of date (odd to say this about a new technology, but the Web is full of stale information).

"How do you authenticate what you're looking at? How do you know this isn't some kind of fly-by-night operation that's put up this Web site?" asks librarian Patricia Wand of American University.

Students typically search only the most obvious parts of the Web, and rarely venture into what is sometimes called the "Dark Web," the walled gardens of information accessible only through specific databases, such as Lexis-Nexis or the Oxford English Dictionary. And most old books remain undigitized. The Library of Congress has about 19 million books with unique call numbers, plus another 9 million or so in unusual formats, but most have not made it onto the Web. That may change, but for the moment, a tremendous amount of human wisdom is invisible to researchers who just use the Internet.

"For a lot of kids today, the world started in 1996," says librarian and author Gary Price.
And yet Berkeley professor Peter Lyman points out that traditional sources of information, such as textbooks, are heavily filtered by committees, and are full of "compromised information." He's not so sure that the robotic Web crawlers give results any worse than those from more traditional sources.

"There's been a culture war between librarians and computer scientists," Lyman says.
And the war is over, he adds.

"Google won."

Advanced Search

In the early days of search engines, finding information was like fishing in a canal: You might hook something good, but you were just as likely to reel in an old tin can or a rubber boot. Now you often find exactly what you want.

One reason Google works so well today is that there's so much for its robotic crawlers to explore. Google initially searched about 20 million Web pages; the company's home page now boasts that it searches 3,307,998,701 pages.

"In 1996, if you tried to Google someone, if Google existed, it wouldn't have been a very satisfying experience," says Seth Godin, author of a number of best-selling e-books. "We hit a critical mass of really valuable stuff that was online, I think, about 2000."

The expansion of the information universe makes the navigational tool all the more valuable. And yet the search function at first seemed to be an unglamorous computer application. The pioneering search engine companies, including Yahoo!, Excite, AltaVista and Lycos, wanted to transform themselves into something snazzier, a "portal," the full gee-whiz Internet Century home page that would offer the user a link to everything between here and Neptune, plus plane tickets.

But the history of computer technology is full of companies that failed to see the potential glory right in front of them. In the early 1980s, IBM thought that the "operating system" within the computer wasn't nearly as important as the hardware, the box itself. And then Microsoft, which benefited from that oversight, became so focused on software programs that it was slow to capitalize on the Internet revolution, leaving Netscape to create the first commercial Web browser. And then almost everyone underestimated Search.

Not Google. When the company debuted in September 1998, it looked like a throwback. This wasn't a portal. The home page showed mostly white space, anchored by a little rectangle, a box, perfectly blank. Fill in blank and get results. This was plain ol' boring Search, without news headlines, plane tickets, e-mail or any other bells and whistles.

But what results! Google has farms of computers working in parallel. You can put in a couple of words and -- gzzzzt! -- get 600,000-plus results within some preposterously brief amount of time. (Google brags about it: "Search took 0.17 seconds." Showoffs!)

Google, the creation of Stanford graduate students Sergey Brin and Larry Page, is like many other search engines in its basic operation. It has powerful software programs that automatically "crawl" the Web, clicking on every possible link, scouting the terrain. What has made Google special is that, in assessing the quality of sites, it takes note of how many other pages link to any given page. This is an old idea from academia, called citation analysis. If many Web sites link to a particular page, the page rises in Google's vaunted "page rank" and is more likely to be on the first page of the search results.

"You're getting the advantage of the group mind," says Paul Saffo, a research director at the Institute for the Future.

This is a key concept: As the Web has grown, it has developed a kind of embedded wisdom. Obviously the Web isn't a conscious entity, but neither is it a completely random pile of stuff. The way one part links to another reflects the preferences of Web users -- and Google tapped into that. Google, in detecting patterns on the Web, harvested meaning from all that madness.
This points the way to one of the next big leaps for search engines: finding meaning in the way a single person searches the Web. In other words, the search engines will study the user's queries and Web habits and, over time, personalize all future searches. Right now, Google and the other search engines don't really know their users.

For example, Saffo isn't really interested in the stuff that most people look for when they do a Web search. He's one of the premier futurists of Silicon Valley and fondly recalls the days, back in the 1980s and early 1990s, the pre-Web era, when the Internet was the reserve of the technological elite who posted their brilliant thoughts on electronic bulletin boards. Now, everyone from about third grade up has an e-mail address and loiters around the Web as though it's the corner 7-Eleven. The results of a Web search reflect the tastes of a broad swath of ordinary Americans who in some cases are still wearing short pants.

"The more people get on the Web, the more the Web becomes the vaster wasteland that is the successor to the vast wasteland of television. I don't care what the majority of people are looking at, because the majority of people are really boring," Saffo says.

He needs a better search engine. He needs one that knows that he's a big-brain tech guru and not an eighth-grader with a paper due.

"The field is called user modeling," says Dan Gruhl of IBM. "It's all about computers watching interactions with people to try to understand their interests and something about them."

Imagine a version of Google that's got a bit of TiVo in it: It doesn't require you to pose a query. It already knows! It's one step ahead of you. It has learned your habits and thought processes and interests. It's your secretary, your colleague, your counselor, your own graduate student doing research for which you'll get all the credit.

To put it in computer terminology, it is your intelligent agent.

Calling Agent 001101

No one knows how the intelligent agents of the future might really work, and once you venture more than a few months out you're already into some seriously fuzzy territory. But you might imagine that this intelligent agent could gradually take on so many characteristics of your mind that it becomes something of a digital doppelganger, your shadow self.

To borrow and slightly distort something from "Star Trek," it's like your personal digital Borg, having absorbed your thoughts and melded them with an existing software program.

Perhaps this digital self could become a commodity, something marketable. Imagine that you have to write a paper for a class about the future of search engines. You don't want to use your own lame, broken-down, distracted, gummed-up-with-stupid-stuff virtual secretary to do your research. You want to download Bill Gates's intelligent agent, or Paul Saffo's, or Sergey Brin's, to help you ask smarter questions and find the best answers.

There are primitive intelligent agents already. Amazon.com makes book recommendations based on your previous purchases and the judgments of others who have liked the same books you've liked. But this form of collaborative filtering is still fairly crude.

Microsoft senior researcher Eric Horvitz describes a variety of new and future technologies in which software is more active, more of an entity, no longer just some inert codes waiting for the user to issue a command. For example, there's a program he already uses called IQ, for "implicit query."

"As you're working, we continue to formulate queries in the background, that the user doesn't even know about. They're happening very quietly," Horvitz says.

But Horvitz is keenly aware that people don't want a program that's too pushy, that's constantly interrupting. Humans have limited powers of attention. Software, says Horvitz, "needs to be endowed with the kind of common courtesies we'd expect from a well-mannered colleague."
And lurking over the future of such programs is the dilemma of privacy. There's valuable information in the way people use the Web, but they may not want others, or even a machine, to pay close attention to every place they venture. How do you create an intelligent agent that knows when to look away? How do you avoid what Horvitz calls the "monster possibilities"?
What everyone wants is a reasonable, discreet intelligent agent, like an English butler. It should be one that can get things accomplished, to take the extra steps even without being prompted.
"I don't think anyone wants a search engine," says Seth Godin. "I think people want a find engine."

Find, and do. Solve problems. Make it so.

"I often use the analogy of Web agents being like travel agents," says James Hendler, a computer science professor at the University of Maryland. "When I go to my travel agent and say where I want to go, they don't usually just say, 'Yes, you can get there.' They give me some options of different ways to get there. They think about some things I might have forgotten. Do I need a car, do I need a hotel reservation? And then they go do it for me."

Computers as a general rule do only what they're told to do. They don't have artificial intelligence in the classic sense. They have no common sense. IBM's Gruhl, the chief architect of a new product called WebFountain, points out that no computer has ever learned what any 2-year-old human knows.

A computer, he says, can become easily confused by the sentence "Tommy hit a boy with a broken leg." The computer doesn't understand that a broken leg is not going to be an instrument used in an attack. "Common sense, how the world works, even something like irony, are very difficult for computers to understand," says Gruhl.

Semantic Discussions

To achieve common sense, the Web needs to go through the infantile process of self-discovery. The Web doesn't really understand itself. There's lots of information on the Web, but not much "information about information," also known as "metadata."

If you're a robotic search engine, you look for words in the text of a page, but ideally the page would have all manner of encoded labels that describe who wrote the material, and why, and when, and for what purpose, and in what context.

Hendler explains the problem this way: If you type into Google the words "how many cows in Texas," Google will rummage through sites with the words "cow" and "many" and "Texas," and so forth, but you may have trouble finding out how many cows there are in Texas. The typical Web page involving cows and Texas doesn't have anything to do with the larger concept of bovine demographics. (The first Google result that comes up is an article titled "Mineral Supplementation of Beef Cows in Texas" by the unbelievably named Dennis Herd.)
Hendler, along with World Wide Web inventor Tim Berners-Lee, is working on the Semantic Web , a project to implant the background tags, the metadata, on Web sites. The dream is to make it easier not only for humans, but also machines, to search the Web. Moreover, searches will go beyond text and look at music, films, and anything else that's digitized. "We're trying to make the Web a little smarter," Hendler says.

But Peter Norvig, director of search quality at Google, points out that the current keyword-driven searching system, clumsy though it may be and so heavily reliant on serendipity, still works well for most situations.

"Part of the problem is that keywords are so good," he says. "Most of the time the words do what you want them to do."

Billions of dollars are at stake in this race to invent the next mousetrap, and Google faces serious challenges. Yahoo! has long had a partnership with Google, using it to power many of its searches, but Yahoo! has since acquired two other search engine companies, and plans to drop Google in favor of its own Web crawlers. Microsoft, meanwhile, is sure to make search a fundamental element of the next version of its operating system , due in 2006 and code-named Longhorn.

Will Google get steamrolled like Netscape?

"We spend most of our time worrying about ourselves and not our competition," says Google's Norvig.

Technology creates a horizon beyond which human destiny is unknowable, because we can't anticipate all the crazy stuff that brilliant people will invent. The author Michael Crichton has pointed out that a person in the year 1900 might have contemplated all the human beings who would be on the planet in the year 2000, and wondered how it would be possible to obtain enough horses for everyone.

And where would they put all the horse droppings?

Specific predictions are usually wrong. But a general trend has emerged over the course of centuries: Information escapes confinement. Information has been able to break free from monasteries, libraries, school-board-sanctioned textbooks, and corporate publishers. In the Middle Ages, books were kept chained to desks. Information is now completely unchained.
It has a life of its own -- and someday perhaps that won't be just a metaphor.

(Source: http://www.washingtonpost.com/ac2/wp-dyn/A42885-2004Feb14?language=printer)

Future search efforts will make Google look like 8-tracks

There is no hotter topic in technology than Google. Cover of Newsweek. Cover of Wired. IPO fever.

What's next? The Google Diet? The Five Googles You Meet in Heaven?

But, you know, much as I couldn't live without it, Google stinks.

Ten years from now — maybe five or even less — we will recall Google circa 2004 and wonder how we could have tolerated it. You know, sort of the way we look back on eight-track tapes.
It's a disaster that I type "turtles" into Google and get 1.9 million results. On the first page, Google serves up ninjaturtles.com, seaworld.com and theturtles.com (devoted to the 1960s band that sang Happy Together). Yet what I want is information about the soccer team I play on, the Turtles — so named because of the speed at which we run.

Google today can't know that's what I'm searching for, but it should. In fact, someday it will, if Google doesn't get roasted by Microsoft or somebody else between now and then.

This is not lost on Google's executives. At a conference last week called PC Forum, Google CEO Eric Schmidt essentially told the audience what he wished Google could become.

"I keep asking for a product called Serendipity," he said, making up the name. This product would have access to everything ever written or recorded, know everything the user ever worked on and saved to his or her personal hard drive, and know a whole lot about the user's tastes, friends and predilections.

"Then when I'm typing a paper, it would know what I'm writing about and say, 'Hey, you forgot this,' " Schmidt said.

That's where search has to go. Eventually, search will be like a direct connection between your brain and all the world's information. It will grasp so much about you and your immediate circumstance that it will often know exactly what you need, perhaps even before you do. It will be an electronic Radar O'Reilly.

No company is there yet. But out in tech's primordial soup, you can see individual pieces evolving.

The first and simplest piece is what you get from Google and other search engines. This is known as the visible Web. It's all the stuff posted on Web sites and open to browsing — billions of pages, some of it valuable, much of it not.

But vast as it is, searching the visible Web is just a beginning. It's like the invention of the wheel — a breakthrough, but a long way from a Porsche 911 Carrera.

Other pieces that are starting to form:

• The invisible Web. This is the stuff you can't readily see and search engines don't usually find. One example is content that's available only by subscription, like stories on Salon.com or video on ABC's paid site. Schmidt says Google is looking at ways to find that content and display a brief description of it. That way the user would know it exists but would still have to go to that site and pay to get it.

Blogs — essentially Web-based diaries — are a growing part of the invisible Web. Blog search engines, such as Technorati, are popping up. But Technorati doesn't search the rest of the Web, just as Google doesn't search blogs.

Books are increasingly a part of the invisible Web. Despite all the content on the Web, vastly more is locked in books. Amazon's "Search Inside the Book" feature gets at some of that. Google doesn't touch it.

• Localization. Google has no idea where you are unless you tell it. So if I search for Hooters — not that I ever do — it can't automatically know I'm in a hotel in Houston and want to find one nearby. Technology is emerging from companies such as Quova that can let a Web site know approximately where you are — but not a specific address, which would rightly make privacy advocates apoplectic. In fact, a lot of these new search functions will stir privacy worries.

• Your hard drive. Sometimes, what you want is somewhere on your hard drive, perhaps buried in notes you took at that 1999 Las Vegas convention you hardly remember. Or maybe it's in an e-mail. If you don't know, it can be hard to find.

You can download software, such as X1 from Idealab, that searches everything in your computer. But it's still not possible to type a search term into something like Google and have it search both the Web and the computer's insides. Microsoft's next operating system, code-named Longhorn, is supposed to do that. It might be the biggest single threat to Google.

• Your life. Google knows nothing about you. It's trying. Google has a new personalization feature. You can inform Google that if you type in "bass," you're looking for a musical instrument, not a fish. But the function is still very limited, which means it doesn't know nearly enough to truly help you, and it has no way to keep learning more.

One new site is taking an interesting approach to this problem. It's called Eurekster. It works, in part, by combining search with letting you set up a network of friends online — sort of a cross between Google and Friendster. By monitoring what you and your friends click on, it can increasingly understand more about your needs and can tailor searches to that.

As Eurekster gets more sophisticated, it could understand that you hang out with plumbers, not Grateful Dead fans — and give you more appropriate results when you search for the word "pipe."

Google recently created its own Friendster-type site, called Orkut. Probably not a coincidence.

Eventually, all these elements will meld together — the visible Web, the invisible Web, localization, your stored content and info about your life. The final piece will be software that can understand what you're typing or reading and constantly look for related content.

A search engine of 2010 will know who you are, where you are and what you're doing, and look across every form of information to automatically find what will help you.

That's when today's Google will seem as quaint as the special effects in an old Godzilla movie.

http://searchenginewatch.com/facts/article.php/2156021

http://www.taipeitimes.com/News/bizfocus/archives/2005/08/21/2003268631

136 TERAFLOPS AND MORE

Currently the world's fastest computer is a machine installed at Lawrence Livermore National Laboratory late last year -- and still growing -- that has reached more than 136 trillion operations a second, or 100,000 times the speed of a fast desktop personal computer. IBM built the machine, Blue Gene/L, and plans to double its speed before the end of the year.

http://www.masternewmedia.org/search_tools_and_technologies.htm

The future of search

Technologies of Cooperation

Posted by Jim_Downing at 03:59 PM

This Time article looks at the future of search engines.

"You land late in the evening in a city where you know nobody. You did not have time to book a hotel, your luggage has not turned up on the carousel -- and the plane's air conditioning gave you a sore throat. What to do?

With your cell phone, you first Google your suitcase -- it has a small implanted chip that responds to radio waves with a GPS locator -- and it turns out that your luggage has been deposited 200 yds. away in the next terminal.

As you walk over,you search for a hotel room; the screen of your cell shows you pictures of several hotels in your price bracket, with views from individual room windows. Your search engine gives you a list of pharmacies that are still open at this hour, and tells you that your favorite blues band will be playing at a festival in the city's park over the weekend.

The engine can search your desktop back home, and it reminds you that a college friend e-mailed you a year ago to say he and his wife were moving to this city (you had forgotten). You decide to invite them to the festival. What you have just tasted is the future of search".

http://www.verdada.com/

August 2009: How Google beat Amazon and Ebay to the Semantic Web
By Paul Ford
A work of fiction. A Semantic Web scenario. A short feature from a business magazine published in 2009.
Please note that this story was written in 2002.

It's hard to believe Google - which is now the world's largest single online marketplace - came on the scene only a little more than 8 years ago, back in the days when Amazon and Ebay reigned supreme. So how did Google become the world's single largest marketplace?
Well, the short answer is “the Semantic Web” (whatever that is - more in a moment). While Amazon and Ebay continue to have average quarterly profits of $1 billion and $1.8 billion, respectively, and are successes by any measure, the $17 billion per annum Google Marketplace is clearly the most impressive success story of what used to be called, pre-crash, “The New Economy.”
Amazon and Ebay both worked as virtual marketplaces: they outsourced as much inventory as possible (in Ebay's case, of course, that was all the inventory, but Amazon also kept as little stock on hand as it could). Then, through a variety of methods, each brought together buyers and sellers, taking a cut of every transaction.
For Amazon, that meant selling new items, or allowing thousands of users to sell them used. For Ebay, it meant bringing together auctioneers and auction buyers. Once you got everything started, this approach was extremely profitable. It was fast. It was managed by phone calls, emails, and database applications. It worked.
Enter Google. By 2002, it was the search engine, and its ad sales were picking up. At the same time, the concept of the “Semantic Web,” which had been around since 1998 or so, was gaining a little traction, and the attention of an increasing circle of people.
So what's the Semantic Web? At its heart, it's just a way to describe things in a way that a computer can “understand.” Of course, what's going on is not understanding, but logic, like you learn in high school:
If A is a friend of B, then B is a friend of A.
Jim has a friend named Paul.
Therefore, Paul has a friend named Jim.
Jim has a friend named Paul.
Therefore, Paul has a friend named Jim.
Using a markup language called RDF (an acronym that's here to stay, so you might as well learn it - it stands for Resource Description Framework), you could put logical statements like these on the Internet, “spiders” could collect them, and the statements could be searched, analyzed, and processed. What makes this different than regular search is that the statements can be combined. So if I find a statement on Jim's web site that says “Jim is a friend of Paul” and someone does a search for Paul's friends, even if Paul's web site doesn't have a mention of Jim on it, we know Jim's considers himself a friend of Paul.
Other things we might know for sure? That Car Seller A is selling Miatas for 10% less than Car Seller B. That Jan Hammer played keyboards on the Mahavishnu Orchestra's albums in the 1970s. That dogs have paws. That your specific model of computer requires a new motherboard and a faster bus before it can be upgraded to a Pentium 18. The Semantic Web isn't about pages and links, it's about relationships between things - whether one thing is a part of another, or how much a thing costs, or when it happened.
The Semweb was originally supposed to give the web the “smarts” it lacked - and much of the early work on it was in things like calendaring and scheduling, and in expressing relationships between people. By late 2003, when Google began to seriously experiment with the Semweb (after two years of experiments at their research labs), it was still a slow-growing technology that almost no one understood and very few people used, except for academics with backgrounds in logic, computer science, or artificial intelligence. The learning curve was as steep as a cliff, and there wasn't a great incentive for new coders to climb it and survey the world from their new vantage.
The Semweb, it was promised, would make it much easier to schedule dentist's appointment, update your computer, check the train schedule, and coordinate shipments of car parts. It would make searching for things easier. All great stuff, stuff to make millions of dollars from, perhaps. But not exactly sexy to the people who write the checks, especially after they'd been burnt 95 times over by the dot-com bust. All they saw was the web - the same web that had lined a few pockets and emptied a few million - with the word “semantic” in front of it.
. . . . .
Semantics vs. Syntax, Fight at 9
The semantics of something is the meaning of it. Nebulous stuff, but in the world of AI, the goal has long been getting semantics out of syntax. See, the trillion dollar question is, when you have a whole lot of stuff arranged syntactically, in a given structure that the computer can chew up, how do you then get meaning out of it? How does syntax become semantics? Human brains are really good at this, but computers, are dreadful. They're whizzes at syntax. You can tell them anything, if you tell it in a structured way, but they can't make sense of it, they keep deciding that “The flesh is willing but the spirit is weak” in English translates to “The meat is full of stars but the vodka is made of pinking shears” or suchlike in Russian.
So the guess has always been that you need a whole lot of syntactically stable statements in order to come up with anything interesting. In fact, you need a whole brain's worth - millions. Now, no one has proved this approach works at all, and the #1 advocate for this approach was a man named Doug Lenat of the CYC corporation, who somehow ended up on President Ashcroft's post-coup blacklist as a dangerous intellectual and hasn't been seen since. But the basic, overarching idea with the Semweb was - and still is, really - to throw together so much syntax from so many people that there's a chance to generate meaning out of it all.
As you know, computers still aren't listening to us as well as we'd like, but in the meantime the Semweb technology matured, and all of a sudden centralized databases - and Amazon and Ebay were prime examples of centralized databases with millions of items each - could suddenly be spread out through the entire web. Everyone could own their little piece of the database, their own part of the puzzle. It was easy to publish the stuff. But the problem was that there was no good way to bring it all together. And it was hard to create RDF files, even for some programmers - so we're back to that steep learning curve.
That all changed - suprisingly slowly - in late 2004, when with little fanfare, Google introduced three services, Google Marketplace Search, Google Personal Agent, and Google Verification Manager, and a software product, Google Marketplace Manager.
. . . . .
Google Marketplace Search
Marketplace Search is a search feature built on top of the Google Semantic Search feature, and it's likely nearly everyone reading will have used it at least once. You simply enter:
sell:martin guitar
to see a list of people buying Martin-brand acoustic guitars, and
buy:martin guitar
to see a list of sellers. Google asked for, and remembered, your postal code, and you could use easy sort controls inside the page to organize the resulting list of guitars by price, condition, model number, new/used, and proximity. The pages drew from Google's “classic,” non-Semantic-Web search tools, long considered the best on the Web, to link to information on Martin models and buyer's guides, as well as from Google's Usenet News archive. Links to sites like Epinions filled in the gaps.
So where did Google Marketplace Search get its information? The same way Google got all of its information - by crawling through the entire web and indexing what it found. Except now it was looking for RDDL files, which pointed to RDF files, which contained logical statements like these:
(Scott Rahin) lives in Zip Code (11231). (Scott Rahin) has the email address (ford@ftrain.com). (Scott Rahin) has a (Martin Guitar). [Scott's] (Martin Guitar) is a model (245). [Scott's] (Martin Guitar) can be seen at (http://ftrain.com/picture/martin.jpg). [Scott's] (Martin Guitar) costs ($900). [Scott's] (Martin Guitar) is in condition (Good). [Scott's] (Martin Guitar) can be described as “Well cared for, and played rarely (sadly!). Beautiful, mellow sound and a spare set of strings. I'll be glad to show it to anyone who wants to stop by, or deliver it anywhere within the NYC area.”
What's important to understand is that the things in parentheses and brackets above are not just words, they're pointers. (Scott Rahin) is a pointer to http://ftrain.com/people/Scott. (Martin Acoustic Guitar) is a pointer to a URL that in turn refers to a special knowledge database that has other logical statements, like these:
(Martin Guitar) is an (Acoustic Guitar). (Acoustic Guitar) is a (Guitar). (Guitar) is an (Instrument).
Which means that if someone searches for guitar, or acoustic guitar, all Martin Guitars can be included in the search. And that means that Scott can simply say he has a Martin, or a Martin guitar, and the computers figure the rest out for him.
Actually, I just lied to you - it doesn't work exactly that way, and there's a lot of trickery with the pointers, and even the verb phrases are pointers, but rather than spout out a few dozen ugly terms like namespaces, URIs, prefixes, serialization, PURLs, and the like, we'll skip that part and just focus on the essential fact: everything on the Semantic Web describes something that has a URL. Or a URI. Or something like that. What that really means is that RDF is data about web data - or metadata. Sometimes RDF describes other RDF. So do you see how you take all those syntactic statements and hope to build a semantic web, one that can figure things out for itself? Combining the statements like that? Do you? Come on now, really? Yeah, well no one does.
So Google connects everyone by spidering RDF and indexing it. Of course, connecting anonymous buyers and sellers isn't enough. There needs to be accountability. Enter the “Web Accountability and Rating Framework.” There were a lot of various frameworks for accountability, but this one was certified, finally, by the World Wide Web Consortium, before the nuclear accident at MIT, and ECMA, and it's now the standard. How does it work? Well:
On Kara Dobbs's site, we find this statement:
[Kara Dobbs] says (Scott Rahin) is (Trustworthy).
On James Drevin's site, we find this statement:
[James Drevin] says (Scott Rahin) is (Trustworthy).
And so forth. Fine - but how do you know how to trust any of these people in the first place? Stay with me:
On Citibank's site:
[Citibank] says (Scott Rahin) is (Trustworthy).
On Mastercard's site:
[Mastercard] says (Scott Rahin) is (Trustworthy).
And inside Google:
[Google Verification Service] says (Scott Rahin) is (Trustworthy).
and if
[Citibank] says (Kara Dobbs, etc) is (Trustworthy).
then you start to see how it can all fit together, and you can actually get a pretty good sense of whether someone is the least bit dishonest or not. Now, this raises a billion problems about accountability and the nature of truth and human behavior and so forth, but we don't have the requisite 30 trillion pages, so just accept that it works for now. And that a lot of other stuff in this ilk is coming down the pike, like:
[The United States Social Security Administration] says (Pete Jefferson) was born in (1992).
Which means that Pete Jefferson can download smutty videos and “adult” video games from the Internet, since he's 19 and has a Social Security number. That's what the Safe Access for Minors bill says should happen, anyway. And don't forget the civil liberty ramifications of statements like these:
[The Sherriff's Department of Dallas, Texas] says (Martin Chalbarinstik) is a (Repeat Sexual Offender).
[The Sherriff's Department of Dallas, Texas] says (Dave Trebuchet) has (Bounced Checks).
[The Green Party, USA] says (Susan Petershaw) is a (Member).
Databases are powerful, and as much as they bring together data, they can intrude on privacy, but rather than giving the author permission to become a frothing mess lamenting the total destruction of our civil liberties at the hand of cruel machines, let's move on.
Anyway, when you think about it, you can see why Google was a natural to put it all together. Google already searched the entire Web. Google already had a distributed framework with thousands of independent machines. Google already looked for the links between pages, the way they fit together, in order to build its index. Google's search engine solved equations with millions of variables. Semantic Web content, in RDF, was just another search problem, another set of equations. The major problem was getting the information in the first place. And figuring out what to do with it. And making a profit from all that work. And keeping it updated....
. . . . .
Google Marketplace Manager
Well, first you need the information. Asking people to simply throw it on a server seemed like chaos - so enter Google Marketplace Manager, a small piece of software for Windows, Unix, and Macintosh (this is before Apple bought Spain and renamed it the Different-thinking Capitalist Republic of Information). The Marketplace Manager, or MM, looked like a regular spreadsheet and allowed you to list information about yourself, what you wanted to sell, what you wanted to buy, and so forth. MM was essentially an “logical statement editor,” disguised as a spreadsheet. People entered their names, addresses, and other relevant information about themselves, then they entered what they were selling, and MM saved RDF-formatted files to the server of their choice - and sent a “ping” to Google which told the search engine to update their index.
When it came out, the MM was a little bit magical. Let's say you wanted to sell a book. You entered “Book” in the category and MM queried the Open Product Taxonomy, then came back and asked you to identify whether it was a hardcover book, softcover, used, new, collectible, and so forth. The Open Product Taxonomy is a structured thesaurus, essentially, of product types, and it's quickly becoming the absolute standard for representing products for sale.
Then you enter an ISBN number from the back of the book, hit return, and the MM automatically fills in the author, copyright, number of pages, and a field for notes - it just queries a server for the RDF, gets it, chews it up, and gives it to you. If you were a small publishing house, you could list your catalog. If you had a first edition Grapes of Wrath you could describe it and give it a lowest acceptable price, and it'd appear in Google Auctions. Most of the smarts in the MM were actually on the server, as Google interpreted what was entered and adapted the spreadsheet around it. If you entered car, it asked for color. If you entered wine, it asked for vintage, vineyard, number of bottles. Then, when someone searched for 1998 Merlot, your bottle was high on the list.
You could also buy advertisements on Google right through the Manager for high-volume or big ticket items, and track how those advertisements were doing; it all updated and refreshed in a nice table. You could see the same data on the Web at any time, but the MM was sweet and fast and optimized. When you bought something, it was listed in your “purchases” column, organized by type of purchase - easy to print out for your accountant, nice for your records.
So, as we've said, Google allowed you to search for buyers and sellers, and then, using a service shamelessly copied from the then-ubiquitous PayPal, handled the transaction for a 1.75% charge. Sure, people could send checks or contact one another and avoid the 1.75%, but for most items that was your best bet - fast and cheap. 1.75% plus advertising and a global reach, and you can count on millions flowing smoothly through your accounts.
Amazon and Ebay - remember them? - doubtless saw the new product and realized they were in a bind. They would have to “cannibalize their own business” in order to go the Google path - give up their databases to the vagaries of the Web. So, in classic big-company style, they hedged their bets and did nothing.
Despite their inaction, before long all manner of competing services popped up, spidering the same data as Google and offering a cheaper transaction rate. But Google had the brand and the trust, and the profits.
It took 2 years for over a million individuals to accept and begin using the new, Semweb-based shopping. During that time, Google had about $300 million in volume - for a net of $4.5 million on transactions. But, just as Ebay and Amazon had once compelled consumers to bring their business to the web, the word-of-mouth began to work its magic. Since it was easy to search for things to buy, and easy to download the MM and get started, the number of people actively looking through Google Marketplace grew to 10 million by 2006.
. . . . .
Google Personal Agent
Now, search is not enough. You need service. You need the computer to help you. So Google also rolled out the Personal Agent - a small piece of software that, in essence, simply queried Google on a regular basis and sent you email when it found what you were looking for on the Semweb.
Want cheap phone rates? Ask the agent. Want to know when Wholand, the Who-based theme park, opens outside of London? Ask the agent. Or when your wife updates her web-based calendar, or when the price of MSFT goes up three bucks, or when stories about Ghanaian politics hit the wire. You could even program it to negotiate for you - if it found a first-edition Paterson in good condition for less than $2000, offer $500 below the asking price and work up from there. It's between you and the seller, anonymously, perhaps even tax-free if you have the right account number, no one takes a cut. Not using it to buy items began to be considered backwards. Just as the regular Google search negotiated the logical propositions of the Semweb, the Personal Agent did the same - it just did it every few minutes, and on its own, according to pre-set rules.
. . . . .
Google Verification Service
Finally, Google realized they could grab a cut on the “Web of Trust” idea by offering their own verification and rating service, $15 a year to answer a questionnaire, have your credit checked, and fill in some bank account information. But people signed up, because Google was the marketplace; the Google seal of approval meant more than the government's.
. . . . .
A Jury of Your Peer-to-Peers
Since all the information was already in RDF format, Google's own strategy came back to bite it. Free clones of Google Marketplace Manager began to appear, and other search engines began to aggregate without the 1.75% cut, trying to find other revenue models. The Peer-to-Peer model, long the favorite of MP3 and OGG traders, came back to include real-time sales data aggregation, spread over hundreds of thousands of volunteer machines - the same model used by Google, but decentralized among individuals. Amazon and Ebay began automatically including RDF-spidered data on their sites, fitting it right in with existing auctions and items for sale, taking whatever cuts they could find or force out of the situation.
In 2006, Citibank introduced Drop Box Accounts for $100/month, then $30, then $15, and $5/month for checking account holders. The Drop Box account is identified by a single number, and can only receive deposits, which can then be transferred into a checking or savings account. They were even URL-addressable, and hosted using the Finance Transfer Protocol. Simply point your browser to account://382882-2838292-29-1939 and enter the amount you want to deposit. There's no risk in giving out a secure drop box number, and no fee for deposits. Banks held the account information of depositors in federally supervised escrow accounts. Suddenly everyone could simply publish their bank account number and sell their goods without any middleman at all.
Feeling the pressure, and concerned, just as the music companies had been ears before, that their lead would slip to the peer-to-peer market, Google dropped its fees to 1%, allowed MM users to use Drop Box accounts, and began to charge $25 a year for the MM software and service for sellers, while still making it free for users. After a nervous few months, Google found that the majority of users who sold more than 10 items per year - the volume users - were glad to buy a working product with a brand name behind it; the peer-to-peer networks were considered less trustworthy, and the connection to Google advertising. Google also realized that they could also offer Drop Box accounts, and tie them to stock and money-market trading accounts, which opened a can of worms that we'll skip over here. If you're interested, you can read The Dragon in the Chicken Coop, by Tom Rawley.
Google's financials can, of course, be automatically inserted into your MM stock ticker; right now they're trading at 25,000 times earnings, heralding news of the “New New New New Economy.” You'll get no such heralding here; while they've pulled it off once, the competition is fierce. Google was the dream company for a little less than the last decade, but they're finally slowing down, and it's high time for a new batch of graduate students too itchy to finish their Ph.D.'s to get on the ball. And I'm sure they will.
. . . . .
A Semantically Terrifying Future?
The cultural future of the Semantic Web is a tricky one. Privacy is a huge concern, but too much privacy is unnerving. Remember those taxonomies? Well, a group of people out of the Cayman Islands came up with a “ghost taxonomy” - a thesaurus that seemed to be a listing of interconnected yacht parts for a specific brand of yacht, but in truth the yacht-building company never existed except on paper - it was a front for a money-laundering organization with ties to arms and drug smuggling. When someone said “rigging” they meant high powered automatic rifles. Sailcloth was cocaine. And an engine was weapons-grade plutonium.
So, you're a small African republic in the midst of a revolution with a megalomaniac leader, an expatriate Russian scientist in your employ, and 6 billion in heroin profits in your bank account, and you need to buy some weapons-grade plutonium. Who does it for you? Google Personal Agent, your web-based pal, ostensibly buying a new engine for your yacht, a little pricey for $18 million, sure. But you're selling aluminum coffeemakers through the Home Products Unlimited (Barbados) Ghost Taxonomy - or nearly pure heroin, you might say - so you'll make up the difference.
Suddenly one of the biggest problems of being a criminal mastermind - finding a seller who won't sell you out - is gone. With so many sellers, you can even bargain. Selling plutonium is as smooth and easy and anonymous (now that you can get Free Republic of Christian Ghana Drop Boxes) as selling that Martin guitar. Couldn't happen? Some people say it can, which explains the Mandatory Metadata Review bill on its way through Congress right now, where all RDF must be referenced to a public taxonomy approved by a special review board. Like the people say, may you live in interesting times. Which people? Look it up on Google.

(Source: http://www.ftrain.com/google_takes_all.html)

What is the Future of Search?

By Stephen Turcotte

A clearer picture of the future of search is developing and, in case you were wondering, that’s great news for search engine marketers.

At the March 2004 Search Engine Strategies conference in New York City, the Walter Cronkite of the search industry, Danny Sullivan, moderated an enlightening session dubbed “Roundtable: The Future of Search”. The panelists included top people from Yahoo!, Google, Ask Jeeves and AOL Search. In attendance were over 570 search engine marketers and major brand marketers from Australia to Madison Avenue. Each of the panelists were asked to provide their opinion where search is heading and how search engine marketing may change.

At this conference the future of search engine marketing started shaping up. The bottom line is that users want information to be accessible, they want more of it, and they want it fast and relevant. Search engines of the future will be personal assistants, butlers, guides and gurus, providing potential answers to our most obscure questions. Search engines of the future will permeate many more aspects of our lives becoming more pervasive, personalized and localized.
For search engine marketers, the key to maintaining prominent listings will be based on a combination of relevance to user queries and ability to pay for the traffic.

Search Pervasiveness

Most of the panelists believed that “access points” to search would grow beyond our desktops suggesting that the next possible venue for search would be our car dashboards and trolley backrests. I’m not crazy about giving drivers more distractions, but the in-car concept is easy to imagine considering the prevalence of OnStar and onboard GPS systems. How hard would it be for a car manufacturer to build a car with Google Inside? The Google Inside mobile could have voice controlled search functionally that searches for results that are pertinent to your current location on a GPS map. I can see my first voice activated search now, “Google Inside, please search for auto body mechanic and personal injury attorney within limping distance.”
Here’s a more sensible prediction - mobile communications device manufacturers bundle voice controlled web searching features into their phones and PDAs. That’s not such big leap either; many of today’s PCS and integrated PDA devices already have voice recognition systems built in. Here are a few other access points we can expect to see search: our kitchen appliances – search for recipe for scallion pancakes, our airplane seats – search for places to go when you arrive at your travel destination, random street corners – do a local area search for a watch repair service.

Localized and Personalized Search

The trend towards personalization is nothing new, but it seems that all the search engines are really starting to roll out services that utilize all the data that they are capturing about us. One of the most thought provoking presentations of the entire conference came from Craig Silverstein of Google when he gave his vision that in 300 years search engines will be more like yeast based search pets that understand our emotions and inferences.

Gerry Cambell, General Manager of AOL Search and Navigation gave a more near future example of inference-based search. He said AOL’s current search learns from their users habits and location and uses that information to provide more relevant search results. "If one of our subscribers searches on ‘pizza,’ our search results will return a list of stores in their ZIP code. In the near future, search results on 'Eagles' will be different for a user who visits sports sites and one who goes to classic rock sites."

At this point a question to consider is how much information should a search engine be allowed to collect about a user before the user’s privacy is violated. Yahoo! Search, for instance, claims to respect users’ privacy by giving the user more control over the results while still extending search to include user intent.

My sentiments on this issue are that I sometimes want the search engine to surprise me with results. Consider going to the library and looking for some particular book on a shelf - you get to check out all kinds of books on the same shelf that you haven’t considered, and which you may find even more interesting than the book you initially searched for. If the search engines of the future know you too well and present more focused search results, these results will only be as good as your search query. Over personalizing search to the point that users are only seeing a narrow page of results would take the adventure and discovery out of search.

Will the Highest Bidder Dominate Search?

Most of the search engine marketing experts I have heard from agree that 2004 is shaping up to be a big year for paid search. And how can you argue with that, a JupiterResearch survey predicts online advertisers will spend 47% of their budgets on search this year, that's a 21% increase over the 2003 ad spend survey. So, what does that mean for the “natural” search engine marketers? Will search engine positioning become a commodity for the highest advertising bidder to spend directly at the search engine venue, or will there continue to be a great value in optimizing websites for natural search traffic? Well, if some of the major search engines get their way, a substantial portion of the advertising dollars will go directly to them in form of paid placements or sponsored listings. But lets be real – it looks plausible for the short term, but if search engines claim that they will interpret our inferences in the future, then they can’t possibly rely solely on paid programs for relevance. After all, the search engine that has been most popular with the users, Google, has utilized the motto of serving the user and providing relevant results as opposed to serving the companies who are trying to generate traffic, and that is the very reason for its popularity. That means that the search engine advertiser will always get the best ROI from being relevant, and that is great news for all the natural listings search engine marketing companies.

Back to the Future

To summarize my projections for the future of search and search engine marketing, lets start by saying that the evolution of search engines is largely determined by user behavior. As users, we already expect great results from majority of the search engines. It is reasonable that we will expect even better, more relevant, results in the future, and that we will want this search functionality to be more accessible. If search is where the consumer is, more companies will want to improve their presence online. The new trend towards paid search will most likely lead to the highest bidder dominating search results in some cases. On the other hand, if this happens, then it will inevitably make natural or organic search engine results more useful to the end user. Paid search will be harder to get into for medium and small players, and they will turn to natural results. In the end, the search engine that gives the most useful search results will dictate the future of search.

(Source: http://www.backbonemedia.com/futureofsearch.asp)

http://www.searchengineguide.com/

The Future of Search Engine Technology

By Andy Beal - January 28, 2004

By now you have probably read numerous articles predicting "What will happen in 2004" or "Can MSN take on Google". While it is always worthwhile to look ahead and consider what may happen this year in the search engine industry, what about the things that we can't quite yet predict? Instead of looking at what will happen this year, perhaps we should look at what must happen in the search engine space if Google, Yahoo and MSN are truly able to revolutionize search and enhance the user experience.

Overcoming The Lack Of Relevant Search Results

Even today, conducting a search on any of the major search engines can be classified as an "enter your query and hope for the best" experience. Google's "I'm Feeling Lucky" button, while designed to take you directly to the number one results, could ironically be a truism for its entire search results (process?). Enter your desired search words into any of the search engines and you often end up crossing your fingers and hoping that they display the type of results you were looking for. Since the recent updates of "Florida" and "Austin", complaints that Google, in particular, is displaying less relevant results have escalated (although mostly by those who lost important positioning that they had assumed was their right to maintain). There is, of course, evidence that the search engines are trying to enhance their search results- so that they can better anticipate the intentions of the searcher. Search for "pizza Chicago" at Yahoo, and you'll see that the top results include names, addresses, telephone numbers and even directions to pizza restaurants in Chicago, a great improvement on previous results. Even when you take everyone's favorite search term example, "windows", you can see that the search engines are at least trying to determine your intent. While Yahoo and Google still display search results saturated with links discussing Microsoft's pervasive operating system, enter your search over at Ask Jeeves and the chirpy English butler will ask you if you meant "Microsoft Windows" or "Windows made out of glass".

Future Search Engine Technology

Smaller search engines have also materialized over the past few weeks, each offering to improve the user experience. Grokker offers an interface that groups search results graphically, improving the way search results are segmented and displayed. Eurekster, combines the social networking elements that are used by sites such as Friendster, and provides results that can be filtered based upon what members of your group are searching. While all of these are interesting and provide a glimpse of the future of search, it will not be the small companies that change the way we search. With Google about to get an influx of cash from its upcoming IPO, Yahoo re-vamping Inktomi and Overture, and Microsoft finally jumping into the search arena, it will be these search engine powerhouses that enhance our search experience and take search engine technology to the next level.

So what is this next level? What technology is it that I speak of, that will revolutionize the way we receive our search engine results? I believe that the search results we receive in just a couple of years from now could make current search engine technology look as archaic and cumbersome as picking up a Yellow Pages book is today. However, in order to achieve this new search nirvana we, as consumers, must quell our fears and trepidations surrounding the protection of our privacy. In order for the search engines to develop technology that will be intuitive and anticipate our every need, we must first relinquish at least some of the privacy that we currently hold so dear. Let's take a look at some of the ways that search technology could improve and you'll soon get the idea why it will require us to cooperate with the search engine providers. "Windows" or "windows"? If you desire to be able to enter a term as ambiguous as "windows" and expect to see relevant results, you'll first need to give up some personal information to the search engines. Google, Yahoo, MSN and Ask already have the means to collect an astonishing amount of information from us, by our use of their toolbars. Don't panic, they currently allow you to disable this information gathering, and even if you do allow it, it is collected anonymously. However, with the technology already in place, why not unleash its full potential? Let's say I let Google track my online activities, allowing it to monitor the web sites I view and keep a log of all of the search queries I enter. This type of information could greatly improve the relevancy of the results displayed to me. For example, two years from now, I could search for "home improvement" on Google. I then find the listing for Lowes.com and visit the site. While I am at their web site, I look at a number of different pages, but I spend a lot of time in the "house windows" section, exploring the different styles and prices. Why not let Google capture all of that useful information? Then, when I go back to Google the following day and search for "windows" it would know that glass windows is more likely to be the type of product I am seeking out. Google would simply have remembered my previous searches, read the HTML and Meta data, located on the Lowes.com pages, and used this to identify the intent of my new search for windows. While I would have to give up some of my privacy, wouldn't it be worth it if I could save myself time and energy by having search engine results more relevant to my desire? You've Got Search In Your Mail Another area with great potential for improving search engine results will likely be developed by Google. You may have heard the rumors that Google is getting set to launch an email client that many expect will be a free service similar to Yahoo Mail or Hotmail. Currently, Yahoo does an adequate job of making search available to all of its email customers. Each page within Yahoo Mail has a search box that makes it easy for you to conduct a search that might be sparked by an email you receive. But why not take it one step further? Google has the technology to really take advantage of search within email. Why else would it even consider entering this arena? Imagine that, in order to use a free Google email account, you allow Google to provide advertisements and track your email activities. Google could change the way that search results and ads are displayed to free email users. For example, let's say you receive an email from your brother, the content of which, among other things, gloats about the brand new P4 desktop computer that they just purchased from Dell. As part of the interface you use to read that email, Google magically displays paid search advertising for desktop computers, including a link that will take you directly to the appropriate page on Dell.com. This information would be quite beneficial to you, as you may be interested in seeing how you too can be a proud owner of a P4 computer. Fantastic targeted advertising for Dell, as they know that if you click on the listing, they are halfway there to converting you into another satisfied customer. This idea is so much closer to reality than you may think. Google already has the advertisers with its AdWords service boasting 150,000 users, eager to spend their advertising dollars. It also has the technology to determine which results to show you within your email interface. Google's AdSense can provide the contextual ad technology that would scan an email's content to determine which ads are the most relevant to display. With this technology in place, a simple provision within any Google Email Terms & Conditions would give the world's largest search engine the necessary permission to serve up relevant ads to all users of its free email service. We could be offered the option of paying a monthly premium in order to not have ads shown when we read our email, but if they are relevant to the content of a received message, why would we want to block them?

From Desktop to Internet

Another development in search engine technology that I can see happening would come from the development of Microsoft's new Longhorn operating system. While I must confess that I am not au fait with the intricate workings of this project, I do know that it will likely use the search technology that MSN is developing. Imagine an operating system that monitors all of your activities -- with your permission, of course. Every file, every image, word document, mp3, even e-books could be monitored by your computer as it endeavors to anticipate your every need. Not only could an integrated search engine allow you to search files located on your hard drive, but it could also use the information it has collected from these files to make your online search experience even more enjoyable. It is quite possible that Longhorn or a future OS (Microsoft, Linux or Mac) could become intelligent enough to know that after listening to one of your favorite songs by the 80's rock band, Heart, your consequent search online for "heart" is more likely to originate from a desire to view the band's fan site, than that pressing need to visit the web site of The American Heart Association. Your all-encompassing search engine would perhaps be a realization of the Ask Jeeves friendly butler, ready to anticipate your every need. To Search Where No-one Has Searched Before When you think about the future of search, it is easy to get excited. Millions (if not billions) of dollars are going to be filling the coffers of the largest search engine providers. They have some of the smartest people in the world working to develop the next great "thing", which will enhance the user experience and serve up better, more relevant search results. Search engine technology is still most definitely in its infancy; how it grows will very much depend upon how much information and privacy the average search engine user is willing to give up. Personally, if I can view search results that more closely match my desired results, I'm willing to give up the name of my favorite pet, my place of birth and my mother's maiden name!

(Source: http://www.alwayson-network.com/comments.php?id=1202_0_1_0_C)
Also see http://www.pandia.com/sw-2004/06-future.html for a probably updated version.

The Future of Search

As the haystack gets bigger, we need better ways to find the needle.

Alex Moskalyuk [ITFacts.biz] POSTED: 10.15.03 @16:14

It's interesting how search suddenly got to be one of the hottest Internet businesses. David Krane from Google likes to tell a story of Larry and Sergey's visit to some major Internet players before settling on their own project. Back then the major sites were in the portal game: provide a start page from which consumers browse to other featured and sponsored locations and—provided you had enough eyeballs and stickiness (both are still part of the Internet marketing lingo)—you were all set in the big portal game.

Google became successful by doing precisely the opposite: allowing the Internet user to leave the Google.com site as soon as possible. However, as Chris Worth notes, "a search engine is quietly becoming a major advertising agency." This puts Google, with projected billion-dollar advertising revenue, among the world's top ten ad agencies. Considering DoubleClick's $300 million in total revenue for 2002, Google and Overture can share the title of largest Internet ad agency.

To succeed in the Internet advertising business over the next few years, you need a superb search engine, even if the search lasts just a few seconds before the user is off to a different site. It would be interesting, then, to collect ideas on what new things we can expect in the Internet search business. Here are just a few of my own.

Beyond keyword searching.

While most of us have excellent search skills and know what Boolean keywords and double quotes do, the quest for information is not always keyword-based. If you want to find out as much as possible about rock music in Pakistan or potato growers in Idaho, keyword- or phrase-based search is indispensable. Quite often, though, we search for facts and answers. Starting from the simple "What time is it now?" to "What was the pre-IPO valuation for Pets.com?" we search for information—for facts that we know or suspect might exist. A search engine that satisfies this type of search would have to go beyond keywords because keywords produce too much noise and don’t allow the software algorithm to define a relevant solution space.

Comprehensive index.

When Alltheweb.com’s index consists of three-something billion pages, it's hard to complain about the index. However, a lot of information is still confined to the paid databases. As a software developer, I find a great wealth of information on sites like Safari where I am a paid subscriber. If closed-access databases that charge customers for access were able to somehow integrate their content into the modern search engines, finding exact results would be a lot easier.Multimedia text-based searches. Back to the keyword-based searching, it would be exciting to have an engine that would parse not only electronic documents, but samples of human speech and digital multimedia (audio and video). Want to find out how frequently a certain keyword was used on corporate VOIP voice mail? Just type into the search window and have the links to the audio files appear.This list is far from complete. Many of the aforementioned technologies have already been pioneered by startups. Northern Light tried to hook up its Web search to the proprietary databases, and Nexidia is currently working on multimedia keyword search technology. Looking at the future of Web search, it's hard not to get excited. What search technologies do you envision developing in the near future?

Alex Moskalyuk is a recent graduate of Eastern Washington University; he's currently involved in embedded systems and home automation devices with Spokane-based Itron.(3819 views) [4 opinions]

Member Comments

Two trends seem clear in this area:

1. An increase in the types of information distinctly indexed and supported (think: web sites, blog posts, RSS feeds, local business listings, news stories, products and comparison data).

2. An increasing focus on enabling you to actually "do" something with the information returned by such searches (think: save, organize, share, transact, post) versus simply getting and forgetting.Check out Verdada's VSOS (disclaimer - it's my company) - www.verdada.com - for one company's approach to embracing the above trends.Also, there is a great piece on a scenario of how Google beats Amazon and eBay to the semantic web that's worthwhile for peeling back the onion further on this one: http://www.ftrain.com/google_takes_all.html

The future of search may be in personal search engines and digital libraries.Check out www.noceans.com

AIS - Associative Indexing Service (Life-time archiving solution)

"Don't delete, don't sort, don't search - group!"

Open source. Also on http://sourceforge.net/projects/ais

WHAT IS IT

Indexing service is a program which allows the addition of keywords to locally or remotely stored resources (such as files, web resources, etc.) It's made for an ability to access information in the future in just one click or CLI invocation. Keywords are supposed to be personal - not necessarily the words that appear in the text (as opposed to automatic indexing) but rather valuable for this particular person associations pointing to different data pieces in huge archives. It works like memo cards, leaving the original files untouched, but adding the desired metadata (keywords) to a database (which can be stored in a personal organizer or cellphones, in the future, and synchronized with huge home databases). This can be seen also as a portable scalable bookmarks database.

The goal is to have linear (or maximum - logarithmic) access to resources from huge archives (terabytes and petabytes - in future) where the hierarchical nature of the file-system is not sufficient for fast information finding (actually - find/grep/sed or search is converges with human life;) and graph representation is necessary - for any information grouping (groups intersection)

Another name for (or view to) the application is "Personalized Search Engine" - because one of interfaces to the metaindex is web interface, similar to search engines interface. Categories (or keywords) are forming any grouping of information so directory names are not carrying any symantic meaning and may be created automatically as incremental backups (example - daily). Keeping automatically-generated indices separate from user associative keywords is important - to make search scalable and fast and it is likely that the same user will use the same associations more than once when searching in the future, so it is his/her "personal shortcut to the information, observed in the past". The same role as directories are playing with small archives.

I like to think about such index metadata as a concentrated information (in some domain) - as a result of work of a particular individuate which has been expressed in grouping and subgrouping of the information taken from the infinite internet. If you want - in mathematical terms it is the OPERATOR OF ACTION (of the specialist) on the informational ENVIRONMENT with the result expressed AS a limited SUBSET of that environment been grouped, WITH assigning of (meaningful) SYMBOLIC VARIABLES to those groups.
screenshots (ver0.3)

To run the application on the local machine (for java version having JVM installed):1)unpack zip or tar.gz2)run server script3)open browser on http://localhost:8080/is

WHY

Don't waste your valuable lifetime on reorganizing archives and searching through the old data...
...when you can easily index all the data when storing it to the archive.

After a mulititude of years of maintaining software archives (old programs, dos games, pieces of code, short records and sayings, books and documents, multimedia files, mail archives, old system configurations and even complete system backups), I realized that the hierarchical nature of the filesystem won't give me the possibility to create arbitrary grouping for all my data (and the creation of multiple soft-links to the same resource - I use linux - does not make life easier). More over, grouping criteria usually change over time. Sure, the solution is maintaining of metadata database. Such metadata database must be scalable because we will be adding to it the rest of our lives ;) It must be robust (sure we'll backup/mirror it against fatal disk crashes). It must be independant of the technology flow: if new technology appears, it must be possible to transform the Index easily and fully automatically - even with the migration to another database implementation, without any pain or risk. So, assuming the following 3 requirements, in the order of significance:* robustness of the Index databases* scalability* flexibility (portability is here)The first version of the Index was plain text files (with names corresponding to keywords) placed into directories (A,B,C...) having no more than the observable number of keys in a directory. If number of keywords for particular letter becomes big and we have performance decrease (due to non-proper file-system use, for example) - we may create another level and another level - corresponding to letters of the key (BTW Solaris OS keeps mails in such way;). Then I've wrote a script which creates such directories automatically, balancing the tree, following the principal: "divide and conquer". Such scripts hide the physical location of the key file in the host filesystem returning the file (path - in the 1st version). Content of such file is just lines, containing bookmarks or URI's for the particular key. (Sure the key should be properly escaped). Such database seems to be very scalable especially when XFS/JFS-like filesystem is used (BTW it seems that even no need in balancing in such case).

What we have in the abovementioned approach is a scalable persistent hash implementation where on the right side we have a unique identificator of a resource (files in case of local archive, or directories for grouped into simple lists files) and due to any way of grouping of such resources we have many-to-many relationship between all the (personal) keywords and all the resources existing in the repository or even in internet (for url bookmarks). Comparing with the traditional grouping approach, where symantic value is put into directories names, we are able to group resources in any way, see, browse and get any resource (from the archive) in a limited time, in one action (and program provides such access in linear time to the size of all the keys) - even if the number of all the resources observed and/or saved in the past is very-very big.
Example of the trivial database with one level of letters distribution and hand-made saving (here I illustrate the very first way of bookmarking - without scripts) - just for illustration:$echo 20041110/books/bind/ >> /mnt/backup/Index/b/bind $echo 20041110/books/bind/ >> /mnt/backup/Index/d/dns where resource (book or tutorial consisting one indivisible set of files) is saved into directory 20041110/books/bind/ (internal, invisible directory name) under 2 keys (which will be likely used for access): 'bind' and 'dns'. Sure, scripts are hiding this process. After some time I decided to make remote access to the archives through web interface which is very similar to search engines interface - for remote access (as well as saving) and in future may be - all file-system operations. The application is still in the development phase although basic operations are already working. There is a possibility to use berkley DB or RDB - for index storing and the persistent mechanizm can be easily switched (corporate customers could use oracle or any another database easily - just by changing one property in config file).

Of course file-based Index repository may be created even using text editors (without using of any tool, script or application). Informaiton is coming each day including today: the sooner one begins using the Index, the more time she/he will gain in the future. Therefore, I can suggest starting to use the file-based (or similar) approach asap: big piles of bookmarks, new mails, documents, viewed pages, etc. arrive quickly and it will be more and more difficult to find time in future to reprocess this information again, to recreate keys for it. And only while reading (the first time probably), did we create the most appropriate key that we'll likely use in the future when searching for this document - different for different people, on their native languages, may be even not included in the vocabulary words. Of course it is easier to place all the information as a whole (under the same root, mount point, disk etc) not forgetting to backup it, synchronize the disks periodically (I use 'rsync -avuzessh arch1 me@remotehost:/mnt/arch1' for that), and check the data integrity. Ideally - you should use RAID and/or mirroring of the disks). Those who are paranoid can put a replica to a remote location (parents home?), put the second disk into a safe, engrave it in stone, use an encripted file-system, etc. ;)

This way, your data-saving process will be more transparent, scalable (remember that computer bus and network are growing slower than storage space, so you will less likely get - how I name it - 'crizis of copying'), you will be more confident with your old data safety, at the same time having instant access to any piece of it. With implementing of remote secure access to the data repository, future applications can transform your archives with index into "Extension of associative memory" (imagine access to all your data from handheld or special glasses with built-in semi-transparent liquid cristals showing an output from your queries;).
-- don't waste your valuable lifetime on reorganizing archives and searching for old data.

Regards, Vasili Gavrilov

(Source: http://srcportal.net/is.html)

http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/FindInfo.html

Quintura Search

Quintura is a new search technology that will be ready for download in October. The company has called their approach to searching “The Way People Search”.

The new search software promises to help users find information on the web easier and faster by defining the context or meaning of the search term.

Quintura also uses dynamic clusterization that builds or changes clusters on the fly depending on user input.

The new search software will also offer visual semantic maps that let you add or subtract keywords and change context with the click of your mouse.

Quintura co-founders Yakov Sadchikov and Alexander Ershov have a background in research and development in the area of neural network and artificial intelligence and are inspired by the Pythagoreans’ philosophy of quintessence. Sounds interesting to us.

Quintura Search is a free software application that operates on top of search engines and web content aggregators.

(Source: http://www.pandia.com/sew/90-quintura-search.html)

A GLIMPSE OF WHAT THE FUTURE OF SEARCH WILL BE LIKE

SEARCH AS KNOWLEDGE MANAGEMENT

VIVISIMO

Solutions

A New Approach is Required

Most enterprises recognize the strategic importance of connecting employees to the content they need, as it directly affects their costs, revenues, productivity, and even competitiveness. The amount of this "information capital" in organizations has grown at an astronomical rate over the last few years as new digital content is being created and older content is being digitized and saved. However, the tools to handle this "information overload" have lagged behind.
Simple search is no longer enough. Enterprises must expand information retrieval systems beyond free-form searching. They need to add the ability to sort and organize results meaningfully, letting users gather more knowledge and insight from content — not just data points.

Vivísimo's search and clustering solutions are based on a powerful new approach that organizes search results into meaningful categories without requiring any preprocessing of documents. Our comprehensive content search and clustering solution requires no preexisting taxonomy, yet works to enhance existing taxonomies where they exist.

Vivísimo's solutions can be broadly adopted across industries. We invite you to explore our industry solutions to learn more about how Vivísimo offerings might benefit your company:
Enterprise
Government
Life Sciences
Publishing
Web Search

Why Vivísimo?

The award-winning Vivísimo clustering technology, developed by Carnegie Mellon research scientists, is unlocking the value of stored information at Fortune 500 companies, top websites, prestigious publishers and government agencies by categorizing search results on the fly into meaningful folders, thus achieving breakthrough improvement in access to relevant information.

Enterprise

If you rely on search-based knowledge and information to get your job done, you may find that you're increasingly being asked for the "big picture," a view of the whole forest instead of a single tree. You are charged with analyzing and synthesizing volumes of data from internal and external sources to provide overviews and to identify trends, relationships and recent activities.
Traditional enterprise search engines attempt to find a needle in a haystack. They try to guess your intent from one- or two-word queries and return the best matches for them. Different types of results are mixed together, making your job more difficult. Traditional search is like leaving you in a room with books piled randomly on the floor. What you need to meet your mandate is a way to organize those books into groups or categories based upon certain similarities. This is what Vivísimo does.

Vivísimo Velocity enables enterprise information seekers to:

See the "big picture" by identifying main themes from the most current results
See more deeply when you get a quick overview of 200 results on the first page
Find more information more quickly by reviewing similar results grouped together
Discover unexpected relationships
View categorized information without taxonomy headaches

Government

Interagency information searches. Interdepartmental information searches. Citizen and business information searches. Which type of search are you responsible for?

As a member of our government, you realize that the government is a vast storehouse of knowledge. Your goals, at the initiative level, may be to:

Improve internal search effectiveness and efficiency
Help constituents quickly find information and answers
Improve decision-making processes
Effectively respond to other governmental organizations and citizens

Search. It's Not So Simple Anymore.

However, with millions of internal and external governmental documents available electronically, simply providing "information" is not an effective way to meet any of those goals. Search product vendors pride themselves on solutions that return thousands of "relevant" documents for any query. To further complicate matters, critical data often resides in separate databases and internet-based sources with varying structures. This means each searcher, internal or external, must search through various information repositories separately. And once they do search, knowledge seekers are presented with long lists of results — sometimes relevant, sometimes not.

In this environment, how can analysts, researchers, service representatives, and citizens gain quick access to the most relevant and complete information available? At what cost do they ignore the results that might be buried deeply within search results pages? While these considerations are always important, they are especially important in government applications such as the military, intelligence communities, law enforcement, etc., where the consequences of "information overlook" can be disastrous.

Vivísimo. We Make Your Mission Possible.

Vivísimo's information retrieval solutions address the problems of "information overload" and "information overlook" faced by citizens, government employees, and partners.
Built upon our award-winning clustering technology, Vivísimo solutions are unique because they offer governmental agencies and departments the ability to provide knowledge seekers with categorized search results without the effort, expense, and time required for traditional taxonomy-based solutions.

Vivísimo's clustering technology intelligently organizes the results from your own search engine and third-party search solutions into clusters or folders without any preprocessing.

Existing Features of 'Advances Search'

LYCOS
Search - Web, People, Yellow Pages, Products, Images & Audio, Discussion, News
Block Offensive Content - Always, Sometimes, Never
Language - Limit results to a specific languages
URL/Site - Return results in specific domain (e.g. wired.com) or top-level domains (e.g. .gov). Multiple domains/sites may be specified, separated by a comma.
Region - Limit results to a specific continent or country.
Date - Limit results to pages published within a specified period of time.

SEARCH ENGINES GALORE

Which Resource to Use for What

See: http://www.noodletools.com/debbie/literacies/information/5locate/adviceengine.html
Also: http://infopeople.org/search/
and http://infopeople.org/search/chart.html

Individual Search Engines

Google (http://www.google.com)
Yahoo (http://www.yahoo.com)
MSN (http://www.msn.com)
AskJeeves (http://www.askjeeves.com)
Lycos (http://www.lycos.com)
AltaVista (http://www.altavista.com)
Hotbot (http://www.hotbot.com)
Excite (http://www.excite.com)
GoJester (http://www.gojester.com)
Clusty: (http://www.clusty.com)

Search Multiple Engines

Dogpile (http://www.dogpile.com)
Monstercrawler (http://www.monstercrawler.com)
Mamma (http://www.mamma.com)
Vivisimo (http://www.vivisimo.com)

Speciality

Humorsearch (http://www.humorsearch.com)

FREE AGENT NATION

Just got referred to the book 'Free Agent Nation'. Haven't read it yet, but seems to hint on the trend I mentioned towards the 'TEN BILLION STRONG ORGANIZATION'.

25 million Americans were working free-lance according to the book, published in 2001. The figures could be higher now. Extrapolate that trend, and combine features like Micro-payments; Online Resumes, Reference Checks and Job History; Outsourcing, insourcing, offshoring, nearshoring, and inshoring; Temping; and the myriad other aspects enabled by the rise of the computer-communication revolution, and you can clearly see that in the future, every person will have, and may prefer, the option to work as a free-lance operator.

Organizations will cease to exist as separate entities, due to merging, partnering, networking, hollowing out, virtualizing, and what not. What we will see is 10 billion humans (my estimate for the population at that time), each working almost independently, for each other, in one giant networked organization - the TEN BILLION STRONG ORGANIZATION.

THE IMPORTANCE OF SEARCH AND MATCH

Search and match seem to be the most important activities of humans. Solve this problem and you solve many of human issues.

Here are some of the situations where you need to search and match. How to do it using technology is the problem.

• Search for patterns in nature, knowledge, (science, discovery, philosophy)
• Search for a suitable school, college or university (education)
• Search for a suitable job, or a suitable candidate for a job (employment)
• Search for buyers or sellers of products and services (business, trade, marketing)
• Search for a suitable mate (romance, love, marriage)
• Search for cures for diseases (medicine)

If one can build a search engine which solves the problem of search and match, it would revolutionise human society. Eliminating the time and effort spent on search would enable people to make a dramatic change in the way they live their lives. Imagine if you could instantly find a buyer for your goods or services - what a jump in business it would bring. If you could find the supplier you want for the goods and services you need, wouldn't it ramp up the satisfaction and comfort level of your life? If you could instantly find a job, what a relief it would be!

I would like to work on such a search engine, one that does not merely look for web-sites based on keyword search, but enables people to link together their needs. It would have to be a collaborative engine, and the knowledge-links built up by many different people. It would utilise the power of existing engines (like search engines, job sites, trade boards, dating sites, etc.) and then mix and match the results. It would have to be an wide-spread collaborative effort like Wikipedia or something like the Open Cyc project where thousands of ordinary people are feeding in and contributing common-sense to an artificial intelligence.

Lets say you are looking for a job. You put in your skills, qualifications and requirements into the search engine, and it goes off, searching multiple job sites like Monster, Workoplis, Resume.ca, etc. You put in your parameters (like where you like or can work, pay range, industry, and so on) and the employer puts in his own parameters (like required certifications, skills, years of experience, etc.). Some of these parameters may be marked essential or mandatory, so if these don't match, they are filtered out, and you are not bombarded with a list of irrelevant job openings. There should be a feature enabling you to refine your search, add or remove criterea or parameters, search within the search results, and to organize the result list into folders or categories ala Gmail labels. That is, you have to be able to slice and dice the search results in any way you want.

This would also apply to business or trade. Suppose you are looking for suppliers of children's winter clothing. The search and match engine takes your requirements (children's warm winter clothing, with parameters for size ranges, price ranges, materials and source countries, for example). It applies the requirements on a number of trade boards, like alibaba.com, itrade.com, and so on, finding the suppliers of these goods. Again you will be able to sift through the results to come up with a short list who you can contact yourself.

Improvements and refinements to the search and match engine will enable it to evolve and become more and more accurate in its results. Adding of ai and some common-sense programming will help to get better results. Ultimately, the goal is to find exactly what you want or need in the least possible time and the least possible effort on your part.

Search and match in this way will revolutionise human society. People will find what they need quickly and easily. Business and trade will be accelerated. Easy employment will enable people to find jobs which better match their skills and temperament, improving productivity. Easy personal matches will help people find better and more satisfying friendship and love, increasing human happiness. Ease in finding knowledge will help in a lot of endeavours, from disaster recovery to space exploration, accelarating progress in science, medicine, economics and almost all other areas.