Democracy is a free and equal people. To have true perception of the state of the nation, the electorate needs access to honest, unbiased information, or failing that; all points of view.

Is corporate control of Internet Indexes a threat to democracy?

In 1835, Alexis de Tocqueville published the first volume of his two part classic ‘Democracy in America’.

Tocqueville was a young French aristocrat and lawyer, living in an unsettled post-revolutionary France, and curious about democracy in America.

America was young and her experiment in modern democracy was unproven. Democracy was an exciting idea, and Tocqueville was curious about how and why it was working out, and where it might lead.

Read the rest of this entry »

An article in the current Scientific American discusses the near future of search:

Science & Technology at Scientific American.com: Seeking Better Web Searches

“Now that “Googling” has become synonymous with doing research, online search engines are poised for a series of upgrades that promise to further enhance how we find what we need.”

If you are interested in learning how to make your own spiders, robots and scrapers - and not just for populating search engines - then be sure to check out Spidering Hacks (2003) by Kevin Hemenway and Tara Calishain.

http://www.oreilly.com/catalog/spiderhks/

Another perspective on the evolution of Google:

A very well done 8 minute Flash item purporting to be a 2014 presentation from the Museum of Media History on how Google and Amazon came together to rule the news world.

EPIC 2014

It was created by Robin Sloan and Matt Thompson.

I wish they’d taken it a little further - it’s 2014 - you’ve got Amazon with its databases of consumer activity, and Google databases of social networks, massive databases of web pages and other resources, and so on - you know what people buy, where they live, what they do, what they talk about, and who they know. You have a database of the entire indexed web at your finger-tips, and you are among the first to know when information becomes available, and one of the few who can adequately mine it.

The mind boggles, it practically boogles at what one could do with this information. Surely it would be more than supplanting traditional news organisations with automated and personalised news reporting. Something more insidious; like tracking people from birth to death, connecting the dots through social networks, or continually eavesdropping on what people are talking about or doing on the Internet. Or simply having the greatest consumer behavior database tied to the store that wants to sell everything.

I mean, with resources like that, how can you not do evil?

A 2001 essay by Cory Doctorow on problems with meta-data. We’re lazy, stupid and can’t be trusted. Seems obvious, really.

Metacrap

But you know - it’s true.

So many ideas presume that users will behave themselves, when the rule is that they won’t. And while users may be stupid, dishonest, and lazy, let’s not forget that users will also be smart and deliberately destructive. So it makes sense to design systems on the premise that people will abuse, misuse, and confuse them. If a design depends on the users (or operators) behaviour, then it’s not robust enough.