Live at the Witch Trials

The Mickey Mouse Protection Act

2012-01-25T14:38:00.000-08:00

The Copyright Term Extension Act (CTEA) of 1998 extended copyright terms in the United States by 20 years ,..., the Walt Disney Company (whose extensive lobbying efforts inspired the nickname "The Mickey Mouse Protection Act")..supported the act

Disney are very pro copyright. Here is Clay Shirky talking about how you can't let kids draw a picture of Mickey Mouse and put it on a cake without getting sued by Disney for copyright violation

They seem oddly pro remixing, homage and parody when it come to other peoples creative output. I am not talking here about the companies cartoons based on traditional folk tales but about their Joy Division T-Shirt.
Disney's description of the shirt:

Inspired by the iconic sleeve of Joy Division's Unknown Pleasures album, this Waves Mickey Mouse Tee incorporates Mickey's image within the graphic of the pulse of a star. That's appropriate given few stars have made bigger waves than Mickey!

This is Disney's photo.

Hook confirmed that he had not given permission for Disney to use the image, adding that it was a legal grey area. "From a legal point of view, the image is in the public domain, as Disney know and, in a funny way, it's quite a compliment for a huge conglomerate like Disney to pick up on a poor little Manchester band that only existed for a couple of years, it's quite startling," he commented. "I'm amazed they're that hard up that they need to prey on little indie bands,
...
The bassist added that though he spends a chunk of his time "policing" Joy Division bootlegs, all he usually required was that wannabe bootleggers made a contribution to an Epilepsy charity in memory of Ian Curtis and called on Disney to do the same.

Joy Division do not get asked for permission nor do they or their charity make any money from this T-Shirt but Disney will sue a shop for printing children's pictures on a cake? If Disney like to control their creative work to such a huge extent why don't they obey the same rules for others work?

'I hope that [Disney] will be as understanding when we start doing Donald Duck shirts' said Hook

From Wesley Morgan Paraham here
*edit "A representative from Disney says, "As soon as we became aware there could be an issue, we pulled it from our shelves and our online store to review the situation further.""

Teach Yourself Synaesthesia

2012-01-13T14:21:00.000-08:00

Can you teach yourself synaesthesia?

A form of synaesthesia in which people experience letters or numbers in colour may be trainable. The discovery could shed new light on how such traits develop.
Synaesthesia is thought to have a genetic component, but some people have reported synaesthetic experiences following hypnosis, so Olympia Colizoli at the University of Amsterdam in the Netherlands, and colleagues, wondered if it might also be possible to acquire synaesthesia through training.

To test the idea, they gave seven volunteers a novel to read in which certain letters were always written in red, green, blue or orange (see picture). Before and after reading the book, the volunteers took a "synaesthetic crowding" test, in which they identified the middle letter of a grid of black letters which were quickly flashed onto a screen. Synaesthetes perform better on the test when a letter they experience in colour is the target letter.

The volunteers performed significantly better on this test after training compared with people who read the novel in black and white.

The findings suggest that natural synaesthesia may develop as a result of childhood experiences as well as genetics, says Colizoli, who presented the findings at the Forum of European Neuroscience in Amsterdam last week.

I liked the idea of training myself to have Grapheme → color synesthesia. It is is a form of synesthesia in which an individual's perception of numbers and letters is associated with the experience of colors. Nabokov had this, he wrote in Speak Memory "Since a subtle interaction exists between sound and shape, I see q as browner than k, while s is not the light blue of c, but a curious mixture of azure and mother-of-pearl". Training yourself to be a poundland Nabokov sounded cool so I wrote a script to do that.

A quick look at stack overflow lead to this question "Changing color of every “r” in html document and I copied Boldewyn's answer. There is probably a better way to do it. If I find it I will post an update.

Warning the ColourChange script which I put here is a greasemonkey script that will make the web look like a geocities Christian site circa 1996


// ==UserScript==
// @name           ColourChange
// @namespace      http://localhost
// @description    change the colour of characters
// @include        *
// ==/UserScript==

  var body = document.getElementsByTagName("body")[0];
var html = body.innerHTML
               .replace(/(^|>[^<a]*)([a])/g, "$1<span style=\"color:red\">a</span>");

body.innerHTML = html;
   var html2 = body.innerHTML.replace(/(^|>[^<o]*)([o])/g, "$1<span style=\"color:violet\">o</span>");
   body.innerHTML = html2;

If people want me to and I remove any bugs I will stick the script up on userscripts to make it easier to install. If you survive the web via the medium of nyan cat let me know.

Many musicians are Sound → color synesthestic Leonard Bernstein, Franz Liszt, Rimsky-Korsakov, Pharrell Williams and Stevie Wonder for example. Mnemonists with extraordinary powers seem to also associate sound with colours. like Solomon Shereshevskii in Luria's A Little Book About a Big (Vast) Memory. One of my favorite bands of last year was Colourmusic though so because of their name, their epic beards and mental videos here is one of their songs

I have set the synaesthesia script to colour a and o as these are letters I mix up. I also mix up p and b but I'll write a script for that if the vowel one helps. I was tested as a kid for dyslexia but they decided I was just stupid.

You could change it to match Dee Adams or Cassidy Curtis or whatever colours match up to letter for you. If you install the script using greasemonkey or you want to and need a hand doing that let me know.

The Irish Religious Copying that Saved Civilisation

2012-01-11T13:20:00.000-08:00

Sweden Formally Recognizes File-Sharing as a Religion

The Church of Kopimism, whose principal tenent is the right to file-share, has been formally recognized as a religious organization in Sweden...
“For the Church of Kopimism, information is holy and copying is a sacrament,” it said in a statement.

This is a pretty good troll but more weirdly there is a history of religious copying in Ireland. One that we may have to thank for western civilisation.

How the Irish Saved Civilisation (on amazon for $8) ultimately argues that it was the Irish love of copying books that saved much of classical literature and combined with their spreading these copies as they set up new monasteries allowed an ember of literacy to be saved until the renaissance reignited it. During the dark ages after the Roman empire fell Irish monasteries with their book copying scriptoriums were one of the few bastions of literacy left in Europe.

This book is full of good anecdotes. For example that for her near contemporaries St Brigid's most famous miracle was performing an abortion.
"Brigid makes the fetus of a nun (whose womb had "through youthful desire of pleasure, .. swelled with child") magically disappear ("without the coming of birth, and without pain")"

I think the book places too much emphasis on St. Patrick and how he converted the Irish to Christianity. I always got the impression St Patrick was sort of a Arthur Guinness character. Used after his death by a giant corporation to try and sell their product to the locals. They both even have a celebration day. The Beer Nut has a great post "Interlude" that goes into the political machinations that went into the creation of the St. Patrick story.

It was a turf war: plain and simple. The independents had all been swallowed by the Big Three who were now each using the political influence they'd garnered along the way to try and crush the other two. The prize was total national dominance of the market, and a hefty slice of the action abroad where the product had been zealously pitched to a receptive customer base, building up a lucrative following among locals and ex-pats alike.

The thrust of the books argument about how the copying Irish rescued the western cannon comes from Columcille. He had a major brain erection for copying any books he came across.

On one occasion when he was at the monastery of Moville he came across Finian's book of the Psalms. Colum Cille decided to copy it secretly. He did, and when he was brought before King Diarmait who was to decide who was the rightful owner of it, Diarmait made his famous decision: “To every cow its calf and to every book its copy”. This might be regarded as the first copyright case in history! Later fighting broke out about the decision. After a battle for it Colum Cille got it back and it has since been known as the 'Cathach' or Battle Book. Colum Cille went into exile as penance and landed in the island of Iona in 563 and there he established his famous monastery

From here "he made one hundred fifty monks the cutoff number for the Iona community, and after they exceeded that, twelve and one monks would set off to establish another foundation in a new setting". These new monasteries would take copies of books with them and produce new analysis and in doing so spread literacy back into Europe. For example "More than half of all our biblical commentaries between 650 ad 850 were written by Irishmen".

I think its fascinating that Colum Cille's 150 monastery size number from the sixth century is the same as Dunbar's number. "Dunbar's number is suggested to be a theoretical cognitive limit to the number of people with whom one can maintain stable social relationships. ... No precise value has been proposed for Dunbar's number. It has been proposed to lie between 100 and 230, with a commonly used value of 150". Colum Cille was obeying modern social network theory in his monastery rules.

Ultimately I think the dark ages are a rorschach test, we know so little about them that the flimsy evidence can be used to argue any point you want to make. For example 'How the Irish saved civilization? By blogging it.' is a blogpost that argues the point that what the book proves is that blogging and such intellectual mashups are what preserved the culture.

While on the subject of dark ages Irish Monasteries "The secret of Kells" is a great film on this era

How the Irish Saved Civilisation is an interesting quick read. I was not convinced that literacy and the classics would not have survived the dark ages without Ireland. But I do think the lesson that we should copy and distribute our really important information around if we want it to be preserved when the barbarians break through the gates is an important one. It is easy to laugh at the Swedish Kopimism party but remember that an Irish monk who was willing to ignore the courts, go to war, murder and got banished from his country to copy things may have saved civilisation.

Peak Farmer

2012-01-09T03:53:00.000-08:00

In what year will the most farmers ever be working? We reached peak baby in 1990 and peak manufacturing employee slightly later.

Mass employment in manufacturing just isn't coming back
'I estimate global manufacturing employment to have been between 150 million and 200 million workers in 2002, with those numbers reflecting a global decline of 20-30 million manufacturing employees in 2002 compared to 1995.'

You can never tell exactly what the future will hold but with the increasing use of robots points to reduced manufacturing jobs for example "Foxconn to rely more on robots; could use 1 million in 3 years". Agricultural employment as a percentage of total employment has been declining since the industrial revolution. The percentage of Americans employed in agriculture has dropped from over 90% at the time of the American revolution to around 2% now for example.

The UN says peak rural population will be sometime between 2020-2025. "However between 2020 and 2025, the total rural population will peak and then start to decline," and "Global rural populations will peak in the 2020s, leading to mass abandonment of rural lands.(Data Source: UN Dept. of Economic and Social Affairs, Population Division)". Not all rural people are farmers and I think the proportion of farmers amongst rural dwellers is decreasing. If the figures of peak rural population being sometime 2020-2025 then that means the most farmers that will ever be living will occur sometime before 2020.

The first time in human history since the agricultural revolution that most people were not employed as farmers would have been some time before urban population passed rural population. The World Population Becomes More Urban Than Rural in around 2007 so we are not long past the majority of people being farmers. 'Agriculture still accounts for about 45 per cent of the world’s labour force, or about 1.3 billion people' according to this 2007 report. Until recently most people were farmers.

The world continues to urbainse, farms continue to mechanise and population continues to rise slower than it used to. All this means that sometime before 2020 farming will become globally a declining employer. If total farming and manufacturing jobs are declining and transport jobs could do the same soon that means we will have to find new things for people to do.

This prediction of declining numbers of farmers globally sometime around 2020 is one that someone with better google foo could disprove quickly, if you can please correct me in the comments.

Most Popular Posts of the Year

2011-12-28T07:32:00.000-08:00

Thanks to everyone who read and commented this year. The most popular posts this year were

Search Engine Deoptimization 16,000 Pageviews. Many people seem to dislike Ryanair and weird legal rules.

Eurovision Voting Fraud 5,500 Pageviews. A rather glib post from 2010 about human rights abuse that got popular when Azerbaijan won the Eurovision.

My favorite posts were

The Dead Zoo Dodo Get Anto the Dodo back to the public. Anto has set up a twitter account

The Kilbrittain Whale. This tourist attraction is brilliantly unhinged.

My favorite books read in 2011

2011-12-21T15:19:00.000-08:00

Books really are kind of magic aren't they? Here is the ones I read this year that changed the way I think and turned me into a different person at the end. All for ten euro and a few hours of my time.

My favorite book was Moonwalking with Einstein. Without memory what would we be? This is a great diverse book on a really interesting topic.

Don't sleep, there are snakes. There is so much in this book from Language to God to child rearing to self sufficiency. It is just fascinating to read about people who see the world completely different to us.

The rational optimist nearly everything is getting better. This book left profoundly optimistic about the future. This is the book I have tried most to persuade my friends to read most this year.

Selfish reasons to have more kids. This book goes through the evidence that once your not an awful parent (one who would be refused an adoption for example) you really don't make much difference to your kids. It is actually a profoundly optimistic argument as this means you don't need to spend your time doing things you dislike to shape your childs path as by the time they are 18 they will end up where they would have anyway. It uses evidence from identical twin adoption studies to show how little parents need to worry about religion, education and all the other things they break their hearts worrying about.

The Great Stagnation By Tyler Cowen. We should have dinner in a pill, Optimus Prime, hover boards and lasers that turn dolls into women by now. Why don't we and what can we do about it?

Next year I plan to read mainly fiction. This year In cold blood by Capote was the one piece of fiction I read that I am sure will stay with me. Everyone says it is great and everyone ain't wrong.

Close but not in my very favorite books were fever by Shah, Adapt by Hardford, What technology wants by Kevin Kelly, Red Plenty by Francis Spufford, Gutenberg, race against the machine, good omens, the unbearable lightness of being, How the irish save civilisation and Launching The Innovation Renaissance by Alex Tabarrok, popular crime by Bill James

How much would a driverless taxi cost?

2011-12-18T12:53:00.000-08:00

This is a post where I do a back of the envelope estimate of when we'll see driverless cars, what will they do to taxi costs and what will that do to unemployment.

There are several estimates to when driverless cars will arrive. The New York time estimates 2020.

Self-Driving Cars
By 2030, Sebastian Thrun predicts, more people will use self-driving cars in their daily commute than manually driven cars.
Submitted by Sebastian Thrun, developer of Google’s self-driving car.
Our readers predict this will occur around 2020, having moved this date 1381 times.

Many similar bets of Driverless cars being regular enough in 2020 and ubiquitous in 2030 exist for example here,

By 2020 - Driverless cars are commercially-available and street-legal somewhere in the United States.
By 2027 - New driverless cars outnumber new cars requiring at least some human control, in the US market.

and here

By 2019, it has begun spreading to public roads, with significant numbers of driverless trucks appearing.

and here

In my post last week, my commentors took me to task on my prediction that cars will drive us in ten years. Some thought Americans would wise up and learn to love mass transit. They don't know Americans.

Others thought the hardware cost would even in ten years remain out of reach. Google did not build an autonomous car by creating the hardware but by harnessing and training good machine learning algorithms. No amount of hardware would have given you a car able to navigate the streets of San Francisco five years ago.

What effect will these cars have? There are all sorts of ideas about how they will alter parking and car ownership. I'm going to try do a back of the envelope here on how much Taxi fares will cost if you don't have to pay the driver.

Taxis cost about 120 cent for a kilometer

For each additional 1/6th of a kilometre or time 28 seconds)

(a) Day time 8am to 10pm €0.15
(b) Night time 10pm to 8am €0.20
(c) Sundays, Public Holidays, Christmas Eve and New Year’s Eve €0.20

Ignoring the pick up costs of about 3.40.

The AA says it costs around 25 cent per kilometer to drive a car in Ireland. About 25 pence per mile in the UK. Taxis charge 1.20 so the majority of the cost looks like the driver. You would have extra costs on top of a normal car with a commercial vehicle. But given the pick up costs a driverless taxi could be about a quater the cost of a taxi with a driver.
The price elasticity of demand should allow an estimate of how this will alter taxi usage. This paper "Estimation of Price Elasticity for Taxi Services in Hassel" gives a PED of -2.644. Though other such as Schaller at -.22 and here of -.6 shorter term. Taxis that cost a quater the current price with a PED of -2.6 would mean about ten times the number of taxi journeys. The long term viability of public transport should take this possibility into account. If by 2030 people will be taking ten times the number of taxi journeys would enough people be using Metro North to make it cost effective?

Transport employs nearly one hundred thousand people in Ireland. Which is about 1 in 20 people who have a job here. Or about a third of the number of unemployed. I doubt everyone who works in transport will lose their jobs overnight. But taxis provide an example of how economic effects could provide a huge incentive to move to driverless cars. So far technological progress has always eventually resulted in new jobs to replace old lost ones. The money people save getting into town for a night now could end up being spent in town and require more employment in restaurants and bars for example.

But I think it is worth considering the possibility that fairly soon we could have nearly a hundred thousand people who earn a decent wage at the moment becoming unemployed in a short period of time. Construction lost 160 thousand people in three years. Transport jobs do not pay as well as construction did. But if the construction change caused most of our current economic issues it would be unwise to ignore a large sudden future change in transport employment.

To put some skin in the game, I am predicting that in 2025 in a period of three years we will see structural unemployment of about 5% of the workforce, half of those that work in transport.

Eradication Game Theory

2011-12-13T06:08:00.000-08:00

Can you crowdsource disease eradictaion?

I mentioned in this post on 2030 that I expect Polio and Guinea worm to be eradicated by then. It becomes a tricky issue when a disease gets really uncommon how you find and treat the last few cases? So far only smallpox and rinderpest have been eradicated. Eradication is great because once its done its done. You would not have to immunise every child for polio any more. Every polio vaccine has a small cost and risk and once thats gone you can go spend the money on something better

India has tried an interesting tack 'Cash awards for info on diseases'

Alert the government on the occurrence of new cases of certain ailments and you may get a cash award! The government has targeted vaccine preventable diseases such as diphtheria, pertussis, measles, tetanus, and leprosy for elimination by 2016. India is also on the verge of being declared polio-free.
District medical and health officer Dr G. Srinivasulu explains that even after being declared polio-free, there should not be a single new case for 14 consecutive months in the country. This is where the reward comes in. “If anybody succeeds in detecting a new polio case meanwhile, the government will give a cash award. Even in case of detection of new leprosy cases, ASHA health workers are given Rs 150-Rs 200,” he said.

There was a competition last year from DARPA to encourage new ways to pool information held my many people.
'MIT wins $40,000 prize in nationwide balloon-hunt contest'

A team from the Massachusetts Institute of Technology won $40,000 in a high-tech scavenger hunt on Saturday by discovering the location of 10 red weather balloons.

"We're giving $2,000 per balloon to the first person to send us the correct coordinates, but that's not all -- we're also giving $1,000 to the person who invited them. Then we're giving $500 whoever invited the inviter, and $250 to whoever invited them, and so on..." it said.

Some similar system that set up a chain of reward could be really useful in disease eradication. Many security protocols rely on a sort of iterative proof of trustworthiness. Something similar could be used to allow steps toward eradication without fear some other country is going to stop efforts.

Assurance contracts are another approach. It is possible that some regions are scared that once a disease is eradicated to their area they will lose funds. Some way to guarantee that funding will not reduce or to reward successful eradication might help here. Something like Dominant assurance contracts might help to change incentives to encourage eradication.

How about a guarantee fund for each of the remaining countries with polio and guinea worm that when who declares them free they get some cash bonus. You could imagine a kickstarter project that gave the minister of health in Nigeria money when polio was declared eradicated from the country.

Another good plan is not to have fake immunisation drives against polio

The juice of the carrot, the smile of the parrot
A little drop of claret - anything that rocks
Elvis and Scotty, days when I ain't spotty,
Sitting on the potty - curing smallpox

We want to discourage deliberately being lax about a disease so you have to be clever about incentives. Anyone have any good idea for how you could bribe people and governments to further incentivise disease eradication?

A Rose Named 20175 Does Not Smell as Sweet

2011-12-09T01:13:00.000-08:00

Would we get more or less innovation without patents? The rose that holds patent no 20175 tells us how little we need patents. The plants have been patentable in America since 1930. 16% of Rose varieties are now patented but this increase in protection to rose growers has not increased the variety of rose varieties.

Launching The Innovation Renaissance by Alex Tabarrok is a short €2 book I read on the bus into work. It details as well as the perils of rose patents other hindrences that block wider innovation

The argument against patents is that they give too much protection to innovators. That in covering too much that is too easily thought of they actually hinder people coming up with new products. I have not seen a coherent argument that (non pharmaceutical) patents have too little protection. Tabarrok does a moderate anti patent argument forward in a clear convincing way though.

Next Tabarrok argues we should have more prizes for innovation. I have a major brain erection for kaggle and science innovation prizes in general. Tabarrok makes a clear case that prizes should be used when new ideas are needed.

Many of the faults in education are also well laid out. "the value of a permanent 25-point increase in scores ...would be 80 trillion" ..."A 25 point increase would bring the united states from about the level of mathematics education in Ireland and Spain to the level in Germany and Australia" How are we in Ireland worse at maths than Australia? Australia is mainly populated with Irish people, and the ones with an unhealthily active interest in sports at that. We have no excuse for the Aussies beating us in Maths. The education section gives several good pieces of advise for Ireland. Pay teachers based on their subject. Pay them on results not just years served. Sponsor college courses that result in more innovation that increases the standard of living more than some of the liberal arts that don't. While on the subject there is a very good blog on Irish education from the teachers side at anseo.

Immigration of high skilled immigrants is discussed next. Again like patents a reasonable achievable change is suggested rather than fully free immigration. I think Ireland should go moneyball on this. Find undervalued players in the market and buy them for our team (this would mean letting persecuted people live here, but it is an argument for another time).

The main thrust of the book is that innovation matters. Instead of a welfare state dividing a pie it might be worth thinking of an innovation state that constantly tried to make things better and increase the size of the pie. This would involve fighting entrenched interests "Few people lobby for innovation because almost by definition, innovation creates present losers and future winners and the present winners are by far the more politically powerful".

The optimistic view that we have in our power to create new and amazing things infuses this book. It is about fixes rather than showing how bad the innovation problem is as Cowen's The Great Stagnation does. None of the suggestions are on the extreme side and all are clearly explained. Because of this I think this book is likely to provide a blueprint for many improvements in the near future.

Should Science Bloggers Have a Session?

2011-12-08T03:26:00.000-08:00

I think it would be really cool if loads of science bloggers talked about the same thing on the same day. Beer bloggers have a monthly "session"

"The Session, a.k.a. Beer Blogging Friday, is an opportunity once a month for beer bloggers from around the world to get together and write from their own unique perspective on a single topic". The beer blogging community really get around the session and seem to delight in sharing stories around one topic. Something similar for science bloggers would be cool, it might already exist but I cannot find it.

If some neglected interesting topic was blogged about once a month that could be really fun. To see the many angles bloggers can take on a subject.

Here is a suggestion World Leprosy day is January 30th.

How about we all write a blog about leprosy for that day?

Off the top of my head here are some possible angles

1. Epidemiology
How is leprosy spread
Where is it common
Can it be eradicated? An irish leprosy charity The Leprosy Mission is here

2. Microbiology
Why can't leprosy be diagnosed until you have symptoms
Why is the Leprosy bacteria so hard to culture
Why do some people take decades to develop symptoms
Why are most people genetically immune from the disease
The reviled drug Thalidamide has a role in modern leprosy treatment why

3. History
Why is leprosy one of the oldest known infectious diseases. How can you tell a mummy has leprosy
How has treatment changed through the ages
How has leprosy effected history? From the king in Braveheart to the Gospels leprosy has been important in history.
Leopardstown I have heard was a 'leper colony'. Was it and what did it look like?

4. Sociology
Why is there such a taboo against leprosy? What can we do to ease this?

5. Medicine
What are the symptoms of Leprosy and why do they occur?

6. Zoology
Armadillos can spread leprosy to humans. Other animals have also been implicated. What does it mean for the animal to carry this disease? How can infection from these animals be minimised?

Thats just a few ideas. None of which I know enough about to be able to write up. But you or someone you know might. It is a neglected disease. I have heard it small pox described as the disease that is gone but not forgotten and leprosy as the disease that is not gone but is forgotten. How about a session for science bloggers on leprosy on January 30th? If you have any ideas or comments or will commit to writing a post please comment below.

Lithium in the water supply

2011-12-03T10:27:00.002-08:00

There is an interesting proposal in the Irish Times

Psychiatrist calls for lithium to be added to water

A consultant psychiatrist last night called on Government to add lithium salts to the public water supply in a bid to lower the suicide rate and depression among the general population.

At a mental health forum on “Depression in Rural Ireland” in Ennistymon, Co Clare, Dr Moosajee Bhamjee said that “there is growing scientific evidence that adding trace amounts of the drug lithium to a water supply can lower rates of suicide and depression”.

I would like to see what this scientific evidence is. The studies I have seen are from Japan where the lithium is naturally occuring. And one from Texas that seems a bit dodgy

Drinking water which contains the element lithium may reduce the risk of suicide, a Japanese study suggests.
Researchers examined levels of lithium in drinking water and suicide rates in the prefecture of Oita, which has a population of more than one million. The suicide rate was significantly lower in those areas with the highest levels of the element, they wrote in the British Journal of Psychiatry.

So taking about 1/1000 of what someone who need treatment takes could reduce suicide? There is an interesting correlation/causation problem with this study. Is it that the lithium in the water is reducing the suicide rate or could the same thing that reduces suicide increases the lithium amounts? One explanation I have heard is that places where it does not rain much water hangs around longer and rubs off rocks picking up lithium. In places where it rains a lot the lithium levels might be naturally lower as the water does not get a chance to pick up much lithium. The kicker here is that places where it rains all the time might be depressing and that could explain the increased suicide rate.

A quick check of this theory throws up some problems. Oita is in the south of Japan. Oita has 94 days of rain per year . It has Average rainfall of 1677mm

Tokyo where people top themselves at the rate of depressed lemmings at a Leonard Cohen concert also has 94 days of rain per year. 'The city of Hiroshima, in western Honshu, averages a sizeable 1,603 mm (63.1 in) of rain each year. Tokyo, further east near the Pacific, receives an annual average rainfall of 1,460 mm (57.5 in). The city of Sapporo, on Hokkaido, averages 1,158 mm (45.6 in) of precipitation per year. The southern end of the Kii Peninsula is known for a heavy annual rainfall exceeding 4,000 mm (157.5 in).'

A quick back of the envelope is not enough to discout the "rain means less lithium but also more depression" explanation. But I still don't think it is time to put lithium in the drinking water.

Takeshi Terao, a coauthor of the paper and a professor at Oita University.
"I do not think [cities should start adding lithium to the water supply], because our study is a preliminary one and further studies are required to establish evidence."
"Lithium does have its negative side effects as well. Some are mild: people often feel thirsty when taking lithium. Other side effects can be more severe, like weight-gain and diabetes and kidney problems. Lithium in the water supply could increase these side effects as well, although Terao's study didn't examine this possibility. Since the dosages are so much smaller, presumably the side effects would be as well, although more research is needed to prove that."

While on the topic of Lithium 7up used to contain lithium

The product, originally named "Bib-Label Lithiated Lemon-Lime Soda", was launched two weeks before the Wall Street Crash of 1929.[49] It contained the mood stabiliser lithium citrate and was one of a number of patent medicine products popular in the late-19th and early-20th centuries.[50] Its name was soon changed to 7 Up; all American beverage makers were forced to remove lithium in 1948.

Syrian Parade

2011-11-30T12:45:00.000-08:00

I've always hated fireworks. They are a boring waste of gunpowder and sky. But I love parades. Particularly crap ones with donkeys and trucks that break down and a collective shambolic happiness. The best parade I was ever at was in the Syrian town of Hama. In Hama stayed in a hostel with a parrot for a receptionist who could say "Welcome. Please wait in the tv room" in one of 6 languages depending on where he thought you were from.

Hama has a weird atmosphere. There is the contrast of really old norias on the river with the rest of the town. These ancient water wheels are in stark contrast to the rest of the city that is just 1980s concrete block apartments. Out in the countryside everything gets old again. They even have beehive huts like in Ireland

Its only when I got back did I read about the Hama Massacre

The Hama massacre (Arabic: مجزرة حماة‎) occurred in February 1982, when the Syrian army, under the orders of the President of Syria Hafez al-Assad, conducted a scorched earth policy against the town of Hama in order to quell a revolt by the Sunni Muslim community against the regime of al-Assad...
Initial diplomatic reports from western countries stated that 1,000 were killed. Subsequent estimates vary, with the lower estimates claiming that at least 10,000 Syrian citizens were killed,[4] the majority civilians, while others put the number at 20,000 (Robert Fisk), or 40,000 (Syrian Human Rights Committee).

It has the same weird cult of personality photos of Assad everywhere that the rest of Syria has. And the same bird like men that are the security agents everywhere in the world

The parade was in the town when I was there. It was something like a Saint Patricks Day Parade 30 years ago and with better weather.

What has always stuck in my mind for some reason was one group of kids form a karate club who happily marched through the town happily waving coloured flags.

I see the the news footage now from Syria of the protests and the Assad regime violent response. I worry about those kids I saw years ago. Those kids would be the perfect age now to be angry protesting. I'm scared to think what has happened to them. They are having a march in Syria right now and its not nearly as happy as the one I saw.

Predictions for 2030

2011-11-29T07:22:00.001-08:00

I have no idea what will happen to the euro in the next month. And whatever happens will have big consequences. Yet I was willing to make predictions for 2030 about lbr, driverless cars, solar power and education in my last post.

The thing is I think I am cheating with technology predictions. Theres a wildly unpopular branch of Marxism that argues for technological determinism. Marx can be interpreted as saying that our technology is inevitable and will change us in unavoidable ways. "The windmill gives you society with the feudal lord: the steam-mill, society with the industrial capitalist"

After reading "What technology wants" by Kevin Kelly I am a convinced technological determinist. There is a podcast from Kelly on technology at econtalk

The book makes a compelling case that

1. You cannot shut yourself off form technological change. The Japanese tried it and failed. The Amish dont try it, they stay about 50 years behind on average but they do not avoid new technology forever.

2. No technology ever dies out.

3. Each new technology is an inevitable consequence of the last. No one person of country can cause or prevent a new technology, though they can shape its exact form.

Almost all patents of significant technologies have multiple very similar independant patents lodged at nearly the same time. Airplanes, radio, transisters, computers, television whatever you can think of about three guys thought of it and implemented it independantly within a few months of each other. This suggests that once the right prior technologies exist the next one is inevitable.

Kevin Kelly: What Technology Wants

Keynes wrote about 2030 in 1930 in the article “The Economic Possibilities for Our Grandchildren.” which has been surprisingly on accurate for the last eight decades. So I am betting it will continue to be for the next two. He made brought claims about future growth rates that have been accurate. I will make similar claims now. Given that I dont think non technological predictions can be made I am going to try anyway
1. Grinding poverty will be gone. This is a low bar of 365 dollars a year in 1990 dollars. Thats really bad. Much worse than medieval England but its one I think we can reach.

2. Polio and guinea worm will be eradicated.

3. World population will be slightly lower than the medium UN estimate of 8321380. The high is 8776486 and the low is 7867332. So I will guess 8250000.

This is just a few predictions based on the world continuing to go the way it has for the last two hundred years. But like I said I cannot predict what will happen to the cash in my pocket over the next two months so two decades away is being ambitious. If you have any predictions for 2030 please put them in the comments. It might be fun for the floating brains in a jar to laugh at them at the time.

Technology of the Year

2011-11-28T07:39:00.003-08:00

Loads of brilliant technologies went mainstream this year. Some new technologies were created but I don't know enough about any of them to make claims for their future. I'm going to describe what I think these old but newly well known technologies mean in reference to the world when my new daughter is 18 in 2030.

Learning by reading
Watson came along from IBM and answered quiz questions really well. Siri did something similar with peoples requests.

This might not seem like a big deal. Not many of us make a living answering quiz questions. But how many of us have jobs that involve finding the right page based on some search and parsing out the right bit of text? A surprising amount of medicine, lawyering, general office power point monkeying involves this. What happens when a cockroach knows everything? Watson is the answer to that as that is about the actual intelligence level it has. What happens when in 2030 the cockroach is a thousand times smarter? I have no idea.

Depressingly most of these systems seem to be owned by big companies like apple, Google and IBM. Most of the data used and the tools that analyse it are open source. It will be a really cool project when someone makes an opensource wikipedia based answering bot.

"if, in like manner, the shuttle would weave and the plectrum touch the lyre without a hand to guide them, chief workmen would not want servants, nor masters slaves." Aristotle wrote in Politics which says that smarter machines will free us from servitude. So far this has been the case but will that continue?.

Driverless cars

Will my daughter ever drive for non fun reasons? People will always drive the same way people still ride horses. Just now they do it for fun rather than transport. Two trends are making it less likely that my daughter will ever have a drivers licence. The increasing sophistication of driverless cars and our decreasing acceptance of the idiocy of youth.

If Ferris Bueller had a day off now, would he spend it on Facebook? Because cars are relatively more expensive and relatively less exciting teenagers are less inclined to drive now. Facebook and youtube provide entertainment for free so you are less inclined to work thousands of hours so you can afford a mildly entertaining car. Cars are now only mildly entertaining as safety nazis have turned them all into a homogenous look that cannot even be modded easily. If you cannot put a spoiler and underlights on your Clio your just not going to look cool as a 17 year old.

Combined with cars being less attractive to teenagers there is increasing control placed over them. New drink drive limits for learners have been imposed here. I have heard talk of curfews for young drivers. We are less inclined to accept the god given right to load a car with 7 of their mates and wrap it around a tree at 4 in the morning. In 2030 well have so many constraints on teenagers to prevent them killing themselves as to remove much of the attraction from driving.

The technology element of this is the driverless car

Hanson wrote

So a huge upcoming policy question is: when will what big cities manage to coordinate to change road law to achieve these huge auto-auto economic gains? Thirty years from now we may look back and lament that big city politics was so broken that no big cities could manage it. Or perhaps history will celebrate how the first big city to do it dramatically increased its importance on the world scene

Cowen said

The typical American spends an average of roughly 100 hours a year in traffic; imagine using that time in better ways — by working or just having fun. The irksome burden of commuting might be lessened considerably. Furthermore, computer-driven cars could allow for tighter packing of vehicles on the road, which would speed traffic times and allow a given road or city to handle more cars

These technologies will come in gradually drive train technology, lane assist, parking assist, crash avoidance are all present in next years s-class Mercedes. Probably legal hangups will delay driverless cars. If I was to guess it will be the old that get them pushed through. The old are increasing in numbers and will continue to vote. The baby boomers won't accept the loss of independence that goes with not being able to drive currently. Legal changes to allow driverless cars could be a vote winner. I'm willing to bet that by 2030 a combination of our scardy-cat nature about risk, the grey vote and improved technology will mean that my daughter may never have to drive.

Solar Power

The exponential improvement in solar power became news this year.

Averaged over 30 years, the trend is for an annual 7 percent reduction in the dollars per watt of solar photovoltaic cells...10 years later, in 2030, solar electricity is likely to cost half what coal electricity does today

This article started a great big argument about why solar is not replacing fracking and such. Including why environmentalists are still worried about co2. What will happen in 2030? I dont think we'll be using enough less energy to prevent climate change that way. I think well still be using fossil fuels to fly planes. New electricity generation methods take decades to become part of the grid, but I am still optimistic about solar.

Education
Education has not changed much in a long time. This is partly due to Baumol's cost disease where a teacher now can only teach the same number of kids as one in 1900. Since that time manufacturing jobs can produce vastly more than they used to. But education is one of those human centric industries that has not changed much.

Since videos came out it has been possible to watch lectures at home. Just giving kids computers does not improve education. If you look at the list here of the best educational strategies Tutorial instruction, Reinforcement, Corrective feedback, Cues and explanation are now part of online educational programs.

Stanford this year ran three computer courses where anyone can sign up for free, get homework graded by computer and do exams at the end. This is a graduate level course in one of the worlds best universities for free. The Khan academy has been around for a while but this year it moved from just a list of videos to include problems. The site is now using machine learning and gamification techniques to improve these problems and thus student learning. They are also using statistical tests to improve the website layout and which version of a explanation gets used. These computer assisted techniques are even being used by humans to get better at Jeopardy as the video below describes.

Interestingly the same guys who got learning by reading and driverless cars to their current state are also important in these changes to education Sebastian Thrun, Andrew Ng and Peter Norvig.
This year technologies became mainstream that finally turn computers into the amazing educational tools they have always promised to be.

Comparison
How did these areas change from when I was born till I was 18?

Cars became safer and drink driving became mildly uncool. The main change here was removing lead from petrol though. The recent drop in crime and increase in IQ may be largely due to removal of leaded petrol. I think driverless cars are important but not as important as this.

Power generation changed a bit, Chernobyl made nuclear uncool but it was probably never economical anyway.

Education did not change much. I did example questions from the year I was born to practice for the leaving cert. Corporal punishment still existed in my time but mainly for nostalgia rather than because they got true joy battering children. I was born after the education system demilitarized. I predict education will change more in my daughters childhood than mine.

Computers did not really visibly exist for people when I was born. By the time I was 18 it was obvious they were going to change the world even if they had not fully done that yet. I think the change from no computers to some was bigger than useful computers to really useful ones that I predict will happen.

Here is the prediction in 2030 for the technologies that became public this year. Many of the cars will be driverless. The world will get a most of its new generational electrical capacity from solar. Education has already changed massively this year just people have not realised it yet. We will each have an assistant that knows everything. I don't know how that will effect employment, my guess is she will end up working in a job that does not exist at the moment.

Memorising Books

2011-11-16T10:25:00.000-08:00

"All of us have photographic memories, but spend a lifetime learning how to block off the things that are really in there." Granger in Fahrenheit 451. In Fahrenheit 451 there is a group of men who memorise books because they want to preserve them if all the physical copies are burned.

This was the natural state of books before the printing press. The Iliad, the Táin Bó Cúailnge and other epic poems were designed to be memorised and recited rather than read. But what books do people memorize now? I have included the text file size to give some idea of how long the book is.

The Bible 4.2 MB
I have never heard of anyone who memorized the bible but it does seem oddly popular. A Plan For Memorizing The New Testament In 5 Years. The New testament is 996 kB in size. I have never met anyone who has memorised the bible. How many people have?

Milton: Paradise Lost John Basinger did this in his 70's. 12 books, 10565 lines and 60,000 words in 8 years. The filesize is about 495 kB.

The Qu'ran 1.1 MB People who have memorised this have a special name of Hafidh. 'The total number of hafidh and hafidhas currently alive in the world has been estimated in the tens of millions.'

'But becoming a hafiz is also believed to bring rewards in the hereafter, guaranteeing the person entrance to heaven, along with 10 other people of his choosing, provided he does not forget the verses and continues to practice Islam.'

'The children, ages 7 to 14, are full-time students, in class 8 a.m. to 5 p.m., Monday through Friday, even in the summer. But they are not studying math, science or English. Instead, they are memorizing all 6,200 verses in the Koran, a task that usually takes two to three years.'

that is three years of full time work? Over 5000 hours or over half way to Gladwell's 10000 hours theory. Presumably once you spend 5000 hours memorising the Qur'an you would spend a considerable amount of time thinking and discussing it. Which means there are millions of near genius level experts in the Qur'an in the world.

I can think of things I would rather be an expert in but it is worth considering what it is about the Qu'ran that devotes this level of commitment to it.

Mao: The Little Red Book 273kb

One of the most read books ever. It still seems remarkably popular given it was written by the man who killed the most people ever.

'Acknowledging that she had never memorized Mao's quotations, Chen Di said the indoctrination of Mao, which was still inscribed on a few school walls today, was impressed in her mind as a primary school girl.' I cannot find a figure on how many people memorized all of this book but at 900 million copies 'sold' and many of them expected to memorise large passages it must be in the millions.

The Torah: Memorizing the Jewish holy book seems to have an important place in the religion but I cannot get any figures on how many people today have done this.

Can you think of any others? Dr Seuss' 'Cat in the Hat' has probably been memorised by many adults just from reading it to their kids a thousand times.

Books designed to be memorised, like the iliad, have tricks involving characters, meter, rhyme and rhythm. I have gotten two books recommended by Josh Foer, The Book of Memory and Memory in Oral Traditions that examine these methods. So hopefully I will blog about these tricks again.

The effort people must go to to memorize a book is incredible. We can now buy books for the cost of a few hours work. Or a kindle that we can get thousands of books on gutenberg for under sixty euro. Books are 0.2% of the average Americans spending. To think that we have such a low cost per hour worked on a physical or digital book now but people will still spend thousands of hours memorizing one is amazing.

When could we see ourselves?

2011-11-06T13:40:00.001-08:00

I've wondered for a while about how far away we could see the earth if we were using our current technology and we had our current light output. At what distance could we see the artificial light the earth produced?

This paper Detection Technique for Artiﬁcially-Illuminated Objects in the Outer Solar System and Beyond provides some sort of evidence on the question

Existing optical telescopes and surveys can detect artiﬁcially illuminated objects comparable in total brightness to a major terrestrial city at the outskirts of the Solar System
...
For this signature to be detectable, the night side needs to have an artiﬁcial brightness comparable to the natural illumination of the day side. Clearly, the corresponding extraterrestrial civilization would need to employ much brighter and more extensive artiﬁcial lighting than we do currently since the global contrast between the day and night sides is a factor ∼ 6 × 10^5 for the present-day Earth

Lets ignore that our telescopes will get better (about 2.5% a year?) so how quickly at current rates will the earth take to be this bright?

“over the last three centuries, and even now, the world spends about 0.72% of its GDP on light. This was the case in the UK in 1700 (UK 1700), is the case in the undeveloped world not on grid electricity in modern times, and is the case for the developed world in modern times using the most advanced lighting technologies.”

But what is that in terms of extra light produced? "In 1700 a typical Briton consumed 580 lumen-hours in the course of a year, from candles, wood and oil. Today, burning electric lights, he uses about 46 megalumen-hours—almost 100,000 times as much."

So that means if we grew at the same rate we have for the last 300 in about 300 years we will be producing enough light that if it was all shone out into space the earth would be as bright at night to an outside observer as it is in the day. I dont think you can make projections this far and it still leaves the question of how long after that ambient external light would be that bright. Still I think its interesting that even with current detection technology as civilisation the 300 years advanced from us would be nearly visible to us. I would guess this prediction will reduce but 300 years is my first estimate to when we could see ourselves.

The Population Surge is Stopping

2011-11-01T03:07:00.000-07:00

Population surge difficult to halt and almost impossible to reverse was published yesterday in the Irish times. It is also availible in blog form here. The article makes some interesting and arguable claims about human environmental damage to the planet. These are based however on claims about human population that do not match the evidence or the UN's demographic predictions.

Today, just like every day for the last 50 years, around half a million babies will be born.

This is not true the figures from the UN are here. In detailed indicators look in births and in select country look in world.
Between 1960-1965 302136 babies were born each day. Between 1985 and 1990 375909 babies and between 2005 and 2001 367320 babies. The 500000 figure is not just wrong but drastically wrong.

The geometric nature of population growth makes it extraordinarily difficult to arrest, and almost impossible to reverse. The last population doubling took only 40 years. Even if global population growth rate drops to just one per cent, today’s seven billion would swell to an unimaginable 14 billion in 70 years.

The growth rate is being arrested. As I have said before in "We have reached Peak Baby" the number of children each woman has has been falling for decades.

"1968 women got 1.87 adult daughters. Ehrlich called it population bomb. Now women get 1.07 daughters, 92% of way to population balance done." Says Hans Rosling. In 43 years we have gone from 1.87 adult daughters to 1.07 a decline of .8 and we only need a decline of .07 more to reach long term balance of the numbers of women. Rural agrarian Bangladesh has reached stasis in adult female population levels for example.Look at the gapminder video here for yourself the number of children per women is declining

Though surprisingly accurate population estimates get better over time. For example the 2050 estimate has been recently honed in

U.N. Raises “Low” Population Projection for 2050
The "low-variant" scenario of population growth now foresees 117 million more people on the planet in 2050 than it did two years ago.
While the "median-variant" scenario, often seen as "most likely," remains almost the same as before - predicting a world with 9.2 billion people by mid-century
...
The high projection, however, foresees some 10.5 billion people - a 295 million person decrease from the previous high projection. The medium projection is 9.2 billion people,

It says something that an decline in the high estimate an over 2.5 times bigger than the increase in the low estimate is not the headline.

There are still countries that have very high birth rates per woman. Senegal for example had 7.5 babies per woman in 1968 4.8 babies and in 2010. But these countries are developing at a speed that is likely to see these birthrates drop rapidly. "Senegal has lower borrowing costs than Ireland." and a GDP growth rate of 4.2%. As these countries that still have a high birthrate develop their birthrate will drop rapidly the way ours and other developed countries did.

The article is in the Irish Times gives the wrong figure for the number of births per day and "unimaginable 14 billion" scare figure. The demographic evidence and historical trends indicate the population will not go to this level. Current UN estimates are for the high population prediction do not match this figure for 2081. There were never that many babies born per day and the birthrate has been falling so fast that we will not reach the 14 billion in 2081 figure.

The Dead Zoo Dodo

2011-10-10T14:26:00.000-07:00

I have mentioned before my weird obsessions with taxidermy, weird tourist destinations and tall dwarfs but I have not mentioned here my first scientific obsession: Dodos

Adam Savage the Mythbuster talks here about the same obsession he has

Like Möbius strips, Dinosaurs and Steve Reich, Dodos are like flypaper for nerds. There is just something about these things that seem to spark off nine year olds and set them off on a lifetime being the makers and doers of nerdy things. It was the dodo in the natural history museum of Ireland, known locally as "the dead zoo" that set me off at this age.

Unfortunately this dodo is no longer viewable to the public. I was in the museum last week, admiring the cool steampunk Victoriana makes the museum as interesting for what it says about the 1800's when it was created as about the animals in it. For example the polar bear still has an obvious bullet mark in its head, you would not see that in a modern museum.

The museum was bigger but large areas are no longer open
Exhibition space does a dodo and vanishes

The museum shut on July 5, 2007, when a flight of the main staircase collapsed. The Department of Arts originally decided to take the opportunity to carry out a €15m renovation and extension, but that was cancelled last year due to the state of the public finances.

Instead the staircase was reinstated and other minor repair works carried out, but two large balconies on the second floor have been shut off from the public due to health and safety concerns. The museum’s biggest attraction, a skeleton of a dodo, is on one of these balconies and, unless it is moved, will no longer be viewable by visitors.

Cuts keep half of 'Dead Zoo' exhibits out of public view

However, some of its best known exhibits, including a dodo skeleton and a piece of moon rock given to the State by the US, will not be on public view for the foreseeable future.

This glass box in the top floor bottom middle holds the Dodo skeleton.

The Dodo is one of the most iconic images of environmentalism before Stuart Brand persuaded Nasa to take a release of the earth the phrase "as dead as a dodo" was one of the best metaphors for the fragility of nature that we had. This sight of these bones is an important warning for us and we should make every effort that nine year olds now get to see them. For this reason I want to see the Dublin NHM Dodo moved down to a floor that is open to the public. Moving a glass box is not going to make a massive improvement to the world, but it should also not be a huge difficulty. A full restoration of the museum would be vastly preferable to this but this would be a relatively simple change to make in the meantime. This dodo is important the Natural History Museum badge has a dodo on it which says to me it is one of the most important and interesting items in the museum.

Here is my favorite song sung by an extinct flightless bird The Mountain Goats- Deuteronomy 2:10

I'm all alone here as I try my tiny song
Claim my place beneath the sky but i won't be here for long
I sang all night the moon shone on me through the trees
No brothers left and there'll be no more after me

A/B testing. Is Khan doing it wrong?

2011-09-15T06:45:00.000-07:00

A/B testing is where you try an old way of doing things and a new way each with a sample of the users and see which one works better. It is frequently used by ads where you test two wordings "Buy new coke!" and "Buy improved coke!" to see which one gets more clicks.

If you ever write a book or need to name a shop you should spend a few quid buying google ads for the two names you are trying to pick "How to start a fight" and "How to win an argument" and see which one gets clicked on more. That one should be the name.

This is one of those embarrasing posts that is probably wrong. But I think the A/B testing used by the Khan academy makes a fairly fundamental mistake. The Khan acedemy has lessons in various subjects. So presumably they want to use A/B testing to see if kids taught "1+1=2" or "1 + 1 = 2" learn more quickly and such.

A/B testing is a useful way to see if little tweaks result in better user experience. In Khan's case learning. It does not substitute for good design vision but can help make some relatively small tweaks. Improving the Khan acedemy and kids education is really important so if there is a bug in their A/B testing they might be making the wrong choices about how to improve their teaching.

For this kind of testing you need to pick the number of test cases in advance. How not to run an A/B test explains why and the effects of looking before all the test is finished. This is an odd feature of frequentist statistics

'However, the significance calculation makes a critical assumption that you have probably violated without even realizing it: that the sample size was fixed in advance. If instead of deciding ahead of time, “this experiment will collect exactly 1,000 observations,” you say, “we’ll run it until we see a significant difference,” all the reported significance levels become meaningless. This result is completely counterintuitive and all the A/B testing packages out there ignore it'

The A/B testing used by Khan seems not to do this as the gae bingo system says

"Controlling and ending your experiments

Typically, ending an experiment will go something like this:

You'll notice a clear experiment winner and click "End experiment, picking this" on the dashboard. All users will now see your chosen alternative."

This seems to be saying that either you should notice what is statistically significant which you won't always or that something can be declared statistically significant before all the samples are tested. Think of it this way. If every test has a 5% chance of being wrong and you think of everytime you look at the A/B test as adding 5% to the chance of being wrong. It is not quite that bad but it gives you a feeling of the problem.

Now there are ways you can tell that something is statistically significant really early in a test. "Bayesian Statistics and the Efficiency and Ethics of Clinical Trials" deals with these. In medical trials you want to know as early as possible if a new treatement is better or worse than an old one. Giving someone the wrong ad wont kill anyone but the wrong cancer drug might. This paper goes through how you would figure this out using Bayesian methods. These methods are also described in chapter 37 of MacKay's 'Information Theory, Inference, and Learning Algorithms'

But looking at the code GAE bingo uses for A/B testing they do not seem to be using these methods. So it looks to me that they are making the mistake of letting you stop a test when you want to. Which in frequentist statistics can be an error.

Also I think Vanity another rails A/B testing framework makes the same assumption
"This experiment will conclude once it has 1000 participants for each alternative, or a leading alternative with probability of 95% or higher:"

The system used by the BBC is based on time and not numbers according to this article. "Example use
For 5 in 100 people to get a two-option test running for 24 hours the function is initialised like this:". Which is not nearly as bad. But it is assuming that at the end of the time period you have had enough users to make a good test.

There is a proper explanation as to what can happen if you stop a trial early in "How not to run A/B testing". But from my reading many of the A/B testing frameworks out there seem to be making this error. Please correct me in the comments.

Addition: Ben from gae bingo got back to me in a comment on their blog.
"You're right that this is an issue, and that's a great blog post. However, this is significantly mitigated by a) letting your experiments run long enough to get a large-ish sample size for your population and b) simply not checking your dashboard constantly and making snap decisions.

We could build stuff into the system to mandate that, but at the moment I believe we'll be able to get solid value out of the existing framework (just like most A/B systems)." This seems fair enough, Khan are going to have such a high volume of users that they will be able to get a large sample size quickly.

War is on its last legs

2011-08-17T12:38:00.000-07:00

Famine is not looking too great either. Pestilence and death are down but not nearly out.

The recent riots in London had all sorts of people claiming the world had gone Mad Max and it was time to shut down facebook and make all teenagers join the army. There is a great summary of the moral panic in this article "Civil disorder and looting hits Britain, We have been here before". I think its interesting how the rioting that probably wont even be visible in the end of the year crime statistics made everyone freak out so much.

For all the talk of moral decay is the world in general a worse place than it used to be? War along with the other horsemen is a pretty good definition of how bad things are. Unless Bob Geldof finally gets that concert to end death set up. Even pestilence would seem to be a big ask to fix. What is the trend for war deaths recently?

Many people have predicted a century of appalling death tolls from war. Robert McNamara the ex US defense secretary and self confessed war criminal (at 3:10 ) for example.

Blight and McNamara project the level of warfare forward into the twenty- first century based on population growth, and suggest a “speculative” but “conservative” estimate of “at least 300 million” fatalities from war in the twenty-first century, of which perhaps 75 million would be military

So how have we done in the last decade on the predicted 3 million deaths a year?

So far they haven't even been close. In fact, the last decade has seen fewer war deaths than any decade in the past 100 years, based on data compiled by researchers Bethany Lacina and Nils Petter Gleditsch of the Peace Research Institute Oslo. Worldwide, deaths caused directly by war-related violence in the new century have averaged about 55,000 per year, just over half of what they were in the 1990s (100,000 a year), a third of what they were during the Cold War (180,000 a year from 1950 to 1989), and a hundredth of what they were in World War II.

So there has been under 2% of the expected war deaths this decade.

The Quaker economist describes the decline with "Just in the last fifty years, the progress towards peace has been startling. If this trend continues — and I am cautiously optimistic that it will — then as pressure from civil society increases and international institutions of conflict resolution mature, warfare as a regular practice may cease sometime late this century."

While talking about sending the four apocalyptic horsemen to their dotage the decline of Famine is also impressive

On the day that a famine is declared in Africa -- thanks as much to Somalia's political dysfunctionality as to a severe drought -- comes news that the world drought severity index has been falling for three decades. You can check for yourself by going here and clicking on "make time series":

There are some fairly serious disclaimers on this famine decline. The first is that agricultural productivity growth is declining at the moment
combine this with a possible loss of farm productivity when global warming kicks in and there could be problems.

While on the subject of how things are not good but significantly better than they used to be the last decade was the best one in human history.

We estimate that between 2005 and 2010, nearly half a billion people escaped extreme hardship, as the total number of the world's poor fell to 878 million people.

Nearly everything is getting better. Including life expectancy which you could take as evidence that even death is not what he was. Oddly all three of these sources give different numbers on 'major armed conflicts' but all agree the number is declining.

If two of the biggest problems mankind has faced for all of history are on the decline so rapidly they could be almost unknown within our life times I would say that is good news. And probably more important evidence of how we are progressing than a few rioters stealing runners.

We are getting better at nearly everything

2011-08-15T12:59:00.000-07:00

How quickly are humans getting better? We tend to think technology is getting better or that humans augmented by technology are improving. New swimming records happen regularly as swim suit technology improves. This post just throws up some evidence about human progress.

The Effect of Testing for Performance Enhancing Drugs on the Progress of World Records in Weightlifting “From 1964 to 1988 the relative strength of the world record holders in those weight classes increased by 21% …The same analysis in other types of sports, where there had been some changes in training methods over the same period of time, revealed that the maximum improvement was only 9%“ So most improvement in weightlifting seems to have been from pharmacological rather than having a wider range of people to select from or improved training mechanism reasons. However what about areas of human endeavour that drug taking seems unlikely to help?

In chess taking steroids seems unlikely to help. Though future nootropics might. "We conclude that there has been little or no ‘inflation’ in ratings over time—if anything there has been deflation. This runs counter to conventional wisdom, but is predicted by population models on which rating systems have been based…The results also support a no answer to question 2. In the 1970’s there were only two players with ratings over 2700, namely Bobby Fischer and Anatoly Karpov, and there were years as late as 1981 when no one had a rating over 2700 (see [Wee00]). In the past decade there have usually been thirty or more players with such ratings."

Even musicians are getting better Virtuosos Becoming a Dime a Dozen "The overall level of technical proficiency in instrumental playing, especially on the piano, has increased steadily over time." One good explanation for this and the chess improvement is just that more people are getting to try these, people who are not as limited by nutrition and disease as they would have been int he past.

This explanation is expanded in this rather good post "Two Hour Marathon in 2045"

"But the pipeline that selects and trains runners behaves, in some ways, like the model. If a person with record-breaking potential is born in Kenya, where running is the national sport, the chances are good that he will be found, he will have opportunities to train, and he will become a world-class runner. It is not a certainty, but the chances are good.

If the same person is born in rural India, he may not have the opportunity to train; if he is in the United States, he might have options that are more appealing.

So in some sense the relevant population is not the world, but the people who are likely to become professional runners, given the talent. As long as this population is growing exponentially, world records will increase linearly.

That said, the slope of the line depends on the parameter of exponential growth. If economic development increases the fraction of people in the world who have the opportunity to become professional runners, these curves could accelerate."

Progress of things like number of children suffering malnutrition and having clean water can really result in increasing the number of great chess or piano players as well as the world running record. We are getting better at loads of things because more people are getting to try them without the poverty induced hinderances they used to have. According to this model it is not that population is increasing exponentially but at the moment population of people who have a chance at being great at something is.

If you have other explanations for why and in what human achievment is progressing I would love to hear them.

Do Budget Cuts and Riots Go Together?

2011-08-09T14:16:00.000-07:00

I saw this paper 'Austerity and Anarchy: Budget Cuts and Social Unrest in Europe, 1919-2009'
with the intro
'In the wake of this week's London riots, some commentators have linked the youth unrest to budget cuts. The authors of CEPR DP8513 explore the historical basis for this view and finds that austerity and violence have tended to go hand in hand.'

I am reading through the paper now. But I wonder if there was an image that could quickly show a connection between the two.

Wikipedia lists the riots since 1950 as

1958 Notting Hill race riots
1970 Garden House riot
1971 Priestlley riots
1975 Chapeltown race riot
1977 Battle of Lewisham
1980 St. Pauls riot
1981 England riots · Brixton riot · Chapeltown race riot · Toxteth riots · Moss Side riot · Handsworth race riots
1985 Brixton riot · Broadwater Farm riot
1987 Chapeltown race riot
1989 Dewsbury race riot
1990 Strangeways Prison riot · Poll Tax riots
1991 Meadow Well riots
1995 Manningham riot · Brixton riot
2001 Bradford riots · England riots · Oldham race riots · Harehills riot
2005 Birmingham race riots
2010 UK student protests
2011 London riots

Now if you look at the government spending as a percentage of GDP here. On top of this graph I put a bar for every riot each year one occurred.

Counting all riots as the same is not fair. Their graph goes from 34->48 whereas the riots go from 0->6. Laying the first on the second is not considered good practice in data visualisation. Also and this is a big one. If GDP drops as in a recession and the percentage of government spending to GDP stays the same total government spending will drop. a fairer graph would look at gdp or government spending adjusted compared to riots not the two combined.

The proper paper says rioting and austerity go hand in hand. I will read it carefully to see how close the link is. But a quick look at the data and no obvious major link jumps out at me.

We have reached Peak Baby

2011-08-05T09:20:00.000-07:00

When will we reach peak babies? In what year will the most children be born? I bet last night a shiny pint that we will reach peak babies in the next three years. That the most babies ever born will be in a year before 2015.

Now I accept that we could never actually know how many children will be born in the future. The bet will end when I present enough evidence to convince those I am betting with rather than with a proof. Demographics is regarded as one of the most predictable of social sciences but some possible future invention could drastically increase the number of babies. We could have brave new world style artificial wombs of some such that vastly increases the birth rate for example.

Hans Rosling the statitician tweeted recently.

I looked up the UN data on this here. In detailed indicators look in births and in select country look in world. The highest birth number in the world was 1985-1990
Period Births per year
1950-1955 97 769
1955-1960 102 894
1960-1965 110 280
1965-1970 118 200
1970-1975 121 715
1975-1980 120 676
1980-1985 129 088
1985-1990 137 207
1990-1995 134 960
1995-2000 132 473
2000-2005 131 644
2005-2010 134 072
2010-2015 135 775
2015-2020 135 396
2020-2025 133 800
2025-2030 132 452
2030-2035 131 991
2035-2040 132 099
2040-2045 131 926
2045-2050 131 127
2050-2055 129 904
2055-2060 128 785
2060-2065 127 998
2065-2070 127 402
2070-2075 126 725
2075-2080 125 823
2080-2085 124 775
2085-2090 123 753
2090-2095 122 837
2095-2100 121 992
Now it could all of a sudden rise up and any prediction for the future is unlikely to be as accurate as historical estimates. Still I think I win the bet.

This does not mean that population will drop. As we are living longer world population is still expected to grow. But it does mean we can say that Malthus was wrong. Human numbers will not grow exponentially barring some disaster.

Crowd Sourced Optimal Fantasy Football Team

2011-08-04T14:06:00.000-07:00

All my fantasy football attempts have the problem that I know nothing about football. So for example Berbatov is unlikely to do as well this season as last season so picking him would seem unwise. But how does someone who knows nothing about football find out who will play well next season?

The wisdom of the crowds is the James Surowiecki about "the aggregation of information in groups, resulting in decisions that, he argues, are often better than could have been made by any single member of the group." This sort of thing does not work if the crowd has a bias in a particular direction.

I decided to look at the people is the "team selected by %" that fantasy premier league shows you. If I rerun the optimisation described in this post but instead of trying to great a team that has the maximum number of points last season I try and get the team whose players have been selected by the most other fantasy football managers.

The idea is that a team with the right number of defenders, goalkeepers, midfielders and forwards, that has at most three players from one team, that costs less than 100 and whose members have been picked most often should be really good.

The most popular team is


      Player Club Pos Price Pts people
7     Al-Habsi  WIG  GK    45 125  199410
30        Bale  TOT MID    80 118   189087
37      Barton  NEW MID    60 131   123865
69      Cahill  BOL DEF    55 105   180751
184      Given  AVL  GK    50   0   215091
205  Hangeland  FUL DEF    65 154   192333
231       Huth  STO DEF    60 138   165684
266    Kompany  MCI DEF    60  95  197352
323      N'Gog  LIV STR    55  48   279932
334 Odemwingie  WBA STR    75 171   218485
421     Suarez  LIV STR    95  68   343361
424    Taarabt  QPR MID    65   0   141086
452      Vidic  MUN DEF    80 148   172773
472   Wilshere  ARS MID    65  93   220381
477 Yaya Toure  MCI MID    80 146   167706

Some of these players were probably picked for the first few games and will be transfered out when they are about to play tougher games.

The average player is picked 26794 times. These players have been picked 200486 people on average.
This team would have scored 1415 points last season. The best team I could have picked for last season would have scored 2332 points. I have updated the dataset to include this people picked statistic.

Fantasy Football Optimization 2

2011-08-01T06:42:00.000-07:00

So it turns out I was thick last post about picking the fantasy football team. As my friend Bren explained to me "you choose 11 players to be "on the pitch" each week, plus a captain who scores double (and a vice captain in case the captain doesn't play). can only have 1 keeper in the first 11, can choose any formation with a minimum of 3 defenders, 3 midfielders and 1 forward."

So the problem is now pick 11 players that will actually play and some really cheap players that won't

So whats the cheapest of each player you could get?
The cheaperst players in each position are about 4 or 4.5 each. For example
Goalkeeper
Moreira SWA 4.0 0
Defenders
Whitbread NOR 4.0 0
Tate SWA 4.0 0
Midfield
Lappin NOR 4.5 0
Gecov FUL 4.5 0
Allen SWA 4.5 0
Strikers
Miller WBA 4.5 6
Hulse QPR 4.5 0
Agyemang QPR 4.5 0

So if you picked from this price of player knowing they would not play that would allow you to spend more money on those that were on the pitch. Now defenders score less points than midfielders and strikers so lets say we only want three of them. And we then buy two cheap ones that we dont play. 2 cheap defenders costs 8 and one goalie for 4 means we now have 88 to pick the remaining 12 players. The case whether you should have 5 midfielders or 3 strikers is less clear. I will try both and see which has a higher score.

Attempt 1. Cheap goalie and 2 defenders and 1 midfielder. Cost 16.5 on 4 non playing players. Trying to have 3 strikers. The constraints would now be

num_goalkeepers <- 1
num_defenders <- 3
num_midfielders <- 4
num_strikers <- 3
max_team_cost <- 835
max_player_from_a_team <- 3

Gives a team of


1         Hart  MCI  GK    70 175
48    Ivanovic  CHE DEF    70 144
49        Huth  STO DEF    60 138
51      Hughes  FUL DEF    50 129
197       Adam  LIV MID    90 192
198    Malouda  CHE MID   105 186
200    Dempsey  FUL MID    85 168
201   N'Zogbia  WIG MID    75 167
210     Jarvis  WOL MID    60 133
386   Berbatov  MUN STR    95 176
388 Odemwingie  WBA STR    75 171

scoring 1779. You would make Adam your captain and he would have double points.

Attempt 2. Cheap non playing goalie and 2 defenders and 1 striker. Trying to have two strikers. Cost 16.5 on 4 non playing players.

num_goalkeepers <- 1
num_defenders <- 3
num_midfielders <- 5
num_strikers <- 2
max_team_cost <- 835
max_player_from_a_team <- 3


1         Hart  MCI  GK    70 175
48    Ivanovic  CHE DEF    70 144
49        Huth  STO DEF    60 138
51      Hughes  FUL DEF    50 129
197       Adam  LIV MID    90 192
198    Malouda  CHE MID   105 186
200    Dempsey  FUL MID    85 168
201   N'Zogbia  WIG MID    75 167
210     Jarvis  WOL MID    60 133
386   Berbatov  MUN STR    95 176
388 Odemwingie  WBA STR    75 171

with a score of 1769. Again Adam would be your captain. It looks like three strikers is a better plan than 5 midfielders. But it is a close run thing.

Just to make things really complicated Bren explained that you should generally have one good player on the subs bench in case one of the rest of the team is injured. "it's quite common for one of the main team to miss a week so you may need to make 1 sub an excellent player (probably defender since they're cheaper)". Having a defender as your good sub has the advantage that you are allowed to play with one forward and three midfielders so if one of your three forwards, four midfielders or three playing defenders gets injured a defender can sub in for any of those.

So assuming we actually want four defenders. We would have one cheap goalie and one cheap defender and one cheap midfielder. At a cost of 12.5 for non playing players.

num_goalkeepers <- 1
num_defenders <- 4
num_midfielders <- 4
num_strikers <- 3
max_team_cost <- 875
max_player_from_a_team <- 3

giving a team of


         Player Club Pos Price Pts
1         Hart  MCI  GK    70 175
46      Cole A  CHE DEF    75 150
49        Huth  STO DEF    60 138
51      Hughes  FUL DEF    50 129
54    Bardsley  SUN DEF    50 123
197       Adam  LIV MID    90 192
200    Dempsey  FUL MID    85 168
201   N'Zogbia  WIG MID    75 167
203    Downing  LIV MID    85 163
386   Berbatov  MUN STR    95 176
388 Odemwingie  WBA STR    75 171
393   Davies K  BOL STR    65 132

1884 points -123 as Bardley wont play = 1761. Basically Bardsley a Sunderland defender is really cheap at 5. This is one more than you pay for a player you don't want to play but if you do need to lay him he gets 123 points a season.
Davies looks a little low there with 132 points. The worst midfielder has 163 points. So what if we get rid of the third striker and try 5 midfielders?


num_goalkeepers <- 1
num_defenders <- 4
num_midfielders <- 5
num_strikers <- 2
max_team_cost <- 875
max_player_from_a_team <- 3


   Player Club Pos Price Pts
1         Hart  MCI  GK    70 175
46      Cole A  CHE DEF    75 150
49        Huth  STO DEF    60 138
51      Hughes  FUL DEF    50 129
53      Distin  EVE DEF    55 124
197       Adam  LIV MID    90 192
200    Dempsey  FUL MID    85 168
201   N'Zogbia  WIG MID    75 167
203    Downing  LIV MID    85 163
210     Jarvis  WOL MID    60 133
386   Berbatov  MUN STR    95 176
388 Odemwingie  WBA STR    75 171

1886-124 for Distin who usually wont play = 1762. So this is my best single fantasy football team of last season.

The more I learn about this game the more nuances it has. Off the top of my head
1. Model that the captain gets double points. In this case Adam the highest scoring player is actually quite cheap. But in the case where he was really expensive you would want to take into account that he can earn double points.
2. Take into account who teams are playing. With a full game by game scoring dataset you can investigate really interesting patterns like if playing top five club means less points and bottom five more. And if so make transfers based on upcoming games. This seems to be where a huge amount of the skill in fantasy football is.
3. Change the code so you can have 3->5 defenders, 3-5 midfielders and 1-3 strikers but only 11 total players.
4. Just picking cheap non playing players is probably wrong. You at least want to pick the best cheap players you can. Which I have not. I am told Shane Ferguson is a good buy so I will probably make him one of the cheap players.

If any of the intuitions about having a good substitute or any of my other assumptions are wrong please correct me.
If you predict different points for players this season. If you want to try this method for next season but dont want to run the program linked to in part one

1. Copy the dataset I have here
2. Put this data into a new google docs spreadsheet.
3. Make your predictions on the number of points they will score. So if you think Berbatov won't score 176 points this season but only 160 change that points value. You can delete players you are not interested in as well.
4. Put your new spreadsheet URL in the comments
and I will run a optimization over your predictions for you.

Fantasy Football Optimisation

2011-07-30T13:15:00.000-07:00

Some friends challenged me to a game called fantasy "football" where you pretend to be a Russian billionaire. Not the bit where you steal natural resources off the population but where you buy a bunch of poncy overpaid foreigners who flounce around and earn insane amounts of money.

While I'm ranting about "football". Why do football ads always say it has "the beauty of a dance". If you like dancing that much go to the ballet

Anyway So I have to pretend to know something about this "football" which I dont but I do know a bit about optimization and more about copying stuff. I remembered this old R package article about optimising for fantasy football. This was written by prasoonsharma and I have just reused his code.

I scraped last seasons scores off the fantasy premierleague website. Some parsing turned this into the correct format. You can download the cleaned up data here

The constraints for this version of the game are slightly different than the one prasoonsharma was playing. This means you need more goalies, have more money to spend on players and other such changes.

The best single team for last season would have been


        Player Club Pos Price Pts
1         Hart  MCI  GK    70 175
5     Al-Habsi  WIG  GK    45 125
48    Ivanovic  CHE DEF    70 144
49        Huth  STO DEF    60 138
51      Hughes  FUL DEF    50 129
54    Bardsley  SUN DEF    50 123
55     Johnson  WOL DEF    50 120
197       Adam  LIV MID    90 192
200    Dempsey  FUL MID    85 168
201   N'Zogbia  WIG MID    75 167
210     Jarvis  WOL MID    60 133
212     Barton  NEW MID    60 131
386   Berbatov  MUN STR    95 176
388 Odemwingie  WBA STR    75 171
393   Davies K  BOL STR    65 132

with a total score of 2224 points. You get to change the team every week in the real game. This optimisation is just if you got to pick one team and leave it.

Picking a fantasy football team involves
1. Predicting how many points each player will score that week
2. Optimising based on this prediction so you pick the team that covers all the rule constraints that scores the most points.

This code carries out the second part. If you have a prediction for how many points players will score that is better than
"exactly the same amount they scored last season" you can change the data with your new prediction and run it (or I can run it for you if you want). A web app that allowed you to enter your player score predictions and ran an optimisation so that the best team that could be picked based on your predictions was then created might be useful. If you save the data in a new google doc and change the Pts values to your player predictions. Then post a link to that google doc in the comments I will run the optimisation for you.

The Kilbrittain Whale

2011-06-28T12:43:00.000-07:00

I love weird crap. I have loads of holiday photos of my wife looking bemused outside some odd tourist destination. Here is one of my wife outside the udderly brilliant Cork Butter Museum last Sunday.

But last week I think I may have reached the Zenith of weird tourist destinations. This acme of travel strangeness is the Kilbrittain Finn whale. This specimen of the second largest species of whale is mounted in the middle of a town several kilometers inland.

How did it get here? The Channel 4 whale autopsy provides some of the answers about this poor animals beaching in 2009. Three minutes in (or 20 seconds in the video below) you see the frankly unbelievable result of a bad autopsy of a whale. Exploding entrails all over the Denmark beach is the sort of image that stays with you.

What this pathological investigation misses though is the Machiavellian intrigue that lead to this particular resting place for the whale. When it landed in 2009 various locals tried to make off with the 50 ton pile of putrifying Balaenoptera physalus. "Apparently, an attempt was made on Saturday night under cover of darkness to steal the jaws but their bulk proved too much for the perpetrators. "

There are stories of a beach based stand off between the communities of Kilbrittain and Courtmacsherry. "Shortly after it died two men from Courtmacsherry, also known as a drinking village with a fishing problem, approached with a chainsaw. Their plan was to saw off its head and mount the jaw bones in the village. However, they were confronted by Kilbrittain residents who told them they were claiming the carcass." Presumably at some point in these tense negotiations two large groups of adults both armed with heavy machinery both realised that they were fighting for control of several dozen tons of rotting whale carcass.

There is no simile that really can convey the logic here. No metaphor can quite sum up the Irish ability to fight over possession of something that no one would want. But if I had go for one description it would be "like a stand off over 50 tons of festering cetacean".

Now having 'won' the whale, some lads from the local abattoir spent 18 months rendering the stinking baleen bag. This year and a half long job is shown in a frankly horrifying pictographic montage beside the the spoiled beluga that now rests in the middle of Kilbrittain park.

There was a great moment on redneck roundup (as nationwide is known inside RTE) about the whale where one of the guys who spent the lifespan of a hamster hacking at the pile of rancid blubber admits they may have gotten the order of some of the bones wrong.

One thing words cannot convey is the stench coming off this attraction. Some flesh still seems to cling to the head parts resulting in an overpowering odour of death.

It was so bad we could not eat the ice-creams we brought to the park. I advise you to go see the Kilbrittain whale while it still has this stench. There is nothing like the smell of dead whale to make you wonder about what some people will go see/fight over.

Really Old Idea

2011-06-17T14:46:00.000-07:00

Sports really annoys me. Particularly at the end when they get boring. When they know they have a minute to go in football they start kicking the ball between them and generally acting the micky. I thought I was dead clever when I thought that they should have random end times of sports. If you do not know when the game will end it is harder to try run down the clock.

Axelrod had the idea of random endings years ago. Many other game theorists have had similar ideas at various times.

Then I heard that random endings have a much longer history. Adapt the new book from Tim Harford explains one really old example. This century IBM got a patent for "a “smooth-finish” auction, an auction that is programmed to end at a random time". This gets over the problem on ebay where at the end of an auction people try not to bid what they value the item at but at just enough to win the auction just before it closes. So instead of people bidding one cent over the current bid with one second left on the auction.

Paul Klemperer pointed out that this idea is quite old. Even though the is a US patent for it.

In a candle auction, the end of the auction is signaled by the expiration of a candle flame, which was intended to ensure that no one could know exactly when the auction would end and make a last-second bid. Sometimes, other unpredictable processes, such as a footrace, were used in place of the expiration of a candle.

Pepys wrote about such an auction in the 1660's
Auction by candle was known in England by 1641, when it is mentioned in the records of the House of Lords.[6] The practice rapidly became popular, and in 1652, John Milton wrote, "The Council thinks it meet to propose the way of selling by inch of candle, as being the most probable means to procure the true value of the goods."

This 350 years old idea is my current record on simple ideas that someone has beat me to. Hartford's book Adapt is full of these sorts of interesting stories. This extract about the Spitfire gives you a good feel for the book.

Malaria Modeling

2011-06-13T14:10:00.000-07:00

There are two very interesting passages in Shah's 'The Fever' that suggest modeling of malaria can have an important role

HIV-positive people are most infective to others when the levels of virus in their bodies are high...Malaria triggers such spikes. Malaria infection, by inducing HIV to replicate, increases the viral load in HIV infected people. According to mathematical models, HIV may be responsible for 980,000 episodes of malaria and malaria responsible for mare than 8,000 Hiv infections in a single district of Kenya. The global effect of the malevolent partnership has yet to be mapped

This paper is the source of this information.

Another interesting point seems to suggest that a scapegoat that is more attractive to mosquitoes than people are.

the greater availability of cow and hog flesh attracted the interest of malaria mosquitoes.Hovering between home and stable ninety-nine out of one hundred times. That bit of anopheline caprice cost Plasmodeum dearly. As cows and sheep sprouted across the English countryside, malaria transmissionground to a halt

Increasing livestock numbers has serious effects on any ecosystem. the pig under the bad scheme should not be promoted without serious analysis. Which leads to a more general problem with complex problems like malaria. I want to talk about these issues in another post. There is an interesting Ted talk on malaria here. A lot of the ideas are impractical which is another reason to talk about practicality later.

Jeffrey Sacks also talks about the important role of livestock that distract mosquitoes in the end of poverty

Another important point is that some types of mosquitoes prefer to bite people, whereas others feed off cattle. Transmitting malaria requires two consecutive human bites: the first for the mosquito to ingest the parasite and the second for the mosquito to infect another person, roughly two weeks later. If the mosquito feeds frequently on cattle rather than on people, the odds are that at least one of the bites, if not both, will be taken from cattle. In India, for example, the predominant type of anopheles tends to bite humans about one third of the time, and cattle the rest. Africa, sadly has another predominating mosquito type which prefers human biting nearly 100 per cent of the time. Mathematically, the chance that an Indian mosquito will feed off two humans in a row is about one in nine, whereas in Africa it's about one in one

While on the subject of development I thought this quote was fascinating Justin Kilcullen head of Troicaire 48:18 in the Frontline 30/05/11 here
"Development aid is what we give to the developing world because we do not give them a decent deal in trade, because we have screwed them on their debts, because the flow of resources...lets change the economic structures that keeps these people poor thats the long term answer". If the head of an aid agency does not think they are the long term answer for development then they are probably not.

For the moment I am going to examine OpenMalaria to see how it can model these issues.

Coding Day Dublin

2011-06-06T06:40:00.000-07:00

Can you do anything useful in a day? This is the premise of code day where computer programmers are invited to work with scientists for a day to try solve a real problem the scientists are trying to deal with. "For those unfamiliar with a code retreat, it’s a hack day with a focus, the angle placed solidly on solving interesting technical problems." Hacking here means writing short programs that do something cool.

On the 28th of May I spent a day in the Science gallery trying to create fixes for various small problems. There were about two dozen people there with about a third of the scientists. Declan, Andrea and Qamir who organise loads of these programmers meet ups around Dublin ran the event and helped make it a really friendly atmosphere.

The small projects that seemed to work best involved visualisation. The problem of taking data and presenting it in an interesting way seems just right for a few hours timeframe. Possibly this is because rather than a complex scientific issue all that has to be understood is what sort of information needs to be conveyed.

It is quite difficult to get up to speed on a problem in such a short period of time. I had a really interesting talk with Scrazzl but even understanding their system took so long I felt like I was wasting their time. This meant we spent a lot of time just understanding the basic background of someones research. Treemetrics were also doing some really interesting Operations research work on forest management. It was really interesting to hear what people were working on but perhaps a day that was focused on one theme might allow the coders to read up and get some understanding in advance. Not that I think all such coder days should have a single theme just that it might be interesting to try one that did.

One of the most interesting projects at the day was fight malaria at home. This is a distributed computer project like SETI at home or folding at home. The aim is to test drug molecules against proteins present in malaria to check for potential binds. This could than guide researchers as to what drugs are worth testing in real life. The guys from the project were really friendly and interesting. If you do have knowledge of BOINC or even just want to learn it it would be a great project to help out on.

Instead of just criticising are very fun and worthwhile day I will give an example of the sort of focused task I am talking about.

Something that is big, widespread and multidisciplinary. I am reading The Fever by Shah at the moment which made me think Malaria was one such problem. Off the top of my head diarrhea diseases, political expenses, maths education and are all areas that people from many areas could come together and do some interesting projects/demonstrations over a day on. But malaria is one of those big thorny problems that a hackday about could be really cool. I have a Google Document of possible ideas here

Bad Taxidermy

2011-06-04T23:51:00.000-07:00

I have had a frankly morbid interest in taxidermy since I was a child. I particularly like bad taxidermy.

My absolute favorite form of taxidermy is when naturalists in the past attempted to reconstruct animals having never seen them. The weird mistakes they made should make raise serious epistemological questions. If animals we know that are quite similar to ones we see are gotten wrong how wrong are our reconstructions of Dinosaurs? More generally miscontructed taxidermy shows how our our ignorance and presumptions can distort our world.

My competition for the best misconstructed animal is between the Horniman walrus and King Fredericks lion.

With the Horniman walrus naturalist were sent the skin of the animal. Not realising it had wrinkles just kept stuffing the skin until it bloated up like some sort of zeppelin.

And the lion of King Frederick I of Sweden in 1731 where they had no idea what a lion looked like. The reconstruction has these weird old persons false teeth and lolcat eyes.

Paris for some unknown reason has particularly excellent taxidermy using shops

1. Deyrolle on rue du Bac.
2. Aurouze which they used in Ratatouille
3. The catacombs these freaked the bejasus out of me.

While I am on the subject I also recommend 'Stuff the World' a documentary on the world taxidermy championships.

The best crap taxidermy animatronics I have seen. And it is not a big category was in Strontium in Scotland. The wildlife museum there has a tunnel you crawl through where various crap animals on rails roll toward you. Strontium was on our travel list of places elements were discovered but that is a whole other blogpost.

If you also like appalling stuffed dead animals "crappy taxidermy" blog is a goldmine. If you know of any animals stuffed by people who have never seen the animal please post it in the comments

Spongebob New Wave Pants

2011-06-03T23:45:00.000-07:00

I was watching Spongebob as usual last Saturday morning and I noticed how most of the characters were modeled on 70's rock musicians. Elvis Costello is a dead ringer for the eponymous hero and Brian Eno could take the wrap of a Squidwort crime if he ever took part in a line up.

Do you know any other 70's rock star look a likes?

How much is technology worth?

2011-05-30T14:32:00.000-07:00

How much would you have to be paid to give up your mobile and your computer? How much is reading these pearls of wisdom, viewing pictures of amusing cats and forum fighting with idiots worth to you?

The brilliant Barking up the Wrong Tree blog it looks like mobiles are worth about 10% of income.

Keeping life satisfaction constant, we subsequently derive substantial GDP per capita estimates equivalent to a 10 percentage point increase in broadband and mobile phone penetration.

Median wage wage in US is about $32,000. So mobiles probably worth about 3 thousand dollars a year to the average American.

The value of the internet seems to vary from $1700 to $3800 according to this slate article

a model based on the value of people's time, ultimately estimating that Internet access is worth 2 percent of full income. Thus, they estimated the value of the welfare gain provided by the Internet ranged as high as $3,800 per person.

This means that in the 15 years that the internet and mobiles have become ubiquitous we are something like $5000 richer per year because of them.

Palaces of memory

2011-05-30T12:23:00.000-07:00

Remembering is a creative process. We tend to think rote memorisation as a dull boring task. However instead we should think of it as "Moonwalking with Einstein". A big challenge to think of the oddest 'most memorable' images you can associate with what you are trying to remember.

The book 'Moonwalking with Einstein' deals with one journalists travels around the world of memory competitions. These competitions have all sorts of weird and wonderful rounds. Remembering binary number, packs of cards, poetry, random word list, matching faces to names and historic dates. It is a very entertaining book, dealing with the history of knowledge, savants, the weird world of self help marketing and the importance of living adventurously.

Many of these competition events seem pretty abstract. Some memory based skill that had some sort of humane self improvement aspect and maybe even some vague practical use would be good to learn though. But if there was one of these skills that would be cool to have I would like to practice the methods described in the book. I read a book, Inside the black room by Jack Allen Vernon, about sensory deprivation experiments in a Californian university. The author describes an Arab immigrant who unusually asked to be put back into sensory deprivation room as he wanted to train himself incase he was ever forced into a similar situation as torture.

Brian Keenan's 'an evil cradling' has vivid, gripping and terrifying descriptions of the madness that stalked him as he was held hostage in isolation in the Lebanon. For example he writes "that the human mind can travel into those dark regions and return exhausted but intact is more a miracle than that word can ever convey". One of the boys own adventure stories I read as a child had the description of an RAF POWs time in the cooler and the physical and mental regime he went through to ease the hardship. One of the parts I remember was that he remembered all the poetry he had learned at school. These tales of how to deal with sensory deprivation have really stayed with me. Maybe if the memory skills in this book could help in such a situation that would seem useful.

One of the expert mnemonists in the book Ed suggests memorising poetry and prose is useful for such a situation of sensory deprivation. "My philosophy of life is that a heroic person should be able to withstand about ten years in solitary confinement without getting terribly annoyed' he said 'an hour of memorization yields ten solid minutes of spoken poetry, and those ten minutes have enough content to keep you busy for a full day.'

I will give myself a challenge to get good at poetry memorisation in a month. The competition seems to be to '15 minute memorization time, 20 minute written recall' for poetry. I will try memorise a poem tomorrow. And again in a month and see if memory training improves poetry memorisation.

As an aside the author mentions the interesting possibility of using "neutropic 'cognitive steroids,'" but never goes into details about how these might aid a memory sports competitor. This is slightly odd given he wrote an article in 2005 (viewable here) about his experience taking one of these drugs. He said 'gym rats have steroids, and overachievers have Adderall. ' it would have been interesting to hear what effect these drugs had on his memory abilities but this book never brooches the subject in a personal way.

Search Engine Deoptimization

2011-03-29T01:44:00.000-07:00

Take a company you have a problem with

1. Imagine you hate Ryanair. I do not btw they are just an example of a company this would work on.

2. You decide you want to harm them commercially

3. You go to their terms and conditions here and see they do not allow linking to their website

"Links to this website. You may not establish and/or operate links to this website without the prior written consent of Ryanair. Such consent may be withdrawn at any time at Ryanair’s own discretion."

There is a list of other companies that do not allow linking listed here. That where I heard of this T&C clause.

4. Now spider all the websites that do link to them. This is easy to do and you now should have a list of many websites. Either from a whois or just from scraping the website get an email address for each site.

5. Send all these linking websites official looking notices that remind them that "You may not establish and/or operate links to this website without the prior written consent of Ryanair" including a link to the official Ryanair terms and conditions. You would have to be careful here. You would not claim to be Ryanair for legal reasons. You do not want to break any laws or threaten anything. Just say you are from some official sounding company "John Smith and associates business compliance coordinators" and remind them of what the companies own policies say. It is not like you are lying.

6. Watch Ryanair drop down search rankings as people delete links to them. Search rankings are mainly based on how many (and what sort of) sites link to you. As they drop down the search engine rankings they will suffer commercially. With my google of "cheap flight dublin london" Ryanair comes up as the third site. Which implies they must get a fair amount of traffic from search engines.

I think this attack should be called Search Engine Deoptimization.

Analog Computer Museum

2011-03-11T13:55:00.000-08:00

This week I was lucky enough to visit the Analog Computer Museum near Frankfurt. Bernd who owns and protects the collection I have known for years via email so I was happy out to finally get to meet him and his wife. These Analog computers carry out calculations using electronic components.

Bernd works as a software archeologist. That is he updates the systems of banks and such that are written in OS' and languages that almost no one knows anymore. If the software running your bank was written int he 60's and you want to get it running on Linux Bernd is the man to call. These languages are so obscure they make the COBOL look like the Beatles. The job seems to pay well enough as he can afford warehouses to keep his computer collection in and a collection of vintage sports cars he drives at terrifying speeds on the local autobahns.

Yes I take my wife to Computer Museums but in fairness shes a physicist and I suspect she thought this rocked as much as I did

These machines are ideal for real time simulation. This Telefunken currently simulating a car was used to train nuclear powerstation operators with simulations of various 'problems'

Next time you call rand() think of this board that was used to generate random numbers through electronics

Old school fractal printed out and taped to the side of a Vax. It is like chalking up a kill on the side of a WW2 fighter plane. Computing from the time where men were men and debuggers were nervous.

I cannot rationalise why these analog computers are so cool (pics here). I think its to do with getting to the bare metal of a computation. Also something of the history of these machines is amazing. Looking at the circuit boards you can imagine national characteristics they describe. The German machines based on mathematical abstractions are beautifully clean (and easy to fix apparently). The Japanese machines were a mess like their remote controls. The British machines had a distinct boffin feel to them. The machines from communist countries really seems like a scramble of clever but under resourced hacks. The American machines were big brash and heavy.

Kevin Kelly claims that a technology never dies

What happens is that most technologies become obsolete, diminish their role, but they don't disappear--more idea-based and can be resurrected more easily. Seems like a silly claim: the hand-axe, the arrowhead--they don't exist any more. And that's what I thought too... Looking on the Internet was able to find an example of whatever the challenger had given to me: brand new steam-powered valve for a steam-powered car; they're making flint axes exactly the same way, using exactly the same tools to the point they are almost indistinguishable from the original; archeological artifacts, making in huge numbers today.

But I really worry that the knowledge of many of the engineers who created these machines will be lost. One thing is that the analog museum really wants donations of machines that may be thrown out. If you see a box of the brand Dornier, TeleFunken, Gould, Aritma, BBC, Electronic Associates Inc, Solartron or even just something you think might be of interest send him an email. Even parts of these machines could be really useful to help repair the computers he has.

I do not think these machines are purely of historic interest. It is possible that in the future analog computers will be used in the real time simulations they excel in. One area I think they could be useful for is search engines. Linear algebra calculations are at the heart of modern search engines. this is described in
The $25,000,000,000∗ Eigenvector the linear algebra behind google and How Google Finds Your Needle in the Web's Haystack

These calculations do not need to be precise (the google algorithm uses random walks) which removes one of the big objections to analog computers.

George Dyson in the this years edge question 'What Scientific concept would improve everybody's cognitive toolkit?' describes a search engine based on analog computers here

Analog computing, once believed to be as extinct as the differential analyzer, has returned. Not for performing arithmetic — a task at which even a pocket calculator outperforms an analog computer — but for problems at which analog computing can do a better job not only of computing the answer, but of asking the questions and communicating the results. Who is friends with whom? For a small high school, you could construct a database to keep track of this, and update it every night to keep track of changes to the lists. If you want to answer this question, updated in real time, for 500 million people, your only hope is to build an analog computer. Sure, you may use digital components, but at a certain point the analog computing being performed by the system far exceeds the complexity of the digital code with which it is built. That's the genius that powers Facebook and its ilk. Your model of the social graph becomes the social graph, and updates itself.

Analog computer methods of calculating eigenvectors have been described decades ago (pdf here).

It is possible that the techniques of analog computers could be the future of search engines. We should make some effort to save these machines, any manuals written on them and if possible some of the knowledge of those who made and used them.

Think Different

2011-03-11T13:03:00.001-08:00

I am fascinated by how people think differently from each other. I think there are big differences in the fundamental methods people use to think, but because we rarely talk about how we think these are not well known.

Feynman talks here about how he heard numbers in his head when he counted and his friend saw numbers visually.

He believed that such fundamental differences explained why we sometimes have such difficulties communicating with each other.

I 'see' numbers as a kind of line and I have heard other people describe other ways they 'see' numbers

Reading is another place people think differently. When I read I 'hear' the words as I read them. This is called subvocalisation and is normal but it slows down reading. "Subvocalization, or silent speech, is defined as the internal speech made when reading a word, thus allowing the reader to imagine the sound of the word as it is read". There are all sorts of online guides as to how to stop subvocalising. My wife does not subvocalise and never has. She describes seeing words as the actual word rather then hearing it and then understanding what it means.

Do left handed people who do that hook handed over the top writing have a fundamentally different view of writing? Do you know of any other basic cognitive tasks that people carry out in completely different ways?

NOW Who is the popular one?

2011-02-28T13:46:00.000-08:00

Someone of the two of you readers were kind enough to nominate this blog for the Irish blog awards

Science/Education 2011 Best Science/Education Blog
Technology 2011 – Best Technology Blog/Blogger

Thank you very much to all of you who read my nonsense especially those who decided it was worth pointing out to other people

On a related trumpet blowing note Tyler Cowen thinks this blog shows that Ireland is going to collapse into anarchy,

@tylercowen
Will the Irish be more or less impatient than the Egyptians?, http://tinyurl.com/4zyyxsg

Which now I think about it may not be a complement

Congratulations to my friends in thebeernut, L. Mulligan grocers and Dungarvan Brewing Company on their nominations.

A no lose lottery would help Haiti

2011-02-28T13:13:00.000-08:00

In this post I want to try and persuade you that there is a form of lottery the Prize Linked Savings scheme that would be popular and fairly cheap to run for the people in Haiti. In my next post I want to explain why such a scheme would actually be useful

"PLS is a kind of savings account that pools some of the interest from all depositors and pays out a big lottery prize every month or so. It combines the thrill of the lottery with the safety of a savings account. It’s sometimes called a “no-lose lottery,” since a depositor is automatically entered into the lottery but can’t lose the original money she deposits."

Lotteries are incredibly popular in Haiti. Probably due to the crushing poverty there the hope and 'chance to dream' lotteries provide is seen as vitally important

"the lottery system in which an astonishing number of Haitians invest their income and their dreams." "At the time of the last official count, in 2005, the streets of Port-au-Prince were home to nearly 2,000 of them" (borlettes lottery shops) "– more than double the number of schools and universities in the city"."All told, Haitians spend as much as $1.5 billion per year on the borlette – a staggering amount in a country whose gross domestic product last year was $6.9 billion"

"You can bet two balls to come up at once, and if that happens you make a ‘marriage’ and win 500 times your bet. If your married numbers are consecutive, you win 1,000 times your bet. The Haitian dream is a bet of US $1,000 that reaps US $10,000 on a consecutive marriage. Many a hopeful future has been built on less." Borlette lottery in Haiti is remarkably popular which is a good indication that a lottery that also acts as a savings scheme will be popular

The overhead cost of these Haitian lotteries is relatively low as they use the numbers and thus the security mechanisms of outside lotteries. These lotteries tend to use the numbers from other countries lotteries for example the New York State Lottery or the Santo Domingo State Lottery

Another expense is the prize won in the lottery. You want your prize to be equivalent to a million euro. This is an amazing life changing amount in Ireland where the average wage is 35K. So a million is about 30 times average income. For Haiti this would be $730 *30= 21 thousand dollars. A million Haitian gourde is about 25 thousand dollars.

If we had a monthly prize of this much that would cost 12*25k=300 thousand a year in prizes. Say you double that with some smaller prizes. 600 thousand which if you were earning 3% interest means you would have to take in 10,000000 dollars. This is a lot of money but in the context of $1.5 billion spent on lotteries in Haiti it is not that much.

If you could take in ten million dollars you could run a no lose lottery in Haiti and use the interest to pay for the prizes. But why would you want to? I will explain in the next post why PLS would be useful to the people of Haiti

The Great Stagnation in Ireland

2011-02-18T04:31:00.000-08:00

There is a new ebook by Tyler Cowen about how growth rates and progress in general has declined since 1973. There is a podcast here where he discusses the argument.

The book, is short you can read it in an evening, and published only in ebook format. It is a bit like an evening with your smart friend as he explains something rather than an economic treatise thats full of equations and aids insomnia. It is well worth the 4 dollars and few hours it takes to read.

Loads of commentators have examined various parts of the argument for stagnating progress. I even joined the argument here. I am going to try give a few datapoints that relate to Ireland that are part of the great stagnation argument here. Ireland has not been stagnating since 1973. In that time we moved from an improvished bachwards country to a modern well off one (and then slipped back somewhat again).
But there some areas where we are stagnating in similar ways, there are areas where we made similar mistakes to the Americans (and the rest of the OECD) and if we can expect a similar decline in growth here that means we should alter our expectations.

Stagnating medicine

We spend too much on medicine and do not get good value for that spending. There is a piece here by Ronan Lyons where he argues convincingly that with our age profile we should be spending a third less than what we currently do on medicine

It is generally agreed that access to the health system in Ireland is bad. If we spend more than we should and people cannot get healthcare as easily as other OECD countries do it is fair to say that Irish healthcare is not improving our quality of life the way it should be

Stagnating education

'In the past decade, funding per student has been increased in real terms by 61 per cent, yet performance has decreased by 15 per cent.

This disconcerting news was reinforced before Christmas when the international ranking of the reading, mathematics and science skills of Ireland’s 15-year-olds was published. Reading skills dropped from fifth to 17th, the sharpest drop among 39 countries, with one quarter of all 15-year-olds classified as effectively illiterate. This was more politely termed in the report as “below the level of literacy needed to participate effectively in society”.'

We are spending more and getting less on education. Literacy is one of those basic measures that is hard to fake. Anyone who leaves school illiterate is a failure of our education system. In this school education area Ireland seems to have stagnating improvement.

Expecting too high growth
One of the major conclusions of the book is that Americans expected growth to continue at previous rates and this facilitated the financial crisis. People believed house prices would go up forever without wages increasing at the same rate. When people realised that they could not afford their debts and no one was going to buy them off them the house of cards collapsed.

In Michael Lewis' Vanity Fair piece makes a similar claim that Ireland expected too much growth
'An Irish businessman named Denis O’Brien sat on the board of the Bank of Ireland in 2005, when it was faced with the astonishing growth of Anglo Irish, which was about to double in size in just two years. “I remember the C.E.O. coming in and saying, ‘We’re going to grow at 30 percent a year,’” O’Brien tells me. “I said, How the fuck are you going to do that? Banking is a 5-to-7-percent-a-year growth business at best.”'

If the financial crisis in America was aided by delusions of future growth the Irish collapse was to an even greater extent.

Expecting growth to be much higher than it will be leads you to paying too much for things. For example buying empty buildings on the grounds that NAMA will make a profit on them eventually.

Other Claims
There is a very interesting section on how the internet might entertain us but does not provide many jobs. There is a similar argument to in "An Exodus Recession?" that because we can sit at home and look at youtube essentially for free the recession both does not feel as bad but is measurably worse as people do not go out and spend money on cinemas etc that employ people.
This does not relate specifically to Ireland though I do think the internet has improved our lives in ways that are hard to measure and that probably help those who like reading and arguing more than they help the average person.

Cowen also claims that increased emphasis on scientific research could again get progress moving faster again. I was watching Bronowski's the ascent of man last night from 1973, the year Cowen claims the stagnation began. Bronowski gives a warning that the next step in mankinds progress is dependent on science and that if we don't prevent scientific progress from stagnation some other civilisation will.

Expecting growth to return now

I have heard many people running for office who claim anyone who questions that we can return to the good old days of 5%+ growth of being unpatriotic. In this narrative our debts are huge but they were in the 1980's as well ans well soon have 5%+ growth that will evaporate our debts for us.

The evidence is that once countries get as rich as we are now their quick growth from playing catchup then slows. Some of us might be able to get more productive now but not nearly the increase that happened in the 90's when we went from working on farms producing vegtables and potatoes to producing Viagra and Pentiums.

We are not going to have the growth rates we had in the 90's we like the rest of the OECD have entered a period of lower growth, what Tyler Cowen calls 'the great stagnation'. Anyone who says we should plan for Ireland growing at a rate higher then the 6.7% interest rate on the ECB/IMF loan are is trying to get you to take on billions of their debt.

The great stagnation a great short book. Though its arguments apply to America the lessons of what they Japan and other countries who developed before Ireland are important for us to learn.

Watson is better off having a three way competition

2011-02-17T08:00:00.000-08:00

It seems non intuitive but I think Watson is more likely to win because its competition are so good. If you look at how well it did on Jeopardy here it seems to show Watson is better having humans splitting the points on questions it cannot answer as it is so quick to answer the ones it can.

When the humans buzz in they are right 19/23 about 83% of the time. Suppose Jennings was up against watson and me and I got zero buzz ins. Jennings may have gotten 11 of the buzz ins that Rutter got. Watson seems so fast on the buzzer that it seems to buzz first for nearly every question it could answer 44/50 times it tries to answer it is first.

This means that Jennings could nearly have doubled the number of answers he had but because there are two people who are really good at the sorts of questions only humans could answer these points are divided amongst the humans. Of the 23 human answers only 6 did Watson try to buzz in on.

There is a similar probability puzzle that might help in a three way duel. "You're a cowboy, and get involved in a three way pistol duel with two other cowboys. You are a poor shot, with an accuracy of only 33%. The other two cowboys shoot with accuracies of 50% and 100%, respectively. The rules of the duel are one shot per cowboy per round. The shooting order is from worst shooter to best shooter, so you get to shoot first, the 50% guy goes second, and the 100% guy goes third, then repeat. If a cowboy is shot he's out for good, and his turn is skipped. Where or who should you shoot first?"

Theres another explanation of a three way duel here.

Sometimes having 2 really good competitors increases your chances of winning.

When will you be able to ask Watson?

2011-02-16T04:27:00.000-08:00

Watson is a computer currently playing the quiz Jeopardy. You can see a video of this here

One useful question is how long this sort of supercomputer like power will take to be seen in everyday life? It has 3000 cores running and answers in less than three seconds. On one machine would take 2 hours to answer a question according to here
So how long to get a desktop computer to be able to answer at this speed? I propose that half the speed improvement in Watson like programs will be from Moore's law and half from algorithm improvements. To go from 7200 seconds to say 2 seconds will require about 13 halvings in speed. I predict this will occur in the time it take processors to improve 7 doublings in computation. You can buy your own supercomputer now on the cloud and depending on how long you are willing to wait you could get an answer to a Watson like answer for costs of around a dollar (if you had the data and software Watson uses).

One comparison here is with chess. "It has been estimated that doubling the computer speed gains approximately fifty to seventy Elo points in playing strength (Levy & Newborn 1991:192)."

If doubling in processing power happens ever two years that would imply about a 30 point increase a year. The actual improvement (in computer v computer games) is described as 'With a 40-point annual improvement due to hardware upgrades, and a 30-point annual improvement due to software upgrades'. Implying improved algorithms are responsible for half the improvement.

This argument is summed up in this explanation as to why a fairly normal computer Deep Fritz in 2002 was an improvement over the supercomputer Deep Blue in 1997 'Deep Fritz has improved considerably over Deep Blue. Despite Deep Fritz having available only about 1.3% as much brute force computation, it plays chess at about the same level because of its superior pattern-recognition-based pruning algorithm'

I claim there is an analogy to Moore's law that says something like.

Once computers get good enough at a task to have a flashy TV challenge from then on algorithms will cause half the improvement in that task

So if I am right I think Watson like computation instead of talking over 25 years to move from twelve refrigerators to something we can use everyday should take less than 10 years. I would take an even money bet that in five years there will be a service you can use from your phone that is a lot closer to Watson than to a search engine we see today.

*update A friend just made this bet with me
"20 euro bet that January 2015 there is a search engine that with a set of quiz questions from one gameshow program (the weakest link, who want to be a millionaire, university challenge) and it has to get 7/10 right". The effects of learning by reading (and autonomous cars) are examined in this rather long but brilliant blogpost

Are Kitchens Better?

2011-01-31T15:27:00.000-08:00

There is a really interesting debate going online right now about whether the rate of improvement in America and the west in general has slowed. The debate has been started by a book by Tyler Cowen called "the great stagnation" which I have yet to read.

Above is a graph from here of how quickly different products became popular
Paul Krugman raises a similar point about how American Kitchens have not improved much from the 1950's here where he points out "By any reasonable standard, the change in how America lived between 1918 and 1957 was immensely greater than the change between 1957 and the present."

You could argue that kitchen equipment is much cheaper than they were in the 1950's. Or that rice cookers, food processors and microwaves are a great improvement. Coffee machines are probably the exception to the stagnation rule as they do seem to have improved vastly. There is also the explanation that if your home kitchen has not improved maybe the kitchen that cooks your food has.

Has the big change been that people do not use their kitchens anymore? In a 1950's kitchen could you order a pizza to be delivered? Dominos started in 1960 according to the history of pizza delivery here so you probably could. I doubt Italian, Chinese, Mexican and Indian food were regularly delivered to ordinary peoples homes though.

Has the technology and labour moved from the domestic kitchen into industrial kitchens that now do 90+% of the work in preparing many of our meals? How much of the supposed lack of improvement could be explained in terms of more processing happening outside the home?

“About 44 percent of food dollars are spent outside the home – a figure that started rising sharply in the 1970s, as more women joined the work force. Full-service restaurant revenue rose 5 to 7 percent a year in the decade leading up to the Great Recession”. So do seem to be eating out more?

Houses with small numbers of people cook less. And houses have less people nowadays, due to demographics. If people eat out more (or order in). Why would they buy some brand new kitchen invention? I have seen people with kitchens filled with shiny new equipment they have never used but I think this is fairly rare.

If the dial up pizza restaurant can have a new dough maker that is more efficient it makes sense for them. I can ring them up now and get the use of their new industrial standard food technology for the minute it takes them to make my pizza. Maybe home technology stagnated and the restaurant/supermarket/Microwave dinner technology improved?

I think it is worth noting that the improvements in Irish kitchens in this time have been huge. The 1918 world Krugman describes is superior to my grandmothers life in the mid fifties rural Ireland. She did not have running water or electricity and in her lifetime between 1950 and 1980 the changes she saw were massive.

Crowdsourcing Taxonomy

2010-12-23T03:56:00.000-08:00

I had an interesting debate on twitter with Panos Ipeirotis and Anand Kulkarni about what counts as a crowdsourcing application.

Wikipeidia defines crowdsourcing as
"Crowdsourcing is the act of outsourcing tasks, traditionally performed by an employee or contractor, to an undefined, large group of people or community (a crowd), through an open call."
And they are even so nice as to give a list of crowdsourcing projects

These seem to fall into a few areas
1. Microwork. MTurk, Freelancer.com, 99designs. You go on mturk you put in a task like "tell me if this sentence is in English" and an amount "1 cent per task" and then you let the people on mturk do the classifications for you.

I think this sort of work will be huge. There are talented people sitting around willing to do something interesting for cash in their spare time just waiting to be harnessed.

Many projects exist to crowdsource advertising ideas. for example IdeaBounty. These advertising ideas seem similar to other microwork projects.

2. Games with a purpose. Galaxy finder, Foldit etc. There is a task like "count the number of spiral galaxies" you put this out there as a 'fun' task and people do it for you free, gratis and for nothing.

This is similar to citizen science projects but explicitly involves the use of fun and game mechanics to get people to do the work.

3. Market mechanisms could be thought of as a form of collective intelligence. In a market the competing forces of supply and demand result on a value being placed on an item. So for example on ebay many people have decided the value of collectible baseball cards. Or on intrade many have decided the probability Obama is to be reelected.

Is market formation a form of crowdsourcing? I think prediction markets are great and should be used more but their placing accurate values on the probability of an event is a byproduct of their action rather than the intention of those operating on it. People would rather if the odds were wrong so they could make money being right.

For some reason I don't think a prediction market is a crowdsourcing application and none are listed on the wikipedia list.

As an aside I read a brilliant blog post two days ago (that I now cannot find) that pointed out that ebay is essentially a storage device. You sell things they go into the ebay and when you want them back you go on the ebay and find the product again and buy it. Your storage costs are usually just the postage and packing charges. This is similar to the argument that there are two ways to produce cars. One is to build factories and hire people and such the other is to send boat loads of cows to Tokyo where magically you get boats filled with cars back in return.

4. Microlending
Kiva and other sites allow you to lend money to business. Say a guy want to plough my field. 10 people give you 5 dollars each so you can buy a plough. The field gets ploughed. They get 6 dollars back after harvest. This again does not seem like crowdsourcing of jobs as you are lending capital not labour.

Artists using a street performer protocol to pay to get an album made is something similar. Kickstarter is one site that does this

5. Wikies, Forums, social search engines, message boards
People on Forums, give their advise and knowledge almost always for free. It is pretty amazing when you think about it. The fact that some very clever and skilled individual is willing to take their time to tell me the arguments to a Java function are wrong is amazing. The wikipedia page does not class these as crowdsourcing but again I cannot see why not.

6. Competitions, kaggle, Netflix, DARPA, Goldcorp and InnoCentive regularly have prizes where they put out a task such as 'improve on our recomendations' or 'make a car that drives itself' and people try to do the best they can at it. I think prizes as a way to encourage innovation are also going to be massively important in the future.

7. Response to events. The guardian set up a webpage to get people to read through all the mp expenses reports to find things that looked excessive and wikileaks have used crowdsourcing to examine the millions of leaked documents and highlight the most interesting.

8. Crime detection. Crowdsourcing has been used for illegal immigrant spotting in Texas and other similar uses.

9. Missing persons. After hurricane Katrina and Steve Fossett's loss crowdsourcing was used to locate missing people.

10. Politics. Oxfam Novib, Moveon.org and other such sites try to use crowdsourcing to create a community based political movement. If these are counted as crowdsourcing ventures I do not see why hobby sites that try promote brewing or knitting are not.

11. Art. Improv everywhere, mechanical olympics, Flash mobs are all listed as crowdsourcing projects.

12. Citizen science. Digitized versions of old weather or other science reports are made available to people to increase our knowledge of past events. Even the open data movement must have some basis in crowdsourcing. If people are not expected to analyse the data there is not much point making it availible to them.

13 Cartography. Open street map, Waze and other projects hope to use peoples GPS systems to build up a map of the environment. This can include real time traffic data.

Reading through wikipedias list of crowdsourcing projects these are the types of projects that occured to me. Most classes are probably wrong and need to be split or combined. Maybe the hierarchy needs to be different. Please comment or write a rebuttal to if you are interested in what projects count as crowdsourcing.

What is the worst thing to know?

2010-11-16T14:35:00.000-08:00

Knowledge is rarely a good thing but sometimes it actively reduces your quality of life. Take coffee as an example. Coffee is an accepted addiction, you cannot go around going 'Sorry I am not with it yet, I have not had my Vodka' but the same excuse for coffee is nearly mandatory as so many of us need coffee to function.

Some strange people learn about coffee though and as a result they no longer can cope with Nescafe instant. Next they notice burnt flavours and incorrect coloration in coffee shop coffees and an epic impossible quest for the 'God Shot'. Expresso starts to ruin their lives.

If you are addicted to a drug becoming snobby about that drug is a really bad idea. You don't see crackheads demanding only the finest Columbian devils dandruff in their rocks. Yet coffee addicts get themselves addicted to a drug then try and make sure they are miserable almost all the time they take the drug.

A normal coffee drinker gets a mild amount of pleasure from instant and when they go to a coffee shop get a fair amount. Someone who has gone to the trouble to learn about coffee lives in a constant state of disappointment except the rare occasions the world champion barista they have taken two hours to go to does not do something almost imperceptibly wrong.

The worst thing to know about I think is fonts. I don't even notice fonts but I have friends who can spot the bad font in a street with 100 other signs. You never hear them happy about fonts all they seem to bring is pain. The entire world just seems to become an annoyance to someone who knows how text should look.

America Is F*cked.......(Graphically at least) from Jess Gibson on Vimeo.

Do you have any nominations for things you are really better off not knowing?

Which Keanu Film

2010-10-25T16:21:00.000-07:00

Bizarrely unaging Keanu Reeves. Stick the answers below in here if you cannot name the film
Ovyy naq Grqf
Gur Qrivyf Nqibpngr
Wbuaal Zarhzbavp
Zl Bja Cevingr Vqnub
Punva Ernpgvba
Gur Zngevk
Guhzofhpxre
Pbafgnagvar
N Fpnaare Qnexyl
Cneragubbq
Fcrrq
Cbvag Oernx

Name the Schwarzenegger Film

2010-10-25T12:52:00.000-07:00

Past below into rot13 to get the answers
Grezvangbe
Ongzna
Urephyrf va Arj Lbex
Gur Ynfg Npgvba Ureb
Cerqngbe
Pbzznaqb
Gehr Yvrf
Raq bs Qnlf
Gbgny Erpnyy
Ehaavat Zna
Renfre

The Travesty of Goats in Trees

2010-10-04T11:46:00.000-07:00

This year I have been happy whiling away the months to the steady and lofty beat of the goats in trees calender

Every month brought a new sylvestral capric delight. So hoping for another fun year of arboreal ungulates i looked up the 2011 calendar. However I was shocked when I looked at this years chronology of tree based mammals.

See the fun loving soaring tribes? Now see this years calendar.

July, April and February are repeats. December does not even have any goats in a tree. The trip of goats in the photo are entirely ground based, what a sham.

I would not lightly give out about the folk at the august Brown Trout publishers but this shoddy display of could lead to dangerous Bovidaen Déjà vu.

Which means I am stuck without a calendar for next year. Any recommendations?

TIL 4

2010-09-29T08:21:00.000-07:00

Odd:Trained monkeys to guard athletes

Music: John Cale - All My Friends - (LCD Soundsystem cover)

Workplace Salaries: At Last, Women on Top

Science:Why squirrels masturbate

Economics:You can see futures in your weight

Funny:Do physicists believe in God?

Til 3

2010-09-28T01:01:00.000-07:00

Design: Airports should let you know your going to die

Music:Brian Eno - By This River

Word: Saturnine meaning sluggish in temperament; gloomy; taciturn

Nerd: Grace Hopper was cooler then i imagined

Tech:Billboards can recognise age and gender

Design: Pizza Boxes can be better

Science: Pierre de Maricourt started science by showing magnetic action at a distance which contradicted Aristotle.

A failed Mturk translation test

2010-09-26T08:53:00.000-07:00

The Mturk is Amazon's platform where you can put up jobs for people to do. These tend to be things like translation or image categorisation that humans are good at but computers are not.

According to Panos Ipeirotis' research mturk workers seem to be highly educated. The majority having degrees and about 10% having postgraduate degrees. Much mturk work is boring and probably does not use the skills these well educated workers have.

A few days ago google announced sponsorship for projects they think are carrying out good work. One winner was

"The Khan Academy will receive $2 million toward funding its work on the "make educational content available online for free" theme. The academy does just that, with a library of over 1,800 videos with lessons on math, science, finance, and history.

Bill Gates is a big fan of the Khan academy "This guy is amazing," he wrote. "It is awesome how much he has done with very little in the way of resources."
.

I am also a huge fan of the Khan academy I think that these videos and other online education videos such as MIT's online courses have amazing potential to transform the education of millions of people.

How far could googles 2 million grant stretch? Obviously it is up to Mr Khan to decide how to use his resources but I thought it would be interesting to see if the mturk could be used to translate one of his mathematics videos. The languages with over 100 million speakers are Mandarin, Spanish, English, Hindi, Bengali, Cantonese, Arabic,Portuguese, Russian and Japanese. 10 languages for 2000 videos would be 20,000 video translations. If each video cost 100 dollars to translate that would spend the 2 million dollars google donated. If this cost can be reduced more languages could be added.

A mathematics video is not something the average person can translate. However we know a large number of turkers have degrees and many are from India where they would likely have an understanding of English and Hindi or Bengali.

I tried an experiment to see if I could get one of Khan's videos translated into Hindi using the mturk at a cost of 5 dollars. Unfortunately I failed. The first person who accepted the task dictated what Khan said into text. Which is useful but not what I was looking for. The second person posted up another video on Solving linear inequalities in English not a translation of Khan's video.

This small experiment tells me that you need to be very clear on the mturk how you ask for a task to be completed. It also says that it might be worthwhile once you find someone who understands and completes the task to encourage them to translate other videos rather than rely on the vagaries of who happens to accept your mturk task.

This experiment ignored the problem of copyright. Khan owns his videos and it is unfair for someone to come along and copy him. I was not trying to steal any glory from Mr Khan with this experiment just to see if the mturk could be used to translate his videos.

Other people have successfuly used mturk to reduce the cost of translations. 'How I reduced translation costs of 200 articles from $9000 to $46' is an interesting article on one successful usage. This tells me that the problem was more likely with my unclear instructions than with the mturk platform. You can even monitor translations taking place in the mturk here so I still think this method would be cost effective. However my simple experiment failed.

TIL 2

2010-09-23T05:07:00.000-07:00

Music: I have had this song stuck in my head for ten years after hearing it once. I finally figured out the name. The Swains - Numberless As The Sands On The Seashore

Science: What happens if you put your hand into the LHC stream?. I also like that bra is a word <

History: You could get excommunicated for believing that god could not create a four dimensional universe or even mulitple universes due to the 297 condemnations of 1277. This is pretty odd to think guys in 1277 were being condemned for things physicists get condemned for today.

Film: The last exorcist is a good horror film. Full of ambiguities and non obvious twists. The wikipedia review is fair "It doesn't fully deliver on the chilly promise of its Blair Witch-style premise, but The Last Exorcism offers a surprising number of clever thrills."

Beer: Homebrew should do more better videos like this one

Maths: Drop a printed map of the land on the land and there must be some common point.

Word of the day: Muckefuck means bad coffee particularly substitute coffee

TIL 1

2010-09-22T08:18:00.000-07:00

Shampoo does not make any sense (Kaye effect)

Medieval love letters mainly involved explanations of previous sexual assaults
"Even when you were unwilling, resisted as much as you could, and tried to dissuade me, since your nature was weaker I often forced you to consent with threats and blows"

Iron and Wine do a great version of "waiting for superman"

There are some fun retro howto documentaries on burning people to death on an industrialised scale

Malware attacks have moved onto nuclear programs

You can get the illicit thrill of buying drugs through sandwiches

Other people dislike Diageo

Word of the day: Götterdämmerung

The Bayeux Tapestry is a Swashbuckling Adventure

2010-09-15T13:01:00.000-07:00

I went to see the Bayeux tapestry a few days ago. If you are ever in Northern France go see it. It is an amazing experience to see a thousand year old hand made item, particularly one that tells a story. Seeing the individual threads and mistakes of something that old is just breathtaking.

But one thing that struck me about it is how fun the story is. I imagined it either as a dry scolarly repetition of the story or an ancient piece of propaganda. Instead it is more of a swashbuckling adventure. Compare one swashbuckling film 'The Princess Bride' with the tapestry and see the similarity. They both have

Kidnapping
Harold gets kidnapped when he lands in France

Quicksand
Harold rescues two men from Quicksand

I really miss quicksand it used to be everywhere "nearly 3 percent of the films in that era (the 60s) showed someone sinking in mud or sand or oozing clay". But you just don't see it anymore. It has sadly gone the way of grappling hooks and that using a knife to slide down a sail trick.

Abseiling
A guy they are trying to catch uses a rope to escape.

Forced Marriage
Williams daughter is married by herself with a priest slapping her.

God's Hand
Monty Python style God's hand comes from the clouds.

Which is a bit like having a miracle man around

Sword Fights

And the tapestry has all sorts of other stuff you see in matinee adventures like horse chases, giant animals, oaths, dragons and divine heavenly light

In Table form you can see the genre cliches

Obviously the Bayeux Tapestry served many purposes at the time. But I think we might underestimate one of those was entertainment. You could remake this story now as 'Indian Jones and the Norman Conquests', it is already in comic book form.

Thoughts on France

2010-09-14T12:41:00.001-07:00

Some things I noticed on a recent trip to France. None are earth shattering.

1. The Rise of the robots. The Hotels, McDonalds the Petrol Stations are all computer booths.

I can think of a few reasons for this. The French all seem to have credit cards so that makes it easier to have card run business. It also means that most shops look at you funny if you try pay with a 50 euro note.

I have heard French labour laws make hiring/firing people difficult so maybe using machines is efficient in a high labour cost country.

Eventually the same effects will happen here and many of these sorts of jobs will be automated in the next few years.

2. Desperado beer is inexplicably popular given how much good booze France has.

You see aisles of the stuff, people in queues have six packs of it. The only explanation I can think of is that wine has become stronger recently leaving a gap in the market for stronger beer. still this stuff is only 5.8% so though strongish it does not fill in the 8% light wine gap.

3. Sophie le giraffe. It is like a secret society. Parents around Ireland have been sneaking in this teething aid. Some friends smuggled back 4 for their friends. I was aware of the child minding powers of savanna ungulates. They have a series of creches here but whoever starts importing these into our homes in Ireland will make a few quid

4. "All our beef is French and Irish" Most restaurants had a sign like this displayed. This was the only non local food that was advertised as such. Our farmers tend to come across as EU sponging yokels but if Irish beef is the one food that is displayed as not French they must be doing something right.

5. An active lifestyle is encouraged which is good. There was via ferrata all over the area of the alps I was in and phone numbers to check canyoning conditions for particular rivers. I presume the French avoid getting constantly sued for this sort of fun.

Basically France is a great spot for your holidays.

Cat Bubbles

2010-08-09T09:47:00.000-07:00

For something to be a bubble its price probably has to increase faster than inflation. People have to believe that other people believe it will increase in price forever. This is the madness of crowds effect like tulip mania in the 1600's where the price of Tulips rose to ridiculous levels.

I have heard arguments recently about higher education in the US being a bubble. But what about Pets? Animal based bubbles have a long and fun history. For example Alpaca's and Ostriches twice have been bubble commodities.

There is a really interesting article on how spending has changed during the recession here. It presents this table

From these figures could pets also be a bubble? The cat has hardly changed in thousands of years. Certainly not enough of a recent efficiency increase to make it worth a 14.4% growth in costs. So keep an eye on the costs of pet ownership especially of the fancier breeds to see if bubble thinking comes along.

Evacuating the Love Parade

2010-07-28T06:00:00.001-07:00

An important issue for crowd safety is how long it would take to evacuate a venue. Like the last article this is not to attempting to figure out the exact right answer to how large entrance/exits need to be but to try see if someone with a limited knowledge and a search engine could easily see in advance that the entrance/exit to the love parade was dangerous. Even large venues can require evacuation in the case of a natural or manmade disaster. These disasters could be weather related as happened here here and here. In the case of man made safety risks venues also are sometimes evacuated. As described here
"On December 12, 2004, the stadium Santiago Bernabeu in Madrid was evacuated because of a bomb threat.... In eight minutes, more than 70,000 people left the premises without incident."

The respected Spiegel says

"Much of the critique has centered around the fact that the tunnel where the panic took place was the only entrance to and the only exit from the party site. "
If this tunnel was the only exit available (another tunnel seems to have been reserved for emergency vehicles) how quickly should people have been able to exit?

This soccer stadium with a capacity of over 25,000 claims 'The target evacuation time for the entire Stadium at full capacity is 8 minutes.'

This document on evacuation planning states
"The maximum Emergency Evacuation Time for sports ground varies between two and a half minutes and eight minutes"

Stadiums do not have as much free space as larger venues but they still allow for moving people onto the pitch as happened at a baseball game after the 1989 San Francisco earthquake. In the case of some natural disasters or terrorist threats a site wide evacuation might be needed so even large sites probably need to be able to be evacuated reasonably quickly.

The acceptable evacuation time for a stadium seems to be under ten minutes. If Spiegel is correct and this tunnel really was the only exit any evacuation would have taken hours which is not be acceptable.

The pictures here give a very good impression of what happened. The width of the tunnel does indeed look like 16 meters. And the 'ramp' described earlier just looks like something people tried to escape up rather than a designated route.

How many people can pass through a tunnel?

2010-07-26T15:39:00.000-07:00

A tragedy happened at the love parade when 19 people lost their lives. This will be investigated by professionals who will come out with detailed analysis on how this incident took place. I want to see what someone in half an hours worth of searching can say about how wide a tunnel you need to fit that number of people.

I worked as a volunteer at the special Olympics some years ago. The job mainly involved making sure spectators could get into and out of the events easily, that emergency services can easily gain access and that spectators and athletes were kept separate. The venues were not huge and the spectators were happy and sober. Still I got some impression of the kinds of things needed to ensure crowd safety at events.

I want to put a figure on how many people could safely fit through the tunnel that was the sole access point to the love parade venue. A 16 meter width is given by most media outlets.

But some say it is 30 meters wide. Most seem to put the number of people there at 1.4 million but others at only 500,000 'an event set up for 250,000 ended up with an estimated 500,000 to 1 million'.

How long does it take to get everyone into a venue? How many people an hour will try enter a venue? I do not think all 1.4m people try enter in an hour. But I would guess most people try to enter a venue in a three hour period. Each hour you could expect over 300,000 people to try and enter through the tunnel. People are unpredictable meaning that you would need to be able to have more than this safely in case loads turn up at the same time.

Who else deals with crowd volumes like this? Every few years during the Muslim Hajj there is a crowd related accident. They are building a new bridge to improve safety. There is a description of this bridge here (pdf).

the new, multi-level bridge structure which will accommodate and ease the flow of 3 million worshippers during a single daylight period. The proposed new Jamarat bridge is a superior structure formed of 4 platform levels... each of the bridge’s 4 floors is roughly 600 m-long, with variable widths (ranging from 60 m to 97 m

The engineers describe these 4 floors as at least 60 meters wide. Each 60 meters in width is supposed to handle .75 million people if scaled linearly to 16 meters that would be 200 thousand people during the day. The new Saudi bridge is designed with a much larger channel to move people than they had in Germany. Pilgrims may act differently to other crowds as they stop to perform religious ceremonies.

Another terrible crowd incident was the Hillsborough disaster where 96 people died. A quick search about football crowd safety found this document. It is a very interesting article and well worth a read

The document states
"the safety limit for crowd density is defined as 40 people in 10 square metres for a moving crowd" the tunnel at 16m wide and 100 meters long should only hold 6400 people. Walking at 38 meters per minute. 100 meters would mean it would take 156 seconds to get through the tunnel. 6400 people every 156 seconds is 147,000 in an hour. This means it would take ten hours for 1.4 million people to safely travel through.

Reuters here states
"Authorities have not yet been able to explain how exactly the tragedy happened -- near a tunnel that led to a ramp into the festival grounds. Most of the victims were found dead on the ramp and none in the tunnel, authorities said"

The football document also in Section '2.8 Wembley Complex Station' gives this equation
No. of Units of Exit Width = Number of Persons (1)
required (each width = 0.55m) Flow Rate (2) x Evacuation Time (3)

"Where number of persons (1) means the maximum number of people that could be expected to be on a platform at any time. Flow rate (2) means 40 persons per minute for escape routes incorporating stairs, and 60 persons per minute for level escape routes (without stairs). Passenger walking speeds should be assumed to be 38 metres per minute for horizontal circulation."

Using the Wembley equation above 16 meters *.55 meters means 29 people can safely span across the tunnel. 40 flow past a minute if the escape route has a stairs and 60 if it is flat. The ramp sounds like it would hinder flow in a similar way to a stairs.

Once again this is all back of the envelope and as more facts come out a much better estimate of how many people could safely negotiate the site will emerge. A reasonable estimate of the number of people who could travel through the tunnel in an hour is (tunnel width/person width)*number or people per minute * 60 minutes
which in a 16 meter wide tunnel with a stairs at the end is (16/.55)*40*60=70,000
Best case scenario with a wider tunnel and no stairs 30 meters/.55 meters *60 people *60=200,000 which is less than the expected number of people.

This calculation indicates that a cursory look at at the venue entrance would give someone grave concerns about safety for a smaller crowd than turned up on the day.

Science grows 2.5% annually why doesn't the economy?

2010-07-20T10:03:00.000-07:00

What causes economic growth? Much of the world still lives in grinding poverty, figuring out a way to get them out of this is important. The best way seems to be for their economy to grow.

There are many theories as to what causes economic growth. The founder of Google claims
"Virtually all economic growth (in the world) was due to technological progress. I think as a society we're not really paying attention to that," Page said. "Science has a real marketing problem. If all the growth in world is due to science and technology and no one pays attention to you, then you have a serious marketing problem."

If technological progress causes growth there should be a link between the amount of scientific progress and the amount of growth. Here is a very interesting article about the rate of the increase in scientific knowledge.

'What I found, using this simple proxy for difficulty, in each field — biology, astronomy, chemistry — was a curve with the same basic shape. In every case, the ease of discovery went down, and in every case it was a curve called an exponential decay ...[I] discovered asteroids get 2.5 percent smaller each year. So while the ease of discovery drops off quickly as early researchers pick the low-hanging fruit, it can continue to “decay” a long time, becoming slightly harder without ever quite becoming impossible"

Say science increases in the level of knowledge by 2.5% a year.

What is the rate of economic growth? 'Since 1820, world development has been much more dynamic. Per capita income rose more than eightfold, population more than fivefold.' 190 years growth 8 fold increase is a growth rate of about 1.1% per annum. Or for two specific countries the UK from 1830 to 2008 had average real GDP 1.97%. The US 1830 to 2009 real GDP of 3.62%.

Other things cause economic growth

-Demographics more people of working age means more stuff gets made.

-Institutions. The rule of law matters. If people think anything the build will be plundered off them they wont build much.

-Governments. Some ways of running a country seem to result in more economic growth than others. Communism was never that good at creating economic growth for example.

-Taxation. At a certain point if you tax people too much they wont work anymore. If you dont tax people enough you might not be able to afford schools. Knowledge of what level of taxation is beneficial to economic growth should improve over time.

-Financial innovation. By getting capital to the people who best use it financial innovation is supposed to help economic growth. If it doesn't we should not spend money bailing out investment banks.

-Foreign aid: Aid to developing countries seems to have a mild positive effect on their economic growth rate though in specific areas it seems to have a negative effect

What else can affect economic growth and how great an effect does it have?

How scared should you be of nut allergies?

2010-06-07T15:42:00.000-07:00

The wife today was selling nuts and parents kept telling her that nuts were banned in their kids school to prevent deaths from nut allergies. How many life years are saved from banning nuts in schools versus those lost through not eating nuts?

I will look at American figures as they are available and have a large sample, which are both handy. "about 150 people die annually from serious allergic food reactions" which compares with "2,000 children drown each year" according to here. So 150 people out of the 2.5 million who die each year in America die from some food allergy. Imagine all these people had 80 years left to live. You have lost 1200 human years.

Now if you happen to be the one with the allergy you would want some reasonable precautions taken. My argument here is a guesstimate on how the over anxiety about how nuts might affect us.

“We try to relieve anxiety about nut allergy by signs saying, ‘this is a nut free zone,’ which suggests that nuts are a clear and present danger,” Dr. Christakis said. “But in doing so, we increase the anxiety.” So imagine this sort of fear of nuts made everyone stop eating nuts. How many life years would be lost? "those eating nuts daily had up to 60% fewer heart attacks than those who ate nuts less than once per month". Given four hundred and fifty thousand people die each year from heart attacks. Preventing 60% of these would be 270 000 lives. Say you only saved one year off each of these that is 270000 life years as opposed to 1200 from banning nuts. This 60% figure seems really high but other studies show massive improvements here and here says '1 ounce of nuts more than 5 times/week can result in a 25 to 39 percent reduction in coronary heart disease risk among people whose characteristics match those of the general adult U.S. population'.

Now just because people who eat nuts don't get heart attacks does not mean the nuts stop the heart attacks. But the googleable studies take this correlation versus causation problem into account. Also banning nuts in schools and telling kids they might kill people wont stop all nuts being eaten. But its not hard to imagine these warnings will severely curtail nut eating.

Not many kids have nut allergies. So a blanket ban without good reason wont save many lives. There is good evidence eating nuts is very good for your health. It is reasonable to assume telling kids nuts could kill them wont encourage them to eat them.

I think this nut fear illustrates a problem many of us have with comparing sudden risks with long term risks. For example when schools ban running by kids because they might be sued but they ignore the long term health effects of kids not exercising. There are risks everywhere but when someone wants to minimise one it is worth asking how dangerous that risk is and what consequences minimising it will have.

Eurovision Voting Fraud

2010-05-24T13:05:00.000-07:00

Forget hanging chads in Florida. Some really important vote rigging is about to happen.

I was investigating the Eurovision Song Contest last week as a prediction problem. No really it is an area of major research into how corruption agreements implicitly develop between countries. There are papers here, here and here.

But then I saw the odds for this year and there is something really odd happening. The odds from Paddy Power are here
Azerbaijan 11/8
Germany 4/1
Israel 9/1
So Azerbaijan a country that no one can spell is hot favorite. Which would not be too suspicious if they had ever had a song that wasn't entirely made up of goats being strangled before. The singer Safura and her Drip Drop has her obvious talents on display but is there something odd going on here?

Wikipedia says

After the conclusion of 2009 Eurovision, it was reported in Belarusian media that unknown people had allegedly paid and bribed students from Belarus to vote for Azerbaijan in the contest, and had even transported them to the Belarusian-Lithuanian border in 10 buses. The cost of this alleged action was claimed to be about 55 million rubles. They purportedly voted for Azerbaijan using Belarusian SIM cards, and then received Lithuanian cards to repeat the process. The Azerbaijani entry "Always" received 10 points from Belarus and 5 points from Lithuania.

So why does an old soviet country, with a pretty dodgy political system, that is 95% Muslim want so badly to win the worlds gayest contest? Turns out its a giant fight with Armenia. There is a war going on between them using the Eurovision as a battleground. I cannot wait till they get Johnny Logan in as a peace negotiator.

The Azerbaijan state broadcaster blocked out the voting number for the 2009 Armenian entry like some corporate logos in a rap video. Then and you can only read this in a full Borat accent "in August 2009, a number of Azerbaijanis who had voted for Armenia's entry during the 2009 Contest were summoned for questioning at the Ministry of National Security in Baku, during which they were accused of being "unpatriotic" and "a potential security threat". One of those summoned, Rovshan Nasirli, said that he was told by his interrogators that they had the names and addresses of all 43 Azerbaijanis who had voted for Armenia".

What is the cause of all this? According to the over 2000 word wiki article on the "Armenia Azerbaijan relations in the Eurovision Song Contest" it is Nagorno-Karabakh an area they both claim.

So it looks like Azerbaijan are willing to buy masses of fake simcards, bus people to other countries, bribe students and track down a few dozen people who voted the 'wrong way'. I wonder what is going to happen this time now that they seem to take the eurovision seriously?

Craig Venter creates synthetic life form

2010-05-21T01:39:00.001-07:00

There are stories about this everywhere. But one quote from wired sticks out

“Over the last five years the field has seen a 100-fold increase in the length of genetic material wholly constructed from raw chemicals,” said synthetic biologist Drew Endy of Stanford University. “This is over six doublings in the max length of a genome that can be constructed.”

Doubling in abilities in less than a year is amazing. Pretty much any exponential improvement like that ends up being a big deal if it carries on for a reasonable length of time. This is 1 million base pairs of synthetic DNA. A human is 3 billion base pairs. So at this growth rate that is less than 12 years. Not that you would artificially create a human or anything but it gives you some impression of how rapidly this technology could develop.

NO DEPOSIT...NO WORRIES

2010-05-02T13:20:00.000-07:00

If we do not learn our lesson from the property bubble and collapse we will wind up back in the same trouble again.

There is a semi ghost estate on my bus route. It is beside the M50 beside Finglas and an area called Scribblestown in murder news reports. The estate is opposite and beside a halting site.

This Heathfield estate seems less than half full. This week I noticed the sign they have flogging houses had a new banner on it. "NO DEPOSIT...NO WORRIES"

We have learned nothing.

We Hate You. Please Save Us

2010-03-30T14:02:00.000-07:00

There is an episode of the Simpson's where Springfield unites against Lisa Simpson who questions the veracity of an angel skeleton. In a very short time, the citizens launch a mob attack against the town's scientific institutions, Moe decides to destroy the skeleton of a woolly mammoth at the Springfield Natural History Museum, yelling "Take that, science!" Predictably, the tusk falls off, crushing him. "Oof, I'm paralyzed," observes Moe. "I only hope medical science can cure me."

Ireland will be saved by science. Here is a document from the Taoiseach about the new smart economy.

One main idea is to "invest heavily in research and development, incentivise multinational companies to locate more R&D capacity in Ireland, and ensure the commercialisation and retaining of ideas that flow from that investment;"

So how does Ireland treat science? Well take the LHC launched fully today. There is a great article here about the disdain this project is meet with by our government. We won't spend the 10 million annually that would make us a part of the project.

Not that no Irish people are helping out but they have to work with foreign research groups if they want to help figure out how the universe works.
“Nobody knows what is going to happen, that is why we are doing it,” stated Dr Steve Myers, the Irish physicist in charge of the largest atom smasher ever built.

So no 10 million annually out of a total of 6.5 billion it will cost to work out how the nature of the universe.

Irish scientists are not valued either. There is a poll of the greatest Irishmen of all time. There is no scientist on the list. Dr Noel Browne is the closest but he is there as a politician. No Walton, Robert Boyle or William Rowan Hamilton.

But you do have Joe Dolan, Colin Farrell, Stephen Gately, Ronan Keating, Daniel O'Donnell and Louis Walsh.

We have space for Daniel O'Donnell but not Nobel prize winner Ernest Walton. We don't have 10 million for CERN but we do have 32000 million for the governments friends.

What can you say? I'll leave it to another Nobel prizewinner

What need you, being come to sense,
But fumble in a greasy till
And add the halfpence to the pence
And prayer to shivering prayer, until
You have dried the marrow from the bone?
For men were born to pray and save:
Romantic Ireland's dead and gone,

September 1913
William Butler Yeats

Influential Books

2010-03-19T02:41:00.000-07:00

Marginal revolution has an article on the 'books that influenced the me most'. There is a follow up article of others lists here.

No one in these lists seems to mention the Bible. And few the Odyssey and Iliad. Aesop and the brothers Grimm also seem to be ignored. Hansel and Gretel, Rapunzel, little red riding hood, Cinderella, Sleeping Beauty, Snow White, Rumpelstiltskin all come from one book. They are pretty much ubiquitious in the lives of children of my age even if they did not read the original.

Is it naive to think that these stories we heard as children probably influence our thinking far more than books read after our teenage years?

I seriously doubt there are many westeners who don't consider the three little pigs as a moral lesson more often than those from anarchy the state and utopia.

The most embarassing book most of the list writers will admit to is something by Ayn Rand which they now disown. But why is Atlas Shrugged allowable on the road to where they are but Rip Van Winkle and the works of Hans Christian Andersen's are ignored?

By most influential do we mean the book your willing to admit to that you think about while debating people? Rather than the ones that are so ingrained in our lives that they serve as a backbone to how society operates.

How much land do you drink?

2010-03-13T13:28:00.000-08:00

Say you drink 35 liters a year of alcohol. 14 liters is the rather high amount the average Irish person does. There is a podcast here where Saul Griffith describes how due to carbon rationing in the future you will have to decide between a shower and a glass of wine each day. So the question of how much effort goes into each brew is important for the future where the amount of land and energy we consume may be heavily taxed and rationed.

What area of land do you need of a crop of apples, barley, wheat, potatoes, grapes or sugar to keep someone in booze for a year? There is a table here of the yield in terms of (kcal/sq m/year) each crop produces

Cane sugar 3500
Beet sugar 1990
Potatoes 1600
Apples 1500
Wheat, cereal 810
Corn 250
Milk 420
Beef 130

So in terms of calories per square meter Cane sugar seems to win. You do have to calculate how efficiently the calories in the crop get into alcohol in the booze you drink.

This guy tries get the alcohol out of sugar beat. "so the theoretical yield of ethanol produced is around 50%alc per weight of sugar"

Each square meter is 2000 kcal according to the earlier table. A kilo of sugar is 4000 kcal. So for 14 liters of alcohol is 28 kilos of sugar. 14 liters is what the Irish person drinks annually. So that is 56 square meters of sugar beat to keep you in (bad) booze for the year.

Now for cider.Say scrumpy is 6.5% abv then one liter of scrumpy is 65 ml of alcohol. So 14 liters is 215 liters of cider. Is that right? Does the average Irish person really drink the equivalant of a pint of scrumpy every day?

Anyway I want about 200 liters of cider which is about 400 kilos of apples. How much area of orchard do I need to get that many apples?

cider and apple juice are both about 400 calories per liter . Growing apples is 1500 kcal per square meter. I estimate 1/2 of that makes it into the juice which is 750kcal. Each liter is 400kcal so 200 liters is 80000 kcal. If a meter is 750 kcal after pressing etc then that is 106 meters squared to keep you in good alcohol for the year.

So back of the envelope it would take about 100 square meters of orchard to keep you in booze for the year.

4.5% Beer seems about 400 kcal per liter. And all grain is to be generous 75% efficient at getting out the sugars from grain. So each liter requires 533 kcal of grain very roughly. Wheat, cereal 810 kcal per square meter per year that is 2/3 of a meter square per liter of beer.
So how many liters of 4.5% beer is 14 liters of alcohol? It is about 310 liters of beer (which is a huge amount!) or back of the envelope 250 square meters of land to keep you in beer if you include malting loses.

Now wine. Each square meter grows about a kilo of grapes. As 1 acre grows about 5 tonnes according to here.

So 112 liters of wine is about 14 liters of alcohol.
3 kilos of grapes produce about 2 liters of wine from looking at various recipes. So 112 liters is about 170 kilos of grapes. So 112 liters is about 170 sq meters of vineyard. So you need about 170 square meters of grapes to keep you in booze for the year.

So in Ireland you could keep you in booze for a year of square meters you would need 56 sugar beat, 100 orchard, 250 barley and in france 170 grapes, roughly.

Drug Delivery Service

2010-03-03T11:43:00.000-08:00

I got a pamphlet in the door today offering a drug delivery service. The drugs are advertised as bath salts and plant food. Head shops selling these legal highs have been burned down in Dublin recently so this could be a way to avoid such attacks.

The Perplexus Rocks

2010-02-14T12:05:00.000-08:00

The perplexus is a 3-d maze where the aim is to move a ball around a series of obstacles. Makezine last issue had a great story on it here. The article described the journey from a school project through decades of effort to a toy and even a giant art piece.
There is an interview here with the creator Michael McGinnis. Usually my wife finds puzzles and games really boring but I have to keep fighting her to get a go on the perplexus. There is a page here on the history of the design. It shows some of the dedication and inventiveness that you can feel went into the puzzle when you play it.

It is not perfect, there are annoying etchings on the top that sometimes hinder viewing. It is slightly off spherical which is not what the designer wanted. It is made out of plastic like all toys. I presume trying to get this level of precision in wood would be nearly impossible. Still this is the best designed toy since the rubik's cube. Similar to the rubik's cube it is mainly aimed at children but not just for them.

This is really pretentious but the puzzle is like moving meditation. I cannot play this without feeling like part of zen and the art of motorcycle maintenance. You never get products anymore that have such obvious dedication built into them. Every step feels like some sort of Japanese mindfullness exercise. Basically the perplexus rocks. I got bought mine in the UK from here. The shipment to Ireland was really quick. You can get it from America here

Analytics X Prize Outside Bets

2010-01-13T16:17:00.001-08:00

Black swans are not predictable but are there fairly rare events that could coour this year in Philadelphia that would skew the homicide rate in an area?

I know of nothing that would cause a sudden drop in the number of homicides in one particular area. Except a mass evacuation. There are a few things that could show up as a sudden rise though.

Terrorist attack. Philadelphia is not a well known terrorist target so any attack there would have to be pretty unpredictable. Terrorist attacks are also very rare so I do not think worth considering in a model.

Going Postal: These seem common enough in America. They have their own list here. I doubt you can predict where they will occur though. Might be worth considering.

Religious nutjobs:The solar temples and the kool aid drinkers in general tend to head off into the sticks before topping each other. So I think there is not much chance of a mass homicide by a cult in
Philadelphia.

Riots:Cities kick off on a fairly regular basis, The LA riots in 1992 resulted in 53 deaths for example. Philadelphia has had riots in the past. Riots in America seem to be mainly caused by racial tension. It might be possible to if not predict them localise where they are most likely to occur. Then submit a higher homicide count for that area in one of the analytics x prize submissions as an outside bet.

Prisoneers: They love a good ruck. In general prison populations have a higher homicide rate. So looking at changes in prisoner ecosystem in Philadelphia might be worth a look

Natural Disasters:After natural disaster people generally think the world turns into something from a Romero film.
The evidence for this is not that strong for example tales of post Katrina anarchy seem to be overblown. Also natural disasters could reduce the homicide rate as people leave an area after one. Philadelphia is unlikely to undergo a natural disaster though.

There are rare events that still occur often enough to make some sort of prediction on. I do not think any of these is worth including in a model with the possible exception of a riot. But predicting that would need more information about riots and Philadelphia than I have at the moment.

Survey the people of Philadelphia

2010-01-12T14:50:00.000-08:00

In order to tell if a zip code in Philadelphia is getting more dangerous maybe we should ask people who live there.

So I created a survey here to ask them here. If you are or know someone living in Philadelphia if you could fill that out it would be great.

The idea is to find areas people think are changing in safety and see this turns out to help predict homicides. If it does this could be used to focus police resource in future to help reduce homicides.

I will release any data that is input as I am not the best person to do analysis on them. People going to the effort to submit a survey deserve to get the most out they possible. I will anonymise any data that does contain personal info before releasing it. There shouldn't be any info like this but I will check through the data in case.

I think ideally such a survey would let user place pins in a map in areas they think are improving/disimproving. Any thoughts on the survey? Or the idea of asking people for their local predictions in general?

Analytics X Prize

2010-01-11T13:57:00.000-08:00

There is a competition here to try and predict what proportion of murders in Philadelphia will occur in each of the cities 47 zip codes. Many people who are interested in these sorts of puzzles have started submitting predictions.

So how would you go about predicting the murder proportion in each zip code?
Well if nothing changes in Philadelphia you would expect each zip code to keep exactly the same proportion of murders, well with some random variation you could not predict. So my first guess is a repeat of exactly what happened last year.

But in the real world things do change. Say the population changes if every person had the same chance of being murdered then the proportion of murders in a zip code would change proportionate to the change in population. If this was the case the prediction problem would become to find out what changes in population will take place over the year.

The dataset I am using is here and some errors in it need to be removed. Each dot is a zip code. It looks like number of murders does roughly follow population but it is not nearly an exact match. So changes in population are important but they are far from the only thing we need to predict.

How expensive the house in an area are or the average income or the number of people per house might help indicate the murder rate. Here I am looking at number of (murders/population)*10000.

So it looks to me a bit like areas with crowded houses could be more likely to have murders.

House cost looks like it is not connected to murder rate. This could be because zip code is too rough grained for this to be a good judge. Maybe the average cost of a house in a block would be a better measure of risk. Philadelphia has even been broken down into 60ft squares here

Does household income look like it is related to murder rate?

So if the graph is a random scattering of dots then it looks like the independant variable on the x-axis has no relation to homicide rate the dependant variable on the y-axis. If the dots form a line (well not just a line but that is another story) then the homicide rate may be related to that independant variable. It really is not this simple but that's the basics.

As Siah pointed out here young black males seem to be murdered out of proportion. The graph above does seem to suggest that predicting changes in ethnicity of a zip code may improve predictions. Age is another important variable and I do not have data on that so that might be the next thing to get.

There are interesting posts already on this puzzle
"Evaluating Spatial Predictions" and "Second Pass at Analytics X Prize" and "Homicides as non homogeneous poisson processes" are very informative.

Dublin pubs and the El Farol Bar problem

2009-12-10T06:36:00.000-08:00

I was in town last night and i noticed how empty the superpubs were. these are the giant warehouse like pubs that need to be packed all the time to pay for the giant rents they have.

They also need to be full to make it look like they are not just giant warehouses. This is a version of the El Farol Bar problem. The game theory is this

* If less than X% of the population go to the bar, It will be too empty and they'll all have a worse time than if they stayed at home.
* If more than X% of the population go to the bar, they'll all have a better time than if they stayed at home.

People realise the pubs will be empty so no one goes. It is a positive feedback problem. Which means these big pubs are in big trouble.

Human Nutrient Needs

2009-11-30T13:33:00.000-08:00

What nutrients does a person need per day?
I am not a dietician or indeed sober so don't take this as actual advice. But what does ten minutes googling give as a list of the needed nutrients?
This is item 1 of the list of things to do for a modern version of Stigler's diet. Wikipedia the universal source of all truth (TM) italic has an article on this here.

It gives the RDA for an average 25-year old male (for ones that use IU which appears to be nonsense so I have taken some figures from here and here)
all figures are in µg


param RDA:=
Vitamin_A 900
Vitamin_C  90000
Vitamin_D  5
Vitamin_K   120
Vitamin_B6   1300
Vitamin_E  15000
Biotin    30
Calcium    1000000
Chloride    2300000
Chromium    35
Choline   550000
Copper  900
Cyanocobalamin  2.4
Fluoride    4000
Folate  400
Iodine  150
Iron   8000
Magsium  400000
Mangase    2300
Molybdenum  45
Niacin 16000
Nickel 1000
Pantothenic_acid    5000
Phosphorus  700000
Potassium    4700000
Riboflavin   1300
Selenium  55
Sodium   1500000
Thiamin 1200
Zinc 11000
;

And the upper limit as


param UL:=
Vitamin_A 3000
Vitamin_C  2000000
Vitamin_D  50
Vitamin_B6   100000
Vitamin_E  1000000
Boron 20000
Calcium  2500000
Chloride 3600000
Choline 3500000
Copper  10000
Fluoride  10000
Folate 1000
Iodine   1100
Iron 45000
Magsium 350000
Mangase 11000
Molybdenum 2000
Niacin 35000
Nickel 1000
Phosphorus  4000000
Selenium  400
Sodium   2300000
Zinc 40000
;

The figures we are interested in are RDA's which must be met and Tolerable upper intake levels which cannot be exceded.
Macro nutrients are in grams


param Macro:
Waterb  3700
Carbohydrates  130
Proteinc  56
Fiber  38
Linoleic 17
alpha-Linolenic 1.6 
end;

Also some nutrients must be minimised Cholesterol, Trans fatty acids, Saturated fatty acids. These will have to be aims of a optimiser for diets.
These are a bit more difficult to deal with so I will ignore them for the moment.
Cholesterol As low as possible
Trans fatty acids As low as possible
Saturated fatty acids As low as possible

Also two types of macro nutrients must be a % of calories.
Added sugar No more than 25% of calories
Fat 20–35% of calories

I am going to guess a 25 year old male needs 2500 calories until I am corrected.
So we have a first approximation of what nutrients people need.

So next up is getting a price and nutrient contents for loads of different foods.

The Diet Problem

2009-11-29T08:20:00.000-08:00

What is the cheapest way to feed yourself? This is not a minor issue our diets and health could be much better if they were optimised to provide the most needed nutrients at the smallest cost and also to provide the most palatable diet that is as healthy as possible.

Stigler in 1939 worked out a near optimal miminum price needed to supply a person with the nutrients they need for a year. The diet consists of five not very pleasant foods so is not intended to be realistic dietry advice.

Still given current knowledge of nutrition a list of prices from various supermarkets could you optimise you shopping basket to provide your family with groceries? Here we want to give people the nutrients they need in a form they will actually eat that is healthy and cheap.

We need a list of
1. What nutrients are needed by a person
2. Foods preferences of people.
3. A price list of foods
4. A Linear program to optimize a shopping basket based on these variables.

All of these requirements need some explanation. I think its best if each gets it's own post. So tomorrow I will have a post on what quantities of nutrients people require. If you have any suggestions please comment.

215 is the first Wikipedia dull number

2009-11-15T13:59:00.000-08:00

I have always wanted my own constant and now for a short period I have one. Imagine numbers were 'interesting' or 'dull'. The first dull number would be interesting because it was the first dull number so no dull number can exist. So all number are interesting.
Now you would think any interesting number would be notable enough to have its own wikipedia page. The first number without a wikipedia page is 215. This should be notable enough to deserve a wikipedia page. So as long as 215 has no wikipedia page it is dull and thus as the first non notable number it is notable. The paradoxes of wikipedia notability were brought up here.

Counting money across cultures

2009-11-08T13:55:00.000-08:00

One great bit in the execrable Inglorious Basterds was where someone noticed that Germans and British people make the sign for three in different ways. I love these cultural comparisons that show how arbitrary much of what we do is. I looked at information from laces here

This is a video showing how money is counted in different countries.

How People Count Cash? - Click here for more free videos

I have heard Japanese people ring out clothes differently to the rest of the world but can find no evidence of this in spite of asking a few of them. Do you know of any other cultural differences?

Matchbox Maths Games

2009-10-29T15:24:00.000-07:00

There is a great article here on Matches on India. No really read it.

In fact, 97% of rural households purchase matches on a monthly basis. Matches are a unique product because of their high, constant demand and low price point.Their ubiquitous presence provides fascinating insights into India's rural distribution networks, and offer potential ways to inform and interact with India's relatively untouched market.

The article then suggests using the matchboxes to spread public health messages

What if that space was used to relay information? Imagine the possibilities of spreading new health/educational information or advertising to 97% of rural families on a monthly basis. Simple pictorial designs would pique interest and accommodate India's vast differences in literacy rates and languages. Awareness of important topics such as the installation of chimneys to reduce smoke inhalation or cleaning and covering water containers to prevent stomach ailments could be spread to households across India, and potentially save lives

I would recommend selling the glamor of flushing toilets and chimneys and such rather than nagging in your images. But i do not know enough about rural Indians and their diseases to advise on what health images to provide them. There are some interesting studies on what does kill these people here.

However what if all you put on the boxes was games? Many people have a ludic philosophy of life. You see a love of games in maths nerds in particular. This love of games I believe adds a cognitive richness that aids intellectual development.

There is a great book called "everything bad is good for you". That claims the increase in IQ in recent decades is due to increased complexity in our culture. I have not studied the Flynn effect enough to be sure it is not caused by nutrition or even to be sure it is important. But I will assume it is and that intellectual challenges improve general cognitive abilities. I will go further out on a limb and claim that such improvements in cognitive abilities would aid rural Indians. Never having been a rural Indian this really is a big assumption.

So what intellectual challenges could fit on a matchbox and be read by an illiterate farmer? How about puzzles and games?

There are some face meltingly brilliant match puzzles here(pdf). Other then puzzles there are games like NIM, dots and boxes, chomp and loads of others you can play with matches. I would imagine if a brand of matches has a game on it that keeps the kids from bothering you this would be a popular feature.

So can you think of some way to explain a puzzle or the rules of a game on a matchbox without using text? Do you think it really could be useful to put mathematical games and puzzles on matchboxes?

A work sceduling problem

2009-10-28T16:25:00.000-07:00

I saw this problem recently

A city authority is considering placing a toll booth on its new bridge. The beginning
times for the shifts are 8am, noon, 4pm, 8pm, midnight and 4am. A collector
beginning a shift at one of the above times works for the next 8 hours.
The following staffing levels during each of the 24-hour periods have been estimated

Hour.................... Minimum Collectors Needed
8am - Noon.............. . 5
Noon – 4pm ................6
4pm – 8pm .................10
8pm - Midnight........... .7
Midnight – 4am ...........4
4am – 8am .................6

Find the minimum number of collectors that need to be hired to begin the 8 hour shifts
at each of the six times.

A GLPK program to calculate this is


var x1 >= 0, integer;
var x2 >= 0, integer;
var x3 >= 0, integer;
var x4 >= 0, integer;
var x5 >= 0, integer;
var x6 >= 0, integer;
/* objective function */
minimize z: x1+x2+x3+x4+x5+x6;

/* Constraints */
s.t. ctr1:x1 + x6 >= 5;
s.t. ctr2:x1+x2>= 6;
s.t. ctr3:x2+x3>= 10;
s.t. ctr4:x3+x4>= 7;
s.t. ctr5:x4+x5>= 4;
s.t. ctr6:x5+x6>= 6;
data;
end;

The answer is 19 and no one starts working at 8am.
This is a simplified version of the program described here. Not a major revelation or anything but always find these puzzle solving programs cool.

Prediction with game theory

2009-10-18T02:48:00.001-07:00

I am going to this talk on Thursday so I am reading the book Predictioneer by Bruce Bueno De Mesquita. He has a ted talk video on his prediction ideas.

The use of game theory in forecasting seems based on the axiom that people are rational. The problem is they are not, here is a list of the different ways they are not. If you know how they are going to be irrational you can alter your models to take this into account. However if you fail to do this you will end up with less accurate models.

In chapter 2 the book claims people are rational and particularly that we are transitive in our preferences. "to be rational.. their preferences must not go in circles. For instance if I like chocolate ice cream better then vanilla and vanilla better then strawberry i presumably like chocolate ice cream better than strawberry." So no rock paper scissors can exist in human preferences.

The problem is they do. There is loads of evidence that people can have intransitive preferences. For example May showed in 1952 that people can have intransitive preferences for wealth, looks and intelligence in a partner. So how can you have a mathematical system modeling the world where one of your axioms is false?

While on the subject of intransitive never play dice with Warren Buffet. There are loads of stories of how he tries to con people with a set of intransitive dice. Edward Thorpe for example. Or Bill Gates as described in Bill Gates Speaks: Insight from the World's Greatest Entrepreneur By Janet Lowe

Buffet once attempted to win a game of dice with Bill Gates using intransitive dice. "Buffet suggested each would choose one dice and discard the other two. They would bet on who would roll the highest number most often. Buffet offered to let Gates pick first. This suggestion instantly aroused Gates curiosity. He asked to examine the dice after which he demanded buffet choose first."

Boy in Balloon, did the balloon have enough lift?

2009-10-16T00:56:00.000-07:00

I love this story, it reminds me of the time myself and my dad nearly burned down the house. We made a tinfoil hot air balloon with a gas camping cooker to power it. The cooker was tipped onto the lino and the melting began. It was my mums fault for leaving us alone in the house together. In general I love these homemade balloon stories, like Larry Walters across California.

Could we have told in advance it was a haox? "Few had raised the issue of whether such a balloon could even lift off with a 50-pound kid inside, and then float the way it did"

So how big was the balloon? "Authorities said the silver balloon, 20-feet long and 5-feet high, at times reached 7,000 feet above the ground while adrift. It was found more than 90 minutes later in a field near Colorado Springs."

Then it would have a diameter of 6 meters and a height of 1.5m high. Taking the balloon as a cylinder then according to here would give it a volume of 42.5 cubic meters.

Gross Lift (Helium) = Volume (cubic meters)*1.05kg
So 42.5 * 1.05=44.625 kg lift.

Now that seems just within the bounds of possibility as the kid weights about 20 kilos and the balloon itself should be light. But could the balloon have gotten to a height of 2000 meters? There are all sorts of tables for these I might work it out later

Another site gives "Helium provides about 9.8 newtons of lift per cubic meter, at sea level and room temperature. That's enough to lift 1kg". So 20 kilos would require 20 meters squared to lift it. Same result as before.

Thanks to the best program on Television we know how many balloons it takes to lift a small child.
"It would require such a large number of balloons (3,500) to lift an average four-year-old girl of 44 pounds (20 kg) just a few feet off the ground that there is no way the myth could have happened unintentionally."

Update: Wired have an article here about how people should have realised the balloon was empty. Their estimation of the carrying capacity of the balloon is similar to mine. they point out that the balloon does not move like it has a heavy weight hanging from the bottom of it.

NAMA, how much is 54 billion euro?

2009-09-16T09:22:00.000-07:00

Nama, the Irish governemts agency to take on bad loans, is paying 54 billion euro to buy property. How much money is that? There is a great visualisation here about how much is a trillion.

So lets picture Croke Park. Its pitch is 144.5 m x 88m. That is much bigger then a soccer or rugby pitch. A 20 euro note is 133 × 72 milimiters. So to pave Croke Park pitch with 20 euro notes requires. So that is 1087 notes long 1223 wide or 1,329,401 notes are needed to pave croke park pitch in 20 euro notes. So that is roughly 2 layers of 20 euro notes being 54 million and paving it two notes deep. A billion is a thousand million so 54 billion is 1000 times this. So 54 billion is paving Croke Park in 20 euro notes 2000 deep.

So if your looking at the game on Sunday imagine there is a 20,000 euro pile of cash on the pitch that you now owe. That is 54 billion divided among the people who pay tax. So you and the person sitting next to you owe a 2000 deep 20 euro sized chunk of the Croke Park pitch.

A comment vaguely asked how much euro coin is this? "Did you know its also enough Euros placed end to end to reach the moon". According to here a 2 euro coin has
Thickness (mm): 2.20
Weight (g): 8.50

So 27 billion 2 euro coins are needed. So in weight that is 229.5 million kilos.
"The maximum gross mass for a 20 ft (6.1 m) dry cargo container is 30,480 kg" so that is 7540 containers full of 2 euro coins.

How long would the stack be? 2.2 * 27 billion millimeters = 59 400 kilometers. Or enough to go around the earth one and a half times. But less than a fifth the distance to the moon.

Ask the waiter

2009-09-15T02:34:00.000-07:00

If you want to know what someone is like ask their waiter. Because as a waiter you are far down the social rung anyone who is just a poseur treats you like dirt. Keith Floyd who died yesterday used to come into an Indian restaurant I worked in. He was much quieter then his media image and a complete gent. As was Liam Neeson BTW. I am not claiming to have known him just that he always treated the waiters well, which I believe is a good sign about someone.

Belinda Carlisle owes me a set of cutlery, she may think I have forgotten but I haven't. She promised she would bring them back and didn't. Try saying "if you cannot trust the singer of the Go-Go's, who can you trust?" to a middle aged Bangladeshi and see if your wages don't get docked.

Apart from an unfortunately forgetful singer I have a long list of who is a complete wanker. I'll post that again.

Prediction Market In Adwords?

2009-09-09T07:39:00.000-07:00

Can we predict how much an adword word will cost in the future? I find prediction markets fascinating but have not talked about them for a while. Here is an interesting game for predicting the popularity of words.

If this prediction market can accurately predict what words people will in the future find interesting and will discuss then you could buy up these adwords in advance and watch their market value (well cost in google) rise. So anyone want to buy a futures contract in the word "Cromulent"?

Fair division in googledocs using Ruby

2009-09-02T13:10:00.000-07:00

I figured out how to access google docs spreadsheets using Ruby's roo library.

So for the spreadsheet
https://spreadsheets.google.co/ccc?key=0AifEYKk1FmScdDUwcTE1c2c4aFF6Wl93cENkWVF1Smc&hl=en
#you can create a object for this document by

        
oo=Google.new("0AoBJp9mMcWUrdDZUY1diRm5KOTd4ZWtKWUpib2V3ZHc"
,user="stuffsplitter",password="thepassword")

where user and password are the username and its password for the account that has the document. 0AoBJp9mMcWUrdDZUY1diRm5KOTd4ZWtKWUpib2V3ZHc is the key
no http, no google.com, no key= and no &hl=en at the end. Good thing I am not an idiot who would take hours to figure this last one out...

Then this data is sent off to the GLPK program described ad nauseum previously.
Then using ruby's standard regular expression methods the solution file fair.sol can be easily parsed. This can be easily written to the googles doc using roo.


o2.default_sheet ='Sheet3'
#match fair.sol output file of glpk. 
  if line=~/^\s+\d+ y\[(\d+).(\d+)\]\s+\*\s+(\d+)/
#   2 x[1,1]       *              0             0             1
   if $3=='1'
   o2.set_value(row, col, $1)
   o2.set_value(row, col+1, $2)
   o2.set_value(row, col+2, $3)

Next thing is to get a web interface for people to add their own fair division problems. There are some guides for doing that here and here. So automated processing is the next task.

By the way if you do have a fair division problem you can mail it to stuffsplitter[at]gmail.com and I will run the processing for you.

Fair division using a spreadsheet and Ruby

2009-08-30T12:41:00.000-07:00

It has taken me a month to get over the idea that someone expected this blog not to be the ranting of an inane balding drunk but actually good. Still it is time to get back to the drunken inane rambling.

There is no point having all these cool algorithms unless people can actually use them. So what is a good way to actually get the data into the program? We'll spreadsheets are used for this sort of thing all the time. Unfortunately I cannot figure out how to get openoffice, excel or gnumeric's solver to carry out this minimax fair division program.

Ruby has a cool library called roo for interacting with spreadsheets. So heres a program to get the data out of a spreadsheet and send it off to GLPK


  require 'rubygems'
  require 'roo'
  
  #open office version
 # oo = Openoffice.new("spreadsheetOO.ods")
 #  o2 = Openoffice.new("spreadsheetOO.ods")
 
 #excell version
   oo = Excel.new("spreadsheetOO.xls")
   o2 = Excel.new("spreadsheetOO.xls")
  oo.default_sheet = oo.sheets.first

 o2.default_sheet ="Sheet2"
st="" #data being written toa file
div= o2.last_column


#start writing data to model file
st<< "data;\n"
st<< "param m := #{oo.last_row-1};\n"

   st<< "param n := #{div};\n"
 #the non divisible is whatever is left
    st<< "param o := #{oo.last_column};\n"
 
st<< "param c :"
 i=1
 (oo.last_column).times{st<< "#{i} " 
 i=i+1}
 st<< ":=\n"
 
# number of people
  i=1

 2.upto(oo.last_row) do |line|  
 st<< "#{i} "
 1.upto(oo.last_column) do |col|
    st<< "#{Integer oo.cell(line,col)} "#Integer
    
   end
   st<< "\n";
   i=i+1
 end
 st<< ";\n"
  st<< "param d :"
 i=1
 (o2.last_column).times{st<< "#{i} " 
 i=i+1}
  i=1
 st<< ":=\n"
  2.upto(o2.last_row) do |line|  
 st<< "#{i} "
 1.upto(o2.last_column) do |col|
    st<<"#{Integer o2.cell(line,col)} "
 end
   st<< "\n";
   i=i+1
 end
 
 
 st<< ";\nend;"

#write data to a file
File.open("data","w")do |file|
file.write(st)
end

#call glpk program on the data
system('./glpsol -m fair.mod -d data -o fair.sol');

The input spreadsheets are openoffice and excelin the first row is the list of items.
Every other row is each persons valuation for each item. The first sheet is the indivisible items and the second sheet are the divisible ones. Unfortunately roo does not allow you write to these spreadsheets so I cannot print out the answer to a spreadsheet.

The roo interface to the google docs spreadsheet does allow cells to be written to. So getting that version up and running is the next task.

Filling knapsacks with candies

2009-07-06T14:51:00.000-07:00

This paper by Marco Dall’Aglio and Raffaele Mosca on using linear programming (and dynamic programming)
it gives two examples of dividing non divisible goods between two people. The paper uses hard candies as a metaphor for any indivisible item.

The valuations are not based out of 100% but just plain numerical values.


candy 1 2 3 4 5 6 7 8 
Alice 12 18 50 40 20 20 10 5 
Bob 5 10 35 30 15 22 30 28

Which as data for glpk looks like this


data;
param m := 2;
param n := 0;
param o := 8;

param d : :=
1      
2      
;
param c : 1 2 3 4 5 6 7 8 :=
1 12 18 50 40 20 20 10 5
2 5 10 35 30 15 22 30 28
;end;

the worst person got 102 of what they wanted

Person alice gets item number 1 3 and 4
Person bob gets item number 2 5 6 7 8
This is the same allocation as the paper come up with.

The next example given is


candy 1 2 3 4 
Alice 32 28 22 18 
Bob 25 25 25 25

which is the data file


data;
param m := 2;
param n := 0;
param o := 4;

param d : :=
1      
2      
;

param c : 1 2 3 4 :=
1 32 28 22 18
2 25 25 25 25
;end;

Alice gets item 1 and 2
Bob gets item 3,4
The worst performing person gets 50.

To use these data files with the fair divider for two people (the code of which is here)
./glpsol -m fair2.mod -d data -o fair.sol

As pointed out in the comments this isn't earth shaking or anything it just shows another use of fair division which can be used when items cannot be divided.

Fair Division of an estate

2009-06-29T12:56:00.000-07:00

This page describes a fair division of an estate between 5 people. The process is described in the "method of Sealed Bids" section. It that requires each participant has money they can use to pay off the others. I think this is a fairly serious drawback as not everyone has loads of money. Also in the case of an estate division it seems slightly galling that the person with more money gets more things, possibly of only sentimental value, then others.

The division works by
"# Step 1. Bidding. Each player produces a sealed bid in which he or she attaches a dollar value to each item in S. A player's fair share is 1/N of his total assessment.

# Step 2. Allocation. Each item in S goes to the highest bidder for that item. If her/his assessed value of the items received exceeds her/his fair share, she/he must pay the difference. If the assessed value of the items received falls short of a fair share, then she/he is paid out of money that others have had to pay.

# Step 3. Dividing the Surplus There is almost always a surplus of cash that is divided equally among the players. "
given results in people getting


Al  BMW, Saab, Miro 
Ben  House  
Cal  Cottage 
Don  Klee
Ed  Yacht

And the minimum receive is $62,500 + $52,100+$18,340=132 940 out of a possible $679,000 or 19% of their desire.

If instead of assuming each player has money to buy stuff off the others I assumed each layer had equal claim to happiness. So I give each persons 100% desire is worth $679000 and those who claimed less then that use the remainder of their allocation as cash.

So this gives a table of desire of this


data;
param m := 5;
param n := 1;
 param o := 7;
 
 /* n divisible items, o non divisible items*/

/*Divorce between Donald and Ivana Trump described in Win-Win solution.
Items to be divided are 
4 divisible items:cash
2 non divisible items: House cottage, bmw,saab,yacht,miro, klee
Item   Al   Ben   Cal   Don   Ed
House  29.46 31.66 28.71 25.77 30.19
Cottage  8.84 7.22 9.2 8.76 8.1
BMW  4.27 3.6 3.68 4.06 4.06
Saab  3.68 2.8 3.33 3.6 2.87
Yacht  17.67 18.4 17.53 19 19.51
Miro  13.99 13.13 7.36 11.05 9.57
Klee  22.09 19.88 14.58 26.36 21.94
cash 0 3.31 15.61 1.4 3.76
*/

param d :  1  2  3  4 5 6 7 :=
      1   29.46 8.84 4.27 3.68 17.67 13.99 22.09
      2   31.66 7.22 3.6 2.8 18.4 13.13 19.88
      3   28.71 9.2 3.68 3.33 17.53 7.36 14.58
      4  25.77 8.76 4.06 3.6 19 11.05 26.36
      5   30.19 8.1 4.06 2.87 19.51 9.57 21.94; 
      
param c :  1 :=
      1   0
      2   3.31
      3   15.61
      4   1.4
      5   3.76; 
      
end;

Running this on the linear programming fair division calculator program(you need to add s.t. wor5{i in I}: ((sum{j in J} c[5,j] * x[5,j])+(sum{k in K} d[5,k] * y[5,k])) >= worst;). gives a the person who gets least 20.5388%


Al BMW Saab Miro 
Ben House 
Cal  0.726381 of the cash  Cottage  
Don Klee
Ed Yacht 0.273619 of the cash

which is an slight improvement of over on the initial allocation. Everyone gets the same items as in the initial allocation but the money is divided amongst those who receive the least rather then everyone.

Fair Division in Treaty Negotiations

2009-06-28T02:44:00.000-07:00

Raiffa in "the art and science of negotiation" describes the Panama Canal treaty negotiations in 1974. He got the value each country attached to each issue from the US negotiating team. In the fair division book chapter 5 describes this negotiation using their Adjusted winner algorithm. One weakness with this algorithm is that all items must be divisible.
There is no way to fix issues as non divisible as there is in my program (you will need to remove s.t. wor3 and s.t. wor4).
So if you take the data as


data;
param m := 2;
param n := 9;
param o := 1;
 
 /*m people,  n divisible items, o non divisible items*/
/*
1 non divisible items expansion routes
                  US Panama
US defense rights  22 9
use rights        22 15
Land and water   15 15
expansion rights  14 3
duration         11 15
expansion routes  6 5
compensation     4 11
jurisdiction      2 7
US military rights  2 7
defense role of Panama   2 13
*/

param c :  1  2  3  4 5 6 7 8 9 :=
      1  22 22 15 14 11 4 2 2 2
      2  9 15 15 3 15 11 7 7 13; 
      
param d :  1 :=
      1   6
      2   5; 
end;

where expansion routes is not divisible.

US gets item number us defense rights, use rights, expansion rights,expansion routes
Panama gets duration, expansion routes, compensation, jurisdiction and us military rights. For Land and water the US gets .1333 of this and Panama get 0.86667 which is ok as land and water are the sorts of things that can be divided up. Each country got 66% of what they wanted, In this case my program gets the same results as the Adjusted winner program.

A better fair division program

2009-06-25T04:56:00.000-07:00

One problem with the wildly unpopular fair division programs discussed previously is that they assume either everything can be cut into pieces or nothing can.
In the real world as Solomon noticed some items cannot be divided up. But some items are easily splittable . So I have improved the fair division program to deal with combined divisible and indivisible items.

Another problem is that the code only dealt with the two person case. I have fixed this by adding a new constraint for each extra person.

The code for this Mathprog program fair.mod is here. I am providing the source code in case anyone can suggest improvements. Also the world is full of decisions that get made for us that we have no way to challenge or understand. So I would like people to be able to see why the reasoning in and results of an arbitration,

The program (glpk) is run by

Put this data in a data file. Call glpk on it.
./glpsol -m fair.mod -d data -o fair.sol
So the data file could be

data;
param m := 4;
param n := 4;
param o := 2;
/*m people, n divisible items, o non divisible items*/

/*
Items to be divided are
4 divisible items: cash, land, cake, gold
2 non divisible items: dog, table
*/

param c : 1 2 3 4 :=
1 20 30 0 20
2 30 20 0 20
3 20 30 0 20
4 0 10 40 40;

param d : 1 2 :=
1 10 20
2 20 10
3 20 10
4 10 0;
end;

Which means person 1 thinks 20% of the value to them in the estate is in the cash, 10% is in the dog etc.

So now you can divide up goods fairly between m people. I have never seen a program that does this before. Using this program the that parties in a will, divorce, international dispute, merger and all sorts of other areas can achieve optimal solutions. The solutions will be proportional, envy-free (in the two person case) and Pareto optimal.

Now to make the dispute resolution program more usable.

Fair Division of Advertising

2009-05-26T12:57:00.000-07:00

Wired today had a really interesting article on how Google uses auctions to sell advertisements here.

it has spawned—dozens of millionaire geeks, billions of auctions, and new ground rules for businesses in a data-driven society.

There is a great paper here(pdf) on the game theory of these online auctions.

But what other mechanisms are there to distribute out advertisements? One way is to use voting which I will describe again. Another method is the algorithmic fair division explained in my last post.

The glpk program given can be used to divide desired ad words between two bidders. You can add more bidders by adding constraints and people. There is an advantage to using this linear programming system of adword selling. It is not designed to maximise earnings but fairness. The procedure guarantees certain criteria described here. One advantage of these criteria is that the least unhappy buyer will be as happy as possible. As Amarillo Slim said "You can shear a sheep many times, but you can skin him only once." If you can keep each party as happy as possible you might maximise revenue over time rather then just for each particular auction. In the given program each party is assumed to have the same resources, 100, a fair division linear program for most applications would give each party a different level of resources.

Dinosaurs are interested in fair division

The book on fair division is this one

So using the previous code but with different data.


/*
Coke and Pepsi want to buy advertisements. They each have 100 dollars to spend.
The adwords they desire are 
1. Refreshing drink 2. Black Fizzy water 
3. cola 4. pop
param c gives how much each company desires an ad on 
each  of these words. Minimax guarantees that
 the company that does worst gets 
the most ads on the topics they want possible.
*/
param c :  1  2  3  4 :=
      1   10 40 10 40
      2   40 20 30 10 ;
end;