Crowdsourcing Taxonomy

I had an interesting debate on twitter with Panos Ipeirotis and Anand Kulkarni about what counts as a crowdsourcing application.

Wikipeidia defines crowdsourcing as
"Crowdsourcing is the act of outsourcing tasks, traditionally performed by an employee or contractor, to an undefined, large group of people or community (a crowd), through an open call."
And they are even so nice as to give a list of crowdsourcing projects


These seem to fall into a few areas
1. Microwork. MTurk, Freelancer.com, 99designs. You go on mturk you put in a task like "tell me if this sentence is in English" and an amount "1 cent per task" and then you let the people on mturk do the classifications for you.

I think this sort of work will be huge. There are talented people sitting around willing to do something interesting for cash in their spare time just waiting to be harnessed.

Many projects exist to crowdsource advertising ideas. for example IdeaBounty. These advertising ideas seem similar to other microwork projects.

2. Games with a purpose. Galaxy finder, Foldit etc. There is a task like "count the number of spiral galaxies" you put this out there as a 'fun' task and people do it for you free, gratis and for nothing.

This is similar to citizen science projects but explicitly involves the use of fun and game mechanics to get people to do the work.

3. Market mechanisms could be thought of as a form of collective intelligence. In a market the competing forces of supply and demand result on a value being placed on an item. So for example on ebay many people have decided the value of collectible baseball cards. Or on intrade many have decided the probability Obama is to be reelected.

Is market formation a form of crowdsourcing? I think prediction markets are great and should be used more but their placing accurate values on the probability of an event is a byproduct of their action rather than the intention of those operating on it. People would rather if the odds were wrong so they could make money being right.

For some reason I don't think a prediction market is a crowdsourcing application and none are listed on the wikipedia list.

As an aside I read a brilliant blog post two days ago (that I now cannot find) that pointed out that ebay is essentially a storage device. You sell things they go into the ebay and when you want them back you go on the ebay and find the product again and buy it. Your storage costs are usually just the postage and packing charges. This is similar to the argument that there are two ways to produce cars. One is to build factories and hire people and such the other is to send boat loads of cows to Tokyo where magically you get boats filled with cars back in return.

4. Microlending
Kiva and other sites allow you to lend money to business. Say a guy want to plough my field. 10 people give you 5 dollars each so you can buy a plough. The field gets ploughed. They get 6 dollars back after harvest. This again does not seem like crowdsourcing of jobs as you are lending capital not labour.

Artists using a street performer protocol to pay to get an album made is something similar. Kickstarter is one site that does this

5. Wikies, Forums, social search engines, message boards
People on Forums, give their advise and knowledge almost always for free. It is pretty amazing when you think about it. The fact that some very clever and skilled individual is willing to take their time to tell me the arguments to a Java function are wrong is amazing. The wikipedia page does not class these as crowdsourcing but again I cannot see why not.

6. Competitions, kaggle, Netflix, DARPA, Goldcorp and InnoCentive regularly have prizes where they put out a task such as 'improve on our recomendations' or 'make a car that drives itself' and people try to do the best they can at it. I think prizes as a way to encourage innovation are also going to be massively important in the future.

7. Response to events. The guardian set up a webpage to get people to read through all the mp expenses reports to find things that looked excessive and wikileaks have used crowdsourcing to examine the millions of leaked documents and highlight the most interesting.

8. Crime detection. Crowdsourcing has been used for illegal immigrant spotting in Texas and other similar uses.

9. Missing persons. After hurricane Katrina and Steve Fossett's loss crowdsourcing was used to locate missing people.

10. Politics. Oxfam Novib, Moveon.org and other such sites try to use crowdsourcing to create a community based political movement. If these are counted as crowdsourcing ventures I do not see why hobby sites that try promote brewing or knitting are not.

11. Art. Improv everywhere, mechanical olympics, Flash mobs are all listed as crowdsourcing projects.

12. Citizen science. Digitized versions of old weather or other science reports are made available to people to increase our knowledge of past events. Even the open data movement must have some basis in crowdsourcing. If people are not expected to analyse the data there is not much point making it availible to them.

13 Cartography. Open street map, Waze and other projects hope to use peoples GPS systems to build up a map of the environment. This can include real time traffic data.


Reading through wikipedias list of crowdsourcing projects these are the types of projects that occured to me. Most classes are probably wrong and need to be split or combined. Maybe the hierarchy needs to be different. Please comment or write a rebuttal to if you are interested in what projects count as crowdsourcing.

What is the worst thing to know?

Knowledge is rarely a good thing but sometimes it actively reduces your quality of life. Take coffee as an example. Coffee is an accepted addiction, you cannot go around going 'Sorry I am not with it yet, I have not had my Vodka' but the same excuse for coffee is nearly mandatory as so many of us need coffee to function.

Some strange people learn about coffee though and as a result they no longer can cope with Nescafe instant. Next they notice burnt flavours and incorrect coloration in coffee shop coffees and an epic impossible quest for the 'God Shot'. Expresso starts to ruin their lives.

If you are addicted to a drug becoming snobby about that drug is a really bad idea. You don't see crackheads demanding only the finest Columbian devils dandruff in their rocks. Yet coffee addicts get themselves addicted to a drug then try and make sure they are miserable almost all the time they take the drug.

A normal coffee drinker gets a mild amount of pleasure from instant and when they go to a coffee shop get a fair amount. Someone who has gone to the trouble to learn about coffee lives in a constant state of disappointment except the rare occasions the world champion barista they have taken two hours to go to does not do something almost imperceptibly wrong.

The worst thing to know about I think is fonts. I don't even notice fonts but I have friends who can spot the bad font in a street with 100 other signs. You never hear them happy about fonts all they seem to bring is pain. The entire world just seems to become an annoyance to someone who knows how text should look.

America Is F*cked.......(Graphically at least) from Jess Gibson on Vimeo.

Do you have any nominations for things you are really better off not knowing?

Which Keanu Film

Bizarrely unaging Keanu Reeves. Stick the answers below in here if you cannot name the film
Ovyy naq Grqf
Gur Qrivyf Nqibpngr
Wbuaal Zarhzbavp
Zl Bja Cevingr Vqnub
Punva Ernpgvba
Gur Zngevk
Guhzofhpxre
Pbafgnagvar
N Fpnaare Qnexyl
Cneragubbq
Fcrrq
Cbvag Oernx

Name the Schwarzenegger Film

Past below into rot13 to get the answers
Grezvangbe
Ongzna
Urephyrf va Arj Lbex
Gur Ynfg Npgvba Ureb
Cerqngbe
Pbzznaqb
Gehr Yvrf
Raq bs Qnlf
Gbgny Erpnyy
Ehaavat Zna
Renfre

The Travesty of Goats in Trees

This year I have been happy whiling away the months to the steady and lofty beat of the goats in trees calender

Every month brought a new sylvestral capric delight. So hoping for another fun year of arboreal ungulates i looked up the 2011 calendar. However I was shocked when I looked at this years chronology of tree based mammals.

See the fun loving soaring tribes? Now see this years calendar.

July, April and February are repeats. December does not even have any goats in a tree. The trip of goats in the photo are entirely ground based, what a sham.

I would not lightly give out about the folk at the august Brown Trout publishers but this shoddy display of could lead to dangerous Bovidaen Déjà vu.

Which means I am stuck without a calendar for next year. Any recommendations?

TIL 4

Odd:Trained monkeys to guard athletes

Music: John Cale - All My Friends - (LCD Soundsystem cover)

Workplace Salaries: At Last, Women on Top

Science:Why squirrels masturbate

Economics:You can see futures in your weight

Funny:Do physicists believe in God?

Til 3

Design: Airports should let you know your going to die

Music:Brian Eno - By This River

Word: Saturnine meaning sluggish in temperament; gloomy; taciturn

Nerd: Grace Hopper was cooler then i imagined

Tech:Billboards can recognise age and gender

Design: Pizza Boxes can be better

Science: Pierre de Maricourt started science by showing magnetic action at a distance which contradicted Aristotle.

A failed Mturk translation test

The Mturk is Amazon's platform where you can put up jobs for people to do. These tend to be things like translation or image categorisation that humans are good at but computers are not.

According to Panos Ipeirotis' research mturk workers seem to be highly educated. The majority having degrees and about 10% having postgraduate degrees. Much mturk work is boring and probably does not use the skills these well educated workers have.

A few days ago google announced sponsorship for projects they think are carrying out good work. One winner was

"The Khan Academy will receive $2 million toward funding its work on the "make educational content available online for free" theme. The academy does just that, with a library of over 1,800 videos with lessons on math, science, finance, and history.

Bill Gates is a big fan of the Khan academy "This guy is amazing," he wrote. "It is awesome how much he has done with very little in the way of resources."
.

I am also a huge fan of the Khan academy I think that these videos and other online education videos such as MIT's online courses have amazing potential to transform the education of millions of people.

How far could googles 2 million grant stretch? Obviously it is up to Mr Khan to decide how to use his resources but I thought it would be interesting to see if the mturk could be used to translate one of his mathematics videos. The languages with over 100 million speakers are Mandarin, Spanish, English, Hindi, Bengali, Cantonese, Arabic,Portuguese, Russian and Japanese. 10 languages for 2000 videos would be 20,000 video translations. If each video cost 100 dollars to translate that would spend the 2 million dollars google donated. If this cost can be reduced more languages could be added.

A mathematics video is not something the average person can translate. However we know a large number of turkers have degrees and many are from India where they would likely have an understanding of English and Hindi or Bengali.

I tried an experiment to see if I could get one of Khan's videos translated into Hindi using the mturk at a cost of 5 dollars. Unfortunately I failed. The first person who accepted the task dictated what Khan said into text. Which is useful but not what I was looking for. The second person posted up another video on Solving linear inequalities in English not a translation of Khan's video.



This small experiment tells me that you need to be very clear on the mturk how you ask for a task to be completed. It also says that it might be worthwhile once you find someone who understands and completes the task to encourage them to translate other videos rather than rely on the vagaries of who happens to accept your mturk task.

This experiment ignored the problem of copyright. Khan owns his videos and it is unfair for someone to come along and copy him. I was not trying to steal any glory from Mr Khan with this experiment just to see if the mturk could be used to translate his videos.

Other people have successfuly used mturk to reduce the cost of translations. 'How I reduced translation costs of 200 articles from $9000 to $46' is an interesting article on one successful usage. This tells me that the problem was more likely with my unclear instructions than with the mturk platform. You can even monitor translations taking place in the mturk here so I still think this method would be cost effective. However my simple experiment failed.

TIL 2

Music: I have had this song stuck in my head for ten years after hearing it once. I finally figured out the name. The Swains - Numberless As The Sands On The Seashore

Science: What happens if you put your hand into the LHC stream?. I also like that bra is a word <

History: You could get excommunicated for believing that god could not create a four dimensional universe or even mulitple universes due to the 297 condemnations of 1277. This is pretty odd to think guys in 1277 were being condemned for things physicists get condemned for today.

Film: The last exorcist is a good horror film. Full of ambiguities and non obvious twists. The wikipedia review is fair "It doesn't fully deliver on the chilly promise of its Blair Witch-style premise, but The Last Exorcism offers a surprising number of clever thrills."

Beer: Homebrew should do more better videos like this one

Maths: Drop a printed map of the land on the land and there must be some common point.

Word of the day: Muckefuck means bad coffee particularly substitute coffee

TIL 1

Shampoo does not make any sense (Kaye effect)

Medieval love letters mainly involved explanations of previous sexual assaults
"Even when you were unwilling, resisted as much as you could, and tried to dissuade me, since your nature was weaker I often forced you to consent with threats and blows"

Iron and Wine do a great version of "waiting for superman"

There are some fun retro howto documentaries on burning people to death on an industrialised scale

Malware attacks have moved onto nuclear programs

You can get the illicit thrill of buying drugs through sandwiches

Other people dislike Diageo

Word of the day: Götterdämmerung

The Bayeux Tapestry is a Swashbuckling Adventure

I went to see the Bayeux tapestry a few days ago. If you are ever in Northern France go see it. It is an amazing experience to see a thousand year old hand made item, particularly one that tells a story. Seeing the individual threads and mistakes of something that old is just breathtaking.



But one thing that struck me about it is how fun the story is. I imagined it either as a dry scolarly repetition of the story or an ancient piece of propaganda. Instead it is more of a swashbuckling adventure. Compare one swashbuckling film 'The Princess Bride' with the tapestry and see the similarity. They both have

Kidnapping
Harold gets kidnapped when he lands in France




Quicksand
Harold rescues two men from Quicksand



I really miss quicksand it used to be everywhere "nearly 3 percent of the films in that era (the 60s) showed someone sinking in mud or sand or oozing clay". But you just don't see it anymore. It has sadly gone the way of grappling hooks and that using a knife to slide down a sail trick.


Abseiling
A guy they are trying to catch uses a rope to escape.




Forced Marriage
Williams daughter is married by herself with a priest slapping her.


The princess bride marriage is also forced

God's Hand
Monty Python style God's hand comes from the clouds.


Which is a bit like having a miracle man around

Sword Fights




And the tapestry has all sorts of other stuff you see in matinee adventures like horse chases, giant animals, oaths, dragons and divine heavenly light


In Table form you can see the genre cliches


Obviously the Bayeux Tapestry served many purposes at the time. But I think we might underestimate one of those was entertainment. You could remake this story now as 'Indian Jones and the Norman Conquests', it is already in comic book form.

Thoughts on France

Some things I noticed on a recent trip to France. None are earth shattering.

1. The Rise of the robots. The Hotels, McDonalds the Petrol Stations are all computer booths.


I can think of a few reasons for this. The French all seem to have credit cards so that makes it easier to have card run business. It also means that most shops look at you funny if you try pay with a 50 euro note.

I have heard French labour laws make hiring/firing people difficult so maybe using machines is efficient in a high labour cost country.

Eventually the same effects will happen here and many of these sorts of jobs will be automated in the next few years.

2. Desperado beer is inexplicably popular given how much good booze France has.


You see aisles of the stuff, people in queues have six packs of it. The only explanation I can think of is that wine has become stronger recently leaving a gap in the market for stronger beer. still this stuff is only 5.8% so though strongish it does not fill in the 8% light wine gap.



3. Sophie le giraffe. It is like a secret society. Parents around Ireland have been sneaking in this teething aid. Some friends smuggled back 4 for their friends. I was aware of the child minding powers of savanna ungulates. They have a series of creches here but whoever starts importing these into our homes in Ireland will make a few quid


4. "All our beef is French and Irish" Most restaurants had a sign like this displayed. This was the only non local food that was advertised as such. Our farmers tend to come across as EU sponging yokels but if Irish beef is the one food that is displayed as not French they must be doing something right.

5. An active lifestyle is encouraged which is good. There was via ferrata all over the area of the alps I was in and phone numbers to check canyoning conditions for particular rivers. I presume the French avoid getting constantly sued for this sort of fun.


Basically France is a great spot for your holidays.

Cat Bubbles

For something to be a bubble its price probably has to increase faster than inflation. People have to believe that other people believe it will increase in price forever. This is the madness of crowds effect like tulip mania in the 1600's where the price of Tulips rose to ridiculous levels.

I have heard arguments recently about higher education in the US being a bubble. But what about Pets? Animal based bubbles have a long and fun history. For example Alpaca's and Ostriches twice have been bubble commodities.

There is a really interesting article on how spending has changed during the recession here. It presents this table



From these figures could pets also be a bubble? The cat has hardly changed in thousands of years. Certainly not enough of a recent efficiency increase to make it worth a 14.4% growth in costs. So keep an eye on the costs of pet ownership especially of the fancier breeds to see if bubble thinking comes along.

Evacuating the Love Parade

An important issue for crowd safety is how long it would take to evacuate a venue. Like the last article this is not to attempting to figure out the exact right answer to how large entrance/exits need to be but to try see if someone with a limited knowledge and a search engine could easily see in advance that the entrance/exit to the love parade was dangerous. Even large venues can require evacuation in the case of a natural or manmade disaster. These disasters could be weather related as happened here here and here. In the case of man made safety risks venues also are sometimes evacuated. As described here
"On December 12, 2004, the stadium Santiago Bernabeu in Madrid was evacuated because of a bomb threat.... In eight minutes, more than 70,000 people left the premises without incident."

The respected Spiegel says

"Much of the critique has centered around the fact that the tunnel where the panic took place was the only entrance to and the only exit from the party site. "
If this tunnel was the only exit available (another tunnel seems to have been reserved for emergency vehicles) how quickly should people have been able to exit?

This soccer stadium with a capacity of over 25,000 claims 'The target evacuation time for the entire Stadium at full capacity is 8 minutes.'

This document on evacuation planning states
"The maximum Emergency Evacuation Time for sports ground varies between two and a half minutes and eight minutes"

Stadiums do not have as much free space as larger venues but they still allow for moving people onto the pitch as happened at a baseball game after the 1989 San Francisco earthquake. In the case of some natural disasters or terrorist threats a site wide evacuation might be needed so even large sites probably need to be able to be evacuated reasonably quickly.

The acceptable evacuation time for a stadium seems to be under ten minutes. If Spiegel is correct and this tunnel really was the only exit any evacuation would have taken hours which is not be acceptable.

The pictures here give a very good impression of what happened. The width of the tunnel does indeed look like 16 meters. And the 'ramp' described earlier just looks like something people tried to escape up rather than a designated route.

How many people can pass through a tunnel?

A tragedy happened at the love parade when 19 people lost their lives. This will be investigated by professionals who will come out with detailed analysis on how this incident took place. I want to see what someone in half an hours worth of searching can say about how wide a tunnel you need to fit that number of people.

I worked as a volunteer at the special Olympics some years ago. The job mainly involved making sure spectators could get into and out of the events easily, that emergency services can easily gain access and that spectators and athletes were kept separate. The venues were not huge and the spectators were happy and sober. Still I got some impression of the kinds of things needed to ensure crowd safety at events.

I want to put a figure on how many people could safely fit through the tunnel that was the sole access point to the love parade venue. A 16 meter width is given by most media outlets.

But some say it is 30 meters wide. Most seem to put the number of people there at 1.4 million but others at only 500,000 'an event set up for 250,000 ended up with an estimated 500,000 to 1 million'.

How long does it take to get everyone into a venue? How many people an hour will try enter a venue? I do not think all 1.4m people try enter in an hour. But I would guess most people try to enter a venue in a three hour period. Each hour you could expect over 300,000 people to try and enter through the tunnel. People are unpredictable meaning that you would need to be able to have more than this safely in case loads turn up at the same time.

Who else deals with crowd volumes like this? Every few years during the Muslim Hajj there is a crowd related accident. They are building a new bridge to improve safety. There is a description of this bridge here (pdf).

the new, multi-level bridge structure which will accommodate and ease the flow of 3 million worshippers during a single daylight period. The proposed new Jamarat bridge is a superior structure formed of 4 platform levels... each of the bridge’s 4 floors is roughly 600 m-long, with variable widths (ranging from 60 m to 97 m

The engineers describe these 4 floors as at least 60 meters wide. Each 60 meters in width is supposed to handle .75 million people if scaled linearly to 16 meters that would be 200 thousand people during the day. The new Saudi bridge is designed with a much larger channel to move people than they had in Germany. Pilgrims may act differently to other crowds as they stop to perform religious ceremonies.

Another terrible crowd incident was the Hillsborough disaster where 96 people died. A quick search about football crowd safety found this document. It is a very interesting article and well worth a read

The document states
"the safety limit for crowd density is defined as 40 people in 10 square metres for a moving crowd" the tunnel at 16m wide and 100 meters long should only hold 6400 people. Walking at 38 meters per minute. 100 meters would mean it would take 156 seconds to get through the tunnel. 6400 people every 156 seconds is 147,000 in an hour. This means it would take ten hours for 1.4 million people to safely travel through.

Reuters here states
"Authorities have not yet been able to explain how exactly the tragedy happened -- near a tunnel that led to a ramp into the festival grounds. Most of the victims were found dead on the ramp and none in the tunnel, authorities said"

The football document also in Section '2.8 Wembley Complex Station' gives this equation
No. of Units of Exit Width = Number of Persons (1)
required (each width = 0.55m) Flow Rate (2) x Evacuation Time (3)

"Where number of persons (1) means the maximum number of people that could be expected to be on a platform at any time. Flow rate (2) means 40 persons per minute for escape routes incorporating stairs, and 60 persons per minute for level escape routes (without stairs). Passenger walking speeds should be assumed to be 38 metres per minute for horizontal circulation."

Using the Wembley equation above 16 meters *.55 meters means 29 people can safely span across the tunnel. 40 flow past a minute if the escape route has a stairs and 60 if it is flat. The ramp sounds like it would hinder flow in a similar way to a stairs.

Once again this is all back of the envelope and as more facts come out a much better estimate of how many people could safely negotiate the site will emerge. A reasonable estimate of the number of people who could travel through the tunnel in an hour is (tunnel width/person width)*number or people per minute * 60 minutes
which in a 16 meter wide tunnel with a stairs at the end is (16/.55)*40*60=70,000
Best case scenario with a wider tunnel and no stairs 30 meters/.55 meters *60 people *60=200,000 which is less than the expected number of people.

This calculation indicates that a cursory look at at the venue entrance would give someone grave concerns about safety for a smaller crowd than turned up on the day.

Science grows 2.5% annually why doesn't the economy?

What causes economic growth? Much of the world still lives in grinding poverty, figuring out a way to get them out of this is important. The best way seems to be for their economy to grow.

There are many theories as to what causes economic growth. The founder of Google claims
"Virtually all economic growth (in the world) was due to technological progress. I think as a society we're not really paying attention to that," Page said. "Science has a real marketing problem. If all the growth in world is due to science and technology and no one pays attention to you, then you have a serious marketing problem."

If technological progress causes growth there should be a link between the amount of scientific progress and the amount of growth. Here is a very interesting article about the rate of the increase in scientific knowledge.

'What I found, using this simple proxy for difficulty, in each field — biology, astronomy, chemistry — was a curve with the same basic shape. In every case, the ease of discovery went down, and in every case it was a curve called an exponential decay ...[I] discovered asteroids get 2.5 percent smaller each year. So while the ease of discovery drops off quickly as early researchers pick the low-hanging fruit, it can continue to “decay” a long time, becoming slightly harder without ever quite becoming impossible"

Say science increases in the level of knowledge by 2.5% a year.

What is the rate of economic growth? 'Since 1820, world development has been much more dynamic. Per capita income rose more than eightfold, population more than fivefold.' 190 years growth 8 fold increase is a growth rate of about 1.1% per annum. Or for two specific countries the UK from 1830 to 2008 had average real GDP 1.97%. The US 1830 to 2009 real GDP of 3.62%.

Other things cause economic growth

-Demographics more people of working age means more stuff gets made.

-Institutions. The rule of law matters. If people think anything the build will be plundered off them they wont build much.

-Governments. Some ways of running a country seem to result in more economic growth than others. Communism was never that good at creating economic growth for example.

-Taxation. At a certain point if you tax people too much they wont work anymore. If you dont tax people enough you might not be able to afford schools. Knowledge of what level of taxation is beneficial to economic growth should improve over time.

-Financial innovation. By getting capital to the people who best use it financial innovation is supposed to help economic growth. If it doesn't we should not spend money bailing out investment banks.

-Foreign aid: Aid to developing countries seems to have a mild positive effect on their economic growth rate though in specific areas it seems to have a negative effect

What else can affect economic growth and how great an effect does it have?

How scared should you be of nut allergies?

The wife today was selling nuts and parents kept telling her that nuts were banned in their kids school to prevent deaths from nut allergies. How many life years are saved from banning nuts in schools versus those lost through not eating nuts?

I will look at American figures as they are available and have a large sample, which are both handy. "about 150 people die annually from serious allergic food reactions" which compares with "2,000 children drown each year" according to here. So 150 people out of the 2.5 million who die each year in America die from some food allergy. Imagine all these people had 80 years left to live. You have lost 1200 human years.

Now if you happen to be the one with the allergy you would want some reasonable precautions taken. My argument here is a guesstimate on how the over anxiety about how nuts might affect us.

“We try to relieve anxiety about nut allergy by signs saying, ‘this is a nut free zone,’ which suggests that nuts are a clear and present danger,” Dr. Christakis said. “But in doing so, we increase the anxiety.” So imagine this sort of fear of nuts made everyone stop eating nuts. How many life years would be lost? "those eating nuts daily had up to 60% fewer heart attacks than those who ate nuts less than once per month". Given four hundred and fifty thousand people die each year from heart attacks. Preventing 60% of these would be 270 000 lives. Say you only saved one year off each of these that is 270000 life years as opposed to 1200 from banning nuts. This 60% figure seems really high but other studies show massive improvements here and here says '1 ounce of nuts more than 5 times/week can result in a 25 to 39 percent reduction in coronary heart disease risk among people whose characteristics match those of the general adult U.S. population'.

Now just because people who eat nuts don't get heart attacks does not mean the nuts stop the heart attacks. But the googleable studies take this correlation versus causation problem into account. Also banning nuts in schools and telling kids they might kill people wont stop all nuts being eaten. But its not hard to imagine these warnings will severely curtail nut eating.

Not many kids have nut allergies. So a blanket ban without good reason wont save many lives. There is good evidence eating nuts is very good for your health. It is reasonable to assume telling kids nuts could kill them wont encourage them to eat them.

I think this nut fear illustrates a problem many of us have with comparing sudden risks with long term risks. For example when schools ban running by kids because they might be sued but they ignore the long term health effects of kids not exercising. There are risks everywhere but when someone wants to minimise one it is worth asking how dangerous that risk is and what consequences minimising it will have.

Eurovision Voting Fraud

Forget hanging chads in Florida. Some really important vote rigging is about to happen.

I was investigating the Eurovision Song Contest last week as a prediction problem. No really it is an area of major research into how corruption agreements implicitly develop between countries. There are papers here, here and here.

But then I saw the odds for this year and there is something really odd happening. The odds from Paddy Power are here
Azerbaijan 11/8
Germany 4/1
Israel 9/1
So Azerbaijan a country that no one can spell is hot favorite. Which would not be too suspicious if they had ever had a song that wasn't entirely made up of goats being strangled before. The singer Safura and her Drip Drop has her obvious talents on display but is there something odd going on here?

Wikipedia says

After the conclusion of 2009 Eurovision, it was reported in Belarusian media that unknown people had allegedly paid and bribed students from Belarus to vote for Azerbaijan in the contest, and had even transported them to the Belarusian-Lithuanian border in 10 buses. The cost of this alleged action was claimed to be about 55 million rubles. They purportedly voted for Azerbaijan using Belarusian SIM cards, and then received Lithuanian cards to repeat the process. The Azerbaijani entry "Always" received 10 points from Belarus and 5 points from Lithuania.

So why does an old soviet country, with a pretty dodgy political system, that is 95% Muslim want so badly to win the worlds gayest contest? Turns out its a giant fight with Armenia. There is a war going on between them using the Eurovision as a battleground. I cannot wait till they get Johnny Logan in as a peace negotiator.

The Azerbaijan state broadcaster blocked out the voting number for the 2009 Armenian entry like some corporate logos in a rap video. Then and you can only read this in a full Borat accent "in August 2009, a number of Azerbaijanis who had voted for Armenia's entry during the 2009 Contest were summoned for questioning at the Ministry of National Security in Baku, during which they were accused of being "unpatriotic" and "a potential security threat". One of those summoned, Rovshan Nasirli, said that he was told by his interrogators that they had the names and addresses of all 43 Azerbaijanis who had voted for Armenia".

What is the cause of all this? According to the over 2000 word wiki article on the "Armenia Azerbaijan relations in the Eurovision Song Contest" it is Nagorno-Karabakh an area they both claim.

So it looks like Azerbaijan are willing to buy masses of fake simcards, bus people to other countries, bribe students and track down a few dozen people who voted the 'wrong way'. I wonder what is going to happen this time now that they seem to take the eurovision seriously?

Craig Venter creates synthetic life form

There are stories about this everywhere. But one quote from wired sticks out

“Over the last five years the field has seen a 100-fold increase in the length of genetic material wholly constructed from raw chemicals,” said synthetic biologist Drew Endy of Stanford University. “This is over six doublings in the max length of a genome that can be constructed.”

Doubling in abilities in less than a year is amazing. Pretty much any exponential improvement like that ends up being a big deal if it carries on for a reasonable length of time. This is 1 million base pairs of synthetic DNA. A human is 3 billion base pairs. So at this growth rate that is less than 12 years. Not that you would artificially create a human or anything but it gives you some impression of how rapidly this technology could develop.

NO DEPOSIT...NO WORRIES

If we do not learn our lesson from the property bubble and collapse we will wind up back in the same trouble again.

There is a semi ghost estate on my bus route. It is beside the M50 beside Finglas and an area called Scribblestown in murder news reports. The estate is opposite and beside a halting site.

This Heathfield estate seems less than half full. This week I noticed the sign they have flogging houses had a new banner on it. "NO DEPOSIT...NO WORRIES"


We have learned nothing.

We Hate You. Please Save Us

There is an episode of the Simpson's where Springfield unites against Lisa Simpson who questions the veracity of an angel skeleton. In a very short time, the citizens launch a mob attack against the town's scientific institutions, Moe decides to destroy the skeleton of a woolly mammoth at the Springfield Natural History Museum, yelling "Take that, science!" Predictably, the tusk falls off, crushing him. "Oof, I'm paralyzed," observes Moe. "I only hope medical science can cure me."

Ireland will be saved by science. Here is a document from the Taoiseach about the new smart economy.

One main idea is to "invest heavily in research and development, incentivise multinational companies to locate more R&D capacity in Ireland, and ensure the commercialisation and retaining of ideas that flow from that investment;"

So how does Ireland treat science? Well take the LHC launched fully today. There is a great article here about the disdain this project is meet with by our government. We won't spend the 10 million annually that would make us a part of the project.

Not that no Irish people are helping out but they have to work with foreign research groups if they want to help figure out how the universe works.
“Nobody knows what is going to happen, that is why we are doing it,” stated Dr Steve Myers, the Irish physicist in charge of the largest atom smasher ever built.

So no 10 million annually out of a total of 6.5 billion it will cost to work out how the nature of the universe.

Irish scientists are not valued either. There is a poll of the greatest Irishmen of all time. There is no scientist on the list. Dr Noel Browne is the closest but he is there as a politician. No Walton, Robert Boyle or William Rowan Hamilton.

But you do have Joe Dolan, Colin Farrell, Stephen Gately, Ronan Keating, Daniel O'Donnell and Louis Walsh.


We have space for Daniel O'Donnell but not Nobel prize winner Ernest Walton. We don't have 10 million for CERN but we do have 32000 million for the governments friends.

What can you say? I'll leave it to another Nobel prizewinner

What need you, being come to sense,
But fumble in a greasy till
And add the halfpence to the pence
And prayer to shivering prayer, until
You have dried the marrow from the bone?
For men were born to pray and save:
Romantic Ireland's dead and gone,

September 1913
William Butler Yeats

Influential Books

Marginal revolution has an article on the 'books that influenced the me most'. There is a follow up article of others lists here.

No one in these lists seems to mention the Bible. And few the Odyssey and Iliad. Aesop and the brothers Grimm also seem to be ignored. Hansel and Gretel, Rapunzel, little red riding hood, Cinderella, Sleeping Beauty, Snow White, Rumpelstiltskin all come from one book. They are pretty much ubiquitious in the lives of children of my age even if they did not read the original.

Is it naive to think that these stories we heard as children probably influence our thinking far more than books read after our teenage years?

I seriously doubt there are many westeners who don't consider the three little pigs as a moral lesson more often than those from anarchy the state and utopia.

The most embarassing book most of the list writers will admit to is something by Ayn Rand which they now disown. But why is Atlas Shrugged allowable on the road to where they are but Rip Van Winkle and the works of Hans Christian Andersen's are ignored?

By most influential do we mean the book your willing to admit to that you think about while debating people? Rather than the ones that are so ingrained in our lives that they serve as a backbone to how society operates.

How much land do you drink?

Say you drink 35 liters a year of alcohol. 14 liters is the rather high amount the average Irish person does. There is a podcast here where Saul Griffith describes how due to carbon rationing in the future you will have to decide between a shower and a glass of wine each day. So the question of how much effort goes into each brew is important for the future where the amount of land and energy we consume may be heavily taxed and rationed.

What area of land do you need of a crop of apples, barley, wheat, potatoes, grapes or sugar to keep someone in booze for a year? There is a table here of the yield in terms of (kcal/sq m/year) each crop produces

Cane sugar 3500
Beet sugar 1990
Potatoes 1600
Apples 1500
Wheat, cereal 810
Corn 250
Milk 420
Beef 130

So in terms of calories per square meter Cane sugar seems to win. You do have to calculate how efficiently the calories in the crop get into alcohol in the booze you drink.

This guy tries get the alcohol out of sugar beat. "so the theoretical yield of ethanol produced is around 50%alc per weight of sugar"

Each square meter is 2000 kcal according to the earlier table. A kilo of sugar is 4000 kcal. So for 14 liters of alcohol is 28 kilos of sugar. 14 liters is what the Irish person drinks annually. So that is 56 square meters of sugar beat to keep you in (bad) booze for the year.

Now for cider.Say scrumpy is 6.5% abv then one liter of scrumpy is 65 ml of alcohol. So 14 liters is 215 liters of cider. Is that right? Does the average Irish person really drink the equivalant of a pint of scrumpy every day?

Anyway I want about 200 liters of cider which is about 400 kilos of apples. How much area of orchard do I need to get that many apples?

cider and apple juice are both about 400 calories per liter . Growing apples is 1500 kcal per square meter. I estimate 1/2 of that makes it into the juice which is 750kcal. Each liter is 400kcal so 200 liters is 80000 kcal. If a meter is 750 kcal after pressing etc then that is 106 meters squared to keep you in good alcohol for the year.

So back of the envelope it would take about 100 square meters of orchard to keep you in booze for the year.

4.5% Beer seems about 400 kcal per liter. And all grain is to be generous 75% efficient at getting out the sugars from grain. So each liter requires 533 kcal of grain very roughly. Wheat, cereal 810 kcal per square meter per year that is 2/3 of a meter square per liter of beer.
So how many liters of 4.5% beer is 14 liters of alcohol? It is about 310 liters of beer (which is a huge amount!) or back of the envelope 250 square meters of land to keep you in beer if you include malting loses.

Now wine. Each square meter grows about a kilo of grapes. As 1 acre grows about 5 tonnes according to here.

So 112 liters of wine is about 14 liters of alcohol.
3 kilos of grapes produce about 2 liters of wine from looking at various recipes. So 112 liters is about 170 kilos of grapes. So 112 liters is about 170 sq meters of vineyard. So you need about 170 square meters of grapes to keep you in booze for the year.

So in Ireland you could keep you in booze for a year of square meters you would need 56 sugar beat, 100 orchard, 250 barley and in france 170 grapes, roughly.

Drug Delivery Service

I got a pamphlet in the door today offering a drug delivery service. The drugs are advertised as bath salts and plant food. Head shops selling these legal highs have been burned down in Dublin recently so this could be a way to avoid such attacks.

The Perplexus Rocks

The perplexus is a 3-d maze where the aim is to move a ball around a series of obstacles. Makezine last issue had a great story on it here. The article described the journey from a school project through decades of effort to a toy and even a giant art piece.
There is an interview here with the creator Michael McGinnis. Usually my wife finds puzzles and games really boring but I have to keep fighting her to get a go on the perplexus. There is a page here on the history of the design. It shows some of the dedication and inventiveness that you can feel went into the puzzle when you play it.



It is not perfect, there are annoying etchings on the top that sometimes hinder viewing. It is slightly off spherical which is not what the designer wanted. It is made out of plastic like all toys. I presume trying to get this level of precision in wood would be nearly impossible. Still this is the best designed toy since the rubik's cube. Similar to the rubik's cube it is mainly aimed at children but not just for them.

This is really pretentious but the puzzle is like moving meditation. I cannot play this without feeling like part of zen and the art of motorcycle maintenance. You never get products anymore that have such obvious dedication built into them. Every step feels like some sort of Japanese mindfullness exercise. Basically the perplexus rocks. I got bought mine in the UK from here. The shipment to Ireland was really quick. You can get it from America here

Analytics X Prize Outside Bets

Black swans are not predictable but are there fairly rare events that could coour this year in Philadelphia that would skew the homicide rate in an area?

I know of nothing that would cause a sudden drop in the number of homicides in one particular area. Except a mass evacuation. There are a few things that could show up as a sudden rise though.

Terrorist attack. Philadelphia is not a well known terrorist target so any attack there would have to be pretty unpredictable. Terrorist attacks are also very rare so I do not think worth considering in a model.

Going Postal: These seem common enough in America. They have their own list here. I doubt you can predict where they will occur though. Might be worth considering.

Religious nutjobs:The solar temples and the kool aid drinkers in general tend to head off into the sticks before topping each other. So I think there is not much chance of a mass homicide by a cult in
Philadelphia.

Riots:Cities kick off on a fairly regular basis, The LA riots in 1992 resulted in 53 deaths for example. Philadelphia has had riots in the past. Riots in America seem to be mainly caused by racial tension. It might be possible to if not predict them localise where they are most likely to occur. Then submit a higher homicide count for that area in one of the analytics x prize submissions as an outside bet.

Prisoneers: They love a good ruck. In general prison populations have a higher homicide rate. So looking at changes in prisoner ecosystem in Philadelphia might be worth a look

Natural Disasters:After natural disaster people generally think the world turns into something from a Romero film.
The evidence for this is not that strong for example tales of post Katrina anarchy seem to be overblown. Also natural disasters could reduce the homicide rate as people leave an area after one. Philadelphia is unlikely to undergo a natural disaster though.

There are rare events that still occur often enough to make some sort of prediction on. I do not think any of these is worth including in a model with the possible exception of a riot. But predicting that would need more information about riots and Philadelphia than I have at the moment.

Survey the people of Philadelphia

In order to tell if a zip code in Philadelphia is getting more dangerous maybe we should ask people who live there.

So I created a survey here to ask them here. If you are or know someone living in Philadelphia if you could fill that out it would be great.

The idea is to find areas people think are changing in safety and see this turns out to help predict homicides. If it does this could be used to focus police resource in future to help reduce homicides.





I will release any data that is input as I am not the best person to do analysis on them. People going to the effort to submit a survey deserve to get the most out they possible. I will anonymise any data that does contain personal info before releasing it. There shouldn't be any info like this but I will check through the data in case.

I think ideally such a survey would let user place pins in a map in areas they think are improving/disimproving. Any thoughts on the survey? Or the idea of asking people for their local predictions in general?

Analytics X Prize

There is a competition here to try and predict what proportion of murders in Philadelphia will occur in each of the cities 47 zip codes. Many people who are interested in these sorts of puzzles have started submitting predictions.

So how would you go about predicting the murder proportion in each zip code?
Well if nothing changes in Philadelphia you would expect each zip code to keep exactly the same proportion of murders, well with some random variation you could not predict. So my first guess is a repeat of exactly what happened last year.

But in the real world things do change. Say the population changes if every person had the same chance of being murdered then the proportion of murders in a zip code would change proportionate to the change in population. If this was the case the prediction problem would become to find out what changes in population will take place over the year.



The dataset I am using is here and some errors in it need to be removed. Each dot is a zip code. It looks like number of murders does roughly follow population but it is not nearly an exact match. So changes in population are important but they are far from the only thing we need to predict.

How expensive the house in an area are or the average income or the number of people per house might help indicate the murder rate. Here I am looking at number of (murders/population)*10000.





So it looks to me a bit like areas with crowded houses could be more likely to have murders.






House cost looks like it is not connected to murder rate. This could be because zip code is too rough grained for this to be a good judge. Maybe the average cost of a house in a block would be a better measure of risk. Philadelphia has even been broken down into 60ft squares here



Does household income look like it is related to murder rate?

So if the graph is a random scattering of dots then it looks like the independant variable on the x-axis has no relation to homicide rate the dependant variable on the y-axis. If the dots form a line (well not just a line but that is another story) then the homicide rate may be related to that independant variable. It really is not this simple but that's the basics.



As Siah pointed out here young black males seem to be murdered out of proportion. The graph above does seem to suggest that predicting changes in ethnicity of a zip code may improve predictions. Age is another important variable and I do not have data on that so that might be the next thing to get.

There are interesting posts already on this puzzle
"Evaluating Spatial Predictions" and "Second Pass at Analytics X Prize" and "Homicides as non homogeneous poisson processes" are very informative.