Tuesday, June 28, 2011

The Kilbrittain Whale

I love weird crap. I have loads of holiday photos of my wife looking bemused outside some odd tourist destination. Here is one of my wife outside the udderly brilliant Cork Butter Museum last Sunday.

But last week I think I may have reached the Zenith of weird tourist destinations. This acme of travel strangeness is the Kilbrittain Finn whale. This specimen of the second largest species of whale is mounted in the middle of a town several kilometers inland.

How did it get here? The Channel 4 whale autopsy provides some of the answers about this poor animals beaching in 2009. Three minutes in (or 20 seconds in the video below) you see the frankly unbelievable result of a bad autopsy of a whale. Exploding entrails all over the Denmark beach is the sort of image that stays with you.


 


What this pathological investigation misses though is the Machiavellian intrigue that lead to this particular resting place for the whale. When it landed in 2009 various locals tried to make off with the 50 ton pile of putrifying Balaenoptera physalus. "Apparently, an attempt was made on Saturday night under cover of darkness to steal the jaws but their bulk proved too much for the perpetrators. "

There are stories of a beach based stand off between the communities of Kilbrittain and Courtmacsherry. "Shortly after it died two men from Courtmacsherry, also known as a drinking village with a fishing problem, approached with a chainsaw. Their plan was to saw off its head and mount the jaw bones in the village. However, they were confronted by Kilbrittain residents who told them they were claiming the carcass." Presumably at some point in these tense negotiations two large groups of adults both armed with heavy machinery both realised that they were fighting for control of several dozen tons of rotting whale carcass.

There is no simile that really can convey the logic here. No metaphor can quite sum up the Irish ability to fight over possession of something that no one would want. But if I had go for one description it would be "like a stand off over 50 tons of festering cetacean".

Now having 'won' the whale, some lads from the local abattoir spent 18 months rendering the stinking baleen bag. This year and a half long job is shown in a frankly horrifying pictographic montage beside the the spoiled beluga that now rests in the middle of Kilbrittain park.

There was a great moment on redneck roundup (as nationwide is known inside RTE) about the whale where one of the guys who spent the lifespan of a hamster hacking at the pile of rancid blubber admits they may have gotten the order of some of the bones wrong.

One thing words cannot convey is the stench coming off this attraction. Some flesh still seems to cling to the head parts resulting in an overpowering odour of death.

It was so bad we could not eat the ice-creams we brought to the park. I advise you to go see the Kilbrittain whale while it still has this stench. There is nothing like the smell of dead whale to make you wonder about what some people will go see/fight over.

Friday, June 17, 2011

Really Old Idea


Sports really annoys me. Particularly at the end when they get boring. When they know they have a minute to go in football they start kicking the ball between them and generally acting the micky. I thought I was dead clever when I thought that they should have random end times of sports. If you do not know when the game will end it is harder to try run down the clock.

Axelrod had the idea of random endings years ago. Many other game theorists have had similar ideas at various times.

Then I heard that random endings have a much longer history. Adapt the new book from Tim Harford explains one really old example. This century IBM got a patent for "a “smooth-finish” auction, an auction that is programmed to end at a random time". This gets over the problem on ebay where at the end of an auction people try not to bid what they value the item at but at just enough to win the auction just before it closes. So instead of people bidding one cent over the current bid with one second left on the auction.

Paul Klemperer pointed out that this idea is quite old. Even though the is a US patent for it.

In a candle auction, the end of the auction is signaled by the expiration of a candle flame, which was intended to ensure that no one could know exactly when the auction would end and make a last-second bid. Sometimes, other unpredictable processes, such as a footrace, were used in place of the expiration of a candle.

Pepys wrote about such an auction in the 1660's
Auction by candle was known in England by 1641, when it is mentioned in the records of the House of Lords.[6] The practice rapidly became popular, and in 1652, John Milton wrote, "The Council thinks it meet to propose the way of selling by inch of candle, as being the most probable means to procure the true value of the goods."


This 350 years old idea is my current record on simple ideas that someone has beat me to. Hartford's book Adapt is full of these sorts of interesting stories. This extract about the Spitfire gives you a good feel for the book.

Monday, June 13, 2011

Malaria Modeling

There are two very interesting passages in Shah's 'The Fever' that suggest modeling of malaria can have an important role

HIV-positive people are most infective to others when the levels of virus in their bodies are high...Malaria triggers such spikes. Malaria infection, by inducing HIV to replicate, increases the viral load in HIV infected people. According to mathematical models, HIV may be responsible for 980,000 episodes of malaria and malaria responsible for mare than 8,000 Hiv infections in a single district of Kenya. The global effect of the malevolent partnership has yet to be mapped

This paper is the source of this information.

Another interesting point seems to suggest that a scapegoat that is more attractive to mosquitoes than people are.
the greater availability of cow and hog flesh attracted the interest of malaria mosquitoes.Hovering between home and stable ninety-nine out of one hundred times. That bit of anopheline caprice cost Plasmodeum dearly. As cows and sheep sprouted across the English countryside, malaria transmissionground to a halt

Increasing livestock numbers has serious effects on any ecosystem. the pig under the bad scheme should not be promoted without serious analysis. Which leads to a more general problem with complex problems like malaria. I want to talk about these issues in another post. There is an interesting Ted talk on malaria here. A lot of the ideas are impractical which is another reason to talk about practicality later.

Jeffrey Sacks also talks about the important role of livestock that distract mosquitoes in the end of poverty
Another important point is that some types of mosquitoes prefer to bite people, whereas others feed off cattle. Transmitting malaria requires two consecutive human bites: the first for the mosquito to ingest the parasite and the second for the mosquito to infect another person, roughly two weeks later. If the mosquito feeds frequently on cattle rather than on people, the odds are that at least one of the bites, if not both, will be taken from cattle. In India, for example, the predominant type of anopheles tends to bite humans about one third of the time, and cattle the rest. Africa, sadly has another predominating mosquito type which prefers human biting nearly 100 per cent of the time. Mathematically, the chance that an Indian mosquito will feed off two humans in a row is about one in nine, whereas in Africa it's about one in one

While on the subject of development I thought this quote was fascinating Justin Kilcullen head of Troicaire 48:18 in the Frontline 30/05/11 here
"Development aid is what we give to the developing world because we do not give them a decent deal in trade, because we have screwed them on their debts, because the flow of resources...lets change the economic structures that keeps these people poor thats the long term answer". If the head of an aid agency does not think they are the long term answer for development then they are probably not.

For the moment I am going to examine OpenMalaria to see how it can model these issues.

Monday, June 06, 2011

Coding Day Dublin

Can you do anything useful in a day? This is the premise of code day where computer programmers are invited to work with scientists for a day to try solve a real problem the scientists are trying to deal with. "For those unfamiliar with a code retreat, it’s a hack day with a focus, the angle placed solidly on solving interesting technical problems." Hacking here means writing short programs that do something cool.

On the 28th of May I spent a day in the Science gallery trying to create fixes for various small problems. There were about two dozen people there with about a third of the scientists. Declan, Andrea and Qamir who organise loads of these programmers meet ups around Dublin ran the event and helped make it a really friendly atmosphere.

The small projects that seemed to work best involved visualisation. The problem of taking data and presenting it in an interesting way seems just right for a few hours timeframe. Possibly this is because rather than a complex scientific issue all that has to be understood is what sort of information needs to be conveyed.

It is quite difficult to get up to speed on a problem in such a short period of time. I had a really interesting talk with Scrazzl but even understanding their system took so long I felt like I was wasting their time. This meant we spent a lot of time just understanding the basic background of someones research. Treemetrics were also doing some really interesting Operations research work on forest management. It was really interesting to hear what people were working on but perhaps a day that was focused on one theme might allow the coders to read up and get some understanding in advance. Not that I think all such coder days should have a single theme just that it might be interesting to try one that did.

One of the most interesting projects at the day was fight malaria at home. This is a distributed computer project like SETI at home or folding at home. The aim is to test drug molecules against proteins present in malaria to check for potential binds. This could than guide researchers as to what drugs are worth testing in real life. The guys from the project were really friendly and interesting. If you do have knowledge of BOINC or even just want to learn it it would be a great project to help out on.

Instead of just criticising are very fun and worthwhile day I will give an example of the sort of focused task I am talking about.

Something that is big, widespread and multidisciplinary. I am reading The Fever by Shah at the moment which made me think Malaria was one such problem. Off the top of my head diarrhea diseases, political expenses, maths education and are all areas that people from many areas could come together and do some interesting projects/demonstrations over a day on. But malaria is one of those big thorny problems that a hackday about could be really cool. I have a Google Document of possible ideas here

Saturday, June 04, 2011

Bad Taxidermy


I have had a frankly morbid interest in taxidermy since I was a child. I particularly like bad taxidermy.

My absolute favorite form of taxidermy is when naturalists in the past attempted to reconstruct animals having never seen them. The weird mistakes they made should make raise serious epistemological questions. If animals we know that are quite similar to ones we see are gotten wrong how wrong are our reconstructions of Dinosaurs? More generally miscontructed taxidermy shows how our our ignorance and presumptions can distort our world.

My competition for the best misconstructed animal is between the Horniman walrus and King Fredericks lion.

With the Horniman walrus naturalist were sent the skin of the animal. Not realising it had wrinkles just kept stuffing the skin until it bloated up like some sort of zeppelin.


And the lion of King Frederick I of Sweden in 1731 where they had no idea what a lion looked like. The reconstruction has these weird old persons false teeth and lolcat eyes.


Paris for some unknown reason has particularly excellent taxidermy using shops

1. Deyrolle on rue du Bac.
2. Aurouze which they used in Ratatouille
3. The catacombs these freaked the bejasus out of me.

While I am on the subject I also recommend 'Stuff the World' a documentary on the world taxidermy championships.

The best crap taxidermy animatronics I have seen. And it is not a big category was in Strontium in Scotland. The wildlife museum there has a tunnel you crawl through where various crap animals on rails roll toward you. Strontium was on our travel list of places elements were discovered but that is a whole other blogpost.


If you also like appalling stuffed dead animals "crappy taxidermy" blog is a goldmine. If you know of any animals stuffed by people who have never seen the animal please post it in the comments

Friday, June 03, 2011

Spongebob New Wave Pants

I was watching Spongebob as usual last Saturday morning and I noticed how most of the characters were modeled on 70's rock musicians. Elvis Costello is a dead ringer for the eponymous hero and Brian Eno could take the wrap of a Squidwort crime if he ever took part in a line up.




Do you know any other 70's rock star look a likes?

Monday, May 30, 2011

How much is technology worth?

How much would you have to be paid to give up your mobile and your computer? How much is reading these pearls of wisdom, viewing pictures of amusing cats and forum fighting with idiots worth to you?

The brilliant Barking up the Wrong Tree blog it looks like mobiles are worth about 10% of income.
Keeping life satisfaction constant, we subsequently derive substantial GDP per capita estimates equivalent to a 10 percentage point increase in broadband and mobile phone penetration.

Median wage wage in US is about $32,000. So mobiles probably worth about 3 thousand dollars a year to the average American.

The value of the internet seems to vary from $1700 to $3800 according to this slate article
a model based on the value of people's time, ultimately estimating that Internet access is worth 2 percent of full income. Thus, they estimated the value of the welfare gain provided by the Internet ranged as high as $3,800 per person.

This means that in the 15 years that the internet and mobiles have become ubiquitous we are something like $5000 richer per year because of them.

Palaces of memory

Remembering is a creative process. We tend to think rote memorisation as a dull boring task. However instead we should think of it as "Moonwalking with Einstein". A big challenge to think of the oddest 'most memorable' images you can associate with what you are trying to remember.

The book 'Moonwalking with Einstein' deals with one journalists travels around the world of memory competitions. These competitions have all sorts of weird and wonderful rounds. Remembering binary number, packs of cards, poetry, random word list, matching faces to names and historic dates. It is a very entertaining book, dealing with the history of knowledge, savants, the weird world of self help marketing and the importance of living adventurously.


Many of these competition events seem pretty abstract. Some memory based skill that had some sort of humane self improvement aspect and maybe even some vague practical use would be good to learn though. But if there was one of these skills that would be cool to have I would like to practice the methods described in the book. I read a book, Inside the black room by Jack Allen Vernon, about sensory deprivation experiments in a Californian university. The author describes an Arab immigrant who unusually asked to be put back into sensory deprivation room as he wanted to train himself incase he was ever forced into a similar situation as torture.

Brian Keenan's 'an evil cradling' has vivid, gripping and terrifying descriptions of the madness that stalked him as he was held hostage in isolation in the Lebanon. For example he writes "that the human mind can travel into those dark regions and return exhausted but intact is more a miracle than that word can ever convey". One of the boys own adventure stories I read as a child had the description of an RAF POWs time in the cooler and the physical and mental regime he went through to ease the hardship. One of the parts I remember was that he remembered all the poetry he had learned at school. These tales of how to deal with sensory deprivation have really stayed with me. Maybe if the memory skills in this book could help in such a situation that would seem useful.

One of the expert mnemonists in the book Ed suggests memorising poetry and prose is useful for such a situation of sensory deprivation. "My philosophy of life is that a heroic person should be able to withstand about ten years in solitary confinement without getting terribly annoyed' he said 'an hour of memorization yields ten solid minutes of spoken poetry, and those ten minutes have enough content to keep you busy for a full day.'

I will give myself a challenge to get good at poetry memorisation in a month. The competition seems to be to '15 minute memorization time, 20 minute written recall' for poetry. I will try memorise a poem tomorrow. And again in a month and see if memory training improves poetry memorisation.

As an aside the author mentions the interesting possibility of using "neutropic 'cognitive steroids,'" but never goes into details about how these might aid a memory sports competitor. This is slightly odd given he wrote an article in 2005 (viewable here) about his experience taking one of these drugs. He said 'gym rats have steroids, and overachievers have Adderall. ' it would have been interesting to hear what effect these drugs had on his memory abilities but this book never brooches the subject in a personal way.

Tuesday, March 29, 2011

Search Engine Deoptimization

Take a company you have a problem with

1. Imagine you hate Ryanair. I do not btw they are just an example of a company this would work on.

2. You decide you want to harm them commercially

3. You go to their terms and conditions here and see they do not allow linking to their website

"Links to this website. You may not establish and/or operate links to this website without the prior written consent of Ryanair. Such consent may be withdrawn at any time at Ryanair’s own discretion."

There is a list of other companies that do not allow linking listed here. That where I heard of this T&C clause.

4. Now spider all the websites that do link to them. This is easy to do and you now should have a list of many websites. Either from a whois or just from scraping the website get an email address for each site.

5. Send all these linking websites official looking notices that remind them that "You may not establish and/or operate links to this website without the prior written consent of Ryanair" including a link to the official Ryanair terms and conditions. You would have to be careful here. You would not claim to be Ryanair for legal reasons. You do not want to break any laws or threaten anything. Just say you are from some official sounding company "John Smith and associates business compliance coordinators" and remind them of what the companies own policies say. It is not like you are lying.

6. Watch Ryanair drop down search rankings as people delete links to them. Search rankings are mainly based on how many (and what sort of) sites link to you. As they drop down the search engine rankings they will suffer commercially. With my google of "cheap flight dublin london" Ryanair comes up as the third site. Which implies they must get a fair amount of traffic from search engines.

I think this attack should be called Search Engine Deoptimization.

Friday, March 11, 2011

Analog Computer Museum

This week I was lucky enough to visit the Analog Computer Museum near Frankfurt. Bernd who owns and protects the collection I have known for years via email so I was happy out to finally get to meet him and his wife. These Analog computers carry out calculations using electronic components.


Bernd works as a software archeologist. That is he updates the systems of banks and such that are written in OS' and languages that almost no one knows anymore. If the software running your bank was written int he 60's and you want to get it running on Linux Bernd is the man to call. These languages are so obscure they make the COBOL look like the Beatles. The job seems to pay well enough as he can afford warehouses to keep his computer collection in and a collection of vintage sports cars he drives at terrifying speeds on the local autobahns.



Yes I take my wife to Computer Museums but in fairness shes a physicist and I suspect she thought this rocked as much as I did



These machines are ideal for real time simulation. This Telefunken currently simulating a car was used to train nuclear powerstation operators with simulations of various 'problems'


Next time you call rand() think of this board that was used to generate random numbers through electronics


Old school fractal printed out and taped to the side of a Vax. It is like chalking up a kill on the side of a WW2 fighter plane. Computing from the time where men were men and debuggers were nervous.

I cannot rationalise why these analog computers are so cool (pics here). I think its to do with getting to the bare metal of a computation. Also something of the history of these machines is amazing. Looking at the circuit boards you can imagine national characteristics they describe. The German machines based on mathematical abstractions are beautifully clean (and easy to fix apparently). The Japanese machines were a mess like their remote controls. The British machines had a distinct boffin feel to them. The machines from communist countries really seems like a scramble of clever but under resourced hacks. The American machines were big brash and heavy.


Kevin Kelly claims that a technology never dies

What happens is that most technologies become obsolete, diminish their role, but they don't disappear--more idea-based and can be resurrected more easily. Seems like a silly claim: the hand-axe, the arrowhead--they don't exist any more. And that's what I thought too... Looking on the Internet was able to find an example of whatever the challenger had given to me: brand new steam-powered valve for a steam-powered car; they're making flint axes exactly the same way, using exactly the same tools to the point they are almost indistinguishable from the original; archeological artifacts, making in huge numbers today.

But I really worry that the knowledge of many of the engineers who created these machines will be lost. One thing is that the analog museum really wants donations of machines that may be thrown out. If you see a box of the brand Dornier, TeleFunken, Gould, Aritma, BBC, Electronic Associates Inc, Solartron or even just something you think might be of interest send him an email. Even parts of these machines could be really useful to help repair the computers he has.

I do not think these machines are purely of historic interest. It is possible that in the future analog computers will be used in the real time simulations they excel in. One area I think they could be useful for is search engines. Linear algebra calculations are at the heart of modern search engines. this is described in
The $25,000,000,000∗ Eigenvector the linear algebra behind google and How Google Finds Your Needle in the Web's Haystack

These calculations do not need to be precise (the google algorithm uses random walks) which removes one of the big objections to analog computers.

George Dyson in the this years edge question 'What Scientific concept would improve everybody's cognitive toolkit?' describes a search engine based on analog computers here
Analog computing, once believed to be as extinct as the differential analyzer, has returned. Not for performing arithmetic — a task at which even a pocket calculator outperforms an analog computer — but for problems at which analog computing can do a better job not only of computing the answer, but of asking the questions and communicating the results. Who is friends with whom? For a small high school, you could construct a database to keep track of this, and update it every night to keep track of changes to the lists. If you want to answer this question, updated in real time, for 500 million people, your only hope is to build an analog computer. Sure, you may use digital components, but at a certain point the analog computing being performed by the system far exceeds the complexity of the digital code with which it is built. That's the genius that powers Facebook and its ilk. Your model of the social graph becomes the social graph, and updates itself.


Analog computer methods of calculating eigenvectors have been described decades ago (pdf here).

It is possible that the techniques of analog computers could be the future of search engines. We should make some effort to save these machines, any manuals written on them and if possible some of the knowledge of those who made and used them.

Think Different

I am fascinated by how people think differently from each other. I think there are big differences in the fundamental methods people use to think, but because we rarely talk about how we think these are not well known.

Feynman talks here about how he heard numbers in his head when he counted and his friend saw numbers visually.


He believed that such fundamental differences explained why we sometimes have such difficulties communicating with each other.

I 'see' numbers as a kind of line and I have heard other people describe other ways they 'see' numbers

Reading is another place people think differently. When I read I 'hear' the words as I read them. This is called subvocalisation and is normal but it slows down reading. "Subvocalization, or silent speech, is defined as the internal speech made when reading a word, thus allowing the reader to imagine the sound of the word as it is read". There are all sorts of online guides as to how to stop subvocalising. My wife does not subvocalise and never has. She describes seeing words as the actual word rather then hearing it and then understanding what it means.

Do left handed people who do that hook handed over the top writing have a fundamentally different view of writing? Do you know of any other basic cognitive tasks that people carry out in completely different ways?

Monday, February 28, 2011

NOW Who is the popular one?


Someone of the two of you readers were kind enough to nominate this blog for the Irish blog awards


Science/Education 2011 Best Science/Education Blog
Technology 2011 – Best Technology Blog/Blogger

Thank you very much to all of you who read my nonsense especially those who decided it was worth pointing out to other people

On a related trumpet blowing note Tyler Cowen thinks this blog shows that Ireland is going to collapse into anarchy,
@tylercowen
Will the Irish be more or less impatient than the Egyptians?, http://tinyurl.com/4zyyxsg

Which now I think about it may not be a complement

Congratulations to my friends in thebeernut, L. Mulligan grocers and Dungarvan Brewing Company on their nominations.

A no lose lottery would help Haiti

In this post I want to try and persuade you that there is a form of lottery the Prize Linked Savings scheme that would be popular and fairly cheap to run for the people in Haiti. In my next post I want to explain why such a scheme would actually be useful

"PLS is a kind of savings account that pools some of the interest from all depositors and pays out a big lottery prize every month or so. It combines the thrill of the lottery with the safety of a savings account. It’s sometimes called a “no-lose lottery,” since a depositor is automatically entered into the lottery but can’t lose the original money she deposits."

Lotteries are incredibly popular in Haiti. Probably due to the crushing poverty there the hope and 'chance to dream' lotteries provide is seen as vitally important

"the lottery system in which an astonishing number of Haitians invest their income and their dreams." "At the time of the last official count, in 2005, the streets of Port-au-Prince were home to nearly 2,000 of them" (borlettes lottery shops) "– more than double the number of schools and universities in the city"."All told, Haitians spend as much as $1.5 billion per year on the borlette – a staggering amount in a country whose gross domestic product last year was $6.9 billion"

"You can bet two balls to come up at once, and if that happens you make a ‘marriage’ and win 500 times your bet. If your married numbers are consecutive, you win 1,000 times your bet. The Haitian dream is a bet of US $1,000 that reaps US $10,000 on a consecutive marriage. Many a hopeful future has been built on less." Borlette lottery in Haiti is remarkably popular which is a good indication that a lottery that also acts as a savings scheme will be popular

The overhead cost of these Haitian lotteries is relatively low as they use the numbers and thus the security mechanisms of outside lotteries. These lotteries tend to use the numbers from other countries lotteries for example the New York State Lottery or the Santo Domingo State Lottery

Another expense is the prize won in the lottery. You want your prize to be equivalent to a million euro. This is an amazing life changing amount in Ireland where the average wage is 35K. So a million is about 30 times average income. For Haiti this would be $730 *30= 21 thousand dollars. A million Haitian gourde is about 25 thousand dollars.

If we had a monthly prize of this much that would cost 12*25k=300 thousand a year in prizes. Say you double that with some smaller prizes. 600 thousand which if you were earning 3% interest means you would have to take in 10,000000 dollars. This is a lot of money but in the context of $1.5 billion spent on lotteries in Haiti it is not that much.

If you could take in ten million dollars you could run a no lose lottery in Haiti and use the interest to pay for the prizes. But why would you want to? I will explain in the next post why PLS would be useful to the people of Haiti

Friday, February 18, 2011

The Great Stagnation in Ireland

There is a new ebook by Tyler Cowen about how growth rates and progress in general has declined since 1973. There is a podcast here where he discusses the argument.

The book, is short you can read it in an evening, and published only in ebook format. It is a bit like an evening with your smart friend as he explains something rather than an economic treatise thats full of equations and aids insomnia. It is well worth the 4 dollars and few hours it takes to read.

Loads of commentators have examined various parts of the argument for stagnating progress. I even joined the argument here. I am going to try give a few datapoints that relate to Ireland that are part of the great stagnation argument here. Ireland has not been stagnating since 1973. In that time we moved from an improvished bachwards country to a modern well off one (and then slipped back somewhat again).
But there some areas where we are stagnating in similar ways, there are areas where we made similar mistakes to the Americans (and the rest of the OECD) and if we can expect a similar decline in growth here that means we should alter our expectations.

Stagnating medicine

We spend too much on medicine and do not get good value for that spending. There is a piece here by Ronan Lyons where he argues convincingly that with our age profile we should be spending a third less than what we currently do on medicine


It is generally agreed that access to the health system in Ireland is bad. If we spend more than we should and people cannot get healthcare as easily as other OECD countries do it is fair to say that Irish healthcare is not improving our quality of life the way it should be

Stagnating education

'In the past decade, funding per student has been increased in real terms by 61 per cent, yet performance has decreased by 15 per cent.

This disconcerting news was reinforced before Christmas when the international ranking of the reading, mathematics and science skills of Ireland’s 15-year-olds was published. Reading skills dropped from fifth to 17th, the sharpest drop among 39 countries, with one quarter of all 15-year-olds classified as effectively illiterate. This was more politely termed in the report as “below the level of literacy needed to participate effectively in society”.'

We are spending more and getting less on education. Literacy is one of those basic measures that is hard to fake. Anyone who leaves school illiterate is a failure of our education system. In this school education area Ireland seems to have stagnating improvement.

Expecting too high growth
One of the major conclusions of the book is that Americans expected growth to continue at previous rates and this facilitated the financial crisis. People believed house prices would go up forever without wages increasing at the same rate. When people realised that they could not afford their debts and no one was going to buy them off them the house of cards collapsed.

In Michael Lewis' Vanity Fair piece makes a similar claim that Ireland expected too much growth
'An Irish businessman named Denis O’Brien sat on the board of the Bank of Ireland in 2005, when it was faced with the astonishing growth of Anglo Irish, which was about to double in size in just two years. “I remember the C.E.O. coming in and saying, ‘We’re going to grow at 30 percent a year,’” O’Brien tells me. “I said, How the fuck are you going to do that? Banking is a 5-to-7-percent-a-year growth business at best.”'

If the financial crisis in America was aided by delusions of future growth the Irish collapse was to an even greater extent.

Expecting growth to be much higher than it will be leads you to paying too much for things. For example buying empty buildings on the grounds that NAMA will make a profit on them eventually.

Other Claims
There is a very interesting section on how the internet might entertain us but does not provide many jobs. There is a similar argument to in "An Exodus Recession?" that because we can sit at home and look at youtube essentially for free the recession both does not feel as bad but is measurably worse as people do not go out and spend money on cinemas etc that employ people.
This does not relate specifically to Ireland though I do think the internet has improved our lives in ways that are hard to measure and that probably help those who like reading and arguing more than they help the average person.

Cowen also claims that increased emphasis on scientific research could again get progress moving faster again. I was watching Bronowski's the ascent of man last night from 1973, the year Cowen claims the stagnation began. Bronowski gives a warning that the next step in mankinds progress is dependent on science and that if we don't prevent scientific progress from stagnation some other civilisation will.


Expecting growth to return now

I have heard many people running for office who claim anyone who questions that we can return to the good old days of 5%+ growth of being unpatriotic. In this narrative our debts are huge but they were in the 1980's as well ans well soon have 5%+ growth that will evaporate our debts for us.

The evidence is that once countries get as rich as we are now their quick growth from playing catchup then slows. Some of us might be able to get more productive now but not nearly the increase that happened in the 90's when we went from working on farms producing vegtables and potatoes to producing Viagra and Pentiums.

We are not going to have the growth rates we had in the 90's we like the rest of the OECD have entered a period of lower growth, what Tyler Cowen calls 'the great stagnation'. Anyone who says we should plan for Ireland growing at a rate higher then the 6.7% interest rate on the ECB/IMF loan are is trying to get you to take on billions of their debt.

The great stagnation a great short book. Though its arguments apply to America the lessons of what they Japan and other countries who developed before Ireland are important for us to learn.

Thursday, February 17, 2011

Watson is better off having a three way competition

It seems non intuitive but I think Watson is more likely to win because its competition are so good. If you look at how well it did on Jeopardy here it seems to show Watson is better having humans splitting the points on questions it cannot answer as it is so quick to answer the ones it can.

When the humans buzz in they are right 19/23 about 83% of the time. Suppose Jennings was up against watson and me and I got zero buzz ins. Jennings may have gotten 11 of the buzz ins that Rutter got. Watson seems so fast on the buzzer that it seems to buzz first for nearly every question it could answer 44/50 times it tries to answer it is first.

This means that Jennings could nearly have doubled the number of answers he had but because there are two people who are really good at the sorts of questions only humans could answer these points are divided amongst the humans. Of the 23 human answers only 6 did Watson try to buzz in on.

There is a similar probability puzzle that might help in a three way duel. "You're a cowboy, and get involved in a three way pistol duel with two other cowboys. You are a poor shot, with an accuracy of only 33%. The other two cowboys shoot with accuracies of 50% and 100%, respectively. The rules of the duel are one shot per cowboy per round. The shooting order is from worst shooter to best shooter, so you get to shoot first, the 50% guy goes second, and the 100% guy goes third, then repeat. If a cowboy is shot he's out for good, and his turn is skipped. Where or who should you shoot first?"

Theres another explanation of a three way duel here.

Sometimes having 2 really good competitors increases your chances of winning.

Wednesday, February 16, 2011

When will you be able to ask Watson?

Watson is a computer currently playing the quiz Jeopardy. You can see a video of this here


One useful question is how long this sort of supercomputer like power will take to be seen in everyday life? It has 3000 cores running and answers in less than three seconds. On one machine would take 2 hours to answer a question according to here
So how long to get a desktop computer to be able to answer at this speed? I propose that half the speed improvement in Watson like programs will be from Moore's law and half from algorithm improvements. To go from 7200 seconds to say 2 seconds will require about 13 halvings in speed. I predict this will occur in the time it take processors to improve 7 doublings in computation. You can buy your own supercomputer now on the cloud and depending on how long you are willing to wait you could get an answer to a Watson like answer for costs of around a dollar (if you had the data and software Watson uses).

One comparison here is with chess. "It has been estimated that doubling the computer speed gains approximately fifty to seventy Elo points in playing strength (Levy & Newborn 1991:192)."

If doubling in processing power happens ever two years that would imply about a 30 point increase a year. The actual improvement (in computer v computer games) is described as 'With a 40-point annual improvement due to hardware upgrades, and a 30-point annual improvement due to software upgrades'. Implying improved algorithms are responsible for half the improvement.

This argument is summed up in this explanation as to why a fairly normal computer Deep Fritz in 2002 was an improvement over the supercomputer Deep Blue in 1997 'Deep Fritz has improved considerably over Deep Blue. Despite Deep Fritz having available only about 1.3% as much brute force computation, it plays chess at about the same level because of its superior pattern-recognition-based pruning algorithm'

I claim there is an analogy to Moore's law that says something like.

Once computers get good enough at a task to have a flashy TV challenge from then on algorithms will cause half the improvement in that task

So if I am right I think Watson like computation instead of talking over 25 years to move from twelve refrigerators to something we can use everyday should take less than 10 years. I would take an even money bet that in five years there will be a service you can use from your phone that is a lot closer to Watson than to a search engine we see today.

*update A friend just made this bet with me
"20 euro bet that January 2015 there is a search engine that with a set of quiz questions from one gameshow program (the weakest link, who want to be a millionaire, university challenge) and it has to get 7/10 right". The effects of learning by reading (and autonomous cars) are examined in this rather long but brilliant blogpost

Monday, January 31, 2011

Are Kitchens Better?

There is a really interesting debate going online right now about whether the rate of improvement in America and the west in general has slowed. The debate has been started by a book by Tyler Cowen called "the great stagnation" which I have yet to read.


Above is a graph from here of how quickly different products became popular
Paul Krugman raises a similar point about how American Kitchens have not improved much from the 1950's here where he points out "By any reasonable standard, the change in how America lived between 1918 and 1957 was immensely greater than the change between 1957 and the present."

You could argue that kitchen equipment is much cheaper than they were in the 1950's. Or that rice cookers, food processors and microwaves are a great improvement. Coffee machines are probably the exception to the stagnation rule as they do seem to have improved vastly. There is also the explanation that if your home kitchen has not improved maybe the kitchen that cooks your food has.

Has the big change been that people do not use their kitchens anymore? In a 1950's kitchen could you order a pizza to be delivered? Dominos started in 1960 according to the history of pizza delivery here so you probably could. I doubt Italian, Chinese, Mexican and Indian food were regularly delivered to ordinary peoples homes though.

Has the technology and labour moved from the domestic kitchen into industrial kitchens that now do 90+% of the work in preparing many of our meals? How much of the supposed lack of improvement could be explained in terms of more processing happening outside the home?

About 44 percent of food dollars are spent outside the home – a figure that started rising sharply in the 1970s, as more women joined the work force. Full-service restaurant revenue rose 5 to 7 percent a year in the decade leading up to the Great Recession”. So do seem to be eating out more?

Houses with small numbers of people cook less. And houses have less people nowadays, due to demographics. If people eat out more (or order in). Why would they buy some brand new kitchen invention? I have seen people with kitchens filled with shiny new equipment they have never used but I think this is fairly rare.



If the dial up pizza restaurant can have a new dough maker that is more efficient it makes sense for them. I can ring them up now and get the use of their new industrial standard food technology for the minute it takes them to make my pizza. Maybe home technology stagnated and the restaurant/supermarket/Microwave dinner technology improved?

I think it is worth noting that the improvements in Irish kitchens in this time have been huge. The 1918 world Krugman describes is superior to my grandmothers life in the mid fifties rural Ireland. She did not have running water or electricity and in her lifetime between 1950 and 1980 the changes she saw were massive.

Thursday, December 23, 2010

Crowdsourcing Taxonomy

I had an interesting debate on twitter with Panos Ipeirotis and Anand Kulkarni about what counts as a crowdsourcing application.

Wikipeidia defines crowdsourcing as
"Crowdsourcing is the act of outsourcing tasks, traditionally performed by an employee or contractor, to an undefined, large group of people or community (a crowd), through an open call."
And they are even so nice as to give a list of crowdsourcing projects


These seem to fall into a few areas
1. Microwork. MTurk, Freelancer.com, 99designs. You go on mturk you put in a task like "tell me if this sentence is in English" and an amount "1 cent per task" and then you let the people on mturk do the classifications for you.

I think this sort of work will be huge. There are talented people sitting around willing to do something interesting for cash in their spare time just waiting to be harnessed.

Many projects exist to crowdsource advertising ideas. for example IdeaBounty. These advertising ideas seem similar to other microwork projects.

2. Games with a purpose. Galaxy finder, Foldit etc. There is a task like "count the number of spiral galaxies" you put this out there as a 'fun' task and people do it for you free, gratis and for nothing.

This is similar to citizen science projects but explicitly involves the use of fun and game mechanics to get people to do the work.

3. Market mechanisms could be thought of as a form of collective intelligence. In a market the competing forces of supply and demand result on a value being placed on an item. So for example on ebay many people have decided the value of collectible baseball cards. Or on intrade many have decided the probability Obama is to be reelected.

Is market formation a form of crowdsourcing? I think prediction markets are great and should be used more but their placing accurate values on the probability of an event is a byproduct of their action rather than the intention of those operating on it. People would rather if the odds were wrong so they could make money being right.

For some reason I don't think a prediction market is a crowdsourcing application and none are listed on the wikipedia list.

As an aside I read a brilliant blog post two days ago (that I now cannot find) that pointed out that ebay is essentially a storage device. You sell things they go into the ebay and when you want them back you go on the ebay and find the product again and buy it. Your storage costs are usually just the postage and packing charges. This is similar to the argument that there are two ways to produce cars. One is to build factories and hire people and such the other is to send boat loads of cows to Tokyo where magically you get boats filled with cars back in return.

4. Microlending
Kiva and other sites allow you to lend money to business. Say a guy want to plough my field. 10 people give you 5 dollars each so you can buy a plough. The field gets ploughed. They get 6 dollars back after harvest. This again does not seem like crowdsourcing of jobs as you are lending capital not labour.

Artists using a street performer protocol to pay to get an album made is something similar. Kickstarter is one site that does this

5. Wikies, Forums, social search engines, message boards
People on Forums, give their advise and knowledge almost always for free. It is pretty amazing when you think about it. The fact that some very clever and skilled individual is willing to take their time to tell me the arguments to a Java function are wrong is amazing. The wikipedia page does not class these as crowdsourcing but again I cannot see why not.

6. Competitions, kaggle, Netflix, DARPA, Goldcorp and InnoCentive regularly have prizes where they put out a task such as 'improve on our recomendations' or 'make a car that drives itself' and people try to do the best they can at it. I think prizes as a way to encourage innovation are also going to be massively important in the future.

7. Response to events. The guardian set up a webpage to get people to read through all the mp expenses reports to find things that looked excessive and wikileaks have used crowdsourcing to examine the millions of leaked documents and highlight the most interesting.

8. Crime detection. Crowdsourcing has been used for illegal immigrant spotting in Texas and other similar uses.

9. Missing persons. After hurricane Katrina and Steve Fossett's loss crowdsourcing was used to locate missing people.

10. Politics. Oxfam Novib, Moveon.org and other such sites try to use crowdsourcing to create a community based political movement. If these are counted as crowdsourcing ventures I do not see why hobby sites that try promote brewing or knitting are not.

11. Art. Improv everywhere, mechanical olympics, Flash mobs are all listed as crowdsourcing projects.

12. Citizen science. Digitized versions of old weather or other science reports are made available to people to increase our knowledge of past events. Even the open data movement must have some basis in crowdsourcing. If people are not expected to analyse the data there is not much point making it availible to them.

13 Cartography. Open street map, Waze and other projects hope to use peoples GPS systems to build up a map of the environment. This can include real time traffic data.


Reading through wikipedias list of crowdsourcing projects these are the types of projects that occured to me. Most classes are probably wrong and need to be split or combined. Maybe the hierarchy needs to be different. Please comment or write a rebuttal to if you are interested in what projects count as crowdsourcing.

Tuesday, November 16, 2010

What is the worst thing to know?

Knowledge is rarely a good thing but sometimes it actively reduces your quality of life. Take coffee as an example. Coffee is an accepted addiction, you cannot go around going 'Sorry I am not with it yet, I have not had my Vodka' but the same excuse for coffee is nearly mandatory as so many of us need coffee to function.

Some strange people learn about coffee though and as a result they no longer can cope with Nescafe instant. Next they notice burnt flavours and incorrect coloration in coffee shop coffees and an epic impossible quest for the 'God Shot'. Expresso starts to ruin their lives.

If you are addicted to a drug becoming snobby about that drug is a really bad idea. You don't see crackheads demanding only the finest Columbian devils dandruff in their rocks. Yet coffee addicts get themselves addicted to a drug then try and make sure they are miserable almost all the time they take the drug.

A normal coffee drinker gets a mild amount of pleasure from instant and when they go to a coffee shop get a fair amount. Someone who has gone to the trouble to learn about coffee lives in a constant state of disappointment except the rare occasions the world champion barista they have taken two hours to go to does not do something almost imperceptibly wrong.

The worst thing to know about I think is fonts. I don't even notice fonts but I have friends who can spot the bad font in a street with 100 other signs. You never hear them happy about fonts all they seem to bring is pain. The entire world just seems to become an annoyance to someone who knows how text should look.

America Is F*cked.......(Graphically at least) from Jess Gibson on Vimeo.


Do you have any nominations for things you are really better off not knowing?

Monday, October 25, 2010

Which Keanu Film



Bizarrely unaging Keanu Reeves. Stick the answers below in here if you cannot name the film
Ovyy naq Grqf
Gur Qrivyf Nqibpngr
Wbuaal Zarhzbavp
Zl Bja Cevingr Vqnub
Punva Ernpgvba
Gur Zngevk
Guhzofhpxre
Pbafgnagvar
N Fpnaare Qnexyl
Cneragubbq
Fcrrq
Cbvag Oernx