Wednesday, December 28, 2011
Search Engine Deoptimization 16,000 Pageviews. Many people seem to dislike Ryanair and weird legal rules.
Eurovision Voting Fraud 5,500 Pageviews. A rather glib post from 2010 about human rights abuse that got popular when Azerbaijan won the Eurovision.
My favorite posts were
The Dead Zoo Dodo Get Anto the Dodo back to the public. Anto has set up a twitter account
The Kilbrittain Whale. This tourist attraction is brilliantly unhinged.
Wednesday, December 21, 2011
My favorite book was Moonwalking with Einstein. Without memory what would we be? This is a great diverse book on a really interesting topic.
Don't sleep, there are snakes. There is so much in this book from Language to God to child rearing to self sufficiency. It is just fascinating to read about people who see the world completely different to us.
The rational optimist nearly everything is getting better. This book left profoundly optimistic about the future. This is the book I have tried most to persuade my friends to read most this year.
Selfish reasons to have more kids. This book goes through the evidence that once your not an awful parent (one who would be refused an adoption for example) you really don't make much difference to your kids. It is actually a profoundly optimistic argument as this means you don't need to spend your time doing things you dislike to shape your childs path as by the time they are 18 they will end up where they would have anyway. It uses evidence from identical twin adoption studies to show how little parents need to worry about religion, education and all the other things they break their hearts worrying about.
The Great Stagnation By Tyler Cowen. We should have dinner in a pill, Optimus Prime, hover boards and lasers that turn dolls into women by now. Why don't we and what can we do about it?
Next year I plan to read mainly fiction. This year In cold blood by Capote was the one piece of fiction I read that I am sure will stay with me. Everyone says it is great and everyone ain't wrong.
Close but not in my very favorite books were fever by Shah, Adapt by Hardford, What technology wants by Kevin Kelly, Red Plenty by Francis Spufford, Gutenberg, race against the machine, good omens, the unbearable lightness of being, How the irish save civilisation and Launching The Innovation Renaissance by Alex Tabarrok, popular crime by Bill James
Sunday, December 18, 2011
There are several estimates to when driverless cars will arrive. The New York time estimates 2020.
By 2030, Sebastian Thrun predicts, more people will use self-driving cars in their daily commute than manually driven cars.
Submitted by Sebastian Thrun, developer of Google’s self-driving car.
Our readers predict this will occur around 2020, having moved this date 1381 times.
Many similar bets of Driverless cars being regular enough in 2020 and ubiquitous in 2030 exist for example here,
By 2020 - Driverless cars are commercially-available and street-legal somewhere in the United States.
By 2027 - New driverless cars outnumber new cars requiring at least some human control, in the US market.
By 2019, it has begun spreading to public roads, with significant numbers of driverless trucks appearing.
In my post last week, my commentors took me to task on my prediction that cars will drive us in ten years. Some thought Americans would wise up and learn to love mass transit. They don't know Americans.
Others thought the hardware cost would even in ten years remain out of reach. Google did not build an autonomous car by creating the hardware but by harnessing and training good machine learning algorithms. No amount of hardware would have given you a car able to navigate the streets of San Francisco five years ago.
What effect will these cars have? There are all sorts of ideas about how they will alter parking and car ownership. I'm going to try do a back of the envelope here on how much Taxi fares will cost if you don't have to pay the driver.
Taxis cost about 120 cent for a kilometer
For each additional 1/6th of a kilometre or time 28 seconds)
(a) Day time 8am to 10pm €0.15
(b) Night time 10pm to 8am €0.20
(c) Sundays, Public Holidays, Christmas Eve and New Year’s Eve €0.20
Ignoring the pick up costs of about 3.40.
The AA says it costs around 25 cent per kilometer to drive a car in Ireland. About 25 pence per mile in the UK. Taxis charge 1.20 so the majority of the cost looks like the driver. You would have extra costs on top of a normal car with a commercial vehicle. But given the pick up costs a driverless taxi could be about a quarter the cost of a taxi with a driver.
The price elasticity of demand should allow an estimate of how this will alter taxi usage. This paper "Estimation of Price Elasticity for Taxi Services in Hassel" gives a PED of -2.644. Though others such as Schaller at -.22 and here of -.6 shorter term. Taxis that cost a quarter the current price with a PED of -2.6 would mean about ten times the number of taxi journeys. The long term viability of public transport should take this possibility into account. If by 2030 people will be taking ten times the number of taxi journeys would enough people be using Metro North to make it cost effective?
Transport employs nearly one hundred thousand people in Ireland. Which is about 1 in 20 people who have a job here. Or about a third of the number of unemployed. I doubt everyone who works in transport will lose their jobs overnight. But taxis provide an example of how economic effects could provide a huge incentive to move to driverless cars. So far technological progress has always eventually resulted in new jobs to replace old lost ones. The money people save getting into town for a night now could end up being spent in town and require more employment in restaurants and bars for example.
But I think it is worth considering the possibility that fairly soon we could have nearly a hundred thousand people who earn a decent wage at the moment becoming unemployed in a short period of time. Construction lost 160 thousand people in three years. Transport jobs do not pay as well as construction did. But if the construction change caused most of our current economic issues it would be unwise to ignore a large sudden future change in transport employment.
To put some skin in the game, I am predicting that in 2025 in a period of three years we will see structural unemployment of about 5% of the workforce, half of those that work in transport.
Tuesday, December 13, 2011
Can you crowdsource disease eradictaion?
I mentioned in this post on 2030 that I expect Polio and Guinea worm to be eradicated by then. It becomes a tricky issue when a disease gets really uncommon how you find and treat the last few cases? So far only smallpox and rinderpest have been eradicated. Eradication is great because once its done its done. You would not have to immunise every child for polio any more. Every polio vaccine has a small cost and risk and once thats gone you can go spend the money on something better
India has tried an interesting tack 'Cash awards for info on diseases'
Alert the government on the occurrence of new cases of certain ailments and you may get a cash award! The government has targeted vaccine preventable diseases such as diphtheria, pertussis, measles, tetanus, and leprosy for elimination by 2016. India is also on the verge of being declared polio-free.
District medical and health officer Dr G. Srinivasulu explains that even after being declared polio-free, there should not be a single new case for 14 consecutive months in the country. This is where the reward comes in. “If anybody succeeds in detecting a new polio case meanwhile, the government will give a cash award. Even in case of detection of new leprosy cases, ASHA health workers are given Rs 150-Rs 200,” he said.
There was a competition last year from DARPA to encourage new ways to pool information held my many people.
'MIT wins $40,000 prize in nationwide balloon-hunt contest'
A team from the Massachusetts Institute of Technology won $40,000 in a high-tech scavenger hunt on Saturday by discovering the location of 10 red weather balloons.
"We're giving $2,000 per balloon to the first person to send us the correct coordinates, but that's not all -- we're also giving $1,000 to the person who invited them. Then we're giving $500 whoever invited the inviter, and $250 to whoever invited them, and so on..." it said.
Some similar system that set up a chain of reward could be really useful in disease eradication. Many security protocols rely on a sort of iterative proof of trustworthiness. Something similar could be used to allow steps toward eradication without fear some other country is going to stop efforts.
Assurance contracts are another approach. It is possible that some regions are scared that once a disease is eradicated to their area they will lose funds. Some way to guarantee that funding will not reduce or to reward successful eradication might help here. Something like Dominant assurance contracts might help to change incentives to encourage eradication.
How about a guarantee fund for each of the remaining countries with polio and guinea worm that when who declares them free they get some cash bonus. You could imagine a kickstarter project that gave the minister of health in Nigeria money when polio was declared eradicated from the country.
Another good plan is not to have fake immunisation drives against polio
The juice of the carrot, the smile of the parrot
A little drop of claret - anything that rocks
Elvis and Scotty, days when I ain't spotty,
Sitting on the potty - curing smallpox
We want to discourage deliberately being lax about a disease so you have to be clever about incentives. Anyone have any good idea for how you could bribe people and governments to further incentivise disease eradication?
Friday, December 09, 2011
Would we get more or less innovation without patents? The rose that holds patent no 20175 tells us how little we need patents. The plants have been patentable in America since 1930. 16% of Rose varieties are now patented but this increase in protection to rose growers has not increased the variety of rose varieties.
Launching The Innovation Renaissance by Alex Tabarrok is a short €2 book I read on the bus into work. It details as well as the perils of rose patents other hindrences that block wider innovation
The argument against patents is that they give too much protection to innovators. That in covering too much that is too easily thought of they actually hinder people coming up with new products. I have not seen a coherent argument that (non pharmaceutical) patents have too little protection. Tabarrok does a moderate anti patent argument forward in a clear convincing way though.
Next Tabarrok argues we should have more prizes for innovation. I have a major brain erection for kaggle and science innovation prizes in general. Tabarrok makes a clear case that prizes should be used when new ideas are needed.
Many of the faults in education are also well laid out. "the value of a permanent 25-point increase in scores ...would be 80 trillion" ..."A 25 point increase would bring the united states from about the level of mathematics education in Ireland and Spain to the level in Germany and Australia" How are we in Ireland worse at maths than Australia? Australia is mainly populated with Irish people, and the ones with an unhealthily active interest in sports at that. We have no excuse for the Aussies beating us in Maths. The education section gives several good pieces of advise for Ireland. Pay teachers based on their subject. Pay them on results not just years served. Sponsor college courses that result in more innovation that increases the standard of living more than some of the liberal arts that don't. While on the subject there is a very good blog on Irish education from the teachers side at anseo.
Immigration of high skilled immigrants is discussed next. Again like patents a reasonable achievable change is suggested rather than fully free immigration. I think Ireland should go moneyball on this. Find undervalued players in the market and buy them for our team (this would mean letting persecuted people live here, but it is an argument for another time).
The main thrust of the book is that innovation matters. Instead of a welfare state dividing a pie it might be worth thinking of an innovation state that constantly tried to make things better and increase the size of the pie. This would involve fighting entrenched interests "Few people lobby for innovation because almost by definition, innovation creates present losers and future winners and the present winners are by far the more politically powerful".
The optimistic view that we have in our power to create new and amazing things infuses this book. It is about fixes rather than showing how bad the innovation problem is as Cowen's The Great Stagnation does. None of the suggestions are on the extreme side and all are clearly explained. Because of this I think this book is likely to provide a blueprint for many improvements in the near future.
The Irish times has an interview with Tabarrok here
Thursday, December 08, 2011
"The Session, a.k.a. Beer Blogging Friday, is an opportunity once a month for beer bloggers from around the world to get together and write from their own unique perspective on a single topic". The beer blogging community really get around the session and seem to delight in sharing stories around one topic. Something similar for science bloggers would be cool, it might already exist but I cannot find it.
If some neglected interesting topic was blogged about once a month that could be really fun. To see the many angles bloggers can take on a subject.
Here is a suggestion World Leprosy day is January 30th.
How about we all write a blog about leprosy for that day?
Off the top of my head here are some possible angles
How is leprosy spread
Where is it common
Can it be eradicated? An irish leprosy charity The Leprosy Mission is here
Why can't leprosy be diagnosed until you have symptoms
Why is the Leprosy bacteria so hard to culture
Why do some people take decades to develop symptoms
Why are most people genetically immune from the disease
The reviled drug Thalidamide has a role in modern leprosy treatment why
Why is leprosy one of the oldest known infectious diseases. How can you tell a mummy has leprosy
How has treatment changed through the ages
How has leprosy effected history? From the king in Braveheart to the Gospels leprosy has been important in history.
Leopardstown I have heard was a 'leper colony'. Was it and what did it look like?
Why is there such a taboo against leprosy? What can we do to ease this?
What are the symptoms of Leprosy and why do they occur?
Armadillos can spread leprosy to humans. Other animals have also been implicated. What does it mean for the animal to carry this disease? How can infection from these animals be minimised?
Thats just a few ideas. None of which I know enough about to be able to write up. But you or someone you know might. It is a neglected disease. I have heard it small pox described as the disease that is gone but not forgotten and leprosy as the disease that is not gone but is forgotten. How about a session for science bloggers on leprosy on January 30th? If you have any ideas or comments or will commit to writing a post please comment below.
Saturday, December 03, 2011
Psychiatrist calls for lithium to be added to water
A consultant psychiatrist last night called on Government to add lithium salts to the public water supply in a bid to lower the suicide rate and depression among the general population.
At a mental health forum on “Depression in Rural Ireland” in Ennistymon, Co Clare, Dr Moosajee Bhamjee said that “there is growing scientific evidence that adding trace amounts of the drug lithium to a water supply can lower rates of suicide and depression”.
I would like to see what this scientific evidence is. The studies I have seen are from Japan where the lithium is naturally occuring. And one from Texas that seems a bit dodgy
Drinking water which contains the element lithium may reduce the risk of suicide, a Japanese study suggests.
Researchers examined levels of lithium in drinking water and suicide rates in the prefecture of Oita, which has a population of more than one million. The suicide rate was significantly lower in those areas with the highest levels of the element, they wrote in the British Journal of Psychiatry.
So taking about 1/1000 of what someone who need treatment takes could reduce suicide? There is an interesting correlation/causation problem with this study. Is it that the lithium in the water is reducing the suicide rate or could the same thing that reduces suicide increases the lithium amounts? One explanation I have heard is that places where it does not rain much water hangs around longer and rubs off rocks picking up lithium. In places where it rains a lot the lithium levels might be naturally lower as the water does not get a chance to pick up much lithium. The kicker here is that places where it rains all the time might be depressing and that could explain the increased suicide rate.
A quick check of this theory throws up some problems. Oita is in the south of Japan. Oita has 94 days of rain per year . It has Average rainfall of 1677mm
Tokyo where people top themselves at the rate of depressed lemmings at a Leonard Cohen concert also has 94 days of rain per year. 'The city of Hiroshima, in western Honshu, averages a sizeable 1,603 mm (63.1 in) of rain each year. Tokyo, further east near the Pacific, receives an annual average rainfall of 1,460 mm (57.5 in). The city of Sapporo, on Hokkaido, averages 1,158 mm (45.6 in) of precipitation per year. The southern end of the Kii Peninsula is known for a heavy annual rainfall exceeding 4,000 mm (157.5 in).'
A quick back of the envelope is not enough to discout the "rain means less lithium but also more depression" explanation. But I still don't think it is time to put lithium in the drinking water.
Takeshi Terao, a coauthor of the paper and a professor at Oita University.
"I do not think [cities should start adding lithium to the water supply], because our study is a preliminary one and further studies are required to establish evidence."
"Lithium does have its negative side effects as well. Some are mild: people often feel thirsty when taking lithium. Other side effects can be more severe, like weight-gain and diabetes and kidney problems. Lithium in the water supply could increase these side effects as well, although Terao's study didn't examine this possibility. Since the dosages are so much smaller, presumably the side effects would be as well, although more research is needed to prove that."
While on the topic of Lithium 7up used to contain lithium
The product, originally named "Bib-Label Lithiated Lemon-Lime Soda", was launched two weeks before the Wall Street Crash of 1929. It contained the mood stabiliser lithium citrate and was one of a number of patent medicine products popular in the late-19th and early-20th centuries. Its name was soon changed to 7 Up; all American beverage makers were forced to remove lithium in 1948.
Wednesday, November 30, 2011
Hama has a weird atmosphere. There is the contrast of really old norias on the river with the rest of the town. These ancient water wheels are in stark contrast to the rest of the city that is just 1980s concrete block apartments. Out in the countryside everything gets old again. They even have beehive huts like in Ireland
Its only when I got back did I read about the Hama Massacre
The Hama massacre (Arabic: مجزرة حماة) occurred in February 1982, when the Syrian army, under the orders of the President of Syria Hafez al-Assad, conducted a scorched earth policy against the town of Hama in order to quell a revolt by the Sunni Muslim community against the regime of al-Assad...
Initial diplomatic reports from western countries stated that 1,000 were killed. Subsequent estimates vary, with the lower estimates claiming that at least 10,000 Syrian citizens were killed, the majority civilians, while others put the number at 20,000 (Robert Fisk), or 40,000 (Syrian Human Rights Committee).
It has the same weird cult of personality photos of Assad everywhere that the rest of Syria has. And the same bird like men that are the security agents everywhere in the world
The parade was in the town when I was there. It was something like a Saint Patricks Day Parade 30 years ago and with better weather.
What has always stuck in my mind for some reason was one group of kids form a karate club who happily marched through the town happily waving coloured flags.
I see the the news footage now from Syria of the protests and the Assad regime violent response. I worry about those kids I saw years ago. Those kids would be the perfect age now to be angry protesting. I'm scared to think what has happened to them. They are having a march in Syria right now and its not nearly as happy as the one I saw.
Tuesday, November 29, 2011
The thing is I think I am cheating with technology predictions. Theres a wildly unpopular branch of Marxism that argues for technological determinism. Marx can be interpreted as saying that our technology is inevitable and will change us in unavoidable ways. "The windmill gives you society with the feudal lord: the steam-mill, society with the industrial capitalist"
After reading "What technology wants" by Kevin Kelly I am a convinced technological determinist. There is a podcast from Kelly on technology at econtalk
The book makes a compelling case that
1. You cannot shut yourself off form technological change. The Japanese tried it and failed. The Amish dont try it, they stay about 50 years behind on average but they do not avoid new technology forever.
2. No technology ever dies out.
3. Each new technology is an inevitable consequence of the last. No one person of country can cause or prevent a new technology, though they can shape its exact form.
Almost all patents of significant technologies have multiple very similar independant patents lodged at nearly the same time. Airplanes, radio, transisters, computers, television whatever you can think of about three guys thought of it and implemented it independantly within a few months of each other. This suggests that once the right prior technologies exist the next one is inevitable.
Kevin Kelly: What Technology Wants
Keynes wrote about 2030 in 1930 in the article “The Economic Possibilities for Our Grandchildren.” which has been surprisingly on accurate for the last eight decades. So I am betting it will continue to be for the next two. He made brought claims about future growth rates that have been accurate. I will make similar claims now. Given that I dont think non technological predictions can be made I am going to try anyway
1. Grinding poverty will be gone. This is a low bar of 365 dollars a year in 1990 dollars. Thats really bad. Much worse than medieval England but it is one I think we can reach.
2. Polio and guinea worm will be eradicated.
3. World population will be slightly lower than the medium UN estimate of 8321380. The high is 8776486 and the low is 7867332. So I will guess 8250000.
This is just a few predictions based on the world continuing to go the way it has for the last two hundred years. But like I said I cannot predict what will happen to the cash in my pocket over the next two months so two decades away is being ambitious. If you have any predictions for 2030 please put them in the comments. It might be fun for the floating brains in a jar to laugh at them at the time.
Monday, November 28, 2011
Learning by reading
Watson came along from IBM and answered quiz questions really well. Siri did something similar with peoples requests.
This might not seem like a big deal. Not many of us make a living answering quiz questions. But how many of us have jobs that involve finding the right page based on some search and parsing out the right bit of text? A surprising amount of medicine, lawyering, general office power point monkeying involves this. What happens when a cockroach knows everything? Watson is the answer to that as that is about the actual intelligence level it has. What happens when in 2030 the cockroach is a thousand times smarter? I have no idea.
Depressingly most of these systems seem to be owned by big companies like apple, Google and IBM. Most of the data used and the tools that analyse it are open source. It will be a really cool project when someone makes an opensource wikipedia based answering bot.
"if, in like manner, the shuttle would weave and the plectrum touch the lyre without a hand to guide them, chief workmen would not want servants, nor masters slaves." Aristotle wrote in Politics which says that smarter machines will free us from servitude. So far this has been the case but will that continue?.
Will my daughter ever drive for non fun reasons? People will always drive the same way people still ride horses. Just now they do it for fun rather than transport. Two trends are making it less likely that my daughter will ever have a drivers licence. The increasing sophistication of driverless cars and our decreasing acceptance of the idiocy of youth.
If Ferris Bueller had a day off now, would he spend it on Facebook? Because cars are relatively more expensive and relatively less exciting teenagers are less inclined to drive now. Facebook and youtube provide entertainment for free so you are less inclined to work thousands of hours so you can afford a mildly entertaining car. Cars are now only mildly entertaining as safety nazis have turned them all into a homogenous look that cannot even be modded easily. If you cannot put a spoiler and underlights on your Clio your just not going to look cool as a 17 year old.
Combined with cars being less attractive to teenagers there is increasing control placed over them. New drink drive limits for learners have been imposed here. I have heard talk of curfews for young drivers. We are less inclined to accept the god given right to load a car with 7 of their mates and wrap it around a tree at 4 in the morning. In 2030 well have so many constraints on teenagers to prevent them killing themselves as to remove much of the attraction from driving.
The technology element of this is the driverless car
So a huge upcoming policy question is: when will what big cities manage to coordinate to change road law to achieve these huge auto-auto economic gains? Thirty years from now we may look back and lament that big city politics was so broken that no big cities could manage it. Or perhaps history will celebrate how the first big city to do it dramatically increased its importance on the world scene
The typical American spends an average of roughly 100 hours a year in traffic; imagine using that time in better ways — by working or just having fun. The irksome burden of commuting might be lessened considerably. Furthermore, computer-driven cars could allow for tighter packing of vehicles on the road, which would speed traffic times and allow a given road or city to handle more cars
These technologies will come in gradually drive train technology, lane assist, parking assist, crash avoidance are all present in next years s-class Mercedes. Probably legal hangups will delay driverless cars. If I was to guess it will be the old that get them pushed through. The old are increasing in numbers and will continue to vote. The baby boomers won't accept the loss of independence that goes with not being able to drive currently. Legal changes to allow driverless cars could be a vote winner. I'm willing to bet that by 2030 a combination of our scardy-cat nature about risk, the grey vote and improved technology will mean that my daughter may never have to drive.
The exponential improvement in solar power became news this year.
Averaged over 30 years, the trend is for an annual 7 percent reduction in the dollars per watt of solar photovoltaic cells...10 years later, in 2030, solar electricity is likely to cost half what coal electricity does today
This article started a great big argument about why solar is not replacing fracking and such. Including why environmentalists are still worried about co2. What will happen in 2030? I dont think we'll be using enough less energy to prevent climate change that way. I think well still be using fossil fuels to fly planes. New electricity generation methods take decades to become part of the grid, but I am still optimistic about solar.
Education has not changed much in a long time. This is partly due to Baumol's cost disease where a teacher now can only teach the same number of kids as one in 1900. Since that time manufacturing jobs can produce vastly more than they used to. But education is one of those human centric industries that has not changed much.
Since videos came out it has been possible to watch lectures at home. Just giving kids computers does not improve education. If you look at the list here of the best educational strategies Tutorial instruction, Reinforcement, Corrective feedback, Cues and explanation are now part of online educational programs.
Stanford this year ran three computer courses where anyone can sign up for free, get homework graded by computer and do exams at the end. This is a graduate level course in one of the worlds best universities for free. The Khan academy has been around for a while but this year it moved from just a list of videos to include problems. The site is now using machine learning and gamification techniques to improve these problems and thus student learning. They are also using statistical tests to improve the website layout and which version of a explanation gets used. These computer assisted techniques are even being used by humans to get better at Jeopardy as the video below describes.
Interestingly the same guys who got learning by reading and driverless cars to their current state are also important in these changes to education Sebastian Thrun, Andrew Ng and Peter Norvig.
This year technologies became mainstream that finally turn computers into the amazing educational tools they have always promised to be.
How did these areas change from when I was born till I was 18?
Cars became safer and drink driving became mildly uncool. The main change here was removing lead from petrol though. The recent drop in crime and increase in IQ may be largely due to removal of leaded petrol. I think driverless cars are important but not as important as this.
Power generation changed a bit, Chernobyl made nuclear uncool but it was probably never economical anyway.
Education did not change much. I did example questions from the year I was born to practice for the leaving cert. Corporal punishment still existed in my time but mainly for nostalgia rather than because they got true joy battering children. I was born after the education system demilitarized. I predict education will change more in my daughters childhood than mine.
Computers did not really visibly exist for people when I was born. By the time I was 18 it was obvious they were going to change the world even if they had not fully done that yet. I think the change from no computers to some was bigger than useful computers to really useful ones that I predict will happen.
Here is the prediction in 2030 for the technologies that became public this year. Many of the cars will be driverless. The world will get a most of its new generational electrical capacity from solar. Education has already changed massively this year just people have not realised it yet. We will each have an assistant that knows everything. I don't know how that will effect employment, my guess is she will end up working in a job that does not exist at the moment.
Wednesday, November 16, 2011
This was the natural state of books before the printing press. The Iliad, the Táin Bó Cúailnge and other epic poems were designed to be memorised and recited rather than read. But what books do people memorize now? I have included the text file size to give some idea of how long the book is.
The Bible 4.2 MB
I have never heard of anyone who memorized the bible but it does seem oddly popular. A Plan For Memorizing The New Testament In 5 Years. The New testament is 996 kB in size. I have never met anyone who has memorised the bible. How many people have?
Milton: Paradise Lost John Basinger did this in his 70's. 12 books, 10565 lines and 60,000 words in 8 years. The filesize is about 495 kB.
The Qu'ran 1.1 MB People who have memorised this have a special name of Hafidh. 'The total number of hafidh and hafidhas currently alive in the world has been estimated in the tens of millions.'
'But becoming a hafiz is also believed to bring rewards in the hereafter, guaranteeing the person entrance to heaven, along with 10 other people of his choosing, provided he does not forget the verses and continues to practice Islam.'
'The children, ages 7 to 14, are full-time students, in class 8 a.m. to 5 p.m., Monday through Friday, even in the summer. But they are not studying math, science or English. Instead, they are memorizing all 6,200 verses in the Koran, a task that usually takes two to three years.'
that is three years of full time work? Over 5000 hours or over half way to Gladwell's 10000 hours theory. Presumably once you spend 5000 hours memorising the Qur'an you would spend a considerable amount of time thinking and discussing it. Which means there are millions of near genius level experts in the Qur'an in the world.
I can think of things I would rather be an expert in but it is worth considering what it is about the Qu'ran that devotes this level of commitment to it.
Mao: The Little Red Book 273kb
One of the most read books ever. It still seems remarkably popular given it was written by the man who killed the most people ever.
'Acknowledging that she had never memorized Mao's quotations, Chen Di said the indoctrination of Mao, which was still inscribed on a few school walls today, was impressed in her mind as a primary school girl.' I cannot find a figure on how many people memorized all of this book but at 900 million copies 'sold' and many of them expected to memorise large passages it must be in the millions.
The Torah: Memorizing the Jewish holy book seems to have an important place in the religion but I cannot get any figures on how many people today have done this.
Can you think of any others? Dr Seuss' 'Cat in the Hat' has probably been memorised by many adults just from reading it to their kids a thousand times.
Books designed to be memorised, like the iliad, have tricks involving characters, meter, rhyme and rhythm. I have gotten two books recommended by Josh Foer, The Book of Memory and Memory in Oral Traditions that examine these methods. So hopefully I will blog about these tricks again.
The effort people must go to to memorize a book is incredible. We can now buy books for the cost of a few hours work. Or a kindle that we can get thousands of books on gutenberg for under sixty euro. Books are 0.2% of the average Americans spending. To think that we have such a low cost per hour worked on a physical or digital book now but people will still spend thousands of hours memorizing one is amazing.
Sunday, November 06, 2011
I've wondered for a while about how far away we could see the earth if we were using our current technology and we had our current light output. At what distance could we see the artificial light the earth produced?
This paper Detection Technique for Artiﬁcially-Illuminated Objects in the Outer Solar System and Beyond provides some sort of evidence on the question
Existing optical telescopes and surveys can detect artiﬁcially illuminated objects comparable in total brightness to a major terrestrial city at the outskirts of the Solar System
For this signature to be detectable, the night side needs to have an artiﬁcial brightness comparable to the natural illumination of the day side. Clearly, the corresponding extraterrestrial civilization would need to employ much brighter and more extensive artiﬁcial lighting than we do currently since the global contrast between the day and night sides is a factor ∼ 6 × 10^5 for the present-day Earth
Lets ignore that our telescopes will get better (about 2.5% a year?) so how quickly at current rates will the earth take to be this bright?
“over the last three centuries, and even now, the world spends about 0.72% of its GDP on light. This was the case in the UK in 1700 (UK 1700), is the case in the undeveloped world not on grid electricity in modern times, and is the case for the developed world in modern times using the most advanced lighting technologies.”
But what is that in terms of extra light produced? "In 1700 a typical Briton consumed 580 lumen-hours in the course of a year, from candles, wood and oil. Today, burning electric lights, he uses about 46 megalumen-hours—almost 100,000 times as much."
So that means if we grew at the same rate we have for the last 300 in about 300 years we will be producing enough light that if it was all shone out into space the earth would be as bright at night to an outside observer as it is in the day. I dont think you can make projections this far and it still leaves the question of how long after that ambient external light would be that bright. Still I think its interesting that even with current detection technology as civilisation the 300 years advanced from us would be nearly visible to us. I would guess this prediction will reduce but 300 years is my first estimate to when we could see ourselves.
Tuesday, November 01, 2011
Today, just like every day for the last 50 years, around half a million babies will be born.
This is not true the figures from the UN are here. In detailed indicators look in births and in select country look in world.
Between 1960-1965 302136 babies were born each day. Between 1985 and 1990 375909 babies and between 2005 and 2001 367320 babies. The 500000 figure is not just wrong but drastically wrong.
The geometric nature of population growth makes it extraordinarily difficult to arrest, and almost impossible to reverse. The last population doubling took only 40 years. Even if global population growth rate drops to just one per cent, today’s seven billion would swell to an unimaginable 14 billion in 70 years.
The growth rate is being arrested. As I have said before in "We have reached Peak Baby" the number of children each woman has has been falling for decades.
"1968 women got 1.87 adult daughters. Ehrlich called it population bomb. Now women get 1.07 daughters, 92% of way to population balance done." Says Hans Rosling. In 43 years we have gone from 1.87 adult daughters to 1.07 a decline of .8 and we only need a decline of .07 more to reach long term balance of the numbers of women. Rural agrarian Bangladesh has reached stasis in adult female population levels for example.Look at the gapminder video here for yourself the number of children per women is declining
Though surprisingly accurate population estimates get better over time. For example the 2050 estimate has been recently honed in
U.N. Raises “Low” Population Projection for 2050
The "low-variant" scenario of population growth now foresees 117 million more people on the planet in 2050 than it did two years ago.
While the "median-variant" scenario, often seen as "most likely," remains almost the same as before - predicting a world with 9.2 billion people by mid-century
The high projection, however, foresees some 10.5 billion people - a 295 million person decrease from the previous high projection. The medium projection is 9.2 billion people,
It says something that an decline in the high estimate an over 2.5 times bigger than the increase in the low estimate is not the headline.
There are still countries that have very high birth rates per woman. Senegal for example had 7.5 babies per woman in 1968 4.8 babies and in 2010. But these countries are developing at a speed that is likely to see these birthrates drop rapidly. "Senegal has lower borrowing costs than Ireland." and a GDP growth rate of 4.2%. As these countries that still have a high birthrate develop their birthrate will drop rapidly the way ours and other developed countries did.
The article is in the Irish Times gives the wrong figure for the number of births per day and "unimaginable 14 billion" scare figure. The demographic evidence and historical trends indicate the population will not go to this level. Current UN estimates are for the high population prediction do not match this figure for 2081. There were never that many babies born per day and the birthrate has been falling so fast that we will not reach the 14 billion in 2081 figure.
Monday, October 10, 2011
Adam Savage the Mythbuster talks here about the same obsession he has
Like Möbius strips, Dinosaurs and Steve Reich, Dodos are like flypaper for nerds. There is just something about these things that seem to spark off nine year olds and set them off on a lifetime being the makers and doers of nerdy things. It was the dodo in the natural history museum of Ireland, known locally as "the dead zoo" that set me off at this age.
Unfortunately this dodo is no longer viewable to the public. I was in the museum last week, admiring the cool steampunk Victoriana makes the museum as interesting for what it says about the 1800's when it was created as about the animals in it. For example the polar bear still has an obvious bullet mark in its head, you would not see that in a modern museum.
The museum was bigger but large areas are no longer open
Exhibition space does a dodo and vanishes
The museum shut on July 5, 2007, when a flight of the main staircase collapsed. The Department of Arts originally decided to take the opportunity to carry out a €15m renovation and extension, but that was cancelled last year due to the state of the public finances.
Instead the staircase was reinstated and other minor repair works carried out, but two large balconies on the second floor have been shut off from the public due to health and safety concerns. The museum’s biggest attraction, a skeleton of a dodo, is on one of these balconies and, unless it is moved, will no longer be viewable by visitors.
Cuts keep half of 'Dead Zoo' exhibits out of public view
However, some of its best known exhibits, including a dodo skeleton and a piece of moon rock given to the State by the US, will not be on public view for the foreseeable future.
This glass box in the top floor bottom middle holds the Dodo skeleton.
The Dodo is one of the most iconic images of environmentalism before Stuart Brand persuaded Nasa to take a release of the earth the phrase "as dead as a dodo" was one of the best metaphors for the fragility of nature that we had. This sight of these bones is an important warning for us and we should make every effort that nine year olds now get to see them. For this reason I want to see the Dublin NHM Dodo moved down to a floor that is open to the public. Moving a glass box is not going to make a massive improvement to the world, but it should also not be a huge difficulty. A full restoration of the museum would be vastly preferable to this but this would be a relatively simple change to make in the meantime. This dodo is important the Natural History Museum badge has a dodo on it which says to me it is one of the most important and interesting items in the museum.
Here is my favorite song sung by an extinct flightless bird The Mountain Goats- Deuteronomy 2:10
I'm all alone here as I try my tiny song
Claim my place beneath the sky but i won't be here for long
I sang all night the moon shone on me through the trees
No brothers left and there'll be no more after me
Thursday, September 15, 2011
If you ever write a book or need to name a shop you should spend a few quid buying google ads for the two names you are trying to pick "How to start a fight" and "How to win an argument" and see which one gets clicked on more. That one should be the name.
This is one of those embarrasing posts that is probably wrong. But I think the A/B testing used by the Khan academy makes a fairly fundamental mistake. The Khan acedemy has lessons in various subjects. So presumably they want to use A/B testing to see if kids taught "1+1=2" or "1 + 1 = 2" learn more quickly and such.
A/B testing is a useful way to see if little tweaks result in better user experience. In Khan's case learning. It does not substitute for good design vision but can help make some relatively small tweaks. Improving the Khan acedemy and kids education is really important so if there is a bug in their A/B testing they might be making the wrong choices about how to improve their teaching.
For this kind of testing you need to pick the number of test cases in advance. How not to run an A/B test explains why and the effects of looking before all the test is finished. This is an odd feature of frequentist statistics
'However, the significance calculation makes a critical assumption that you have probably violated without even realizing it: that the sample size was fixed in advance. If instead of deciding ahead of time, “this experiment will collect exactly 1,000 observations,” you say, “we’ll run it until we see a significant difference,” all the reported significance levels become meaningless. This result is completely counterintuitive and all the A/B testing packages out there ignore it'
The A/B testing used by Khan seems not to do this as the gae bingo system says
"Controlling and ending your experiments
Typically, ending an experiment will go something like this:
You'll notice a clear experiment winner and click "End experiment, picking this" on the dashboard. All users will now see your chosen alternative."
This seems to be saying that either you should notice what is statistically significant which you won't always or that something can be declared statistically significant before all the samples are tested. Think of it this way. If every test has a 5% chance of being wrong and you think of everytime you look at the A/B test as adding 5% to the chance of being wrong. It is not quite that bad but it gives you a feeling of the problem.
Now there are ways you can tell that something is statistically significant really early in a test. "Bayesian Statistics and the Efficiency and Ethics of Clinical Trials" deals with these. In medical trials you want to know as early as possible if a new treatement is better or worse than an old one. Giving someone the wrong ad wont kill anyone but the wrong cancer drug might. This paper goes through how you would figure this out using Bayesian methods. These methods are also described in chapter 37 of MacKay's 'Information Theory, Inference, and Learning Algorithms'
But looking at the code GAE bingo uses for A/B testing they do not seem to be using these methods. So it looks to me that they are making the mistake of letting you stop a test when you want to. Which in frequentist statistics can be an error.
Also I think Vanity another rails A/B testing framework makes the same assumption
"This experiment will conclude once it has 1000 participants for each alternative, or a leading alternative with probability of 95% or higher:"
The system used by the BBC is based on time and not numbers according to this article. "Example use
For 5 in 100 people to get a two-option test running for 24 hours the function is initialised like this:". Which is not nearly as bad. But it is assuming that at the end of the time period you have had enough users to make a good test.
There is a proper explanation as to what can happen if you stop a trial early in "How not to run A/B testing". But from my reading many of the A/B testing frameworks out there seem to be making this error. Please correct me in the comments.
Addition: Ben from gae bingo got back to me in a comment on their blog.
"You're right that this is an issue, and that's a great blog post. However, this is significantly mitigated by a) letting your experiments run long enough to get a large-ish sample size for your population and b) simply not checking your dashboard constantly and making snap decisions.
We could build stuff into the system to mandate that, but at the moment I believe we'll be able to get solid value out of the existing framework (just like most A/B systems)." This seems fair enough, Khan are going to have such a high volume of users that they will be able to get a large sample size quickly.
Allen Downey has a great review of the problem with simulations here
Wednesday, August 17, 2011
The recent riots in London had all sorts of people claiming the world had gone Mad Max and it was time to shut down facebook and make all teenagers join the army. There is a great summary of the moral panic in this article "Civil disorder and looting hits Britain, We have been here before". I think its interesting how the rioting that probably wont even be visible in the end of the year crime statistics made everyone freak out so much.
For all the talk of moral decay is the world in general a worse place than it used to be? War along with the other horsemen is a pretty good definition of how bad things are. Unless Bob Geldof finally gets that concert to end death set up. Even pestilence would seem to be a big ask to fix. What is the trend for war deaths recently?
Many people have predicted a century of appalling death tolls from war. Robert McNamara the ex US defense secretary and self confessed war criminal (at 3:10 ) for example.
Blight and McNamara project the level of warfare forward into the twenty- first century based on population growth, and suggest a “speculative” but “conservative” estimate of “at least 300 million” fatalities from war in the twenty-first century, of which perhaps 75 million would be military
So how have we done in the last decade on the predicted 3 million deaths a year?
So far they haven't even been close. In fact, the last decade has seen fewer war deaths than any decade in the past 100 years, based on data compiled by researchers Bethany Lacina and Nils Petter Gleditsch of the Peace Research Institute Oslo. Worldwide, deaths caused directly by war-related violence in the new century have averaged about 55,000 per year, just over half of what they were in the 1990s (100,000 a year), a third of what they were during the Cold War (180,000 a year from 1950 to 1989), and a hundredth of what they were in World War II.So there has been under 2% of the expected war deaths this decade.
The Quaker economist describes the decline with "Just in the last fifty years, the progress towards peace has been startling. If this trend continues — and I am cautiously optimistic that it will — then as pressure from civil society increases and international institutions of conflict resolution mature, warfare as a regular practice may cease sometime late this century."
While talking about sending the four apocalyptic horsemen to their dotage the decline of Famine is also impressive
On the day that a famine is declared in Africa -- thanks as much to Somalia's political dysfunctionality as to a severe drought -- comes news that the world drought severity index has been falling for three decades. You can check for yourself by going here and clicking on "make time series":
There are some fairly serious disclaimers on this famine decline. The first is that agricultural productivity growth is declining at the moment
combine this with a possible loss of farm productivity when global warming kicks in and there could be problems.
While on the subject of how things are not good but significantly better than they used to be the last decade was the best one in human history.
We estimate that between 2005 and 2010, nearly half a billion people escaped extreme hardship, as the total number of the world's poor fell to 878 million people.
Nearly everything is getting better. Including life expectancy which you could take as evidence that even death is not what he was. Oddly all three of these sources give different numbers on 'major armed conflicts' but all agree the number is declining.
If two of the biggest problems mankind has faced for all of history are on the decline so rapidly they could be almost unknown within our life times I would say that is good news. And probably more important evidence of how we are progressing than a few rioters stealing runners.
Monday, August 15, 2011
The Effect of Testing for Performance Enhancing Drugs on the Progress of World Records in Weightlifting “From 1964 to 1988 the relative strength of the world record holders in those weight classes increased by 21% …The same analysis in other types of sports, where there had been some changes in training methods over the same period of time, revealed that the maximum improvement was only 9%“ So most improvement in weightlifting seems to have been from pharmacological rather than having a wider range of people to select from or improved training mechanism reasons. However what about areas of human endeavour that drug taking seems unlikely to help?
In chess taking steroids seems unlikely to help. Though future nootropics might. "We conclude that there has been little or no ‘inflation’ in ratings over time—if anything there has been deflation. This runs counter to conventional wisdom, but is predicted by population models on which rating systems have been based…The results also support a no answer to question 2. In the 1970’s there were only two players with ratings over 2700, namely Bobby Fischer and Anatoly Karpov, and there were years as late as 1981 when no one had a rating over 2700 (see [Wee00]). In the past decade there have usually been thirty or more players with such ratings."
Even musicians are getting better Virtuosos Becoming a Dime a Dozen "The overall level of technical proficiency in instrumental playing, especially on the piano, has increased steadily over time." One good explanation for this and the chess improvement is just that more people are getting to try these, people who are not as limited by nutrition and disease as they would have been int he past.
This explanation is expanded in this rather good post "Two Hour Marathon in 2045"
"But the pipeline that selects and trains runners behaves, in some ways, like the model. If a person with record-breaking potential is born in Kenya, where running is the national sport, the chances are good that he will be found, he will have opportunities to train, and he will become a world-class runner. It is not a certainty, but the chances are good.
If the same person is born in rural India, he may not have the opportunity to train; if he is in the United States, he might have options that are more appealing.
So in some sense the relevant population is not the world, but the people who are likely to become professional runners, given the talent. As long as this population is growing exponentially, world records will increase linearly.
That said, the slope of the line depends on the parameter of exponential growth. If economic development increases the fraction of people in the world who have the opportunity to become professional runners, these curves could accelerate."
Progress of things like number of children suffering malnutrition and having clean water can really result in increasing the number of great chess or piano players as well as the world running record. We are getting better at loads of things because more people are getting to try them without the poverty induced hinderances they used to have. According to this model it is not that population is increasing exponentially but at the moment population of people who have a chance at being great at something is.
If you have other explanations for why and in what human achievment is progressing I would love to hear them.
Tuesday, August 09, 2011
I saw this paper 'Austerity and Anarchy: Budget Cuts and Social Unrest in Europe, 1919-2009'
with the intro
'In the wake of this week's London riots, some commentators have linked the youth unrest to budget cuts. The authors of CEPR DP8513 explore the historical basis for this view and finds that austerity and violence have tended to go hand in hand.'
I am reading through the paper now. But I wonder if there was an image that could quickly show a connection between the two.
Wikipedia lists the riots since 1950 as
1958 Notting Hill race riots
1970 Garden House riot
1971 Priestlley riots
1975 Chapeltown race riot
1977 Battle of Lewisham
1980 St. Pauls riot
1981 England riots · Brixton riot · Chapeltown race riot · Toxteth riots · Moss Side riot · Handsworth race riots
1985 Brixton riot · Broadwater Farm riot
1987 Chapeltown race riot
1989 Dewsbury race riot
1990 Strangeways Prison riot · Poll Tax riots
1991 Meadow Well riots
1995 Manningham riot · Brixton riot
2001 Bradford riots · England riots · Oldham race riots · Harehills riot
2005 Birmingham race riots
2010 UK student protests
2011 London riots
Now if you look at the government spending as a percentage of GDP here. On top of this graph I put a bar for every riot each year one occurred.
Counting all riots as the same is not fair. Their graph goes from 34->48 whereas the riots go from 0->6. Laying the first on the second is not considered good practice in data visualisation. Also and this is a big one. If GDP drops as in a recession and the percentage of government spending to GDP stays the same total government spending will drop. a fairer graph would look at gdp or government spending adjusted compared to riots not the two combined.
The proper paper says rioting and austerity go hand in hand. I will read it carefully to see how close the link is. But a quick look at the data and no obvious major link jumps out at me.
Friday, August 05, 2011
When will we reach peak babies? In what year will the most children be born? I bet last night a shiny pint that we will reach peak babies in the next three years. That the most babies ever born will be in a year before 2015.
Now I accept that we could never actually know how many children will be born in the future. The bet will end when I present enough evidence to convince those I am betting with rather than with a proof. Demographics is regarded as one of the most predictable of social sciences but some possible future invention could drastically increase the number of babies. We could have brave new world style artificial wombs of some such that vastly increases the birth rate for example.
Hans Rosling the statitician tweeted recently.
I looked up the UN data on this here. In detailed indicators look in births and in select country look in world. The highest birth number in the world was 1985-1990
Period Births per year
1950-1955 97 769
1955-1960 102 894
1960-1965 110 280
1965-1970 118 200
1970-1975 121 715
1975-1980 120 676
1980-1985 129 088
1985-1990 137 207
1990-1995 134 960
1995-2000 132 473
2000-2005 131 644
2005-2010 134 072
2010-2015 135 775
2015-2020 135 396
2020-2025 133 800
2025-2030 132 452
2030-2035 131 991
2035-2040 132 099
2040-2045 131 926
2045-2050 131 127
2050-2055 129 904
2055-2060 128 785
2060-2065 127 998
2065-2070 127 402
2070-2075 126 725
2075-2080 125 823
2080-2085 124 775
2085-2090 123 753
2090-2095 122 837
2095-2100 121 992
Now it could all of a sudden rise up and any prediction for the future is unlikely to be as accurate as historical estimates. Still I think I win the bet.
This does not mean that population will drop. As we are living longer world population is still expected to grow. But it does mean we can say that Malthus was wrong. Human numbers will not grow exponentially barring some disaster.
Thursday, August 04, 2011
All my fantasy football attempts have the problem that I know nothing about football. So for example Berbatov is unlikely to do as well this season as last season so picking him would seem unwise. But how does someone who knows nothing about football find out who will play well next season?
The wisdom of the crowds is the James Surowiecki about "the aggregation of information in groups, resulting in decisions that, he argues, are often better than could have been made by any single member of the group." This sort of thing does not work if the crowd has a bias in a particular direction.
I decided to look at the people is the "team selected by %" that fantasy premier league shows you. If I rerun the optimisation described in this post but instead of trying to great a team that has the maximum number of points last season I try and get the team whose players have been selected by the most other fantasy football managers.
The idea is that a team with the right number of defenders, goalkeepers, midfielders and forwards, that has at most three players from one team, that costs less than 100 and whose members have been picked most often should be really good.
The most popular team is
Player Club Pos Price Pts people
7 Al-Habsi WIG GK 45 125 199410
30 Bale TOT MID 80 118 189087
37 Barton NEW MID 60 131 123865
69 Cahill BOL DEF 55 105 180751
184 Given AVL GK 50 0 215091
205 Hangeland FUL DEF 65 154 192333
231 Huth STO DEF 60 138 165684
266 Kompany MCI DEF 60 95 197352
323 N'Gog LIV STR 55 48 279932
334 Odemwingie WBA STR 75 171 218485
421 Suarez LIV STR 95 68 343361
424 Taarabt QPR MID 65 0 141086
452 Vidic MUN DEF 80 148 172773
472 Wilshere ARS MID 65 93 220381
477 Yaya Toure MCI MID 80 146 167706
Some of these players were probably picked for the first few games and will be transfered out when they are about to play tougher games.
The average player is picked 26794 times. These players have been picked 200486 people on average.
This team would have scored 1415 points last season. The best team I could have picked for last season would have scored 2332 points. I have updated the dataset to include this people picked statistic.