Friday, November 20, 2015

Bombing Back to the Stone Age

There is a common meme that is easy to find with a twitter search

If you read Jared Diamond or Stephen Pinker they talk about the really high levels of violence in the stone age.
Or to describe it with statistics

"By many estimates, 10 to 20 percent of all Stone Age humans died at the hands of other people.
This puts the past 100 years in perspective. Since 1914, we have endured world wars, genocides and government-sponsored famines, not to mention civil strife, riots and murders. Altogether, we have killed a staggering 100 million to 200 million of our own kind. But over the century, about 10 billion lives were lived — which means that just 1 to 2 percent of the world’s population died violently. Those lucky enough to be born in the 20th century were on average 10 times less likely to come to a grisly end than those born in the Stone Age. And since 2000, the United Nations tells us, the risk of violent death has fallen even further, to 0.7 percent."


To reduce violence don't send people back to the stone age.

Tuesday, November 17, 2015

How Good Will the Upcoming Stephen King Adaptations be?

This is the third and final post in a series on Stephen King adaptations. (The first is Are Stephen King films better than the books?, and the second Do Stephen King's Better Books Make Better Films?)
According to IMDB the Stephen King Novels below are currently being adapted
Titlegood doubled
The Talisman8.2
Rose Madder7.22
Lisey's Story7.22
Mr. Mercedes7.74
Gerald's Game6.86
Cell7.22
11.22.638.52
The Dark Tower8
Three of these books have exactly the same rating on Goodreads. The correlation between the ratings and his books and his films previously discussed I will use to predict future movies ratings. The analysis gives a 95% confidence prediction intervals of
Titleestimatelower boundupper bound
The Talisman7.56.48.6
Rose Madder5.954.87.0
Lisey's Story5.954.87.0
Mr. Mercedes6.795.78.8
Gerald's Game5.374.26.4
Cell5.954.87.0
11.22.638.056.99.15
The Dark Tower7.26.18.2
The upcoming films with their predicted IMDB ratings are in red in the graph below. Three are predicted to have a rating of 5.95.

There is an interesting 538 podcast here about a company that predicts film earnings. He mentions the correlation between film quality and earnings and film quality. This whole topic is an interesting challenge for prediction.

Stephen King is unique among authors in the number and variety of adaptations his works have gone through. So he is possibly the only author this could be even tried with. I am really looking forward to 11.22.63 and the Talisman now. And if Gerald's Game beats the predicted IMDB score that is a bonus.




Code


mydata = read.csv("King.csv")  
library(ggplot2)
attach(mydata)     # attach the data frame 
king.lm = lm(imdb ~ good.doubled)

Call:
lm(formula = imdb ~ good.doubled)

Coefficients:
 (Intercept)  good.doubled  
      -5.821         1.628  

upcoming = read.csv("upcoming.csv")  
predict(king.lm, upcoming, interval="predict")

> predict(king.lm, upcoming, interval="predict")

Title,Publication date,Pages,imdb,goodreads,good doubled,clr
Carrie,05/04/1974,199,7.4,3.89,7.78,1
Salem's Lot,17/10/1975,439,6.8,3.97,7.94,1
The Shining,28/01/1977,447,8.4,4.12,8.24,1
The Stand,Sep-78,823,7.3,4.32,8.64,1
The Dead Zone,Aug-79,428,7.3,3.88,7.76,1
Firestarter,29/09/1980,426,6,3.8,7.6,1
Cujo,08/09/1981,319,6,3.61,7.22,1
The Running Man,May-82,219,6.6,3.74,7.48,1
Christine,29/04/1983,526,6.6,3.69,7.38,1
Pet Sematary,14/11/1983,374,6.6,3.86,7.72,1
Thinner,19/11/1984,309,5.7,3.6,7.2,1
It,15/09/1986,1138,6.9,4.12,8.24,1
Misery,08/06/1987,310,7.8,4.06,8.12,1
The Tommyknockers,10/11/1987,558,5.4,3.42,6.84,1
The Dark Half,20/10/1989,431,5.9,3.71,7.42,1
Needful Things,Oct-91,690,6.2,3.84,7.68,1
Dolores Claiborne,Nov-92,305,7.4,3.76,7.52,1
The Green Mile,March–August 1996,400,8.5,4.39,8.78,1
Bag of Bones,22/09/1998,529,5.8,3.84,7.68,1
Dreamcatcher,20/03/2001,620,5.5,3.53,7.06,1
Under the Dome,10/11/2009,1074,6.8,3.89,7.78,1
Shawshank Redemption,10/11/2009,181,9.3,4.51,9.02,1
Stand by me,10/11/2009,80,8.1,4.25,8.5,1
The Mist,10/11/2009,230,7.2,3.88,7.76,1
The Langoliers,10/11/2009,230,6.1,3.71,7.42,1
Apt Pupil,1983,179,6.7,3.8,7.7,1
Hearts in Atlantis,2000,640,6.9,3.77,7.54,1
The Talisman,na,na,7.5,4.1, 8.2,2
Rose Madder,na,na,5.95,3.61, 7.22,2
Lisey's Story,na,na,5.95,3.61, 7.22,2
Mr. Mercedes,na,na,6.79,3.87, 7.74,2
Gerald's Game,na,na,5.37,3.43, 6.86,2
Cell,na,na,5.95,3.61, 7.22,2
11.22.63,na,na,8.05,4.26, 8.52,2
The Dark Tower,na,na,7.2,4,8,2


mydata = read.csv("King.csv")
attach(mydata)
p1 <- ggplot(mydata, aes(x=good.doubled, y=imdb)) +
    geom_point(colour = factor(clr),shape=1,size=2) +    # Use hollow circles
    geom_smooth(method=lm,se=FALSE)           
            
p1 <- p1 + ylab("IMDB Ratings")  
p1 <- p1 + xlab("GoodReads Ratings")  
p1 <- p1 + ggtitle("Upcoming Stephen King Adaptations")
p1 <- p1 + annotate("text", x = 6.97, y = 5.26, label = "The Tommyknockers", size=3, colour="blue3")  
p1 <- p1 + annotate("text", x = 7.85, y = 7.42, label = "Carrie", size=3, colour="blue3") 
#Salem's Lot,17/10/1975,439,6.8,3.97,7.94,1
p1 <- p1 + annotate("text", x =8.0 , y =6.9, label = "Salem's Lot", size=3, colour="blue3") 
p1 <- p1 + annotate("text", x = 7.8, y = 6.5, label = "Pet Sematary", size=3, colour="blue3") 
p1 <- p1 + annotate("text", x = 8.34, y = 8.51, label = "The Shining", size=3, colour="blue3") 
p1 <- p1 + annotate("text", x = 8.62, y = 8.2, label = "Stand By Me", size=3, colour="blue")
p1 <- p1 + annotate("text", x = 8.9, y = 8.4, label = "The Green Mile", size=3, colour="blue3") 
p1 <- p1 + annotate("text", x = 9.0, y = 9.09, label = "Shawshank\nRedemption" , size=3, colour="blue3") 
p1 <- p1 + annotate("text", x = 8.75, y = 7.3, label = "The Stand", size=3, colour="blue3") 
p1 <- p1 + annotate("text", x = 8.27, y = 6.85 , label = "It", size=3, colour="blue3") 
p1 <- p1 + annotate("text", x = 8.19, y = 7.74, label = "Misery", size=3, colour="blue3") 
p1 <- p1 + annotate("text", x = 8.05, y = 6.7, label = "Under the Dome", size=3, colour="blue3") 
#Under the Dome,10/11/2009,1074,6.8,3.89,7.78,1
p1 <- p1 + annotate("segment", x = 7.9, xend = 7.79, y = 6.7, yend = 6.8, colour = "blue3") 
#Dolores Claiborne,Nov-92,305,7.4,3.76,7.52,1
p1 <- p1 + annotate("text", x = 7.5, y = 7.3, label = "Dolores Claiborne", size=3, colour="blue3") 
#The Dark Half,20/10/1989,431,5.9,3.71,7.42,1
p1 <- p1 + annotate("text", x = 7.5, y = 5.81, label = "The Dark Half", size=3, colour="blue3")
#Bag of Bones,22/09/1998,529,5.8,3.84,7.68,1
p1 <- p1 + annotate("text", x = 7.78, y = 5.7, label = "Bag of Bones", size=3, colour="blue3")
#The Dead Zone,Aug-79,428,7.3,3.88,7.76,1
p1 <- p1 + annotate("text", x = 7.5, y = 7.72, label = "The Dead Zone", size=3, colour="blue3")
p1 <- p1 + annotate("segment", x = 7.76, xend = 7.5, y = 7.33, yend = 7.66, colour = "blue3") 
#The Mist,10/11/2009,230,7.2,3.88,7.76,1
p1 <- p1 + annotate("text", x = 7.76, y = 7.1, label = "The Mist", size=3, colour="blue3")
#Firestarter,29/09/1980,426,6,3.8,7.6,1
p1 <- p1 + annotate("text", x = 7.71, y = 6, label = "Firestarter", size=3, colour="blue3")
#The Langoliers,10/11/2009,230,6.1,3.71,7.42,1
p1 <- p1 + annotate("text", x = 7.56, y = 6.11, label = "The Langoliers", size=3, colour="blue3")
#Cujo,08/09/1981,319,6,3.61,7.22,1
p1 <- p1 + annotate("text", x = 7.23, y = 6.1, label = "Cujo", size=3, colour="blue3")
#The Running Man,May-82,219,6.6,3.74,7.48,1
p1 <- p1 + annotate("text", x = 7.48, y = 6.7, label = "The Running Man", size=3, colour="blue3")
#Christine,29/04/1983,526,6.6,3.69,7.38,1
p1 <- p1 + annotate("text", x = 7.38, y = 6.5, label = "Christine", size=3, colour="blue3")
#Thinner,19/11/1984,309,5.7,3.6,7.2,1
p1 <- p1 + annotate("text", x = 7.25, y = 5.6, label = "Thinner", size=3, colour="blue3")
#Needful Things,Oct-91,690,6.2,3.84,7.68,1
p1 <- p1 + annotate("text", x = 7.83, y = 6.2, label = "Needful Things", size=3, colour="blue3")
#Dreamcatcher,20/03/2001,620,5.5,3.53,7.06,1
p1 <- p1 + annotate("text", x = 7.2, y = 5.5, label = "Dreamcatcher", size=3, colour="blue3")
#The Talisman,na,na,7.52,4.1, 8.2,2
p1 <- p1 + annotate("text", x = 8.36, y = 7.52, label = "The Talisman", size=3, colour="red3")
#Rose Madder,na,na,5.93,3.61, 7.22,2
p1 <- p1 + annotate("text", x = 7.07, y = 5.93, label = "Rose Madder", size=3, colour="red3")
#Lisey's Story,na,na,5.93,3.61, 7.22,2
p1 <- p1 + annotate("text", x = 7.07, y = 6.05, label = "Lisey's Story", size=3, colour="red3")
#Mr. Mercedes,na,na,6.78,3.87, 7.74,2
p1 <- p1 + annotate("text", x = 7.60, y = 6.81, label = "Mr. Mercedes", size=3, colour="red3")
#Gerald's Game,na,na,5.34,3.43, 6.86,2
p1 <- p1 + annotate("text", x = 7.03, y = 5.36, label = "Gerald's Game", size=3, colour="red3")
#Cell,na,na,5.93,3.61, 7.22,2
p1 <- p1 + annotate("text", x = 7.28, y = 5.92, label = "Cell", size=3, colour="red3")
#11.22.63,na,na,8.05,4.26, 8.52,2
p1 <- p1 + annotate("text", x = 8.63, y = 8.05, label = "11.22.63", size=3, colour="red3")
#The Dark Tower,na,na,7.2,4,8,2
p1 <- p1 + annotate("text", x = 8.16, y = 7.2, label = "The Dark Tower", size=3, colour="red3")
p1 <- p1 + ylab("IMDB Ratings")  
p1 <- p1 + xlab("GoodReads Ratings")
p1
ggsave("plot.png", width=10, height=10, dpi=100)
detach(mydata)

Friday, November 13, 2015

Do Stephen King's Better Books Make Better Films?

My last post (and data) got a bit popular on reddit. Some people noticed that Stephen King films ratings and the books ratings seemed highly correlated.

I think Max is right on this. So I made a graph showing how movie and book ratings are correlated

Now the actual correlation figure is


 cor(good.doubled, imdb, use="complete")
[1] [1] 0.8766 so it looks like highly rated King books make highly rated films

It could be that a good film makes people read and rate highly a book. But my basic conclusion is Stephen King's highly rated books make higher rated films















Appendix: Code for the Graph


mydata = read.csv("King.csv")
attach(mydata)
cor(good.doubled, imdb, use="complete")
p1 <- ggplot(mydata, aes(x=good.doubled, y=imdb)) +
    geom_point(shape=1) +    # Use hollow circles
    geom_smooth(method=lm,   # Add linear regression line
                se=FALSE)    # Don't add shaded confidence region
p1 <- p1 + ylab("IMDB Ratings")  
p1 <- p1 + xlab("GoodReads Ratings *2")  
p1 <- p1 + ggtitle("Stephen King: Books vs. Movies Correlation")
p1 <- p1 + geom_text(aes(label=ifelse(good.doubled>0,as.character(Title),'')),hjust=0,just=0,size=2, position = "jitter")
p1
ggsave("correlate.png")





Other Correlations I changed the publication data column into a column of Years since 1974.


 cor(good.doubled, Years, use="complete")
[1] -0.1879052 Not a strong correlation about Kings adapted books get better or worse since he started
> cor(good.doubled, Pages, use="complete")
[1] -0.02715659 no relationship between the length of a book and it being rated highly or lowly.
> cor(Years, Pages, use="complete")
[1] 0.482048 King's adapted books may be getting a bit longer over time

Wednesday, November 11, 2015

Are Stephen King films better than the books?

Rejoice, Snobs: The Book IS Better Than The Movie
The literary originals have higher ratings than the film adaptations 74 percent of the time
. This recent piece claims books are usually better than their films versions.

This article reminded me of a claim that Stephen King films are better than his books. So I extracted the ratings from IMDB and Goodreads. I doubled the Goodreads ratings to make them out of 10.


I included miniseries. Carrie was made twice but I only included the original. The Langoliers, Maximum Overdrive, Lawnmower man and Secret garden seem not to have independent English language printed book versions.

 

The Shining is one of two films with a higher rating than the book. King famously hates the film

"Are you mystified by the cult that's grown around Kubrick's Shining?
I don't get it. But there are a lot of things that I don't get. But obviously people absolutely love it, and they don't understand why I don't. The book is hot, and the movie is cold; the book ends in fire, and the movie in ice."

 

To make this a fair test films and book would have to be graded on a curve. By what the average rating of each is. There is a good discussion on how to normalise the original comparison here. But I think this graph is enough to show Stephen King books are better than his films.

Less than 8% of King adaptations are rated higher compared to 26% usually for book adaptations. To get King to have the average quality of adaptations, sixish Stephen King Films being rated better than is books. This would require Dolores Claiborne, The Green Mile, Stand By Me, Misery and maybe Carrie to be much worse books or these already highly rated films to be much better. I am going to call this Myth Busted*, Stephen King adaptations are less successful than the average adaptation.

 

TitlePublication datePagesimdbgoodreadsgood doubled
CarrieApril 5, 19741997.43.897.78
'Salem's LotOctober 17, 19754396.83.977.94
The ShiningJanuary 28, 19774478.44.128.24
The StandSep-788237.34.328.64
The Dead ZoneAug-794287.33.887.76
FirestarterSeptember 29, 198042663.87.6
CujoSeptember 8, 198131963.617.22
The Running ManMay-822196.63.747.48
ChristineApril 29, 19835266.63.697.38
Pet SemataryNovember 14, 19833746.63.867.72
ThinnerNovember 19, 19843095.73.67.2
ItSeptember 15, 198611386.94.128.24
MiseryJune 8, 19873107.84.068.12
The TommyknockersNovember 10, 19875585.43.426.84
The Dark HalfOctober 20, 19894315.93.717.42
Needful ThingsOct-916906.23.847.68
Dolores ClaiborneNov-923057.43.767.52
The Green MileMarch–August 19964008.54.398.78
Bag of BonesSeptember 22, 19985295.83.847.68
DreamcatcherMarch 20, 20016205.53.537.06
Under the DomeNovember 10, 200910746.83.897.78
Shawshank Redemption19821819.34.519.02
Stand by me1982808.14.258.5
The Mist19832307.23.887.76
Apt Pupil19831796.73.87.7
Hearts in Atlantis20006406.93.777.54

   
 mydata = read.csv("King.csv")  
 library(ggplot2)  
 p1 <- ggplot(mydata, aes(x = good.doubled, y = imdb))  
 p1 <- p1 + geom_abline(intercept=0, slope=1)  
 p1 <- p1 + geom_point(shape=1)  
 p1 <- p1 + ylim(5, 10)  
 p1 <- p1 + xlim(5, 10)  
 
 p1 <- p1 + ylab("IMDB Ratings")  
 p1 <- p1 + xlab("GoodReads Ratings *2")  
 p1 <- p1 + ggtitle("Stephen King: Books vs. Movies")  

p1 <- p1 + annotate("text", x = 7.3, y = 5.3, label = "The Tommyknockers", size=3, colour="blue3")  
p1 <- p1 + annotate("text", x = 7.95, y = 7.37, label = "Carrie", size=3, colour="blue3") 
p1 <- p1 + annotate("text", x = 8, y = 6.5, label = "Pet Sematary", size=3, colour="blue3") 
p1 <- p1 + annotate("text", x = 7.9, y = 8.54, label = "The Shining", size=3, colour="red3") 
p1 <- p1 + annotate("text", x = 8.75, y = 9.4, label = "Shawshank Redemption", size=3, colour="red3")
p1 <- p1 + annotate("text", x = 9.18, y = 8.4, label = "The Green Mile", size=3, colour="blue3") 
p1 <- p1 + annotate("text", x = 8.72, y = 8, label = "Stand By Me", size=3, colour="blue3") 
p1 <- p1 + annotate("text", x = 8.92, y = 7.3, label = "The Stand", size=3, colour="blue3") 
p1 <- p1 + annotate("text", x = 8.3, y = 6.85 , label = "It", size=3, colour="blue3") 
p1 <- p1 + annotate("text", x = 8.32, y = 7.7, label = "Misery", size=3, colour="blue3") 
p1 <- p1 + annotate("text", x = 8.18, y = 6.7, label = "Under the Dome", size=3, colour="blue3") 

p1 <- p1 + annotate("text", x = 9.28, y = 5.0, label = "Books Higher Rated", size=5, colour="blue3") 
p1 <- p1 + annotate("text", x = 5.7, y = 9.90, label = "Films Higher Rated", size=5, colour="red3") 



p1 <- p1 + annotate("text", x = 6.9, y = 6, label = "Cujo", size=3, colour="blue3") 
p1 <- p1 + annotate("segment", x = 7.05, xend = 7.2, y = 6.0, yend = 6.0, colour = "blue3") 


p1 <- p1 + annotate("text", x = 6.45, y = 5.52, label = "Dreamcatcher", size=3, colour="blue3") 
p1 <- p1 + annotate("segment", x = 6.8, xend = 7.05, y = 5.5, yend = 5.5, colour = "blue3") 


p1 <- p1 + annotate("text", x = 6.80, y = 7.4, label = "Dolores Claiborne", size=3, colour="blue3")
p1 <- p1 + annotate("segment", x = 7.28, xend = 7.5, y = 7.4, yend = 7.4, colour = "blue3") 

p1 <- p1 + annotate("text", x = 7.92, y = 5.72, label = "The Dark Half", size=3, colour="blue3")
p1
ggsave("plot.png")
*This needs proper statistical significance testing so not really busted.

Tuesday, November 03, 2015

Odd Mathematical Patents #1: Gömböc

The Gömböc is a self righting shape. A bit like a turtle that will always rotate back to standing up if placed on its back. This shape was patented under US D614077 S1 and a cease and destist has been send to someone giving plans to 3D print one.
Prof. Domokos has politely requested that I remove this file. I have respectfully declined to do so, as I do not believe it violates any of their established legal rights, and I believe it may have actual value for researchers interested in the Gömböc and mono-monostatic bodies in general.

Gömböc's are cool. And the inventors were very ingenious. You can buy one from the patent holders here. Patenting a shape though strikes me as weird.