Heatmap of the world

On the 14th of June 2016 I made a heatmap of global temperatures since 1850. This got popular on reddit with 15.4K points


The picture as I said in the first comment on the post was inspired by Ed Hawkins spiral animation of world temperatures and was an attempt to show everything in one picture.
A few outlets picked up the picture. The weather channel put it on facebook without crediting me.

Tableau I found out last year use it to advertise themselves but with no credit to me the original design

Ed Hawkins tweeted about the graph


And in June 2016 we discussed further as I switched Time to the Y axis and made other changes. My point is he tweeted about "'heat-map' representation of global temperature changes since 1850" two years before he published his heat map representation (using stripes). My graphs used the Hadcrut-4 and HadCET datasets

In May 2018 he produced a climate graph (below) with only years not months in the heatmap. At no point has he said that my heatmaps (which were inspired by his global temperatures spiral animation) might have inspired him.




I had produced a heatmap graph with lines for years that he saw in Sept 2017

Hawkins graph has deservidely become famous. And heatmaps are one of the most common visualisation types, which is evidence he could have come up with it on his own anyway. I would like if my ones did inspire him that it got mentioned somewhere. But the big deal here is that global warming is being shown in a popular eye catching way.

February is Hot

At the moment it is really hot. 20°C is ridiculously hot for this time of year.

Central England has weather data going back a long time. 1772 for daily average temperature and 1878 for maximum and minimum temperatures on each day.

2 times February the 25th since 1772 had an average temperature for the day above 10°C in Central England 1790 with 10.7°C, and 1922 with 11°C

The maximum temperatures observed in central England on the 25th of February were 1922 with 14.1,1953 with 13.1 and 1976 with 13.1 For those 140 years the maximum daily temperature averaged 6.6C.

I put the code to work all this out here. Graph of Feb 25th max temperature shows how weird 20°C is

It takes a while for the HADCET data to be updated. There is a nearby weatherstation http://www.weathercast.co.uk/world-weather/weather-stations/obsid/99060.html when I figure out the relationship between the two it will be possible to do a comparison between today and all the previous years.

English soccer is not normal

Are wins in football normally distributed? If they are not it might affect how we should calculate the probabilities of teams winning.

Baseball wins seem not to follow a normal distribution

There is a great R Package dataset of football results by James Curley here. This engsoccerdata has a function to generate soccer league tables of many countries over a long time period.

       league<-maketable_all(df=england[,])
       
 

team GP W D L gf ga gd Pts Pos

1 Manchester United 962 604 209 149 1856 847 1009 2021 1

and create a new column for the percentage of wins

       league<-league %>% 
    mutate(PercentW = W / GP)

p<-ggplot(data=league, aes(league$PercentW)) + geom_histogram()
#binwidth=20
p<-p + ggtitle("Percentage wins\n in English football league") +   xlab("Percentage Wins") + ylab("Number of Teams")
p<-p+theme_update(plot.title = element_text(hjust = 0.5))
p<-p + theme_bw()
       
 

       library(fitdistrplus)
library(logspline)
x<-league$PercentW
fit.norm <- fitdist(x, "norm")
plot(fit.norm)
       
 

       shapiro.test(x)       
 

Shapiro-Wilk normality test

data: x W = 0.96276, p-value = 0.0006663 Which means English football wins really do not have a normal distribution.

Goals per game are also not normally distributed. But I dont think anyone expectes them to be

       
league<-league %>% 
    mutate(GoalsPgame = gf / GP)
shapiro.test(league$GoalsPgame)

 

Shapiro-Wilk normality test data: x W = 0.92134, p-value = 4.818e-07

And for France

Shapiro-Wilk normality test

data: leagueF$PercentW W = 0.98522, p-value = 0.4699 so French football wins do not have might have (thanks for Paulfor the correction in the comments) a normal distribution. I must check the other leagues in the dataset as behaviour this different is odd.

The Oldest Irishman Joke

John Scotus Eriugena (c. 815 – c. 877) was on the five punt note. And won a Michael D Higgins look alike competition in 870AD.

The French king sat opposite him and asked.

'Quid distat inter sottum et Scottum?' (What separates a sot [drunkard] from an Irishman?),

Eriugena replied, 'Tabula tantum' (Only a Table).

I think the gag setup still works 1200 years later.

What separates man from the animals?

The Liffey

Ireland Carbon reduction by 2030

How are we going to meet Ireland's carbon reduction aims in the next eleven years?

How much do we need to reduce Carbon by?

We have committed to 'reducing greenhouse gas emissions by 40 per cent (compared to 1990 levels)'. Which is 60% of 56 million tonnes our carbon output by 40%. Which is 33.6 million tonnes of around 42.5% of our current carbon usage.

How would we do this?

On a personal level not an industrial one what would I have to do to reduce my carbon output by 40%? Please correct me if I am wrong here but roughly it seems to be

1. Get rid of my car. (as seen in Show Table: 2.5) 20% of our carbon output is Transport. Electric cars do not seem to be vastly more efficient than petrol ones.

2. Become Vegan. This seems to reduce food Carbon by about 40% .

3. Insulate the house and install energy efficient appliances. This seems to make less impact than cars and meat.

4. Sign up to airtricity. This one is easy enough.

How would this happen?

This all seems asking a lot. On transport Public transport subsidies would have to increase massively immediately. Even then 30% of Irish people live in the countryside which is really hard to do without a car.

It could be that rewilding the countryside means that we do not need to reduce carbon outputs by as much. But that would be a huge undertaking by itself.

Why is all this not being talked about? If I am totally wrong here please explain in the comments how to reduce by 40% in an easier way. If it is that we have to go vegan, live in cities and massively boost public transport starting immediately that seems to be a massive task.

Trumps Year 1

I posted here a year ago four things that if Trump improved he'd be keeping major campaign promises. Measures for a successful Trump

1. Better healthcare. Cover Everybody, cost less and have lower deducables - No things got Worse. America's uninsured rate is on the rise.

2. "If Trump can deliver an average of more than 3% over his 4 years in office I think an impartial observer would agree the economy has done well." Didnt happen in year 1. "The economy grew 2.3 percent in 2017, better than the year before, but far short of what Trump is promising. Officials had focused on 3 percent GDP growth as proof his economic policies were working, and Trump has said it could go far beyond that target."

3. A balanced Budget. Didn't happen 'the House of Representatives passed its version of a tax reform bill that added an estimated $1.4 trillion to the federal deficit over the next decade.' The vote this year was not for an immediate reduction in the deficeit.

4. Infrastructure improvements didn't obviously happen but in fairness this would take time. So it cant be called based on the data that only goes up to 2016

Three of these measures did not succeed this year.

The Best Books I Read in 2017

I read 60 books so far this year. These are my favorites

Non fiction

1. The Toaster Project- This seems like you could get it across in a Ted talk. Things are far more complicated then you imagine and getting them as cheap as we do is amazing. But its really funny to follow him for a long time and see the frustrations he goes through learning this. Secondly I found it oddly cheering. No one does anything all we do is add a tiny bit to what the millions around us do so don't take yourself so seriously.

Runners up:

A Farewell to Alms: A Brief Economic History of the World why do loads of us live long well feed lives? doesn't answer the question but gives loads of evidence of what didn't cause it.

Corruption: What Everyone Needs to Know?. corruption as a basic dishonesty seemed really important this year. Why it happens and at can be done about it in a short easy read.

Also very good: Other Minds: The Octopus, Silk Roads, Vermeer's Hat, Fifty Things that Made the Modern Economy, Stubborn Attachments

Fiction

We Have Always Lived in the Castle A creepy southern Gothic story with a very unreliable narrator. I have read loads of Shirley Jackson this year.

Runners up: The Handmaid's Tale

also very good The Death of Ivan Ilych, High-Rise and loads of Ross MacDonald books.

I havent done one of these in year so my favorite non fiction in 2015 was Wood, G.D.: Tambora: The Eruption That Changed the World and in 2016 was Money: The Unauthorised Biography by Martin Felix

I had great fun reading this year. Next year I must stay off twitter and chess.com and read more.

Truncated Prime Pictures

Is it just the last digit that causes primes not to be normal numbers? As in is it the lack of ending in 2,4,6,8,0 that means the walk on them doesn't look random like pi does? This post looks at the same walks as the other recent posts but with the least significant dropped.

I made the graph dropping the last digit from each prime. The walk looks different but not that different. similar to some of the other walks I calculated

I made a barchart of the counts of digits in prime numbers up to 10138889.

The same graph not including the final digit also shows this truncated view is not normal either. A graph of the truncated counts on all numbers up to 10158737 is 60 million digits

This really does look normal. So it looks like Prime numbers except the least significant digit might be normal but making a walk with them still shows them moving from the origin at too quick a rate to be really a random walk in the digits.

I made a heatmap of which numbers follow each other in the primes up to ten million digits.

and the primes except the last digit

Code is here

More Prime Pictures

Smarandache Sequence as a horizontal color representation. First digit is the first pixel top left. Each number is its own color. Move one step to the right and each 1000 drop down a step. Code is here

Thsi idea is from Walking on real numbers though they did not look at the prime number concatenated.

Using the first 11066873 prime digits if the previous number was smaller go up a step if it was larger go down a step. The path crosses zero the 204 times.

Using the first 2010484 prime digits if the previous number was smaller go up a step if it was larger go down a step. The path crosses zero the 197 times.

I am still not sure these pictures mean anything.

Number Sequence Walks in One Image

To show each stage of the pattern these number sequences make I have changed my script here to zoom out each order of magnitude. This means the path up to 100,1000,10000,100000 and 1000000 will all show up as similar sized. Though coloured differently.

Prime Numbers

Champernowne Number is similar but there are some differences.

even Numbers

Odd Numbers

Nearly Primes (numbers that are not divisible by 2,3,5,7,11,13,17,19) are added tot he string to walk. This walk will be all primes and a few numbers that aren't. The graph is more similar to the prime graph (and I got the scaling a bit wrong at the start )but I am still surprised at how different it is.

Zooming in on the loop in Smarandache Sequence

What is going on in the loop here at the bottom right that causes the turtle to change direction? In the Ten thousand digits path it is some of the orange and light purple parts below

to draw just this loop part of the image the code is here. Zoomed in the path looks like this

This image contains the 279 primes from 14503 up to 17183

In the 100K image it seems to be the over 2000 primes between 152003 and 178361 that make up this loop part

Fractal Pattern in the prime numbers

The Smarandache–Wellin number is made by sticking together all the prime numbers.

2 add on 3 add on 5 add on 7... to make 2357111317192329

A reddit user improved my earlier code with generators. That old code is here. And using the generators from this code I got a prime number checker from here.

With 10k steps this looks like

100k steps

and a million steps (digits)

Five million digits

1000 digits from primes

100 digits looks like an album cover

Here every one tenth of the total distance changes the colour so roughly the blue part on the top left is the previous image. These are primes in the base 10 which is probably why the pattern repeats like that.

These seem really similar to each other. The Champernowne constant seems to do something similar, so maybe most sequences of natural numbers stuck together make repeating patterns like this. I tested the even and off numbers and they kind of do.

even numbers 100k digits

odd numbers 100k digits

The code is here so you can play with it, find bugs and recreate the images. Thanks to the help to the reddit user who recreated the images in Mathematica, code here as a sanity check.

Numbers from a sequence in the base ten make a similar pattern that gets ten time bigger each round. Which doesn't sound surprising said like that. Still the pictures look cool I think.

Champernowne constant might form a fractal

Making each digit of Champernowne constant a step in a logo turtle picture seems to form a fractal picture.

10k steps

100k steps

2 million steps

It seems to form this circle with another below and to the left over and over again.

Making Turtle pictures with Numbers made from Numbers

The post Random Walks with Number Digits looked at pictures made from the digits of numbers. Go through each digit in turn. If it is 0 take a step to the right. If it is one go 36 degrees down from that. If it is 2 72 degrees etc etc. For base ten numbers this makes a picture that looks like brownian motion but with only 10 directions not any possible direction.

In this post I look at numbers made from numbers.

The Copeland–Erdős constant is made by sticking together all the prime numbers into a number 2 then 3 then 5 then 7 then 11 etc.. 0.235711131719232931374143 A033308. One of these turtle logo pictures made from this number has a very odd pattern. With 1.5 million digits from prime numbers this pattern is made. The Copeland–Erdős constant is normal so I would expect the loop to eventually come back around.

Champernowne constant is made with 1 then 2 then 3 then 4 etc.. 0.12345678910111213141516 makes a similar picture but it is much more curvy. 600k digits looks like

Random Walks with Number Digits

Say you took each digit of a number and used that to decide which direction to take a step. What path would you follow?

Similar to last post I took Pi, e, Sqrt(2) and random numbers. I multiplied each digit by 36. There are 10 digits to go into the 360 degrees you can go. For 0 step right. For 1 36 degrees down from 0. For 2 72 degrees etc.

Here are the pictures with a description of which number made each at the end. If any look different that might be an indication the digits are not that random.

.

After drawing this I realised it looked like Dragon Curves. Googling Pi pictures and Golden curves revealed this book. so someone had the idea before.

My code for all these pictures is here. The picture is on reddit and people seemed to like it.

And now other numbers

E first 700k

e next million

Sqrt(2)

Using a random number generator, just for comparison

Catalan Constant

These look like pictures of brownian motion which in effect they are.

Pi Digits High Low Game

Suppose you take a long number and for each digit if it is bigger then the previous one increase a counter by one. If it is less then the previous number reduce the counter by one. You keep a running total and graph that total. Noting when it passes 0. This total number will go up and down and can get to zero many times. If you play this game with random numbers in a million digits on average the number of times you will have crossed 0 is 1594.4 and the standard deviation of the number of times crossed 0 is 1207.3. Though as the number of times zero is crossed cannot be less then 0 this is a bit odd.

       
import random

numcrossed=[]
j=0
while j < 200:
	i = 0
	last=0
	total=0
	x=[]
	y=[]
	crossed=0
	while i < 1000000:
		ran= random.randint(0, 10)
		if ran==last:
			total=total
		elif ran>last:
			total=total+1
		else:
			total=total-1
		if total==0:
			x.append(i)
			y.append(total)
			crossed=crossed+1        
		i=i+1        
		last=ran
	numcrossed.append(crossed)
	j=j+1
       
 

If instead of random numbers the digits of pi are used. This is what the path of total counts looks like

       
file = open("pi1000000.txt", "r") 
#3.14159265358979323846264338327950 pi2.txt
x = []
y = []
text=file.read() 

pi = list(text)
total =0
i = 0
crossed=0
i=0

while i < len(pi):
	if pi[i]>pi[i-1]:
		#print(pi[i])
		total=total+1
	if pi[i]
 

Pi has a 0 total 657 times. Which is more than 51 out of 200 random million long sequences did in my tests. None of this means anything. Going up or down based on digits in a base ten number but i like these pattern sort of sequences.

E crosses 0 1725 times

sqrt2 crosses 0 1300 times

and with 2 million digits

The python code for visualisation is 
       
import numpy as np
import matplotlib.pyplot as plt

plt.scatter(x, y, alpha=0.5, color='green')
plt.title('Sqrt 2 High Low Game')
plt.show()