Category: Daten

The Most Beautiful Place

Similar in spirit and methodology to my last post, I asked 482 people about the most beautiful place they have ever been to. Click on the image below to get to the map showing the results, zoom in to see all markers. Yellow markers include comments. Original data is here.

Yes, I’ll write about other things soon.

The Happiest Age

What is your age? And, over the course of your life (past, present, and future), at which age do you think you were/are/will be the happiest? Can you answer the second question? I can’t, but I was curious what people might say. Over the last two weeks, I used Amazon’s crowdsourcing service to ask 672 people these two questions. Here are the results:

Graph age vs expected happiest age

Each dot is at least one person. If more than one person gave the same answer, the dot is bigger. Click on the graph for a better version, or take a look at the complete dataset.

I don’t know what to do with the graph, but lots of people wrote short comments explaining their choices which I really enjoyed reading. Here are some I liked, each with the age of the author and the age at which he/she expects to be the happiest:


An 18-year-old: 25
Because that’s when i’ll have a stable job, good income and my own house (hopefully).

A 25-year-old: 18
I wish I could stay 18 or ever.


A 33-year-old: 53
Will have met financial freedom and retirement goals.

A 46-year-old: 53
By the time I’m 53, I feel like I will be old enough to truly know myself and young enough to be physically fit.

A 53-year-old: 53
I was diagnosed with cancer around 2.5 years ago, went through the chemo and radiation. I’m finally at a point where I am grateful and happy that it was found in time to do something about it, and not worrying every second that it will come back. Not exactly a near-death experience, but as close as I want to come. Makes you REAL happy to be alive and to try and appreciate even the small things.


A 24-year-old: 40
I have already had a lot of happiness in my life and am very grateful for said happiness but I noticed that as my father ages, he seems to find more and more joy in the smaller things in life. Even in tough times he seems to maintain a more positive attitude – maybe it’s because he’s retired or he’s realized that worrying does’t accomplish much. I hope to achieve his level of wisdom someday and look forward to more happiness and fulfillment later in life.

A 53-year-old: 40
This was the point in my life that I had gotten through college, was married, had children and a career. I spent many hours at my childrens games (football, softball, etc) and and loved every minute if it! Financially, things were getting easier as my husband and I advanced in our careers so we could do more things such as travel and not have to worry so much about being able to afford it. It seemed that the hard work of college and “paying my dues” as I began my career were finally beginning to pay off.


A 21-year-old: 30
At 30, still young but old enough to be really developing my career.

A 30-year-old: 21
Life at my fingertips…


A 26-year-old: 35
I love the family life and independence; by the age mentioned I hope to have less day-to-day worries and more kids, but still be in a really good shape to enjoy it. Plus, I hope to be much more stable professionally.

A 34-year-old: 25
It’s amazing what experience, debt, and growing older can do to your outlook on life. I thought a decade ago I would be happer a decade later, and I am finding out that’s not really the case. I’m not unhappy by any means, but the more responsibilities we accrue in life, the easier it is to rate our happiness by different things.


A 23-year-old: 21
In college – so far it was the best time of my life. Hopefully it won’t be!

A 35-year-old: 21
College years were the best


An 18-year-old: 28
Done with college, can settle down, new job, etc

A 28-year-old: 22
I was a college student then. That were happiest years in my life, because my character shaped and tempered. Though any college assignment felt hard, I can enjoyed the hardship. Sincerely, I dreamed several times about my college years when I slept. I really missed that moment of life.


A 26-year-old: 15
15 was a great year. I was still to young to care and honestly thought the world was at my feet. I spent the school year hanging out with friends and of course school. Summer I spent most of it at my uncles enjoying time with my younger cousins and the cute boy down the street. Ahh… life before a I ever had a job.

Another 26-year-old: 26
As someone who has spent most of her short life daydreaming, I have learned not to waste my time measuring happiness or planning how to create it. I try and make the best of the present time and hope that I continue to do that for the rest of my life. Interesting question!

Yet another 26-year-old: 32
I feel by that time I will have finished grad school and be working in a field that I love. I will be more comfortable with who I am and my place in the world by that time.


A 26-year-old: 30
The day i get married will be the happiest day of my life.

A 41-year-old: 26
I met and married my husband at 26 years old. It was the best time of my life.

A 70-year-old: 30
We had a very happy marriage and two beautiful daughters. Although we are still married, things often got complicated and stressful but never hopeless.


A 23-year-old: 8
Childhood was a time of innocence; no worries, no bills, no thought as to cause and effect… You walk around with your fingers in your nose picking wedgies and thinking about the playground never wondering about world hunger war terrorism or even when companies may go bankrupt and cancel your favorite television show.


A 44-year-old: 25
The age I married my beloved husband and set out on our new life together. We are still together and still very happy. The adventures we share and have shared have brought so much joy to my life.

A 46-year-old: 34
I was happiest when I was single and working at the beginning of my professional career.


A 27-year-old: 45
Having children and watching them grow will give me the greatest joy.

A 38-year-old: 45
at that age most of my kids will be grown and hopefully I will be able to quit my job by then and do some of the things i would like to be able to do in my life


A 29-year-old: 16
I desperately miss the imagined knowledge and unknown ignorance of being in high school

Another 29-year-old: 17
Got married at 16 have 4 kids…so life has been challenging, wouldn’t trade them for anything…but would like to go back and have less responsability even for a little while :)


A 29-year-old: 26
This is the age that I came to the realization that I had finally found what I wanted. Everything just seemed to be coming together.

A 50-year-old: 26
At 26 i had my only child…and i was going to college, met a man of my dreams and felt like i could do it all..then i got involved with my child and man everything i wanted went out the window with in the first year..do i regret it sometimes would i change it not on my life


A 54-year-old: 23
I know for sure that as I age I get unhappier. At this point I’m thinking about how much longer I have as compared to having my whole life in front of me.

A 31-year-old: 50
I said 50 is the age I would be the happiest because it seems like the older I get the happier I am. And I hope that when I’m 50 I will continue to get more happy with each passing year and I hope to still be in great health.


A 44-year-old: 32
I finally had my child after trying for 10 years and my life was complete then.

A 43-year-old: 61
My youngest child will be of age and hopefully off to college \u0026 then I will be free to do what I want with my life.

A 39-year-old: 40
We had always known we wanted to adopt 2 special children and our hearts goal was that it would be by the time I am 40 and my husband 45. Our second and final adoption will be finalized in 2009 and I will be 40 years old.


A 46-year-old: 65
I am looking forward to 65, so I can retire, and actually take some time out to enjoy life.

A 65-year-old: 34
I was well on my way with a career wide open, had purchased my first house and had a great social life. Since then I have been up and down, but never so enthusiastic about life as I was then.


A 53-year-old: 22
We don’t appreciate things at a young age, but as we grow older we always wish we knew then what we know now.

A 22-year-old: 22
Two young daughters make my life the happiest! :)


A 30-year-old: 25
I was earning, was healthy, carefree. Looked as if there is enough time in life


A 62-year-old: 19
I was young, gorgeous, full of life and in love

This last one made me stop and stare at the screen for some time, simultaneously not knowing anything about this person and yet so much.

Update: David Sturman did some statistical analysis of the data that is worth reading.

Find Similar Users on del.icio.us

Download: delicious_mates.py

On the social bookmarking site del.icio.us, you can add other users to your network to see their recent bookmarks aggregated on one page. There are two kinds of people in my network: 1. Friends and 2. Users I don’t know personally, but who regularly post interesting links. People who bookmark the same things that I bookmark are likely to have similar interests and are thus likely to continue bookmarking interesting things in the future. Since the information on who bookmarked what URL is public, the process of finding people with similar interests can be automated.

Urban Hafner brought up this idea three years ago, but I could not find an implementation. Yesterday, I wrote a short Python (2.5) script that implements the following ideas:

  1. Look at every link in your bookmarks: Who bookmarked the same page? Add these users to a list of people possibly similar to you.
  2. The more bookmarks another user has in common with you, the higher your similarity.
  3. The smaller the number of people who bookmarked a page, the more significant the fact that another user has this bookmark in common with you.
  4. If a user has lots of bookmarks, common bookmarks are less remarkable. The percentage of common links counts.

Here is what you need to do to find people whose interests are similar to yours:

  1. Download delicious_mates.py
  2. Run python ./delicious_mates.py
  3. Wait — this takes some time.

Then, what you will see will look something like this:

andreas> python ./delicious_mates.py
Your del.icio.us username? andreas.s
Your del.icio.us password?

Fetching list of bookmarks ... (485)

Fetching list of users for each bookmark ...
    1. http://en.wikipedia.org/wiki/Langton's_ant (7)
    2. http://www.nytimes.com/2008/05/13/science/13coat.html?_r=2&partner=rssnyt&emc=rss&oref=slogin&oref=login (0)
    3. http://sifter.org/~simon/journal/20080509.2.html (2)
    4. http://www.intercult.su.se/cultaptation/tournament.php (16)
    5. http://www.pnas.org/cgi/content/abstract/0801268105v1 (49)
    6. http://atlas-conferences.com/c/a/n/i/15.htm (0)
	[..]
    480. http://lifeboat.com/ex/main (129)
    481. http://prize.hutter1.net/ (137)
    482. http://www.psg.com/~dlamkins/sl/cover.html (194)
    483. http://www.idsia.ch/~juergen/ (157)
    484. http://sl4.org/wiki/ShannonInformation (1)
    485. http://www.scottaaronson.com/writings/ (13)

Finding 50 candidates from list of 49937 users ...
    rainer (42/9367) ok
    siggiB (36/1006) ok
    fhtagn (35/2471) ok
    anissimov (35/6281) ok
    jbone (41/19982) ok
    invisibleandpink (16/203) ok
    irchans (16/610) ok
    ferrouswheel (14/946) ok
	[..]
    eggywat (14/4290) ok
    Cunya (8/3133) ok
    getpost (13/9787) ok
    hannu (12/3915) ok
    lispmeister (11/2967) ok
    dean.vanniekerk (12/5873) ok
    rgrant (9/1769) ok

Top 50 del.icio.us mates:
username             weight               # common bookmarks   # total bookmarks    % common
——————————————————————————————————————————————————————————————————————————————————————————————
siggiB               54.92937             36                   1022                 3.52250
invisibleandpink     53.05635             16                   203                  7.88177
fhtagn               20.61878             36                   2474                 1.45513
irchans              15.61606             16                   611                  2.61866
fogeli               12.85109             16                   628                  2.54777
ferrouswheel         7.99177              14                   946                  1.47992
rainer               7.60971              43                   9423                 0.45633
anissimov            6.46154              34                   6309                 0.53891
pdorrell             4.10251              19                   3041                 0.62479
jefallbright         3.87392              12                   1193                 1.00587
ladro                3.76458              12                   1526                 0.78637
miguel1626           3.04347              11                   1039                 1.05871
jbone                2.62672              41                   20039                0.20460
hartmut              2.15612              9                    1316                 0.68389
asciilifeform        2.04304              10                   1581                 0.63251
tmalin               1.89627              12                   2181                 0.55021
herrmann             1.84991              21                   5410                 0.38817
jas0nm               1.82646              20                   3808                 0.52521
[..]

If you look at the script, you will find a few settings you might want to change. For each of these holds: The higher you set them, the more time it takes for the script to finish.

  • MAX_MATES

    is the maximum number of similar users the script suggests.

  • MAX_BOOKMARKS

    defines how many of your bookmarks the script will look at.

  • BOOKMARK_FILTER

    defines which types of bookmarks are analyzed. Remove

    , "no"

    from

    {"shared" : [None, "yes", "no"]}

    to exclude private bookmarks.

  • MATE_MIN_BOOKMARKS

    sets a minimum for the number of bookmarks a del.icio.us user needs to have before he can be considered to be similar to you.

  • MATE_MIN_COMMON

    sets a minimum to the number of bookmarks a user has to have in common with you to be included in the list of similar users.

The script needs two Python modules, the parser BeautifulSoup and Michael Noll’s del.icio.us Python API. If the script does not find one of the modules, it will download the missing module to the current directory and import it from there. If you don’t like this because you believe this is a security nightmare (which it is), don’t run delicious_mates.py install the two modules beforehand.

Do you know some Javascript and have spare time? I would love to see the script converted into a direc.tor-like bookmarklet. The need to download and run a Python script makes finding similar users more complicated than it chould be.

The feature I like best about the online bookshelf LibraryThing is its Unsuggester: Name a book you have read and it suggests those books that are least likely to be on your bookshelf. I like it because it is a means to counteract the temptation to adjust your sources of information such that whatever you read reinforces your point of view. Seeing how easy it is to give in to this temptation, is a script that makes it easier to surround yourself with like-minded people just one more sign of a general trend towards biased, largely isolated online communities?

Machine Learning from Scratch

Ich habe einige Zahlen, will die nächste in der Folge vorhersagen und weiß nichts über die Herkunft der Zahlen. Kann ich eine Vorhersage treffen? (Nein.)

1010101010101010101010101010101010101010_

Nun habe ich dieselben Zahlen, darf aber annehmen, dass einfache Erklärungen wahrscheinlicher sind als komplizierte. Kann ich jetzt eine Vorhersage treffen? (Nein.)

1010101010101010101010101010101010101010_

Ich nehme zusätzlich an, dass die Folge auf irgendeine Weise berechnet werden kann. Kann ich jetzt die wahrscheinlichste nächste Zahl vorhersagen? (Nein, nicht in endlicher Zeit. Das allgemeine Vorhersageproblem ist unlösbar.)

1010101010101010101010101010101010101010_

Kann ich einen Algorithmus schreiben, dessen Ausgabe gegen die wahrscheinlichste nächste Zahl konvergiert, wenn die Laufzeit gegen unendlich geht? (Ja. Dovetailer über alle möglichen Programme, angefangen mit dem kürzesten; die Ausgabe des kürzesten Programms, das die Zahlen und eine zusätzliche ausgibt, ist die aktuelle Hypothese.)

Wir Menschen treffen jeden Tag Vorhersagen. Der Vorteil, den wir gegenüber derzeitigen Algorithmen haben, liegt in den zusätzlichen Annahmen, die wir unbewusst machen, in der Art von inductive bias, mit dem uns die Evolution ausgestattet hat. Die Aufgabe von Forschern auf dem Gebiet des maschinellen Lernens ist es, den Suchraum von Algorithmen derart einzuschränken, dass die Laufzeit der Algorithmen polynomiell wird, und dabei möglichst wenige Lösungen für Vorhersageprobleme auszuschließen, die für unsere Welt relevant sind.

Amazon Web Services

Wissen ist Macht, das wusste bereits der Philosoph Francis Bacon im 16. Jahrhundert. Und es gilt heute mehr denn je. Unternehmen wie Google speichern nicht grundlos jedes Bit an Information, das sie in die Hände bekommen. Mit Daten eröffnen sich Möglichkeiten: Data Mining, d.h. das Entdecken neuer Zusammenhänge in alten Daten, und der Einsatz neuronaler Netze sind nur zwei davon. Generell gilt: Je mehr Informationen, desto besser. Read on »