- List of tutorials from Peter Orbanz.
- Foundation of nonparametric bayesian methods tutorial by Peter Orbanz at Machine Learning Summer School (MLSS), Cambridge 2009
- Nonparametric Bayesian Models by Yee Whye Teh at MLSS 2009
- International Conference on Learning Representations 2013
– A Deep Learning Tutorial by Richard Socher, Yoshua Bengio and Chris Manning.
– Selective Sharing for Multilingual Dependency Parsing. I always like the work from MIT people. This is an interesting paper on multilingual learning. The model does not assume the existing of parallel corpus, which makes it more practical. Moreover, it can transfer linguistic structures between unrelated languages.
– Unsupervised Morphology Rivals Supervised Morphology for Arabic MT. Very close to my thesis. In fact, I just need to read Mark Johnson’s 2009 paper Improving nonparameteric Bayesian inference: experiments on unsupervised word segmentation with adaptor grammars.
According to Yann LeCun, this 3 weeks summer school will cover many stuff in Deep Learning.
Last night, I ran into an interesting question on Complexity while surfing the Internet. Here it is:
Let A be the language containing only the single string w where: w = 0 if God does exist and w = 1 otherwise. Is A decidable? Why or why not? (Note: the answer does not depend on your religious convictions.)
So what’s the proof. I come up with my proof that A is undecidable.
Consider a Turing Machine M used to decide A. If A is decidable, the Turing Machine M will stop and output God, the supreme being, in the output tape.
Forget about religious viewpoint, from logical point of view, let’s assume that God is the supreme being, who holds the most super power and can create anything. So, where does God come from? There must be another supreme being, who created God. The Turing Machine M, therefore continues working to output The One, who created God. But then, who created The One? Turing Machine M definitely enters infinite loop seeking for the original supreme being. Thus, A is undecidable.
Hey, does the proof sound familiar? Of course, it’s an analogue version of Turtles all the way down. In case you’ve never heard of this philosophical story, here is one of those version, found in the first page of my book Plato and a Platypus Walk into a Bar, Philogagging chapter.
Dimitri: If Atlas holds up the world, what holds up Atlas?
Tasso: Atlas stands on the back of a turtle.
Dimitri: But what does the turtle stand on?
Tasso: Another turtle.
Dimitri: And what does that turtle stand one?
Tasso: My dear Dimitri, it’s turtles all the way down!
(Kleene – Recursion Theorem) For any computable function
there exists an with .
We define a partial recursive function by . By s-m-n theorem or parameter theorem, we can find a recursive function such that
Let , choose then we have
Using recursion theorem, we have a beautiful proof for Rice’s Theorem.
Let be a class of partial recursive functions. Set is recursive, if and only if or contains all partial recursive functions.
or is trivial. We proof the case and .
Exist and . If is recursive, the following function is also recursive:
By recursion theorem, . We consider 2 cases:
- , by index property we have , so by the definition of . We get contradiction.
- , by index property , so by the definition of . Again we arrive to contradiction.
Why’s recursion theorem interesting? A typical application of recursion theorem is to find a function that can print itself. Let’s say print(*program*) is that function. The output of print(*program*) is not *program* but the function that excuses print(*program*). In other words, we could ask if there is a function which can be self-conscious. I like to call it function-ception, which can be related to inception: does one know that he is dreaming in his dream?
Kleene recursion theorem offers an answer to our question: there is a function-ception, or a program that can print itself. We’ll construct such that function.
Let , is the projection function .
By s-m-n theorem, there is total recursive function such that . Let . Apply Kleene recursion theorem, there is a number such that .
For each :
My work here is done!
How many fixed points does a recursive function have?
Intuitively, we can see that a (total) recursive function has infinite fixed points. Recall the proof for Kleene recursion theorem, when we choose . There is an infinite number of having that property. Analogically , there is an infinite number of ways to implement (write code) a function (by adding comments, using different variable names, …)
Now let’s work on some exercises using recursion theorem.
Lemma: There is a number , such that
We need to find a function that is only defined for input and undefined on the rest of natural numbers
I’m doing this in reversing way (how my thought leads me to the solution). By recursion theorem, .
Let is the function that it is only defined for such that . Such an can be constructed as following:
My work here is done. There is an index such that , by s-m-n theorem there is a total recursive function . Let , and the last step is trivial.
Lemma: There is a number , such that
With the same line of proof for the previous lemma, we only need to construct a partial recursive function . One can easily recognize is a projection function
I started Clojure past few days. Thanks to Jo and Milos, my nerdy classmates who are big fans of functional programing. Now I’m getting into it.
I heard of functional programming years ago, I played with Haskell but not much. Probably, the main reason I started liking Clojure because of Computational Complexity course I’ve taken at Charles University.
To be honest, I hated this course at the beginning. Machine Turing is not that bad, NP-complete and reduction techniques are pretty much fun, but recursive function and λ-calculus are so fucked up. I did hate all the lambda notations and hazy proofs with old math style.
I still hate λ-calculus if I don’t have to study for the exam. After several group studies with Jo, and he kept telling me how much he likes Scala and functional programing I thought “Hey, that’s pretty much similar to what we’re studying, you know, recursive functions and other stuff”
Yes, I like Clojure because I like λ-calculus, recursive function, partial recursive function, Kleene theorem and all other stuff. I decided to learn Clojure seriously, so I come up with an idea that I’m gonna blog about it. Of course, not only Clojure, FP but all other stuffs I like such as Computational Complexity, Machine Learning, Computational Linguistics, Lomography and so on.
Why is the blog named MyLomoWalk? I love Lomography, and Lomography is the art of coincidence. I’m a big fan of uncertainty, which often leads to coincidence. MyLomoWalk is my walk in the uncertain universe (or multiverse?)