pvigier's blog
computer science, programming and other ideas
Pychain Part 1 - Computational Graphs
Welcome in this big tutorial on neural networks!
Our goal is to write our own deep learning framework like TensorFlow or Torch. We are going to learn in-depth how neural networks work, all the mechanics behind them.
We will get our hands dirty and code everything! In this tutorial, we will use Python3 and scipy but I hope that the code and the ideas are clear enough so that you can adapt the code to your favorite language.
First, I show you the plan. In this part, we are going to quickly introduce neural networks and then, we will introduce computational graphs in order to model them. In the end of this part, we are going to use our implementation to learn some non-linear functions.
In the second part, we will deal with a more serious problem. We are going to build an optical character recognition system upon the MNIST database. It is a classical problem in machine learning, we have to do it.
Then, we will tackle recurrent neural networks and show how to model them with our library. To apply our new knowledge, we will try to learn a formal grammar generated by an automaton.
In part 4, we will go further with recurrent neural networks and introduce the well-known LSTM cell. We will briefly compare it with fully-connected recurrent neural networks.
To approach part 6, some more efficient optimization algorithms are necessary. Consequently, we will discuss them in part 5.
Have you ever read this fabulous article by Andrej Karpathy? Yes? Cool, because, we are going to reproduce his results with our own library in part 6. Amazing, isn’t it?
Finally, parts 7 and 8 are going to be theoretical appendices for the most curious readers.
Is it all? Maybe not! Stay tuned!
If you are ready, let’s go!
Hello World!
Welcome on my new website!
I will try to keep it up-to-date. And to write a post as soon as I have an idea or discovered anything interesting. Stay Tuned!
A short word on the technologies which proudly power this website. As you can see, the website is gracefully hosted by GitHub. Thanks guys! This website is completely static, I use Jekyll to generate the pages. By the way, I found this page very interesting to get started with Jekyll. And the design is made with Bootstrap.
I try Disqus as commenting system. I’ve seen a lot of blogs using it and it was very easy to install. Finally, I use Google Webmasters and Google Analytics to manage my metrics.
I hope you’ll find anything interesting here!
