Over the last five months I had done nothing but program neural nets. The purpose of that was not that I had anything specific in mind when I started, but merely to get good at writing efficient numerical code. It had its ups and downs and now I am think I am rather decent at it, good enough that I have no need to rely on mainstream libraries for my machine learning needs. Before I transition to more practical matters using the skills that I carefully developed, what I want to do is write about my experiences and teach how to write your own basic reverse AD GPU library from scratch in F#.

Writing numerical code is difficult and error prone, so much that knowing the right data structures can make a world of difference. Using correct design patterns can turn a indecipherable messes that were my first attempts at LSTMs into a relatively straightforward affair.

I think knowing then what I do now would have saved me a lot time.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s