The Year of Hacker School

A year back this time, I landed in New York city amidst a lot of snow fall. I had gotten into Hacker School for the Winter'14 batch, and a lot of new things were in store for me. I was gonna be living on my own in a new country for the first time. When I got to my subletted apartment, the incessant sirens of fire engines and ambulances that make you wonder how many things could possibly be wrong in this city - intimidated me. But I grew the most in those three months.

I fell in love with New York, my own independent life, my amazing roommate, new friends and all the stuff that I'd never have tried otherwise. Most of all, Hacker School. I've tried explaining Hacker School to a lot of people - and I always like to say - it's an amazing place to learn whatever you want, work with others, meet the experts and talk to them and all that, but that's not what makes it special. What makes it unique is the learning environment it seeks to create - with all the social rules and embracing diversity - making it the place where everyone can thrive - no matter who they are or where they come from.

I spent most of my time there learning Clojure, among other things. I built small simple things, but I realized I had so much more focus. I was putting in the effort to dig into the tools I used, understand how things worked underneath. Other people read my code and helped me make it better. I was no more the programmer who was trying to put together different things and libraries and make an "app". It felt good.

When I came back to India in May, I made some elaborate plan for this year that doesn't matter anymore. But it involved having to wait for stuff to happen for a few more months, and I thought what better to do than go back to Hacker School once again. And so in August I was back in New York, all ready to start a new batch in September.

This time around I got to contribute to Hacker School's internal site, Community and implement search for it, do fancier stuff in Clojure like writing a bittorrent client. I played around with Distributed Systems which I realized I love working on very much. I went to two of the best tech conferences - StrangeLoop and Clojure/conj - and met people I admire and people who've helped me out so much.

I made a lot of friends at Hacker School. Friends who took me dancing. Friends who helped me out in very difficult times. Friends who threw the most amazing Christmas potluck dinner ever. And friends who I can always turn to for anything.

I finished my second batch of Hacker School in November, and came back home in January. And then I realized, all of 2014, I'd done Hacker School! I've wondered if I was doing the right thing by investing a whole year in it - but seeing how much I've grown - it was pretty damn worth it.

I have new exciting beginnings coming up - more on that later.

The week of Strange Loop

I was at St.Louis last week to attend Strange Loop, thanks to a generous diversity scholarship by all it's awesome sponsors. Strange Loop is definitely the most exhilarating, informative and inclusive programming conference that I've been to.

Fun

One highlight of Strange Loop for me was that there were so many Hacker Schoolers there, past and present, attending, talking or running workshops. It's my first ever conference in the US and the first time I was out of New York, and it felt super nice to have familiar faces all around.

I reached St.Louis on Tuesday, 16th, and got together with Pablo and Lita, my W'14 batchmates for dinner at Pi's Pizza. We did some exploring and walking around, and it was super fun catching up with them.

Dinner with Pablo and Lita

Wednesday evening, before the preconference party, there was a small meetup where over 50 women attending the conference came - Lambda Ladies go bowling, at Flamingo Bowl. It was awesome fun to bowl, and by some twisted luck I tied at the top of my team bowling :'D It was also great connecting with everyone there.

Flamingo Bowl

And then the best party ever happened at the City Museum St.Louis. It's the singular best museum ever and has amazing slides, caves, climbable art, airplanes you climb into and sit on the cockpit, ferrous wheels 10 stories atop. You can only walk out of it feeling like a kid. I've been to geek conference parties before, and outside of ones at Hacker School, never has one been this fun and inclusive, considering I don't even drink!

City Museum Party

The conference happened on Thursday and Friday, and I'll get to that later, but both days in the evenings, I went to dinners put together by Hacker Schoolers/people at the conference. On thursday I went to one organized by the awesome folks at TwoSigma, at 1111 Mississipi, to which over 40 people came, and we thoroughly took the restaurant folks by storm! Friday evening was a small dinner at a nice Thai place with Hacker School folks.

Hacker School Dinner

I also went to a baseball game between the Cardinals and Reds on Friday evening, with Pablo and Denise, and except for the fact we din't understand the game and clapped and cheered whenever the rest of the crowd did, it was super great :D

Baseball game

Learning

If you were thinking after reading all the above, did you go to the conference or not? Oh yes I totally did. And it's surprising as I write this that despite all the activity, I actually went to so many talks and learnt so much that my mind was blown.

All talks are here

Some of the favorites among talks I went to, in no particular order were:

Where Kyle Kingsbury enthusiastically talks about Jepsen, the unlimited grant he got from Comcast, what he did with it, and his experiments on RabbitMQ, etcd, and ElasticSearch, with drawn slides.

Where Rich Hickey talks excellently about how ridiculous it is that when we program, processes on data in our code don't happen in steps like in the real world, but happen like do something, make a copy of the data, do something to the copy, and so on. Transducers introduced in Clojure changes that, where a process can be defined as a series of steps, like a real world recipe.

Where Joe Armstrong, one of the co creators of Erlang, tells you how we've bloated the world of code with tons of code, and his dream for a compressor machine that can just eat away duplicate code and make it smaller, with the ideas of distributed hash tables like in Bittorrent, and hashing as in Git - Gittorrent may be?

Where Will Wilson, an engineer at FoundationDB talks how they built a simulation framework that uses a single threaded concurrency system and simulate network, disk, and all kinds of possible failures and exhaustively, and deterministically test a distributed system.

Where Peter Bourgon, distributed systems engineer talks in detail about how they built an AP system compliant to ACID 2.0 using Redis shards at its core at SoundCloud, with eventual consistency using CRDTs.

Being the most spirited speaker ever, Julia Evans (Hacker Schooler :D) talks about whys and hows for kernel hacking, demos and explains a fun kernel module that rickrolls every time you open a file, and how she built her own little OS than can now read input from the keyboard and display it.

Where Ian Davis, engineer at Prismatic talks about how they moved to using ClojureScript, and Om, a cljs wrapper around React, the frontend JS framework, and what was hard and the great benefits they reaped out of the shift.

And, of course when Sam Aaron and Carin Meier in the final keynote, made robots dance to live music by writing Clojure!!!

Robots dance

StarStruck

Learning and all the fun aside, the best part of Strange Loop for me was to actually meet and talk to some of my favorite programmers and people who I look up to, like Rich Hickey, Kyle Kingsbury and Joe Armstrong. I definitely left there starstruck :D (Disclaimer: This para is over 10 notches toned down on the level of excitement I actually feel for this!)

So, best programming conference ever, and I'm already looking forward to going next year. If you missed it this year, I have one word for you, Go!

Second stint at Hacker School - Devlog Week 1

It's been super exciting returning to Hacker School for another batch. I'd started with Clojure the last time, and this time's goal was to become better at it, and also learn about distributed systems. The first week went pretty well, and here's a summary.

Writing a Bittorrent client is an oft taken up project in Hacker School, given the array of concepts it covers. To get the feel of building a larger system using Clojure, and understanding how to manage concurrency and communication, I decided to build one too. After a brief walkthrough about the steps involved with Alan, one of our facilitators, I got started with first building a parser for Bencode - the format in which torrent file data is stored. Once I had the encoder/decoder done, I got started on the UDP tracker protocol. I'm still on that, but I can successfully send and receive UDP messages now.

  • Work on community.hackerschool.com

The community project is a new internal Hacker School site, as a replacement to the use of mailing lists for communication. It's built with Ruby on the server side and Om/Clojurescript for the frontend - and that's interesting to me, and I've been making tiny contributions - 2 so far, thanks to Zach, who's been so patiently explaining stuff to me and helping me out.

As part of the friday job prep related activities, this week's task was to spend 2-3 hours building a small command line phonebook app, that can create phonebooks, and do add, delete, update and lookup on it's entries. I built a simple one with clojure using files to persist data - this exercise was a lot of fun!

That's all for this week. Fingers crossed for my Bittorrent client to do wonders by the next ;)

Subtle Changes after Hacker School

Coming to Hacker School in Feb sure made me a better programmer, but there are a few other things that have changed. Subtle but profound.

  • Subtle-isms

Hacker School has a social rule revolving around subtle-isms, which bans racism, sexism and other forms of bias. Now I never identified myself as any kind of subtle-ist, but now I notice these things when people talk around me more. I notice it more when I talk. When I say something about someone, or when someone says something, at events I go to, when talking to friends, I can tell myself or say it aloud, Hey, that's racist, or Hey, that's sexist, even with things that have become so ingrained it never occured to me in the past that it was one of those subtle-isms. For instance, referring to group of people as "guys", or involuntarily deciding all people who were from South East Asia to be Chinese, are things I now realise as wrong, and have my gaurd up against.

  • Feigning Surprise

The other social rule which is against feigning surprise - The number of times I've heard feigned surprise and the number of times I did it myself before, unconsciously even, took me by surprise. When someone asked what some simple thing was, my automated response would begin with, oh you don't know? It's just this simple thing you know... I wouldn't be condescending, but that's how I was responding! Once I came to Hacker School and learnt this little rule, I was now replying with, Okay, so, this is how it works. And I took it back home with me. When a school kid came home to ask some Math doubt to my brother, I told him before he was going to teach her, Hey, don't feign surprise if she doesn't know something you think is trivial. Teach with patience. And I'm happy I did that.

  • Fear of judgement

Hacker School and to some extent, all of New York, have been places that are completely free of judgement. Places to make mistakes and recover without any shame, because no one was judging you. I always had a fear about what others think of what I do, and that has controlled my whole life. Now having functioned in these environments, I feel so much more secure, even if to make a fool of myself - I can tell myself I'm learning, and it doesn't matter at all what others think. And that freedom in my head is life changing.

Being a part of this inclusive community is definitely one of the best decisions I've made. Hacker School <3.

Devlog: Week 11: The end nears

This week wasn't a very productive week for me. I've been too restless thinking about leaving home next week and life after Hackerschool. Anyway, some notes from the week.

  1. I understood how to build a full stack Clojure webapp with a ring server and compojure router on the server end and Om/ClojureScript on the frontend.

  2. I deployed this helloworld webapp that I built on to Heroku, my first app on Heroku. It was mostly simple because Heroku can read the project.clj file and automatically build and deploy. However there were a few gotchas on lein versions that led to ClassNotFound errors.

  3. Heroku uses git for version control, but I use git too! Files like generated JavaScript files needed to go on Heroku's git, but not my repo on Github. I was super confused about how to manage two different remote endpoints. Then I figured I could make a deployment branch and commit and push all the compiled resources there. One little problem was that Heroku only redeploys the app when pushing to master. Then I learnt this little trick.

git push heroku deploy:master

This pushes my deployment branch on to Heroku's master, so that problem solved.

  1. I've been trying to learn datomic, the transactional immutable database. Kevin Lynagh, the Hackerschool resident for last week, gave me a 2 minute pitch on the architecture, and I've since been playing around with the in memory storage and figuring out how stuff works. This week I hope to plugin a datomic db to my app and learn how to provision storage for datomic on Heroku too.

  2. The Hackerschool job fair happened this Thursday. Although I wasn't really looking to get a job, it was one awesome experience. Around 25 companies, most of them represented by Hackerschool Alumni came in and pitched in 2 minutes why we should work for them and it was followed by loads of networking with 150 people in the room trying to talk louder than each other. It was awesome meeting these companies and learning of their tech stack, culture and exciting problems.

That was last week. Hoping to shake out of my reverie and do more this last week.

Devlog: Week 10: Countdown timer up and running

My first goal this week was to deploy the countdown timer app I've been working on in Om. I spent a while trying to figure out how to do it, tried using Heroku et.al and had no clue whatsoever what to do. But thanks to being at Hacker School, I asked folks, and soon with David Nolen's help, I figured that given my app was entirely on the frontend, I could just advance compile the app to JS and host it as a static site.

I also had a Jquery plugin, and these additional JS dependencies are called "externs", and this blog talks about dealing with them. Anyway, so I set up my different dev and release builds, and used advanced optimization and externs options for the release build.

:cljsbuild {
    :builds [{:id "dev"
              :source-paths ["src"]
              :compiler {
                :output-to "cheating_countdown_dev.js"
                :output-dir "out"
                :optimizations :none
                :source-map true}}
             {:id "release"
              :source-paths ["src"]
              :compiler {
                :output-to "cheating_countdown_release.js"
                :optimizations :advanced
                :pretty-print false
                :preamble ["react/react.min.js"]
                :externs ["react/externs/react.js"
                          "resources/js/jquery.min.js"
                          "resources/js/jquery.datetimepicker.js" ]}}]}

Then I setup a static app on Webfaction and wrote a tiny bash script to compile and copy the contents to my server.

lein cljsbuild once release

scp -r resources index.html cheating_countdown_release.js <SERVER ADDRESS>:<STATIC APP DIRECTORY>

Then I had my app up and running here: http://countdown.madhuvishy.in. So far so good.

I added a couple of features, one - Undo. Undo was not something one looked to have in a countdown timer, but given I was using Om, I had to test time travel. And it was so simple to do. I had to decide what to store in the global app state and local component state in order to do this, and move the deadline to app state and remaining time to local state, which I had the other way round before to make undo happen nicely.

Other was the drunk mode, which is kinda pointless, but meh, in case you want to mask reality and see random times being countdown to, use it, or not.

So there's my countdown timer, the full code's here.

The other interesting thing this week was the Monday talk, that for the first time happened at Etsy. The talk was by one of Etsy's product managers on how to remain a hacker while at work. The talk was great, but even more awesome was the tour of Etsy's office. Everything in the office was spiked with creativity and was artistic. From floors to desks to walls to ceilings, it was all handmade by local artists or Etsy sellers, and was incredible. Definitely an office space anyone would love to work at!

I also spent a little bit of time pairing with fellow HackerSchooler Will Sommers on his Online Github repo viewer app in Om, leading to discussions on the philosophy behind what to store in local and global state in Om, which was insightful and fun.

I attended my first Meetup event today! The NYC Clojure Users group put together a hackday hosted at Pivotal Labs. It was awesome meeting other clojure devs and hearing about stuff they were working on. I spent my time figuring out the web stack in Clojure and how to put Ring, Compojure, EDN, Datomic etc together to build a webapp. I thought I'd build a URL shortener with all this stuff, but got as far as Hello world today. Rest in next.

Ooh Sun

This weekend's been so much fun. Thanks to the temperatures being around 20°C the whole time. In the spirit of being liberated to walk around without any winter ammunition whatsoever, you can pretty much guess what I was up to. Well yeah. Walking. Quite a bit. But other things too.

I discovered the awesomeness of walking without the jacket when walking to HackerSchool on Friday. As soon as I went in, got some work done and was already restless to be out. Went on a long walk back and forth to Union square on the usual Hot chocolate mission.

Talking of being restless, this whole week's been like that for me, because the weather out is so good, and I'm itching to step out as soon as the clock strikes 6. Among random window shopping walks on Canal, my Thursday which started with salad for lunch, ummm progressed to hot chocolate and cookie to Pizza at Amici II in Soho's Little Italy and culminated with sweet chocolate crepe in Eight turn crepe. So much for healthy lunch.

Back to Friday. Cutting across my pointless wanderings on Avenue of the Americas, friends from college lured me to the Madame Tussaud's museum near Times Square. Though expensive to visit, I had an amazing time at the museum, dumb copycat posing of celebrities and watching the 4D Marvel Superhero movie :D

DSC01472

DSC01623

On Saturday, I stepped out to Union Square and lunched with Laura at Pret, a kinda nice place that makes handmade rolls and sandwiches. Basking in the awesome weather, we pointlessly strolled to the West Highway after dropping in at Chelsea Market, walked downtown, made our way up to Broadway, and back to Union Sq. On instinct we decided to go up to the Grand central station, and, the lights, the lovely architecture and the grandeur, made the visit worth our while. I headed home a couple of hours later after more unintentional walking thanks to the chaotic subway schedules in the weekends in NYC.

Although now should be the part where I was extremely tired and crashed, the day got better when HaSc-er Patrick invited us all to a play by the Drunk Shakespeare(!?) Society, at Quinn's bar, a short walk from my place. Shakespeare? Drunk? I had to see that! I headed up there, and it was pretty much the craziest thing I've ever seen.

What happened there stays there. All there is to say is it was nuts. And that it involved an auctioned throne for king and queen and a dance off on Michael Jackson's Beat it instead of the final war between Macbeth and Macduff. No more.

I quietly lazed at home all day on Sunday, and went out to a nice little concert where HackerSchooler Jeff Fowler was playing the guitar and bass. It was at a rather quiet new bar in Brooklyn called Friends and Lovers.

DSC01705

<Awesome music and awesome weekend />

Devlog: Week 9: Getting started with Om

I started this week in the enthusiasm of going back to Clojure, and learning something new. I've been wanting to toy with ClojureScript, and there's been interest in Om, a ClojureScript wrapper over Facebook's React JS, and decided I'd pick it up and build something.

With perfect timing, the awesome David Nolen, who built Om, gave a talk on Monday night at Ebay on Om, and the philosophy behind how Om apps are written, and how they could be extended to regular frontend building. Om benefits from the great decisions React's engineers made, and the immutable data structures built into ClojureScript, but, it is almost, esoteric. It took me quite sometime for stuff to click in my mind, and I'm still not sure if I have it all.

With help from fellow HackerSchooler @briankimisms, I got started on building a countdown timer in Om. You choose a date and time, and the timer counts down to it, and stops at 0. Pretty simple. I might also build a drunk version of the timer that will show you a speeded up version of the timer in case you just wanna speed up time and not really wait for something to happen. Don't ask me why a timer would do any such thing. It's drunk.

I should deploy what I have to a server, and have been trying to figure out how exactly to do it in Heroku, and may be later in Webfaction, but that's for the coming week. The code so far - https://github.com/madhuvishy/cheating-countdown, and this is how it looks:

Leaving NYC in...

Apart from this, I code reviewed my ray tracer with HackerSchooler Carl Vogel, and got super valuable feedback on writing better idiomatic Julia. I'm yet to incorporate his suggestions into my code, but should have that done soon.

I attended a video conference talk on OmChaya, a full fledged application built using Om, by Sean Grove, and, . His talk gave me a better understanding of how well Om and in turn React could be put to use to solve real problems, like user activity history tracking, etc, and got me more enthusiastic about Om.

Let's see where I get with it next week. Till then.

Girls days out

When I left to New York, I was pretty sad that I had a real sedentary life at Chennai. Sleep very late, get up very late, work from home, eat, watch TV, sleep late, was my daily schedule. When I came here, it all turned topsy turvy. My levels of activity have gone up so much, and I’m amazed about that. In the spirit of that, to keep reminding myself of my life here, I decided to get a FitBit. My FitBit flex arrived on April 1st, and I did not go out of the way to boost my stats, but this pretty much sums up my weeks here :)

Screen Shot 2014-04-06 at 10.10.52 PM

Woah, what the hell did I do in the weekend? Here goes.

My friday started by visiting the set of the Night of the museum, or rather, the American Museum of Natural History. Scenes from the movie playing in my head as I went around the museum, it’s by far the most amusing and interesting museum I’ve been to in New York. The ocean life and biodiversity sections were breathtaking, and the mineral and meteorite sections caught me in awe. I went to HackerSchool after the visit to the museum, and in the evening, given that the weather was super warm and perfect, took a super long walk home.

On Saturday, I had late awesome lunch at Gazalas' Palace with Laura, and went on the Staten Island ferry and to Brooklyn Bridge again.

Sunday was kinda great. Laura and I visited Roosevelt Island, a small tiny island reachable by train, part of NYC, but a completely different world altogether. Quiet, solitary, long walkways by the river, parks, cut off from the noises of Manhattan, it's a beautiful place to be in. After a long walk by the East river, we took the famed cable car across the river and got off at East Side Manhattan. My childlike excitement for my first cable car experience gave through to awe for the breathtaking view from the heights.

DSC01445

We then had South Indian food at Lexington Av, and the day already having been brightened up, headed to shop downtown. Later in the evening, we went to the closing concert of Unsound, an experimental music festival that originated in Poland. The venue was at Issue Project Room, a nice music venue at Brooklyn, New York.

It was unlike any concert I've ever been to. Everybody just sat and listened. There were no vocals. No one knew the tunes being played. There was just music. There was electro-acoustic music, free-jazz drumming and synthesized music. The kind I've never listened to. It drew you away from the rest of the world, but din't draw you into the music. It just leaves you hanging. In your own world of thoughts. Amidst a different kind of music that was silence.

...

Devlog: Week 8: Writing a Ray Tracer in Julia

It is real long since I studied coordinate geometry or computer graphics. This last week, dusting the very little remains of what I knew, I decided to work on a ray tracer. When I started, I did not know what a ray tracer was. Slowly, I understood that it was one algorithm that is used to construct a 3D scene on a 2D screen. Broadly, this is how it works

Define the eye/camera/viewpoint

Define the screen in m*n pixels

Shoot a ray for each pixel in the screen from the eye. For each ray,
    Find nearest intersected surface
    Find a color for the point based on surface reflection/refraction or the light source and shading model
    Plot the color on the pixel

I decided to build one that would trace spheres, so I had to define 3 major types - Vector, Ray, Sphere, and a helper type Intersection to store where an object intersected with the screen.

type Vector
    x::Real
    y::Real
    z::Real
end

type Ray
    origin::Vector
    direction::Vector
end

type Sphere
    center::Vector
    radius::Real
    color::Vector
end

type Intersection
    point::Vector
    distance::Real
    normal::Vector
    object
end

Interestingly when I went about defining methods for the vector, I found Julia had inbuilt support for finding dot and cross products, so all I had to do was piggyback on those implementations and build my vector class. Also, thanks to Julia's multiple dispatch mechanism, overriding inbuilt operators was as simple as this:

function +(a::Vector, b::Vector)
    return toVec(toList(a) + toList(b))
end

That done, I went about implementing the core algorithm, and a couple of simple shading models, being Lambertian and the Blinn Phong shading models.

Once I'd implemented it all, and fixed all bugs that produced seemingly awesome but wrong images, the main loop consisted of this:

for x in 1:imageWidth, y in 1:imageHeight
    traceWorker(x, y, imageArray, cameraPos, lightSource, objects) 
end

//See full implementation <a href="https://github.com/madhuvishy/ray-tracer/blob/master/tracer.jl#L119">here</a>

I used Images.jl plugin, to build the image, and ImageView to display it, and when I ran my code with the above main loop I got something like this:

☁  ray-tracer [master] ⚡ time julia tracer.jl

julia tracer.jl  11.55s user 0.26s system 98% cpu 11.933 total

But having used Julia, I had to put it's parallel processing capabilities to test. And because each of the calculations in ray tracing are independent of each other, I could put this, to use. And now my loop became...

@parallel for x in 1:imageWidth, y in 1:imageHeight
    traceWorker(x, y, imageArray, cameraPos, lightSource, objects) 
end

//See full implementation <a href="https://github.com/madhuvishy/ray-tracer/blob/master/tracer.jl#L119">here</a>

Sweet! Let's check the new running time.

☁  ray-tracer [master] time julia tracer.jl

julia tracer.jl  6.44s user 0.11s system 99% cpu 6.566 total

Yay! Down by almost 50%. Pretty sure there are more ways to optimize my code, but this is how far I've gotten till now.

Now, for our final showdown.

Screen Shot 2014-03-31 at 5.18.54 PM

Woah, but no...

Screen Shot 2014-04-01 at 11.17.16 AM

Ummm, circles or spheres?

Screen Shot 2014-04-03 at 5.14.02 PM

Yes this.

Thanks to fellow HackerSchooler Lita Cho for patiently explaining how ray tracers work :) Check it out at: https://github.com/madhuvishy/ray-tracer/

Share