2015, Finished

31 December 2015

On New Years’ Eve I think on the events and patterns of the year, as a reflective practice. I learn more when I use hindsight to my advantage.

I’ve looked at the work I didn’t do. Now I’ll look at what I did.


The unexpected is a source of inspiration and joy. Sometimes it is a source of meaningless interruptions. I have found that people are a source of inspiration, and advertising is invariable noise. I use technology to filter the latter out of my life.

  • Web ads, including Facebook ads (via Ghostery, Adblock Plus and Social Fixer)
  • Television ads (I don’t have a TV, just a laptop & projector)

Lesson: Technology, like all tools, can be used to further your goals. Identify what you care about (fewer interruptions, privacy), and chances are there’s a tool for you.


I like to anticipate obstacles and opportunities by being well informed. I ended up reading over a thousand blog posts, research papers, newspaper articles, and books this year. I lack an eidetic memory, so I maintain a personal knowledgebase.

I keep track of the most useful resources, categorized by topics such as health, food, education, government, companies, privacy, computer security, intellectual property, software development, data science, finance, environmental engineering, productivity, communication, job searches, scientific research, and our ever-growing police state.

Lesson: Latency is king. The amount of information I can access depends on time. Computing latency works the same way.

  • CPU registers - immediate recall. I can continue doing what I was doing without pause.
  • L2 cache - I can find answers in less than a minute using a personal knowledgebase.
  • Main Memory - I can find answers in less than an hour by reading articles/posts/papers found via Google searches.
  • Disk - The information is not available online. It will take me hours or days to get the answer from books or experts.

Many useful tools (Google, StackOverflow, cookbooks) are successful because they move knowledge a lower latency.


I was able to complete one big side project this year, analyzing the personnel expenses of the University of California (UC) system. Spoiler: most of the money goes to administrators, not teachers or students.

I can’t publicly show the work I do at my job. Practically all of my work results are derived from student activity, and even non-PII data about students is restricted by FERPA laws unless lawyers say otherwise.

Luckily I can talk about what I do, without the results, on my work blog.

I have made some (very modest) headway into tackling the problem of student debt at the UW. I didn’t anticipate that all of the challenges would be political/organizational, rather than technical.

Lesson: Don’t expect to change a company’s culture without the risk of getting fired. Work on analysis projects that have clean data sets, to save time.


My learning this year was focused on a small number of skills.

  • Data science, with an emphasis on feature extraction and nonlinear classifiers.
  • Neural networks, including 10-layer nets (‘deep learning’)
  • Medicine, specifically genomic analysis and epidemiology.
  • Communication. No project is done alone, and people are more complicated than any algorithm.

Brain Food

A large number of the best engineers I’ve met are accomplished musicians. Learning new programming languages, patterns and business rules require focused effort to fit new ideas into your own understanding of the world. It’s the same process as learning new music. I play in a Brazilian samba band for fun.

Lesson: Work on a craft or hobby. It’ll make you a better engineer, and it’s a blast.

Here’s to Another Great Year!


2015, Unfinished

30 December 2015

2015 is at an end. Each year I think on the events and patterns of the year, as a reflective practice. I grow and learn more when I use hindsight to my advantage.

Let’s start with the projects and work I didn’t do. I don’t have regrets, but I don’t if I would make the same choices, either.


As an aspiring polymath, I am always learning something new (and forgetting something old; cache eviction happens to humans too). Sadly, I was not able to learn a few things I aspired to:

  • JavaScript and React.js
  • Scala, in more depth
  • C / C++
  • Bayesian inference

Lesson: I didn’t devote significant time to these things. I didn’t even start learning most of them. I always felt I had something more important to do. So the real question is whether my sense of judgment is wise, day-to-day.


When I’m not spending time at work doing data science, I’m doing it at home, on side projects. My life is awesome that way.

Lesson: These projects didn’t get far because I didn’t have the time/inclination to pursue them over other ideas. Several of them, notably around health-care and people’s behavior, are limited by a lack of data that’s easily accessible.


I didn’t update my blog much; I was usually chatting with on Twitter. I did maintain draft blog posts, mostly as a personal knowledgebase.

Lesson: This year I was more intentional about communication, and focused on back-and-forth communication (Twitter) rather than one-way communication (a blog).


A few lessons and changes are apparent from this year.

  • For analysis projects, make sure I care enough to start one. Use cleaned-up / available data sets when possible
  • Be intentional with larger-scale projects and efforts. There isn’t time to do very many of them.
  • For everything else, go for incremental improvement over what others have done.
  • Change the format of my blog to be more collaborative. Hosting IPython notebooks would be a great start.