Maintaining momentum

Collaboration in an unstructured world

While I haven’t made any decisions yet, I’m currently considering heading off into a much less structured career path which could be variously described as “independent scientist”, “entrepreneuer”, “contractor” or “self-employed open-source developer”. Along those lines, I’ve been wondering what makes work good for me and how I can put the pieces together reliably. I think the critical piece will be to proactively communicate with lots of people.

At the moment, I work as a machine learning engineer for QuantCo, working together with business people, economists, statisticians and other software engineers to build useful software systems for things like ecommerce pricing or insurance fraud detection. Jobs like this are inherently collaborative. Half the job is to work together with other people to figure out what exactly you should be doing and how to do it. There are strategy discussions, design discussions, pair programming sessions, stand ups and just plan old hang-out-and-chats.

By comparison, as an independent scientist, there’s not necessarily anyone there that is depending on me, helping me or working together with me. Even as a faculty member within the structure of academia, there’s a lot of individualism to science. A single research group is often quite disconnected from the rest of the community. Papers aren’t normally communicated until they’re done. A lot of people simply don’t care about the work you’re doing. Seminars or conferences can be confrontational. People don’t often say what they really think about some research outside of their small social network.

So, as I’ve been considering this potential path, I’ve been thinking a lot about how to establish the kind of collaboration I want. And I think part of the solution is to really focus on the people. While the true goal of my work is scientific progress combined with personal fulfullment, a reasonable proxy goal might be to make it all about people and engagement: people that value/use my work, people that talk to me, people that work with me. I doubt the end result will be very different between these objective functions and the people-focused version is going to do a better job of maintaining my momentum.

Along those lines, these are some rough ideas I’ve had:

Three pillars of good work

While discussing some career ideas with a friend recently, she asked me to back up and think about what basic components make work good and productive for me. Below was my attempt at a response:

  1. A medium collaboration level. Maybe ~10-40% of time is spent communicating and working together with good people who are invested in the work I’m doing and I’m invested in their work.
  2. Work that is intrinsically motivating. This is a fairly broad category. That could be building something that I think people want, could be fixing something that is broken, could be learning something new.
  3. Enough control. I don’t need to be master of the universe, but without some control, people will treat you poorly, projects will get cancelled and any number of other bad things will happen.

I think I’ve had two of these things at many points and that was super motivating and resulted in high happiness and high productivity:

I took away two lessons from this list. First, when I’ve felt unfocused or unable to get good work done, it was because something was fundamentally wrong and not because I needed to push harder. You can’t push through a fundamentally flawed mathematical method or a flawed business plan. When things went from going great to going badly in the situations above, I probably should’ve made changes instead of just languishing or pushing against a brick wall. Second, any time I can think of where I had two of these things was a really good time for my motivation and happiness. I don’t know what would happen if I had all three of these things and maintained that situation for a few years. Probably something really good. I should probably try to find that!

Thanks to Liz Santorella, Ruth Byers, Brendan Meade, Reena Joubert, and Phoebe DeVries for helping me crystallize some of the ideas in this post and for providing some edits. The ideas still are pretty rough and hypothetical, so if you want to chat please get in touch!

Posts BIEBook