Why I left Microsoft Data & AI team and become a researcher in Blockchain

Overview & Incentive

This would be my first article sharing mostly my thoughts and learnings based on my working experiences in startups and big companies like Microsoft, and two hot topic in computer science, AI and Blockchain. I hope to provide insights through comparing and contrasting by having practical experiences.

I would also like to summarize and document my thinking process of making the choice which I have struggled for the past month.

The Background

Last summer, for a university research project, I started to become interested in and learn Blockchain because of its novelness concepts and the cool name. However, even though I later had some project experiences in startups and teaching experiences in universities, I found it hard for undergraduate students like me to find opportunities interning for large corporations in blockchain. Thus, in order to gain the experiences in big companies, I accepted the offer of a one-year placement program as a Data & AI Consultant for 6-month’s training in Microsoft and 6-month’s developing an AI solution for and implementing in HSBC.

The Reason

I found being a Data & AI engineer not suitable for me and made up my mind to pursue a PhD in Blockchain after graduation because of the following reasons, listing out from two aspects:

Engineering vs Research

To me, the biggest difference between the two is the freedom of investing time and effort creating new value that only you in the world can develop.

In large corporations, most of the time, engineers wait for the pre-sale team to sign deals with customers and are assigned jobs afterwards to fulfil the customers’ need. In startups, engineers may be the seller of the product they develop themselves. However, both are expecting engineers to utilize existing tools and methodologies. The standard architect for specific customers is often reused to minimize risks and speed up the delivery. They don’t have the time and funding to develop new algorithms or bear the risk of failure in experimenting.

If being a researcher, your main job is to make sure you are updated with the latest trend and findings by reading as much as you can, develop a new solution combing and building on top of the best practices and discoveries to a particular problem. If in the end you have a valuable discovery, you can publish it in journals and conferences, in the hope of benefiting other researchers with your work.

I found the later one more interesting and also challenging but worths the effort to achieve:)

Be a researcher or an engineer first?

Consulting my mentor, a Data Scientist in Microsoft with 9 years of PhD experiences, I learned the benefit of being a researcher first and then an engineer is that you will have the connections of people that are really the expert in this area by attending conferences and sharing your findings, and it is very unlikely to meet them in person once you’ve entered the working space. She encourages me to learn as a researcher first to build up my knowledge base before implementing.

Although there are still many cases that people start working for a few years before going for research, it is worth noting that the level of difficulty of applying for a postgraduate degree may increase as the time you left college grows. You may lost the professors’ contact for recommendation letters and the resources available for students to support you.

To me, being a researcher first gives me more time in discovering my true passion and more freedom in which topic I want to dive deep in. I can still work on the projects that interest me without being restricted by the responsibility of a full time job.

Data Science vs Blockchain

From my understandings, data science interacts with the data that is static while participants in blockchain interact with protocols that is dynamic to how people with different incentives behave. Data Scientists gain insights of data through statistics and interpret the result with a reasonable story of the data’s behaviour and relationships, while Blockchain researchers develop consensus algorithms which guarantees consistency even when there exists untruthful participants.

From an engineer’s perspective, one of the most critical and time consuming part of data science is data exploration and data cleansing as you may have GBs and TBs of data to be processed. The cleansed data is then fed into machine learning libraries, one can imagine this as a super machine that takes a particular format of input and generates the desired result after some time and computing power. For similar customer needs, the set of the machine learning libraries are mostly identical. The most difficult part is making the messy data obtained from customers into the desired input format to for the super machine. We then try out different machines to find one that gives us the best result. We will need to further understand how the machine works in order to explain the result to the customer.

Blockchain to me is something combing Psychology, Economics, Database, Computer Networking, and Cryptography. Each element of its concepts is not a new invention in history but their combination is. Also, it is compelling because it gives us the hope to swipe off the traditional centralized parties like banks and the government, which sometimes gave us a bad impression on exploiting people by money laundering or social stratification.

Decentralization implies that everyone is a boss of his/her own. We are equal and no one can ever manages me or force me to do things that deviates from my incentive. I fell in love with the dream that Blockchain is trying to realize and I would be more than honored to make a contribution towards this grand vision.

Conclusion

The above is my personal view on how I see research and engineering, Blockchain and Data Science from my experiences so far. There is still a long way to go and feel free to discuss with me if I interpreted something wrongly.

Thank you for all the friends that support me during the journey:) I will never be so brave without your consultations and encouragement!!