It's time to build data pipelines

Desperate times call for uplifting speeches from venture capitalists, apparently. Some six weeks into the COVID-19 induced economic crisis Marc Andreessen has blessed us with the glorious essay "IT’S TIME TO BUILD" (sic).

For those of you who have not read it yet, here's my brief rundown:

The corona virus made us realise there are certain scalable products and services our society desperately needs but simply does not have enough of (surgical masks etc.), and the reason for that is not a lack of technical know-how or missing financial means for production, but that we consciously have chosen in the past not to build more of them. Andreessen is segueing from healthcare to housing, education and transportation to name some examples for sectors that we should literally expand both in scale and accessibility so we're better off collectively. After a spirited request to the political decision-makers on the right and the left he argues, although building is difficult, it will lead to the next wave of economic-idealogical prosperity in the US. Here's the finishing punchline:

Our nation and our civilization were built on production, on building. Our forefathers and foremothers built roads and trains, farms and factories, then the computer, the microchip, the smartphone, and uncounted thousands of other things that we now take for granted, that are all around us, that define our lives and provide for our well-being. There is only one way to honor their legacy and to create the future we want for our own children and grandchildren, and that’s to build.

First and foremost, I feel the confusion of Vicki Boykis, the queen of Normcore Tech (who also happens to be a master at curating artwork for her newsletter):

What are we, the people, supposed to make of an essay that is asking, nay, demanding that we build while at the same time ignoring what that the author has previously built - or, rather, invested in, has created as many, and maybe more, societal problems as it’s solved? The situation online as a result of many of these startups is so bad even the inventor of the internet doesn’t like it anymore.

Let's put that very valid argument aside for a second and detach the message from the messenger. Furthermore, let's also ignore our current economic realities, the ideological landscape of the Western hemisphere and the systemic structure of our society. Let's ignore the cheap argument of going borderline Sheryl Sandberg ("we need to do better") before the closing HATERS-WILL-SAY-IT'S-PHOTOSHOP!-style pre-emptive defence at the end of the original post ("I expect this essay to be the target of criticism.").

Ben Thompson comes to a much more positive conclusion after putting the piece into a broader context:

I do believe that It’s Time to Build stands alone: the point is not the details, or the author, but the sentiment. The changes that are necessary in America must go beyond one venture capitalist, or even the entire tech industry. The idea that too much regulation has made tech the only place where innovation is possible is one that must be grappled with, and fixed. And yet, Andreessen himself said that we need to demand more from one another. We need to figure out how to fix Wisconsin, not flee from it. We need to figure out how to build real businesses that build real things, not virtualize everything. And we need to start fighting for not just infinite upside, but the sort of minute changes in cities, states, and nations that will make it possible to build the future.

Moving from individual action to cooperation to arrive at collaboration is a tremendous part of what made humankind prosper in the first place. Building is a manifestation of an agreement about what we as a species need for the future and how we should invest our collective resources and yes, it presumes a shared desire and a grand vision. Building as a process carries an inherent value that is able to give purpose and hopefully even meaning to our lives. In this context building becomes the enabler of prosperity.

What does this have to do with data engineering, you ask?

A couple of weeks ago COVID-19 has derailed our timeline for starting our new school for data engineering. While we were trying to get used to the new rules of lockdown-life and attempting to reorganise our daily routines, we've decided that there is no way that we can sit back and watch how this story unfolds without us doing anything about it. We are firm believers of the transformational power of education and that sharing our professional and personal experience in the realm of data is something that can help individuals and businesses improve their competences, which as a result will make them more resistant to the economic impact of the crisis and possibly more attractive on the job market on the long run. So we came up with an idea that we're working on right now. I'll share more on this in a couple of weeks (sign up for the newsletter and follow us on instagram and LinkedIn).

And this is why the below paragraph of Marc Andreessen's essay has resonated with me so strongly, it comes from the same place as our intention. I am doing my best to avoid sounding overly pathetic and I won't compare our pseudo-altruistic relief effort to the ones essential workers are coping with on a daily basis. Still, I am excited about the next steps of this journey and encourage everyone to think about this for a minute:

Every step of the way, to everyone around us, we should be asking the question, what are you building? What are you building directly, or helping other people to build, or teaching other people to build, or taking care of people who are building? If the work you’re doing isn’t either leading to something being built or taking care of people directly, we’ve failed you, and we need to get you into a position, an occupation, a career where you can contribute to building.

The right time to build

It might sound controversial to some and a bit of a cliche to others, but I am actively trying to convince myself that starting a company during a recession is a good idea. There are plenty of well-known companies that made it and there are honest founders to tell their heroic stories of success. On a separate but related note I'd also like to introduce you to my old friend, survivorship bias... but on the other hand: