Pipeline Data Engineering Academy home blog pages letters

Skills Gap in Data Engineering

Most data professionals realise very early in their journey that accessing the knowledge that they really need to solve data engineering problems is hard to come by. The other thing they don’t necessarily see is how short-sighted a lot of courses are, and how most of the technical content they provide is going to be rendered useless in a year or two.

Whenever people talk about the so called skills gap, they focus on addressing problems in a systemic scale and try to come up with a solution that is meant to address multiple dimensions at the same time: time, social factors, economic trends, technical advancement, legal changes just to name a few. This is complicated by the conflicting agendas of the involved stakeholders: employers, learners, government institutions and lastly, educational institutions.

I was lucky enough to be involved in an action group organised by Emerge, Coursera, Ufi and Filtered, that set itself the goal of coming up with a green paper that “examines how we can develop the necessary skills and establish new pathways into jobs”. My contribution in this work was very humble, but this should not stop you from reading the study. I don’t want to miss the opportunity to share this with you so everyone can have a sneak peek into the world of structural challenges that educators and edtech entrepreneurs have to tackle in the upcoming years.

READ THE FULL REPORT: Developing skills and establishing new pathways into jobs (.pdf)

You see, there is a strategic and a tactical level to addressing the challenges of the near future, but they can’t and shouldn’t be separated. Otherwise this will lead to fundamentally broken outcomes for learners: using wrong didactical methods, developing meaningless curricula, abusing technology for the wrong purposes, just to name a few. But false marketing messages (“profession of the next decade” — does this ring a bell?) are just as much of a problem, they can lead masses towards an unsustainable career direction.

Long story short, ultimately I tend to come back to this quote from John Alexander Smith, Professor of Moral Philosophy at Oxford from 1914:

“Gentlemen, you are now about to embark on a course of studies that (will) form a noble adventure…Let me make this clear to you. ..nothing that you will learn in the course of your studies will be of the slightest possible use to you in after life – save only this – that if you work hard and intelligently, you should be able to detect when a man is talking rot, and that, in my view, is the main, if not the sole purpose of education.”

The faster the world moves, the more you’ll have to learn to avoid being misled. Lifelong learning is not a trend.