As a data engineer at the company, you will play a vital role in supporting this project. You will be responsible for designing, building, and maintaining the data infrastructure and pipelines that enable the collection, processing, and analysis of diverse data sets. By ensuring the availability, reliability, and security of our data assets, you will empower our team to derive meaningful insights and make data-driven decisions.

Working closely with cross-functional teams, you will contribute to the development of advanced analytics capabilities, leveraging cutting-edge technologies and methodologies. Your expertise in data engineering will help us streamline data workflows, optimize data storage and retrieval, and facilitate seamless integration with analytics and visualization tools.

We are looking for candidates who are passionate about data engineering, have a solid understanding of modern data technologies and tools, and possess strong problem-solving and analytical skills. The ideal candidate should have experience in Azure data services (preferably Synapse, Databricks, ADF, Azure Storage, and Purview) and a proven track record of delivering high-quality data solutions in a fast-paced environment.

• 4 years or more of experience in the field;
• Most of the work will be in SQL, so being skilled in SQL is a requirement;
• Some work will probably be in Python, so at least one of the profiles should be skilled enough to write custom pipelines if necessary;
• Good understanding of data modeling, normalization, fact vs dimension tables and so on;
• Experience with llms, open AI, Automl, DBT, Azure DataBricks, Databricks PySpark, Azure prompt flows and/or similar technologies is important;
• Works well in a setting where he or she needs to make sure the principles and requirements are met, but where speed is valued over extreme scalability;
• Actively use AI pair programmer, such as Github Copilot.

Nice to have:

• Familiar with data/dev ops principles (versioning, ci/cd, automated tests, etc);
• Familiar with the Microsoft data stack (know about the possibilities in data factory, purview, anomaly detection, key vault, etc);
• Advantage if they have experience with dbt (https://www.getdbt.com/), but it is mostly just SQL so can easily be learned if they know SQL and data modeling;
• We might want to do some work in Spark, so knowing that is an advantage, but not a requirement;
• Be a driving force for good data governance.

Join the project and become part of a dynamic and collaborative team that values innovation, integrity, and excellence. Together, we can make a significant impact on understanding the global economy and market trends, shaping our investment strategies, and driving the success of our portfolio companies.

#LI-TR1