Data Engineer

Location: On-site in Abu Dhabi

We’re looking for an experienced, hands-on Data Engineer capable of designing, building, and operating data pipelines and models that power analytics and applications across investment teams and Middle/Back Office of one of our clients.

The ideal candidate has familiarity with financial markets (securities, prices, corporate actions, positions/holdings) and thrives in ambiguous environments - proactively shaping solutions rather than waiting for tickets. You will own data end-to-end: from ingesting vendor and internal sources, to modeling in the lakehouse, to ensuring data is discoverable, reliable, and cost-efficient.

You will collaborate closely with BAs/PMs and quants, anticipate downstream needs, and propose pragmatic architectures that balance speed, governance, and scalability.

Responsibilities

• Participate in requirements clarification and sprint planning sessions
• Design technical solutions and implement them
• Build robust ETL pipelines using PySpark for extraction and transformation
• Optimize and tune existing ETL processes for performance, scalability, and reliability
• Write unit and integration tests
• Support QA teammates during acceptance testing
• Resolve production incidents as a 3rd line engineer

Required Skills

• Minimum 5 years of experience in IT/Data
• Proficiency in PySpark for distributed computing
• Strong Python skills for ETL development
• Strong expertise in writing and optimizing complex SQL queries
• Experience with PostgreSQL, MySQL, Oracle, or Snowflake preferred
• Experience with data warehousing concepts and platforms (ideally Databricks)
• Familiarity with ETL tools and processes
• Experience with Data Modeling
• Proficiency with Git
• Experience with Prometheus, Grafana, or custom monitoring scripts
• Experience implementing data validation, cleansing, and quality frameworks (ideally Monte Carlo)
• Familiarity with Azure DevOps, Jira, and/or Confluence
• Strong debugging and troubleshooting skills for complex data issues
• Strong written and verbal communication skills
• Ability to explain technical concepts to non-technical stakeholders
• Understanding of the Investment Data domain (desired)