Lead Data Engineer

Lead Data Engineer

Experience Range: 10 - 18 years

Location: Bengaluru, India

Primary Skills Required: Java/Python, Scala, Spark, Kafka, Airflow, SQL, Databricks, Snowflake, Hive, AWS

Required Qualifications:

• Minimum of 8-10 years of relevant data science, Business Intelligence, or data analytics experience preferably with a digital media or digital advertising company
• Experience delivering products in an agile software development environment including sprint work in Jira
• Excellent interpersonal skills with the ability to collaborate and partner across multiple teams and organizations
• Strong SQL experience including writing/optimizing complex queries against distributed data stores such as Aurora, Hadoop, Hive, Oracle, and Snowflake
• Experience with data pipeline and workflow management tools: Apache NiFi or Apache Airflow
• Expert level knowledge in at least one programming language such as Python, R, Scala, Java.
• Demonstrated experience working with business intelligence tools such as MicroStrategy, Tableau, Business Objects, Looker, etc.

Category Technologies

Languages

Scala, Python, SQL

Data Processing

Apache Spark, Flink, Airflow

Platforms & Tools

Databricks, Hive, Presto, Snowflake

Cloud

AWS (S3, EMR, Glue, Lambda)

• Demonstrated ability to effectively communicate complex concepts to technical and unacquainted audiences
• Experience in working with large datasets (Terabytes or more)
• Strong presentation and documentation skills including experience working with Confluence, Power Point, and Lucid Chart
• Strong analytical and critical thinking skills
• Enjoy working collaboratively with a talented group of people to tackle challenging business problems so we all succeed (or fail fast) as a team.

Preferred Qualifications

• Experience with digital advertising systems and ecosystems
• Experience using data analytics to triage and troubleshoot advertising workflows
• Experience with network topologies, systems architecture overviews, and entity relationship diagrams (ERD)