Qureos Mentor

WellDoc, Inc.

<div><div>Job Purpose
 
We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. As a Data
 
Engineer, you will play a key role in the design, development, and optimization of our data
 
pipelines, supporting the ingestion, transformation, and analysis of large and complex datasets.
 
This role involves working with a variety of data sources, including Azure SQL Database, Google
 
Analytics, Google Play Store, Apple App Store, Salesforce, and other cloud-based services, to
 
ensure seamless data flow across our platforms. You will work extensively with Databricks,
 
leveraging its workflows, cluster management, and Unity Catalog to streamline and govern data
 
processes.
 
Responsibilities
 
Data Ingestion &amp; Integration:
 
<ul><li>Ingest data from a variety of sources such as Azure SQL DB, Google Analytics, Google
</li></ul>Play Store, Apple App Store, Salesforce, and others.
 
<ul><li>Develop and optimize ETL/ELT pipelines to transform data from CSV, JSON, SQL tables,
</li></ul>and APIs into usable formats.
 
<ul><li>Work with REST APIs to pull data from various external sources and integrate it into our
</li></ul>data ecosystem.
 
Data Transformation &amp; Modeling:
 
<ul><li>Design and implement efficient data transformation processes to cleanse, aggregate, and
</li></ul>enrich data.
 
<ul><li>Apply industry best practices for data modeling to ensure scalability, performance, and
</li></ul>data integrity.
 
<ul><li>Collaborate with data analysts and data scientists to provide clean, high-quality datasets
</li></ul>for reporting and analysis.
 
Databricks &amp; Cluster Management:
 
<ul><li>Utilize Databricks for data processing, transformation, and orchestration tasks.
</li></ul></div><ul><li>Manage and optimize Databricks clusters for performance, reliability, and cost-
</li></ul>effectiveness.
<ul><li>Implement Databricks workflows to automate and streamline data pipelines.
</li><li>Use Unity Catalog for data governance and metadata management, ensuring compliance
</li></ul>and data access control. 
 
<div>Required Skills &amp; Experience
 
Experience:
 
<ul><li>5+ years of hands-on experience in data engineering or a related field.
</li><li>Proven experience with Databricks and Databricks workflows, including cluster
</li></ul>management and data pipeline orchestration.
 
<ul><li>Strong experience in data ingestion from SQL databases (Azure SQL DB), APIs (Google
</li></ul>Analytics, Google Play Store, Apple App Store, Salesforce), and file-based sources (CSV,
 
JSON).
 
Technical Skills:
 
<ul><li>Proficiency in SQL for data manipulation and transformation.
</li><li>Experience with Python or Scala for writing and managing data workflows.
</li><li>Working knowledge of REST APIs for data integration.
</li><li>Experience in data transformation using Apache Spark, Delta Lake, or similar
</li></ul>technologies.
 
<ul><li>Knowledge of cloud platforms such as Azure, with a focus on Azure SQL DB.
</li><li>Familiarity with Unity Catalog for metadata management and governance.
</li></ul>Data Engineering Best Practices:
 
<ul><li>Understanding of data architecture, data pipelines, and the ETL/ELT process.
</li><li>Experience in data modeling, optimizing queries, and working with large datasets.
</li><li>Familiar with data governance, metadata management, and data access controls.
</li></ul>Preferred Skills (Optional):
 
<ul><li>Knowledge of Apache Kafka or other real-time streaming technologies.
</li><li>Experience with Data Lake or Data Warehouse technologies.
</li><li>Familiarity with additional data transformation tools such as Apache Airflow or dbt.
</li><li>Understanding of machine learning workflows and data pipelines.</li></ul></div></div>

Data Engineer

Join WellDoc, Inc. and explore 1   exciting career opportunities. They're hiring for various roles with options for remote or on-site work. Apply today and become part of their innovative team.

WellDoc, Inc.

ABOUT_COMPANY

Data Engineer