data warehouse engineer

Pavago

Job Title: Data Engineer
 Position Type: Full-Time, Remote
 Working Hours: U.S. client business hours (with flexibility for pipeline monitoring and data refresh cycles)
 
About the Role: 
Our client is seeking a Data Engineer to design, build, and maintain reliable data pipelines and infrastructure that deliver clean, accessible, and actionable data. This role requires strong software engineering fundamentals, experience with modern data stacks, and an eye for quality and scalability. The Data Engineer ensures data flows seamlessly from source systems to warehouses and BI tools, powering decision-making across the business.
 
Responsibilities:Pipeline Development: 
<ul>
<li>Build and maintain ETL/ELT pipelines using Python, SQL, or Scala.</li>
<li>Orchestrate workflows with Airflow, Prefect, Dagster, or Luigi.</li>
<li>Ingest structured and unstructured data from APIs, SaaS platforms, relational databases, and streaming sources.</li>
</ul>
Data Warehousing: 
<ul>
<li>Manage data warehouses (Snowflake, BigQuery, Redshift).</li>
<li>Design schemas (star/snowflake) optimized for analytics.</li>
<li>Implement partitioning, clustering, and query performance tuning.</li>
</ul>
Data Quality & Governance: 
<ul>
<li>Implement validation checks, anomaly detection, and logging for data integrity.</li>
<li>Enforce naming conventions, lineage tracking, and documentation (dbt, Great Expectations).</li>
<li>Maintain compliance with GDPR, HIPAA, or industry-specific regulations.</li>
</ul>
Streaming & Real-Time Data: 
<ul>
<li>Develop and monitor streaming pipelines with Kafka, Kinesis, or Pub/Sub.</li>
<li>Ensure low-latency ingestion for time-sensitive use cases.</li>
</ul>
Collaboration: 
<ul>
<li>Partner with analysts and data scientists to provide curated, reliable datasets.</li>
<li>Support BI teams in building dashboards (Tableau, Looker, Power BI).</li>
<li>Document data models and pipelines for knowledge transfer.</li>
</ul>
Infrastructure & DevOps: 
<ul>
<li>Containerize data services with Docker and orchestrate in Kubernetes.</li>
<li>Automate deployments via CI/CD pipelines (GitHub Actions, Jenkins, GitLab CI).</li>
<li>Manage cloud infrastructure using Terraform or CloudFormation.</li>
</ul>
What Makes You a Perfect Fit: 
<ul>
<li>Passion for clean, reliable, and scalable data.</li>
<li>Strong problem-solving skills with debugging mindset.</li>
<li>Balance of software engineering rigor and data intuition.</li>
<li>Collaborative communicator who thrives in cross-functional environments.</li>
</ul>
Required Experience & Skills (Minimum): 
<ul>
<li>3+ years in data engineering or back-end development.</li>
<li>Strong Python and SQL skills.</li>
<li>Experience with at least one major data warehouse (Snowflake, Redshift, BigQuery).</li>
<li>Familiarity with pipeline orchestration tools (Airflow, Prefect).</li>
</ul>
Ideal Experience & Skills: 
<ul>
<li>Experience with dbt for transformations and data modeling.</li>
<li>Streaming data experience (Kafka, Kinesis, Pub/Sub).</li>
<li>Cloud-native data platforms (AWS Glue, GCP Dataflow, Azure Data Factory).</li>
<li>Background in regulated industries (healthcare, finance) with strict compliance.</li>
</ul>
What Does a Typical Day Look Like?
 A Data Engineer’s day revolves around keeping pipelines running, improving reliability, and enabling teams with high-quality data. You will:
 <ul>
<li>Check pipeline health in Airflow/Prefect and resolve any failed jobs.</li>
<li>Ingest new data sources, writing connectors for APIs or SaaS platforms.</li>
<li>Optimize SQL queries and warehouse performance to reduce costs and latency.</li>
<li>Collaborate with analysts/data scientists to deliver clean datasets for dashboards and models.</li>
<li>Implement validation checks to prevent downstream reporting issues.</li>
<li>Document and monitor pipelines so they’re reproducible, scalable, and audit-ready.</li>
</ul>
In essence: you ensure the business has accurate, timely, and trustworthy data powering every decision.
 
Key Metrics for Success (KPIs): 
<ul>
<li>Pipeline uptime ≥ 99%.</li>
<li>Data freshness within agreed SLAs (hourly, daily, weekly).</li>
<li>Zero critical data quality errors reaching BI/analytics.</li>
<li>Cost-optimized queries and warehouse performance.</li>
<li>Positive feedback from data consumers (analysts, scientists, leadership).</li>
</ul>
Interview Process: 
<ul>
<li>Initial Phone Screen</li>
<li>Video Interview with Pavago Recruiter</li>
<li>Technical Task (e.g., build a small ETL pipeline or optimize a SQL query)</li>
<li>Client Interview with Engineering/Data Team</li>
<li>Offer & Background Verification</li>
</ul>

Data Engineer

Algo

<div>Position: Data Engineer (Azure Platform)

Location: Lahore, Pakistan (on-site)<div> 
Job Description 
<ul><li>Design and develop scalable data warehouse solutions to support analytics and reporting.</li><li>Build and maintain efficient Azure Data Pipelines using Azure Data Factory for ETL processes.</li><li>Develop and optimize Spark jobs using Databricks and PySpark for large-scale data transformations.</li><li>Coordinate with onsite and offshore teams to ensure seamless project execution and business continuity.</li><li>Develop and maintain dashboards and reports using BI tools such as Power BI, Tableau, MicroStrategy, or Qlik.</li><li>Manage the presentation and access layers to ensure secure and efficient data access.</li><li>Work directly with clients and stakeholders to understand requirements and deliver actionable data solutions</li></ul></div> 
Qualifications
<ul><li>BS in Computer Science or a related field with 4-5 years of experience in BI and data management projects.</li><li>Strong expertise in database and data warehouse design, support, and maintenance.</li><li>Proficiency in writing complex SQL and SparkSQL queries.</li><li>Experience in Python and its data-related frameworks.</li><li>Hands-on experience with Databricks and PySpark for data engineering tasks.</li><li>Strong proficiency in Power BI; experience with other BI tools is a plus.</li><li>Experience with Azure Data Platform (Data Factory, Azure SQL, Synapse, Data Lake) is preferred.</li><li>Ability to generate and maintain comprehensive technical documentation.</li><li>Excellent communication and interpersonal skills.</li><li>Strong analytical and problem-solving abilities.</li><li>Willingness to learn and adapt to new technologies and project requirements.</li><li>Experience in data analysis is a plus.</li><li>Ability to work independently and handle critical situations confidently.</li><li>A collaborative, team-oriented mindset.</li></ul></div>

Data Engineer (Azure Platform)

Higher Logic

<div>About Higher Logic:

Higher Logic exists to connect people, knowledge, and ideas to improve lives and organizations. Through our community solutions, we empower growth-stage businesses and associations to build a more engaged and meaningful relationship with their customers and members. Higher Logic serves more than 3,000 customers, representing over 350,000 online communities with greater than 200 million users in more than 42 countries worldwide.

Our team is a thriving community of authentic people with diverse experiences and perspectives. We are honest and genuine and believe that sharing ideas and feedback openly helps us to be successful. We believe that diversity and empathy bring us #AllTogether to make us stronger. We learn from failure and successes because both help us to grow and learn. We strive to understand the why behind our actions and bring our best selves to work every day.

JOB SUMMARY

The Data Solutions Engineer supports Higher Logic's strategic data initiatives by designing, developing, and scaling data pipelines and models that enable analytics, business intelligence, and AI workflows, using both structured and unstructured data. This role is critical in building our data warehouse and enabling AI capabilities across the organization and reports directly to the Chief Data Officer. The engineer works closely with internal stakeholders across departments to ensure data is reliable, accessible, and actionable for both operational and strategic use.

Help design and build the next generation of our data platform. You'll play a key role in architecting scalable, performant, and governed data systems that power analytics, marketing automation, and customer insights across thousands of clients. 
Our environment spans AWS, Databricks, and Power BI, with data flowing continuously from multiple operational systems (AMS, CRM, LMS, events, behavioral streams) into a modern lakehouse architecture. If you thrive at the intersection of data engineering, architecture, and governance &mdash; this is the role for you. 


ESSENTIAL DUTIES AND RESPONSIBILITES:

<ul><li>Architect and build new data pipelines and lakehouse patterns to handle ~1 TB/day of streaming and batch data.</li>
<li>Establish data governance and lineage foundations, designing systems that make data traceable from source through warehouse and back into our platforms.</li>
<li>Build and optimize pipelines that power both analytics (Power BI) and operational workloads (e.g., marketing automation).</li>
<li>Design performant warehouse schemas and ETL/ELT processes for analytical and reverse ETL use cases.</li>
<li>Lead performance and cost optimization initiatives across Glue, Databricks, and S3.</li>
<li>Implement data quality, documentation, and testing frameworks using dbt.</li>
<li>Develop infrastructure as code using Terraform and automate deployments through GitHub Actions.</li>
<li>Partner closely with analysts, platform teams, and business stakeholders to enable data visibility, traceability, and self-service analytics.</li>
<li>Act as a senior member of the data team, setting best practices, mentoring others, and helping shape the data platform's future.</li>
</ul>
KNOWLEDGE AND ABILITIES REQUIRED:

<ul><li>5+ years of experience in data engineering, including architecting scalable data platforms in AWS.</li>
<li>Deep hands-on experience with S3, Glue, Databricks, Postgres/MSSQL, and streaming pipelines.</li>
<li>Strong background in data modeling, warehouse performance tuning, and ELT patterns.</li>
<li>Demonstrated experience implementing data governance, quality frameworks, and ideally designing lineage systems.</li>
<li>Proficiency with Terraform and CI/CD (GitHub Actions) for data infrastructure.</li>
<li>Excellent communication skills &mdash; able to translate business needs into data architecture.</li>
<li>Comfortable operating as a senior team member in a growing data org, influencing architecture and mentoring others.</li>
</ul>
Nice to haves:

<ul><li>Experience with tools like OpenMetadata, DataHub, or Collibra for lineage and cataloging.</li>
<li>Exposure to reverse ETL patterns and marketing automation platforms.</li>
<li>Certifications in AWS Data, Databricks, or related fields.</li>
<li>Experience in multi-tenant or high-scale SaaS environments.</li></ul> 


-

You'll earn:
<ul>
<li>Competitive compensation.</li>
<li>Comprehensive health benefits package.</li>
<li>401(k) plan with employer match.</li>
<li>Healthcare and dependent-care flexible spending account.</li>
<li>Company short-term and long-term disability insurance.</li>
<li>Company culture that recognizes its employees.</li>
<li>Room for growth and development and management that cares about your professional growth and will help you achieve your goals.</li>
<li>Significant advancement opportunities for outstanding performers.</li></ul></div>

Data Solutions Engineer

marketrove 

Job Description:* We're seeking an experienced SAP BW / BI professional to join a remote project focused on migrating SAP BW logic to Google Cloud BigQuery. This is a unique opportunity to work on data migration, ETL transformation, and cloud data warehouse modernization. Key Responsibilities:- Analyze and reverse engineer existing SAP BW ETL workflows, InfoCubes, DSOs, and transformations, documenting data flows, business rules, and transformations. - Design and implement equivalent ETL/data pipelines in GCP BigQuery, ensuring seamless data integration and workflow migration. - Collaborate with business stakeholders to validate migrated data and reports, ensuring accuracy and consistency. - Optimize BigQuery queries, tables, and views for performance, scalability, and cost-effectiveness. - Support testing, UAT, and deployment of migrated data workflows, ensuring smooth project delivery. - Provide guidance on best practices for cloud-based data warehousing, leveraging industry expertise. Requirements: Strong SAP BW / BI experience (ETL, Data Modeling, InfoCubes, DSO, Queries) with a deep understanding of data migration and ETL processes. Hands-on experience with Google Cloud Platform (BigQuery, Dataflow, Dataform, Pub/Sub) is a plus, but not mandatory. Solid grasp of data warehouse architecture and data modeling principles. Proven experience in reverse engineering and documenting existing BW logic. Excellent problem-solving, analytical, and communication skills, with the ability to work independently in a remote environment. Preferred Qualifications:- Experience with cloud-based data warehouse migration projects, preferably SAP BW to BigQuery. Strong SQL skills and experience with scripting languages like Python for ETL automation. Familiarity with Agile methodologies and collaborative tools like Jira, Confluence, or Git.Job Type: Full-timeWork Location: Remote

SAP BW to GCP BigQuery Migration Consultant / Data Engineer (Remote Contract)

Secure Tech Cards Pvt. Ltd

Secure Tech Consultancy Position: Senior Data Engineer (For Our Client)We&rsquo;re hiring an experienced Senior Data Engineer to design, build, and maintain advanced data systems for our client in the Fintech domain. The ideal candidate will have strong expertise in ETL development (Python, Talend), data warehousing, and SQL optimization across multiple databases.Key Responsibilities:<ul><li>Design and maintain complex ETL/ELT pipelines and data architectures.</li><li>Build scalable data solutions for analytics and business intelligence.</li><li>Optimize SQL queries and data warehouse models (Star, Snowflake).</li><li>Manage and integrate data across Teradata, Oracle, Postgres, MySQL, MongoDB, Gauss DB, and IBM DB.</li><li>Automate processes using Linux bash scripting.</li><li>Collaborate with cross-functional teams to deliver data-driven insights.</li></ul>Requirements:<ul><li>4+ years in Data Engineering or BI.</li><li>Strong command of Python, Talend, SQL, and data modeling.</li><li>Experience in Linux scripting and familiarity with Microstrategy.</li><li>Bachelor&rsquo;s or Master&rsquo;s in Computer Science, Engineering, or related field.</li></ul>If you&rsquo;re passionate about building high-performing data systems, apply now to join Secure Tech Consultancy&rsquo;s expert team.Job Type: Full-timePay: From Rs180,000.00 per monthWork Location: In person

Senior Data Engineer

sonnen Egypt

<div>
 
 Standort
 
 
 
 Cairo, EG
 
 
 
 Stellen-ID
 
 : 4758
 
 
 At the earliest date possible we are looking for support for our #teamsonnen
 
 
 Clean and affordable energy for all is one of the greatest challenges of our time. Sonnen has been committed to this mission since 2010 and is now represented in Europe, the USA and Australia with over 1,000 team members globally. In Germany, our offices are located in the beautiful Wildpoldsried (Allg&auml;u) and the vibrating heart of Berlin Kreuzberg.
 
 
 At sonnen, we are building the energy system of the future. To this end, we are developing new technologies and business models that have not existed before. Whether intelligent power storage, virtual power plants or digital services - we set the trends of a new energy world. That means, among other things, the inexpensive production and storage of renewable energy and the intelligent networking of producers and consumers. sonnen thereby gives people around the world the opportunity to become independent of fossil fuels and conventional energy supplier.
 
 
 We are growing steadily and are looking for people who want to join us in our vision of clean and affordable energy for all while making a positive impact on climate change.
 
 
 We are seeking a Data Engineer (m/f/d) to join our team and play a pivotal role in enabling our Business Intelligence and Data Science teams. As a Data Engineer, you will be responsible for maintaining and enhancing our AWS-based analytics platform, provisioning data from various sources, and enabling the development and stable operation of analytical solutions. We&rsquo;re looking for someone who shares our values, who isn&rsquo;t afraid to challenge existing processes, and who would like to work on innovative solutions in a fast-moving environment. If this sounds like you, then don&rsquo;t hesitate to get in touch.
 
 
 
 What you will do to help drive the energy transition:
 
 
 
 <ul>
 <li>
 (Senior) Define and design the architecture of our analytics platform, ensuring scalability, performance, and reliability.
 </li>
 <li>
 You design and build scalable data pipelines, ETL processes, and data lake and data warehousing solutions to support our analytics solutions.
 </li>
 <li>
 You develop and maintain data integration processes with a variety of structured and unstructured data sources.
 </li>
 <li>
 You implement, test, and maintain infrastructural components of sonnen&rsquo;s analytics platform and deploy and maintain data models.
 </li>
 <li>
 You implement data quality checks and monitoring mechanisms to ensure accuracy, completeness, and consistency of data.
 </li>
 <li>
 You define coding principles and support data scientists and analysts in implementing algorithms into operational solutions.
 </li>
 <li>
 Adhere to stringent quality assurance and documentation standards, utilizing version control and code repositories such as Gitlab.
 </li>
 <li>
 You stay up to date with emerging trends and technologies in data engineering and assess their potential applications to our business needs.
 </li>
 <li>
 (Senior) Share your knowledge by coaching/pair coding with more junior engineers.
 
 
 </li>
 </ul>
 
 What You Will Need On That Journey
 
 
 
 <ul>
 <li>
 Bachelor&rsquo;s degree in computer science or a related technical field.
 </li>
 <li>
 +3 years of relevant employment experience in roles such as Data Scientist, Data Engineer, or Data Architect.
 </li>
 <li>
 +3 years of experience with ETL tools, Data Modeling, and Data Architecture.
 </li>
 <li>
 Experience with Big Data technologies such as Hadoop, Hive, and Spark.
 </li>
 <li>
 Ability to translate organizational goals into technical solutions.
 </li>
 <li>
 Experience with Infrastructure as Code (experience in Terraform and Ansible are a plus).
 </li>
 <li>
 Proficiency in Python, SQL, and NoSQL.
 </li>
 <li>
 Excellent communication and project management skills.
 
 
 </li>
 </ul>
 
 Stellensegment:
 
 Data Warehouse, Business Intelligence, Computer Science, Quality Assurance, Project Manager, Technology
</div>

Data Engineer (m/f/d)

Systems Limited

<div>We are seeking a Senior Data Engineer with strong expertise in Informatica and SQL to design, develop, and optimize data pipelines and ETL workflows. The ideal candidate will have hands-on experience in data integration, data warehousing, and performance tuning across enterprise environments.
<h3 class="jobSectionHeader">
Responsibilities:</h3>
<ul>
<li>Design, build, and maintain ETL workflows using Informatica PowerCenter (or Informatica Cloud).
</li>
<li>Develop and optimize SQL queries, stored procedures, and views for data extraction and transformation.
</li>
<li>Work closely with data modelers, analysts, and business teams to understand data requirements.
</li>
<li>Ensure data quality, integrity, and consistency across multiple systems.
</li>
<li>Perform root cause analysis on data and ETL issues, ensuring timely resolution.
</li>
<li>Support data warehouse development and manage data integration across different sources.
</li>
<li>Monitor and enhance ETL performance, implementing best practices for scalability and reliability.
</li>
<li>Collaborate with infrastructure and DevOps teams for deployment and job scheduling (e.g., Autosys, Control-M).
</li>
</ul><h3 class="jobSectionHeader">Requirements</h3>
<ul>
<li>8&ndash;10 years of experience in Data Engineering or related roles.
</li>
<li>Strong hands-on experience with Informatica PowerCenter / IICS.
</li>
<li>Advanced proficiency in SQL (Oracle, SQL Server, or similar).
</li>
<li>Solid understanding of data warehousing concepts (Star/Snowflake schema, fact/dimension modeling).
</li>
<li>Experience with performance tuning of ETL processes and database queries.
</li>
<li>Familiarity with data governance, data quality, and metadata management practices.
</li>
<li>Experience working in cloud-based or hybrid data environments (Azure, AWS, or GCP) is a plus.
</li>
<li>Strong problem-solving skills and ability to work in cross-functional teams.</li></ul></div><div></div>

Principal Data Engineer

Search 10000+ Data warehouse engineer jobs near you this October. New roles added regularly across leading industries. Apply today.