big data engineer

BCforward

<div>
 
 
 
 Presently we are unable to sponsor and request applicants to apply who are authorized to work without sponsorship*
 
 
 
 
 
 
 
 
 
 Employment TYPE: Contract W2 - NO C2C
 
 
 
 
 
 
 
 
 Job Title: Big Data Engineer
 
 
 
 
 Location:
 
 Wilmington, DE - Onsite
 
 
 
 
 
 
 About BCforward
 
 
 
 BCforward is a leading global IT consulting and workforce solutions firm providing services and support to Fortune 500 and government clients. Founded in 1998, BCforward has grown with our customers needs into a full-service business solutions provider. With delivery centers and offices across North America and India, we take pride in building long-term relationships and delivering excellence through innovation, collaboration, and integrity.
 
 
 
 
 
 
 Job Description
 
 
 
 We are seeking a
 
 Big Data Engineer
 
 to join our team. The ideal candidate will have strong experience in
 
 Java/J2EE development, Agile delivery, and CI/CD practices
 
 and a proven ability to
 
 design, build, test, deploy, and maintain secure, resilient applications
 
 .
 
 
 
 
 
 
 Responsibilities:
 
 
 
 - Design, develop and maintain scalable and large-scale data processing pipelines and infrastructure on the cloud following engineering standards, governance standards and technology best practices
 
 
 - Develop and optimize data models for large-scale datasets, ensuring efficient storage, retrieval, and analytics while maintaining data integrity and quality.
 
 
 - Collaborate with cross-functional teams to translate business requirements into scalable and effective data engineering solutions.
 
 
 - Demonstrate a passion for innovation and continuous improvement in data engineering, proactively identifying opportunities to enhance data infrastructure, data processing and analytics capabilities.
 
 
 
 Required Skills &amp; Qualifications:
 
 
 
 - Strong analytical problem solving and critical thinking skills
 
 
 - Proficiency in at least one programming language ( Python, if not Java or Scala)
 
 
 - Proficiency in at least one distributed data processing framework ( Spark or similar)
 
 
 - Proficiency in at least one cloud data Lakehouse platforms (AWS Data lake services or Databricks, alternatively Hadoop),
 
 
 - Proficiency in at least one scheduling/orchestration tools ( Airflow, if not AWS Step Functions or similar)
 
 
 -Proficiency with relational and NoSQL databases.
 
 
 - Proficiency in data structures, data serialization formats (JSON, AVRO, Protobuf, or similar), and big-data storage formats (Parquet, Iceberg, or similar),
 
 
 -Experience working in teams following Agile methodology
 
 
 - Experience with test-driven development (TDD) or behavior-driven development (BDD) practices, as well as working with continuous integration and continuous deployment (CI/CD) tools.
 
 
 
 Preferred Skills:
 
 
 <ul>
 <li>
 Experience with cloud platforms and related services.
 </li>
 <li>
 Exposure to test automation and performance engineering.
 </li>
 </ul>
 
 
</div>

Big Data Engineer

MASTER-WORKS

<div>Nationality: Any Arabic Nationality to achieve better communication and understand our Arabic data content better.
Data Engineer Roles and Responsibilities:
1. Data Collection and Integration: Data engineers collect data from various sources, including databases, APIs, external data providers, and streaming sources. They must design and implement efficient data pipelines to ensure a smooth flow of information into the data warehouse or storage system.
2. Data Storage and Management: Once the data is collected, data engineers are responsible for its storage and management. This involves choosing appropriate database systems, optimizing data schemas, and ensuring data quality and integrity. They also must consider scalability and performance to handle large volumes of data.
3. ETL (Extract, Transform, Load) Processes: ETL is a fundamental process in data engineering. Data engineers design ETL pipelines to transform raw data into a format suitable for analysis. This involves data cleansing, aggregation, and enrichment, ensuring the data is usable for data scientists and analysts.
4. Big Data Technologies: In today's data landscape, dealing with big data is the norm rather than the exception. Data engineers work with big data technologies such as Hadoop and Spark to efficiently process and analyze massive datasets.
5. NoSQL Databases: In addition to traditional relational databases, data engineers often work with NoSQL databases like MongoDB and Cassandra, which are well-suited for handling unstructured or semi-structured data.
6. Cloud Computing: Cloud platforms like AWS, Azure, and Google Cloud have become the backbone of modern data infrastructure. Data engineers leverage these platforms to build scalable and cost-effective data solutions.
7. Distributed Systems: Data engineering often involves distributed systems architecture to handle huge data volumes and ensure fault tolerance. Understanding how distributed systems work is essential for data engineers. 8. Streaming Data: Real-time data processing is crucial in many industries. Data engineers work with streaming technologies like Apache Kafka to handle and analyze data as it flows in 
 
 
Requirements 
Nationality: Any Arabic Nationality to achieve better communication and understand our Arabic data content better.
preferably 5+ years of experience.</div>

Data Engineer

Maaloomatiia

<div>
 
 Job Title
 
 
 
 
 Cloudera Data Engineer
 
 
 
 
 Job Scope
 
 
 
 We are seeking an experienced
 
 Cloudera Data Engineer
 
 with strong hands-on expertise in
 
 Cloudera CDP, Hadoop, Spark, Hive, and HDFS
 
 . The ideal candidate will be responsible for building and maintaining big data pipelines, optimizing data workflows, and supporting production environments to ensure reliability and performance.
 
 
 
 Main Duties And Responsibilities
 
 
 
 <ul>
 <li>
 Design, build, and maintain big data pipelines using Cloudera CDP components.
 </li>
 <li>
 Develop and optimize ETL/ELT workflows using Spark, Hive, and Hadoop ecosystems.
 </li>
 <li>
 Manage and maintain HDFS storage, ensuring high availability and performance.
 </li>
 <li>
 Perform production support activities, including issue resolution, monitoring, and performance tuning.
 </li>
 <li>
 Work with data scientists, analysts, and other engineering teams to support data-driven initiatives.
 </li>
 <li>
 Ensure data governance, security, and compliance across big data environments.
 </li>
 <li>
 Troubleshoot cluster-level issues related to resource utilization, scheduling, and configuration.
 </li>
 <li>
 Participate in solution design, architecture discussions, and technical documentation.
 
 
 
 </li>
 </ul>
 
 Position Requirements
 
 
 
 <ul>
 <li>
 Strong hands-on experience with Cloudera CDP.
 </li>
 <li>
 Proficiency in Hadoop ecosystem: HDFS, YARN, MapReduce, Hive, Impala.
 </li>
 <li>
 Strong experience working with Apache Spark (PySpark or Scala preferred).
 </li>
 <li>
 Solid understanding of distributed systems and big data architecture.
 </li>
 <li>
 Experience with production support and troubleshooting in big data environments.
 </li>
 <li>
 Strong SQL skills and experience optimizing large-scale queries.
 </li>
 <li>
 Familiarity with Linux, shell scripting, and automation tools.
 </li>
 <li>
 Experience with version control (Git) and CI/CD workflows is a plus.
 
 
 
 </li>
 </ul>
 
 Education
 
 
 
 Bachelor's degree in Computer Science or any other related field
 
 
 
 Experience
 
 
 
 <ul>
 <li>
 Experience with cloud platforms (AWS, Azure, GCP) for big data workloads.
 </li>
 <li>
 Exposure to Kafka, NiFi, Oozie, or similar data orchestration tools.
 </li>
 <li>
 Knowledge of Docker, Kubernetes, or containerized environments.
 </li>
 <li>
 Understanding of security, encryption, and data governance concepts.
 </li>
 </ul>
</div>

Cloudera Data Engineer

NorthBay Solutions

<div>
 &nbsp;
 
 
 
 
 
 
 
 Job Title:
 
 
 
 
 
 
 
 
 
 
 
 
 Data Engineer (Snowflake)
 
 
 
 
 
 
 
 
 
 
 
 
 
 Location:
 
 
 
 
 
 
 
 
 
 
 
 
 Hybrid - Lahore, Karachi, Islamabad
 
 
 
 
 
 
 
 
 
 
 
 
 
 Experience:
 
 
 
 
 
 
 
 
 
 
 
 
 6-8 years (Minimum 4-6 years of Snowflake experience)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 Job Category:
 
 
 
 
 
 
 
 
 
 
 
 
 Data Engineering
 
 
 
 
 
 
 
 
 
 
 
 
 
 Key Responsibilities:
 
 
 
 
 
 
 <ul>
 <li>
 
 
 
 
 
 
 Database Design &amp; Development:
 
 
 
 
 
 
 <ul>
 <li>
 
 
 
 
 
 
 Utilize Snowflake to design, develop, and optimize data architecture.
 
 
 
 
 
 
 </li>
 <li>
 
 
 
 
 
 
 Work with PostgreSQL and other databases to manage and manipulate data.
 
 
 
 
 
 
 </li>
 <li>
 
 
 
 
 
 
 Lead conversion projects from SQL-based platforms to Snowflake.
 
 
 
 
 
 
 </li>
 </ul>
 </li>
 <li>
 
 
 
 
 
 
 Programming &amp; Scripting:
 
 
 
 
 
 
 <ul>
 <li>
 
 
 
 
 
 
 Develop robust applications and scripts using C#, Java, and Python, or other relevant technologies.
 
 
 
 
 
 
 </li>
 <li>
 
 
 
 
 
 
 Write efficient Snowflake functions and stored procedures using JavaScript.
 
 
 
 
 
 
 </li>
 <li>
 
 
 
 
 
 
 Implement big data solutions using Spark and PySpark (Plus).
 
 
 
 
 
 
 </li>
 </ul>
 </li>
 <li>
 
 
 
 
 
 
 Cloud Integration:
 
 
 
 
 
 
 <ul>
 <li>
 
 
 
 
 
 
 Implement data storage solutions using Amazon S3 (AWS).
 
 
 
 
 
 
 </li>
 <li>
 
 
 
 
 
 
 Integrate and manage data pipelines using AWS Glue (Plus).
 
 
 
 
 
 
 </li>
 <li>
 
 
 
 
 
 
 Utilize AWS Lambda for serverless computing (Plus).
 
 
 
 
 
 
 </li>
 <li>
 
 
 
 
 
 
 Contribute to cloud data strategies involving GCP (Plus).
 
 
 
 
 
 
 </li>
 </ul>
 </li>
 <li>
 
 
 
 
 
 
 Additional Database Management:
 
 
 
 
 
 
 <ul>
 <li>
 
 
 
 
 
 
 Work with Aurora and MSSQL for various data management tasks (Plus).
 
 
 
 
 
 
 </li>
 </ul>
 </li>
 </ul>
 
 
 
 
 
 
 Required Skills:
 
 
 
 
 
 
 <ul>
 <li>
 
 
 
 
 
 
 Languages:
 
 
 
 
 
 
 
 
 
 
 
 
 C#, Java, Python, or other relevant technologies
 
 
 
 
 
 
 </li>
 <li>
 
 
 
 
 
 
 Database:
 
 
 
 
 
 
 
 
 
 
 
 
 Snowflake (Design, Develop), PostgreSQL
 
 
 
 
 
 
 </li>
 <li>
 
 
 
 
 
 
 Cloud Tools:
 
 
 
 
 
 
 
 
 
 
 
 
 Amazon S3 (AWS)
 
 
 
 
 
 
 </li>
 <li>
 
 
 
 
 
 
 Scripting:
 
 
 
 
 
 
 
 
 
 
 
 
 JavaScript (for Snowflake functions and stored procedures)
 
 
 
 
 
 
 </li>
 </ul>
 
 
 
 
 
 
 Preferred Skills:
 
 
 
 
 
 
 <ul>
 <li>
 
 
 
 
 
 
 Big Data Technologies:
 
 
 
 
 
 
 
 
 
 
 
 
 Spark, PySpark
 
 
 
 
 
 
 </li>
 <li>
 
 
 
 
 
 
 Cloud Tools:
 
 
 
 
 
 
 
 
 
 
 
 
 AWS Glue, AWS Lambda, GCP
 
 
 
 
 
 
 </li>
 <li>
 
 
 
 
 
 
 Database:
 
 
 
 
 
 
 
 
 
 
 
 
 Aurora, MSSQL
 
 
 
 
 
 
 </li>
 </ul>
 
 
 
 
 
 
 Conversion:
 
 
 
 
 
 
 
 
 
 
 
 
 SQL to Snowflake code conversion
 
 
 
 
 
 
</div>

Data Engineer(Snow Flake)

Ovise

<div>
 
 
 Data Engineer &ndash; Real-Time Data &amp; Cloud (NYC Area/Fairfield County, CT)
 
 
 
 
 
 
 Ovise is partnered with a global, technology-led FinTech organisation looking for a
 
 Data Engineer
 
 to help build and scale a real-time data platform powering analytics and reporting.
 
 
 
 
 
 You&rsquo;ll design and develop
 
 scalable ETL pipelines
 
 , process high-volume streaming data, and build automated data solutions using modern cloud technologies.
 
 
 
 
 
 
 Hedge Fund or Big Tech candidates are highly desirable here.
 
 
 
 
 
 
 
 Key Responsibilities
 
 
 <ul>
 <li>
 Build and optimise
 
 ETL pipelines and data workflows
 
 </li>
 <li>
 Process streaming and large-scale datasets
 </li>
 <li>
 Develop data transformations using
 
 Python &amp; PySpark
 
 </li>
 <li>
 Design scalable
 
 data models and cloud-based data solutions
 
 </li>
 <li>
 Implement
 
 automation, monitoring, and data quality checks
 
 </li>
 </ul>
 
 
 
 
 
 Requirements
 
 
 <ul>
 <li>
 3+ years&rsquo; experience in
 
 Python-based data engineering
 
 </li>
 <li>
 Strong experience with
 
 ETL, PySpark, Pandas or similar frameworks
 
 </li>
 <li>
 Experience with
 
 AWS data services (S3, Glue, Athena, EMR)
 
 </li>
 <li>
 Strong
 
 SQL and data modelling
 
 knowledge
 </li>
 <li>
 Experience working in
 
 Linux environments
 
 </li>
 </ul>
 
 
 
 
 
 Bonus:
 
 experience with real-time data, BI tools, or financial data environments.
 
</div>

The Bunton Real Estate Co.

Job Overview We are seeking a dynamic and innovative Data Engineer to join our team and drive the development of robust data solutions that empower data-driven decision-making. In this role, you will design, build, and maintain scalable data pipelines and architectures, leveraging cutting-edge technologies to handle large volumes of data efficiently. Your expertise will enable seamless integration of diverse data sources, optimize data storage, and facilitate advanced analytics to support strategic initiatives across the organization. If you thrive in a fast-paced environment and are passionate about transforming complex data into actionable insights, this opportunity is for you!Duties<ul><li>Develop and implement scalable ETL (Extract, Transform, Load) processes using tools like Informatica, Talend, and custom scripting to ensure high-quality data flow across systems </li><li>Design and maintain data warehouses and lakes utilizing platforms such as Azure Data Lake, Hadoop, and Apache Hive to support analytics and reporting needs </li><li>Build and optimize data pipelines using Spark, Python, Java, and Bash scripting for efficient processing of big data workloads </li><li>Collaborate with cross-functional teams to integrate linked data sources and develop RESTful APIs for seamless data access </li><li>Perform database design and management with Microsoft SQL Server, Oracle, and other relational databases to ensure data integrity and performance </li><li>Support model training activities by preparing datasets, implementing feature engineering techniques, and ensuring data quality for machine learning projects </li><li>Conduct analysis skills assessments to identify opportunities for process improvements and automation within the data ecosystem </li><li>Participate in Agile development cycles to deliver iterative enhancements aligned with business priorities</li></ul>Requirements<ul><li>Proven experience designing and managing large-scale big data solutions using Hadoop ecosystem components such as Apache Hive and Spark </li><li>Strong proficiency in SQL programming with Microsoft SQL Server, Oracle, or similar relational databases </li><li>Hands-on experience with cloud platforms like AWS (Amazon Web Services) and Azure Data Lake for scalable storage solutions </li><li>Expertise in ETL tools including Informatica, Talend, or equivalent frameworks; familiarity with Shell Scripting (Bash) is a plus </li><li>Proficiency in programming languages such as Python and Java for developing robust data processing applications </li><li>Knowledge of linked data concepts and experience integrating diverse datasets from multiple sources </li><li>Familiarity with analytics tools like Looker for creating insightful dashboards and reports </li><li>Understanding of database design principles to develop efficient data warehouses supporting enterprise analytics </li><li>Experience working within Agile methodologies to deliver continuous value through iterative development cycles </li><li>Strong analysis skills to interpret complex datasets and translate findings into actionable insights</li></ul>Join us as a Data Engineer to be at the forefront of transforming raw data into strategic assets! Your expertise will help shape innovative solutions that drive business growth while working in a collaborative environment that values your technical prowess. Embark on a journey where your skills make a real impact&mdash;apply today!Pay: $120,629.77 - $145,274.56 per yearBenefits:<ul><li>401(k)</li><li>Dental insurance</li><li>Health insurance</li><li>Paid time off</li><li>Vision insurance</li></ul>Work Location: Remote

Search 10000+ Big data engineer jobs near you this April. New roles added regularly across leading industries. Apply today.

10000 big data engineer jobs Near Me

Big Data Engineer

Data Engineer

Cloudera Data Engineer

Data Engineer(Snow Flake)

Data Engineer(Snow Flake)

Data Engineer

Data Engineer