big data engineer

Hexaware Technologies

Role Big Data Engineer Responsibilities Implement Data integration and Data Warehouse based solutions using Big Data Technologies. Should be highly proficient in the use of Big Data / Open Source Technologies and standard techniques of Data Integration; Data Manipulation with hands-on contribution. Should be able to develop cost efficient and performant data pipelines in the cloud platform In-depth understanding of modern big data technology; including Data modeling and machine learning skills Knowledge of real time data streaming and aggregation architectural patterns and practice Essential Skills: 3+ Years hands on knowledge on SQL as well as SQL/NoSQL databases Proficient in programming languages such as Python and PySpark/Scala/Java Experience with Apache Spark in On-prem / Cloud (Databricks / EMR / DataProc HDInsight) Working knowledge of API development Experience in building data pipelines in ETL/ELT methods Experience with integration of data from multiple data sources Experience in one of the NoSQL databases; such as HBase; Cassandra; MongoDB

Big Data Engineer

EXL Service

<div><p>Job Description: Independently and collaboratively design, develop, and maintain high-performance, cloud-native data solutions and ETL pipelines in complex enterprise environments. Analyze system specifications, business requirements, and data models to build scalable and secure platforms for analytics, reporting, and real-time processing.</p></div>
<br><div><ul><li>Responsibilities: Architect and implement modern data lake and data warehouse solutions using AWS S3, Glue, and Athena.
</li><li>Develop event-driven and batch data pipelines with Kafka, AWS Lambda, and PySpark.
</li><li>Manage CI/CD workflows using Jenkins, Git, and Terraform.
</li><li>Automate data ingestion, transformation, and provisioning processes, ensuring data quality, lineage, and governance across environments.
</li><li>Collaborate with cross-functional stakeholders to deliver secure, high-quality data products for insurance and BFSI business domains.
</li><li>Contribute to enterprise-level projects, driving innovation and continuous improvement in data platform capabilities.
</li><li>Position may work at various and unanticipated worksites throughout the United States. Telecommuting permitted.</li></ul></div>
<br><div><p>Qualifications: Requires Master&rsquo;s degree in information technology, Computer Science, Software Engineering, or related IT field plus Six (6) years of Related professional experience. Experience must include: Six (6) years of experience in the following: Designing and developing data pipelines using Spark (PySpark/Scala), Hadoop, Hive, and Kafka; Scripting and programming with Python, Shell Scripting, and Java. Five (5) years of experience with the following: Data warehouse and data lake architectures, including data modeling, ETL, and data governance; DevOps tools, including Terraform, Jenkins, and CI/CD pipelines; SQL and NoSQL databases, including PostgreSQL, Oracle, SQL Server, MongoDB, and DynamoDB. Four (4) years of experience in monitoring and observability using Splunk, CloudWatch, or similar tools. Three (3) years of experience in the following: Data visualization and reporting tools such as Power BI and Microsoft Fabric. Two (2) years of experience with Agile methodologies, Scrum processes, and collaboration in cross-functional teams. One(1) year of experience in the following: AWS Cloud technologies including Glue, Lambda, Step Functions, Athena, CloudFormation, S3, RDS, and DynamoDB.
</p><p>40 hours/week, 9:00am-5:00pm, Salary range: $190,000 to $195,000 per year.
</p><p>To apply: Send resume and cover letter to us.careers@exlservice.com. Must cite job title and code EXL69 in response. This notice is subject to ExlService.com, LLC's employee referral program. EEO/Minorities/Females/Vets/Disabilities.</p></div>

Senior Big Data Engineer

<div><p><b>Title:</b></p>
Big Data Systems Engineer<p></p><p><b>
Belong, Connect, Grow, with KBR!</b></p><p>
KBR&rsquo;s National Security Solutions (NSS) team provides high-end engineering and advanced technology solutions to our customers in the intelligence and national security communities. In this position, your work will have a profound impact on the country&rsquo;s most critical role &ndash; protecting our national security.</p><p></p><p>
KBR is seeking a Big Data Systems Engineer to join our team. The successful candidate will be part of the KBR team supporting the Test Resource Management Center&rsquo;s (TRMC) Big Data (BD) and Knowledge Management (KM) Team deploying BD and KM systems for DoD testing Ranges and various acquisition programs.</p><p></p><p><b>
Responsibilities:
</b></p><ul><li>The Big Data Systems Engineer will work on the deployment and integration of a highly visible data analytic project called Cloud Hybrid Edge-to-Enterprise Evaluation Test &amp; Analysis Suite (CHEETAS) at multiple DoD ranges and labs.</li><li>
As a Big Data Systems Engineer, you will be a critical part of our technical team responsible for deploying CHEETAS within customer environments. You will be the frontline interface that customers will have when first experiencing CHEETAS within their DoD Range and lab environments.</li><li>
This position will require you to work closely with system administrators and software developers to communicate, document and ultimately resolve deployment issues as they arise.</li><li>
You will deploy CHEETAS within disparate DoD testing Ranges and acquisition programs environments (on different non-standard hardware stacks and integrated into different existing ecosystems) sometimes located within DoD vaults with no outside internet connectivity.</li><li>
Work on the deployment and integration of a highly visible data analytic project called Cloud Hybrid Edge-to-Enterprise Evaluation Test &amp; Analysis Suite (CHEETAS) at multiple DoD ranges and labs</li><li>
Deploy CHEETAS within customer environments</li><li>
Work closely with system administrators and software developers to communicate, document and ultimately resolve deployment issues as they arise</li><li>
Deploy CHEETAS within disparate DoD testing Ranges and acquisition programs environments (on different non-standard hardware stacks and integrated into different existing ecosystems) sometimes located within DoD vaults with no outside internet connectivity</li></ul><p></p><p><b>
Work Environment:
</b></p><ul><li><b>Location:</b> Remote - The candidate can either work in one of KBR&rsquo;s facilities or work from home, with a stable internet connection.</li><li><b>
Travel Requirements:</b> This position is anticipated to require travel of 25% with surges possible up to 50% to support end users located at various DoD Ranges and Labs across the US (including Alaska and Hawaii).</li><li><b>
Working Hours:</b> Standard</li></ul><p></p><p><b>
Basic Qualifications/Knowledge:</b></p><ul><li>
Must have an active TS/SCI Security Clearance to be considered for this position.</li><li>
This position requires a bachelor's degree in a STEM Computer Science, Data Science, Statistics or related, technical field, and 10 years of DoD experience. <b>Entry level Big Data Engineers will NOT be considered due to the breadth of knowledge necessary to be successful in the position.</b></li><li>
Previous experience must include three (3) years of hands-on experience in the integration with and configuration of: SQL Server Big Data Cluster, CentOS, Ubuntu, RedHat, Windows Server, VMWare, etc.)</li><li>
Previous experience must include five (5) years of hands-on experience in big data environments.</li><li>
Must be adept at deploying and configuring Big Data and Knowledge Management tools in an enterprise environment.</li><li>
Must have extensive technical expertise in the configuration and troubleshooting of big data ecosystems.</li><li>
Must have excellent written and verbal communication skills and be comfortable assisting customers with installation and configuration of their big data infrastructure.</li><li>
Must have strong troubleshooting skills and the ability to become a CHEETAS deployment subject matter expert.</li><li>
Must be comfortable working with a wide range of stakeholders and functional teams at various levels of experience.</li><li>
Excellent interpersonal skills, oral and written communication skills, and strong personal motivation are necessary to succeed within this position.</li><li>
Experience with installation, configuration, integration with and usage of the following tools and technologies: NFS, SMB, S3, SQL Server, Windows Server, Windows 10/11, Linux (CentOS, Ubuntu, RedHat).</li><li>
Must be prepared to learn new business processes or CHEETAS application nuances every Agile sprint release (roughly every 6 weeks) prior to deploying to customer sites.</li><li>
Ability to problem solve, debug, and troubleshoot while under pressure and time constraints is required.</li><li>
Ability to communicate effectively about technical topics to both experts and non-experts at both the management and technical level is required.</li><li>
Ability to work independently and provide appropriate recommendations for optimal design, analysis, and development.</li><li>
Excellent verbal communications skills are required, as the Integration Engineer will be in frequent contact with the project technical lead, be taking direction from various government leads, and will frequently be interacting with end users to gather requirements and implement solutions while away from other team members.</li><li>
Excellent testing, debugging and problem-solving skills are required to be successful in this position.</li><li>
Experience designing, building, integrating with and maintaining both new and existing big data systems and solutions.</li><li>
Ability to speak and present findings in front of large groups.</li><li>
Ability to document and repeat procedures.</li><li>
This position is anticipated to require travel of 25% with surges possible up to 50% to support end users located at various DoD Ranges and Labs across the United States.</li></ul><p></p><p><b>
Preferred Qualifications:
</b></p><ul><li>Experience working in government/defense labs and their computing restrictions.</li><li>
Experience working with major DoD acquisition programs.</li><li>
Knowledge of the Test and Training Enabling Architecture (TENA), the Joint Mission Environment Testing Capability (JMETC) and distributed testing and training.</li><li>
Experience with working in distributed team environment.</li><li>
Ability to teach and mentor engineers with a variety of skill levels and backgrounds.</li><li>
Knowledge of DoD cybersecurity policies.</li></ul><p></p><p><b>
Basic Compensation:
</b></p><p><b>$142,400 - $180,000 (For the Denver, CO Area only)</b></p><p><b>
$148,900 - $200,000 (For the Los Angeles, CA Area Only)</b></p><p><b>
$148,900 - $200,00 (For the Washington, DC Area Only)</b></p><p></p><p>
The offered rate will be based on the selected candidate&rsquo;s working location, knowledge, skills, abilities and/or experience, clearance level, contract affordability and in consideration of internal parity.</p><p></p><p><b>
Additional Compensation:</b></p><p>
KBR may offer bonuses, commissions, or other forms of compensation to certain job titles or levels, per internal policy or contractual designation. Additional compensation may be in the form of a sign on bonus, relocation benefits, short-term incentives, long-term incentives, or discretionary payments for exceptional performance.</p><p></p><p>
Belong, Connect and Grow at KBR<br>
<br>
At KBR, we are passionate about our people and our Zero Harm culture. These inform all that we do and are at the heart of our commitment to, and ongoing journey toward being a People First company. That commitment is central to our team of team&rsquo;s philosophy and fosters an environment where everyone can Belong, Connect and Grow. We Deliver &ndash; Together.</p><p></p><p>
KBR is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, disability, sex, sexual orientation, gender identity or expression, age, national origin, veteran status, genetic information, union status and/or beliefs, or any other characteristic protected by federal, state, or local law.</p></div><p></p>

Big Data Systems Engineer

DS Technologies Inc

<div><div><div><b>About US: </b>We are a company that provides innovative, transformative IT services and solutions. We are passionate about helping our clients achieve their goals and exceed their expectations. We strive to provide the best possible experience for our clients and employees. We are committed to continuous improvement and innovation, and we are always looking for ways to improve our services and solutions. We believe in working <b>collaboratively</b> with our clients and employees to achieve success.</div>
<div></div>
<div>DS Technologies Inc is looking for <b>Engineer II &ndash; Big Data / ETL Engineer (Apptio) </b>role<b> </b>for one of our premier clients.<br>
</div><div><b>Job Title: Engineer II &ndash; Big Data / ETL Engineer (Apptio)</b><br>
<b>Location: New York, NY (NYC Metro Area only)(Hybrid)</b><br>
<b>Position Type: Contract</b><br>
<b>Only W2</b><br>
<br>
<b>Job Summary:</b><br>
We are looking for a highly skilled Big Data / ETL Engineer who can work as a standalone developer and write production-ready code from scratch. The ideal candidate has strong expertise in Python, SQL, ETL development, and data warehousing, with hands-on experience building scalable data pipelines in enterprise environments.<br>
<br>
<b>Key Responsibilities:</b>
</div><ul><li>Design, develop, and maintain ETL pipelines using Python from scratch</li>
<li>Write efficient, reusable, and scalable code following OOP principles</li>
<li>Develop and optimize complex SQL queries, stored procedures, and database objects</li>
<li>Perform data modeling including schemas and entity relationships</li>
<li>Build and manage Airflow DAGs</li>
<li>Ingest, cleanse, govern, and report data from Big Data platforms</li>
<li>Work with multiple SQL dialects (SQL Server, Oracle, DB2, PostgreSQL)</li>
<li>Collaborate with stakeholders while operating independently</li>
</ul><div><b>Required Skills &amp; Qualifications:</b></div>
<ul><li>Strong Python (ETL pipeline development from scratch)</li>
<li>Expert SQL and RDBMS concepts</li>
<li>MS SQL Server (SSIS, SSRS)</li>
<li>Oracle PL/SQL</li>
<li>Solid understanding of OOP principles</li>
<li>Apptio experience</li>
<li>Airflow (DAG development)</li>
<li>Experience with Data Warehousing &amp; Big Data</li>
<li>Node.js, JSON, and NoSQL databases</li>
<li>Ability to work as an independent contributor</li>
<li>Must be local to NYC metro area</li>
</ul><div><b>Nice to Have:</b></div>
<ul><li>PySpark / Apache Spark</li>
<li>Databricks Delta Lakehouse or BigQuery</li>
<li>Exposure to cloud-based business tools and platforms</li>
</ul>
</div><p>Flexible work from home options available.</p></div>

Engineer II – Big Data / ETL Engineer (Apptio)

Tata Consultancy Services (TCS)

<div></div><div><div><div><div><p>TCS Hiring Azure Data Engineer at PAN India locations!!</p>
<p>
Experience Range- 5 to12 Years</p>
<p>
Key Technical Skillsets-</p>
<p>
</p><ul><li>Strong proficiency in Python, Pyspark.</li>
<li>Data Ingestion and Integration: Ingest data from various sources such as databases, files, APIs, and streaming platforms into Azure. For example, you could use Azure Data Factory to orchestrate the extraction, transformation, and loading (ETL) of data from an on-premises SQL Server database to Azure Blob Storage.
</li><li>Data Transformation and Processing: Perform data transformation and processing tasks to prepare the data for analysis and reporting. Azure Databricks or Azure Synapse Analytics can be used for large-scale data transformations using tools like Apache Spark.</li>
<li>Data Storage and Management: Select and manage the appropriate storage solutions for your data. For instance, you could use Azure SQL Database or Azure Cosmos DB for structured data, Azure Data Lake Storage for big data and unstructured data, or Azure Blob Storage for file storage.</li>
<li>Data Warehousing: Implementing data warehousing solutions is a common use case for Azure Data Engineers. Azure Synapse Analytics (formerly Azure SQL Data Warehouse) offers scalable and distributed analytics capabilities for handling large datasets.</li>
<li>Data Modeling and Analysis: Collaborate with data scientists and analysts to design and implement data models for analytical workloads. Azure Analysis Services or Azure Databricks can be used for building and deploying models to support interactive data analysis.</li></ul><p><br>
</p></div></div></div><div><div><div><b>Location</b></div>
<div>Hyderabad</div>
</div><div><div><b>Job Function</b></div>
<div>CONSULTANCY</div>
</div><div><div><b>Role</b></div>
<div>Engineer</div>
</div><div><div><b>Job Id</b></div>
<div>404819</div>
</div><div><div><b>Desired Skills</b></div>
<div>Azure | Big Data | SQL</div>
</div></div><p>Desired Candidate Profile
</p><div><div><b>Qualifications</b> : BACHELOR OF TECHNOLOGY</div></div></div><div></div>

Azure Data Engineer

MANTECH

<div><p><b>MANTECH </b>seeks a motivated, career and customer-oriented <b>Data Engineer</b> to join our team in <b>Chantilly, VA.</b></p><p><br>
The Data Engineer will leverage their development skills and experience to support the successful designing, ingesting, cleansing, transformation, loading, and display of significant amounts of data.</p><p><b><br>
Responsibilities include but are not limited to:</b></p><ul><li>
Designing, implementing, and optimizing data extraction, cleansing, transformation, loading, replication/distribution, and large-scale ingest systems in a Big Data environment.</li><li>
Optimizing all stages of the data lifecycle, from initial planning to ingest through final display and beyond.</li><li>
Developing custom solutions or code to ingest and exploit new and existing data sources.</li><li>
Developing data profiling, deduping logic, and matching logic for analysis.</li><li>
Organizing and maintaining Data Layer documentation while working with data scientists to craft pipelines for AI/ML workflows.</li><li>
Collaborating with teammates, service providers, vendors, and users to develop more efficient methods.</li><li>
Effectively articulating the risks and constraints associated with software solutions based on the environment.</li></ul><p><b><br>
Minimum Qualifications:</b></p><ul><li>
High School Diploma with 4+ years of relevant data engineering, software development, and/or programming experience.</li><li>
Demonstrated data analysis, parsing, and programming language experience such as Python or Java.</li><li>
Significant experience with SQL and database management.</li><li>
Experience with the full data lifecycle, from ingest through display, in a Big Data environment.</li><li>
Hands-on experience with Java-related technologies, such as JDK, J2EE, EJB, JDBC, and/or Spring.</li><li>
Experience with RESTful APIs, data pipelining systems like Apache Airflow, and performing ETL tasks in a Linux environment.</li></ul><p><b><br>
Preferred Qualifications :</b></p><ul><li>
Experience deploying systems that leverage AI/ML technology.</li><li>
Experience publishing results in BI dashboards.</li><li>
Self-starter with excellent people skills able to work across teams and drive change.</li></ul><p><b><br>
Clearance Requirements:</b></p><ul><li>
An active TS/SCI with Polygraph is required for this position.</li></ul><p><b><br>
Physical Requirements:</b></p><ul><li>
Must be able to remain in a stationary position 50% of the time.</li><li>
Needs to occasionally move about inside the office to access file cabinets, office machinery, etc.</li><li>
Frequently communicates with co-workers, management, and customers, which may involve delivering presentations.</li><li>
Must be able to exchange accurate information in these situations.</li></ul></div>

Data Engineer

VeeRteq Solutions Inc.

<div><p><b>Roel: Data Engineer Databricks</b></p>
<p><b>Location Chicago, IL (Local candidates only)</b></p><br>
<p><br>
<b>Experience 10+ Years</b></p>
<p>We are looking for a Data Engineer with strong experience in Databricks and data engineering technologies.</p>
<p></p><p><b>Note Final round will be in person</b></p>
<p></p><p><b>Key Responsibilities</b><br>
Build and maintain scalable data pipelines<br>
Work with Databricks for data processing and analytics<br>
Perform data transformation and optimization<br>
Collaborate with cross functional teams</p>
<p>Mandatory Skills<br>
Strong Databricks experience<br>
Good SQL and data engineering skills<br>
Experience with ETL and big data tools<br>
Knowledge of cloud platforms is a plus</p>
<p></p><p><b>Note Final round will be in person</b></p>
<p></p><p><b>pooja</b><b> </b><b>1 (609) 423-6588</b></p></div>

Data Engineer Databricks

Search 10000+ Big data engineer jobs near you this April. New roles added regularly across leading industries. Apply today.