big data developer

Citi

<div><p>The Applications Development Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to contribute to applications systems analysis and programming activities.<br>
<br>
<b>Responsibilities:</b></p><ul><li>
Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements</li><li>
Identify and analyze issues, make recommendations, and implement solutions</li><li>
Utilize knowledge of business processes, system processes, and industry standards to solve complex issues</li><li>
Analyze information and make evaluative judgements to recommend solutions and improvements</li><li>
Conduct testing and debugging, utilize script tools, and write basic code for design specifications</li><li>
Assess applicability of similar experiences and evaluate options under circumstances not covered by procedures</li><li>
Develop working knowledge of Citi&rsquo;s information systems, procedures, standards, client server application development, network operations, database administration, systems administration, data center operations, and PC-based applications</li><li>
Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.</li></ul><p><b><br>
Qualifications:</b></p><ul><li>
0-2 years of relevant experience</li><li>
Experience in programming/debugging used in business applications</li><li>
Working knowledge of industry practice and standards</li><li>
Comprehensive knowledge of specific business area for application development</li><li>
Working knowledge of program languages</li><li>
Consistently demonstrates clear and concise written and verbal communication</li></ul><p></p><p>
Key Skills:</p><p></p><p>
Strong programming skills in <b>PySpark in a BigData</b> environment.</p><p>
Familiarity with big data processing tools and techniques.</p><p>
Experience with the Hadoop ecosystem including <b>Hive</b>, <b>HDFS</b>, Sqoop, <b>Spark</b>, Impala, Scala, etc.,</p><p>
Well versed with shell scripting and Autosys scheduler.</p><p>
Good understanding of distributed systems.</p><p>
Should be familiar with data warehouse concepts.</p><p>
Experience with streaming data platforms.</p><p>
Excellent analytical and problem-solving skills.</p><p>
Experience with writing complex <b>SQL</b> queries.</p><p>
Knowledge on data modeling and data design is essential.</p><p>
Should be independent and resourceful dealing with risks/issues and resolving them in a timely manner.</p><p>
Excellent communication and articulation skills.</p><p><b>
Education:</b></p><ul><li>
Bachelor&rsquo;s degree/University degree or equivalent experience</li></ul><p></p><p><br>
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.</p><p></p><p>
-</p><p><b>
Job Family Group:
</b></p>Technology<p>
-</p><p><b>
Job Family:</b></p>
Applications Development<p>
-</p><p><b>
Time Type:</b></p>
Full time<p>
-</p><p><b>
Most Relevant Skills
</b></p>Please see the requirements listed above.<p>
-</p><p><b>
Other Relevant Skills
</b></p>For complementary skills, please see above and/or contact the recruiter.<p>
-</p><p><i>
Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.</i></p><p></p><p><i><br>
If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review </i><i>Accessibility at Citi</i><i>.</i><br>
<br>
<i>View Citi&rsquo;s </i><i>EEO Policy Statement</i><i> and the </i><i>Know Your Rights</i><i> poster.</i></p></div>

Big Data Developer

PROZ TECHNOLOGIES

<p>Indeed Job Post:</p><p><b>Job Title:</b> Big Data Platform Developer<br/><b>Location:</b> Doha, Qatar</p><p><b>Job Overview:</b><br/>We are hiring a <b>Big Data Platform Developer</b> skilled in Python, Java, and big data technologies to design and optimize our client&rsquo;s big data platform. You will be responsible for data ingestion, normalization, visualization, and automated reporting.</p><p><b>Key Responsibilities:</b></p><ul><li>Parse, inject, and normalize large datasets</li><li>Develop dashboards, heat maps, and visualization reports</li><li>Build and automate data pipelines for real-time analytics</li><li>Generate detailed reports for stakeholders</li><li>Collaborate with data scientists &amp; cybersecurity teams</li><li>Maintain technical documentation</li></ul><p><b>Qualifications:</b></p><ul><li>Bachelor&rsquo;s in Computer Science, Data Science, or related field</li><li>4&ndash;6 years of big data experience (Hadoop, Spark, Kafka)</li><li>Strong coding skills in Python &amp; Java</li><li>Proficiency in SQL/NoSQL databases, REST APIs, and cloud platforms (AWS/Azure/GCP)</li><li>Familiarity with Tableau, Power BI, or other visualization tools</li><li>Strong problem-solving and collaboration skills</li></ul><p><b>About Proztec:</b><br/>Proz Technologies (PROZTEC), based in Doha, Qatar, is a leading ICT firm with 300+ employees. Be part of a growing team delivering innovative data-driven solutions.</p><p>Apply now by sending your CV to Recruitment@proztec.com</p><p>Job Type: Full-time</p><p>Pay: QAR15,000.00 - QAR18,000.00 per month</p>

Big Data Platform Developer

BrightEdge

<div><div><div><b>About Us</b></div><div>
BrightEdge is a leading SEO and content performance marketing platform that transforms online content into tangible business results. Our platform processes massive amounts of data to provide actionable insights to our clients. We're looking for a talented Big Data Engineer to join our Professional Services to help us scale and optimize our data processing capabilities.
</div><div><b>Role Overview
</b></div><div>As a Big Data Engineer at BrightEdge, you will design, build, and maintain high-performance data pipelines that process terabytes of data. You'll work on optimizing our existing systems, identifying and resolving performance bottlenecks, and implementing solutions that improve the overall efficiency of our platform. This role is critical in ensuring our data infrastructure can handle increasing volumes of data while maintaining exceptional performance standards.</div></div><div><div><h3 class="jobSectionHeader"><b>
Key Responsibilities</b></h3><ul><li><ul><li>
Design and implement scalable batch processing systems using Python and big data technologies</li><li>
Optimize database performance, focusing on slow-running queries and latency improvements
</li><li>Use Python profilers and performance monitoring tools to identify bottlenecks
</li><li>Reduce P95 and P99 latency metrics across our data platform
</li><li>Build efficient ETL pipelines that can handle large-scale data processing
</li><li>Collaborate with data scientists and product teams to understand data requirements
</li><li>Monitor and troubleshoot data pipeline issues in production
</li><li>Implement data quality checks and validation mechanisms</li><li>
Document data architecture and engineering processes
</li><li>Stay current with emerging big data technologies and best practices</li></ul></li></ul></div></div><div><div><h3 class="jobSectionHeader"><b>
Qualifications</b></h3><ul><li><ul><li><b>
Required</b></li><li>
Bachelor's degree in Computer Science, Engineering, or related technical field
</li><li>4+ years of experience in data engineering roles</li><li>
Strong Python programming skills with focus on data processing libraries
</li><li>Experience with big data technologies (Spark, Hadoop, etc.)
</li><li>Proven experience optimizing database performance (SQL or NoSQL)
</li><li>Knowledge of data pipeline orchestration tools (Airflow, Luigi, etc.)
</li><li>Understanding of performance optimization techniques and profiling tool</li></ul></li></ul></div></div><div><div><h3 class="jobSectionHeader"><b>
Preferred</b></h3><ul><li><ul><li>
Master's degree in Computer Science or related field
</li><li>Experience with SEO data or web crawling systems</li><li>
Experience with Clickhouse Database
</li><li>Knowledge of distributed systems and microservices architecture
</li><li>Familiarity with container orchestration (Kubernetes, Docker)
</li><li>Experience with real-time data processing
</li><li>Contributions to open-source projects
</li><li>Experience with machine learning operations</li></ul></li></ul></div></div></div><div></div>

Big Data Engineer- Professional Services

TransUnion

<div><p>TransUnion's Job Applicant Privacy Notice
</p><p></p><p><b>What We'll Bring:
</b></p>The Role:
<br>
We are seeking a talented and experienced Senior Data Engineer/Big Data Developer to join our growing team. In this role, you will be responsible for designing, developing, and maintaining our Big Data infrastructure and pipelines. You will work with large datasets to build scalable solutions for data processing, analytics, and machine learning initiatives. You will collaborate with data scientists, analysts, and other engineers to deliver high-quality, reliable, and performant data solutions that drive business decisions.
<br>
Responsibilities:
<br>
<ul><li>Design, develop, and maintain robust and scalable Big Data pipelines using technologies like Hadoop, Spark, and Scala.
</li><li>Develop and maintain data ingestion, processing, and storage solutions on cloud platforms such as AWS and GCP.
</li><li>Write high-quality, well-documented, and testable code in Java, Scala, Python, and SQL.
</li><li>Optimize data processing performance and ensure data quality.
</li><li>Collaborate with data scientists and analysts to understand their requirements and build data solutions that meet their needs.
</li><li>Work with large datasets and perform data analysis to identify trends and patterns.
</li><li>Implement data governance and security best practices.
</li><li>Contribute to the evolution of our Big Data architecture and technology stack.
</li><li>Troubleshoot and resolve issues in the data pipelines and infrastructure.
</li><li>Stay up-to-date with the latest Big Data technologies and trends.
</li><li>Participate in code reviews and contribute to the team's knowledge sharing.
</li><li>Work in a fast-paced, agile environment.
</li></ul><p></p><p><b>What You'll Bring:
</b></p><p>Qualifications:
</p><ul><li>Bachelor's degree in Computer Science, Engineering, or a related field.
</li><li>4+ years of experience in software development with a focus on Big Data technologies.
</li><li>Strong proficiency in Java, Scala, Python, and SQL.
</li><li>Experience with Hadoop ecosystem components such as HDFS, MapReduce, YARN, and Hive.
</li><li>Extensive experience with Apache Spark for large-scale data processing.
</li><li>Experience building and deploying data pipelines on cloud platforms (AWS and/or GCP).
</li><li>Solid understanding of data modeling, data warehousing, and ETL concepts.
</li><li>Experience with data analytics and visualization tools is a plus.
</li><li>Strong understanding of distributed systems principles and architecture.
</li><li>Proficiency in Unix/Linux environments.
</li><li>Excellent problem-solving and analytical skills.
</li><li>Strong communication and collaboration skills.
</li><li>Experience with data governance and security best practices.
</li><li>Experience with Agile development methodologies.
</li></ul><p></p><p><b>Impact You'll Make:
</b></p><p>Bonus Points:
</p><ul><li>Experience with data streaming technologies like Kafka or Kinesis.
</li><li>Experience with NoSQL databases like Cassandra or MongoDB.
</li><li>Experience with machine learning frameworks like TensorFlow or PyTorch.
</li><li>AWS or GCP certifications.
</li><li>Contributions to open-source projects.
</li></ul><p></p>This is a hybrid position and involves regular performance of job responsibilities virtually as well as in-person at an assigned TU office location for a minimum of two days a week.
<p></p><p>TransUnion Job Title
</p><p></p>Developer, Data Analysis and Consulting</div>

Sr. Analyst Data Development

Luxoft

<div><div><h5 class="jobSectionHeader"><b>Project description
</b></h5><p>LUXOFT is one of the major software services companies world-wide. We deliver professional software services in multiple business verticals such as finance, automotive, and digital transformation. Supporting large vehicle manufacturers in the area of development, production and aftersales is LUXOFT's ambition. The software inside a vehicle was traditionally expected to be a very controlled and self-contained environment. Equipping cars with perception and machine intelligence changes a lot in the overall picture of the vehicle manufacturer. LUXOFT's goal is to empower our customers, with deep domain knowledge and smart solutions, to develop cars of the future. Great mechanical engineering is not the only criteria anymore for delivering the best cars in the world. Using state of the art software technologies are key to provide the requested user experience. For the various software development projects and environments in automotive, we are looking for Software Developer. Join our international team.
</p></div><div><h5 class="jobSectionHeader"><b>Responsibilities</b></h5>
<div><div><p>Design, implement, and maintain scalable data pipelines to process large-scale vehicle fleet data using PySpark on AWS cloud infrastructure.
</p></div><div><p>Collect, clean, and transform raw telematics and sensor data to enable meaningful analysis.
</p></div><div><p>Develop workflows to isolate specific driving scenarios based on business requirements and engineering questions.
</p></div><div><p>Build and automate computation of Key Performance Indicators (KPIs) to monitor vehicle behavior and system performance.
</p></div><div><p>Collaborate with data scientists, software engineers, and domain experts to translate analytical questions into efficient data engineering solutions.
</p></div><div><p>You will perform code analysis, optimize algorithms, refactor and optimize code
</p></div><div><p>You will develop robust and effective code and ensure high quality of the code by performing unit test as well as simulation on real data (software in the loop)
</p></div></div></div><div><h5 class="jobSectionHeader"><b>Skills</b></h5>
<p>Must have
</p><div><div><p>Academic degree in computer science, mechatronics, electronic engineering, telecommunication engineering or similar qualification, PhD welcome
</p></div><div><p>Python knowledge
</p></div><div><p>Strong OOP and Python background in more recent versions
</p></div><div><p>Very good understanding of big data analysis preferably in Autonomous Driving sector
</p></div><div><p>You have great problem solving skills and are able to find creative ways to get responses from the data
</p></div><div><p>You are hardworking and have team spirit
</p></div><div><p>Strong communication skills in English
</p></div></div><p>Nice to have
</p><div><div><p>Jupyter Notebook experience
</p></div><div><p>CI/CD experience
</p></div><div><p>AWS Quicksight
</p></div><div><p>Docker
</p></div><div><p>Automotive SW industry
</p></div><div><p>Agile, SCRUM methodology
</p></div></div></div><div><h5 class="jobSectionHeader"><b>Other</b></h5>
<div><div><div><p>Languages</p>
<p>English: B2 Upper Intermediate
</p></div></div><div><div><p>Seniority</p>
<p>Regular</p></div></div></div></div></div>
<br><div><div><div><p>Cairo, Egypt
</p></div><p>Req. VR-116757
</p></div><div></div><div><div><p>Data Science
</p></div><div><p>Automotive Industry</p>
</div><div><p>25/08/2025
</p></div><div><p>Req. VR-116757</p></div></div></div><p></p>

Data Engineer

Apple

<div>Apple is where individual imaginations gather together, committing to the values that lead to great work. Every new product we build, service we create, or Apple Store experience we deliver is the result of us making each other&rsquo;s ideas stronger. That happens because every one of us shares a belief that we can make something wonderful and share it with the world, changing lives for the better. It&rsquo;s the diversity of our people and their thinking that inspires the innovation that runs through everything we do. When we bring everybody in, we can do the best work of our lives. Here, you&rsquo;ll do more than join something - you&rsquo;ll add something. As a BigData Software Engineer, you are an integral part of a small data-centric team driving large-scale data infrastructure and processes development, implementation, and improvement. Our organization thrives on collaborative partnerships. Join and play a key role in developing and driving the adoption of Data Mesh and data-centric micro-services. Apple's Manufacturing Systems and Infrastructure (MSI) team is responsible for capturing, consolidating and tracking all manufacturing data for Apple&rsquo;s products and modules worldwide. Our tools enable teams to confidently use data to shape the next generation of product manufacturing at Apple. We seek a practitioner with experience building large-scale data platforms, analytic tools, and solutions. If you are passionate about making data easily accessible, trusted, and available across the entire business at scale, we'd love to hear from you.
<br>
<br>
<b>Description</b><br>
<br>
As a BigData Engineer, you will lead the development and implementation of our cloud-based data infrastructure. You will work closely with cross-functional teams to understand data requirements, design scalable solutions, and ensure the integrity and availability of our data. The ideal candidate will have a deep understanding of cloud technologies, data engineering best practices, and a proven track record of successfully delivering complex data projects. - Lead the design and implementation of cloud-based data architectures. - Collaborate with data scientists, analysts, and business stakeholders to understand data requirements. - Implement data security and privacy best practices. - Optimize data storage, retrieval, and processing for performance and cost efficiency. - Lead efforts to ensure data quality, consistency, and accuracy. - Mentor and provide technical guidance to junior data engineers. - Stay current with industry trends and emerging technologies in cloud data engineering.
<br>
<br>
<b>Minimum Qualifications</b><br>
<ul><li>
Bachelor&rsquo;s or Master's degree in Computer Science or a related field
</li><li>Proven experience as a Big Data developer, with a focus on cloud platforms
</li><li>Strong proficiency in cloud services such as AWS, Azure, or Google Cloud
</li><li>Expertise in data modelling and data integration
</li><li>Experience with big data technologies (Hadoop, Spark, etc.)
</li><li>Proficient in programming languages such as Python, Java, Scala, GoLang
</li><li>Knowledge of data warehouse solutions (Redshift, BigQuery, Snowflake, Druid)
</li><li>Experience with streaming data technologies (Kafka, Kinesis)
</li></ul><br>
<b>Preferred Qualifications</b><br>
<ul><li>
Certification in cloud platforms (AWS Certified Big Data - Specialty, Microsoft Certified: Azure Data Engineer, etc.)
</li><li>Experience building data streaming solutions using Apache Spark, Storm, Link, Flume
</li><li>Experience using ETL tools and technologies like Reltio, Stibo, Informatics, Apache Atlas
</li><li>Knowledge of machine learning and data science concepts
</li><li>Contributions to the open source community
</li><li>Familiarity with data governance and compliance
</li><li>Strong problem-solving and analytical skills
</li>Excellent communication and leadership skills<br>
<li>
</li></ul><br>
<br>
Submit CV</div>

Big Data Software Engineer - Manufacturing Systems & Infrastructure

اورنچ مصر

<div><p>About the job Sr. Supervisor Data Engineer</p>
<div><p><b>Purpose of the job</b></p>
<p>
As a data engineer you'll work as s a Big Data Developer, you will be working on Data Ingestion activities to bring large volumes of data into our Big Data Lake. The candidate will play a vital role in building new data pipelines from various structured and unstructured sources into Hadoop. You will be working closely with data consumers and source owners to create the foundation for data analytics and machine learning activities.</p>
<p><b>
Duties and responsibilities</b></p><ul><li>
Identifying data ingestion patterns and build framework to efficiently ingest the data to our Data Lake</li><li>
Performance tuning ingestion jobs to improve throughput</li><li>
Improving CI/CD process by automating build, test and deployment framework</li><li>
Build high-performance algorithms, prototypes, and proof of concepts</li><li>
Research opportunities for data acquisition and new uses for existing data</li><li>
Develop data set processes for data modeling, mining and production</li><li>
Integrate new data management technologies and software engineering tools into existing framework</li><li>
Work with In-memory database tools (Redis, Riak)</li><li>
Collaborate with data architects, modelers and IT team members on project goals</li></ul>
<p><b>
Education</b></p><ul><li>
Bachelor degree of Engineering in Computer Systems, or Computer Science.</li></ul>
<p><b>
Experience</b></p><ul><li>
5+ years working with Java based applications and open source technologies</li><li>
3+ years data engineering experience supporting high-volume, high-velocity data streams.</li><li>
Prior experience in working with Hadoop technologies such as HBase, Hive, Spark is a plus<br>
</li><li>
Performance tuning of Java applications and messaging technologies is a plus</li><li>
Prior experience with relational databases such as MS SQL, Oracle and DB2</li><li>
Knowledge and understanding of SDLC and Agile/Scrum procedures, CI/CD and Automation is required</li><li>
Ability to use containers like dockers or Kubernetes is a plus</li><li>
Ability to write SQL queries and use tools such as Hadoop, Tableau, QlikView, and other data reporting tools.</li><li>
Experience in transactional and data warehouse environments using MySQL, Hive, or other database systems. Must deeply understand joins, subqueries, window functions, etc.</li><li>
Strong background in designing relational databases like Postgres and NoSQL database like Mongo DB
<br>
or Cassandra</li></ul>
<p><b>
Skills and abilities</b></p><ul><li>
Very good English both written &amp; spoken.</li><li>
Good computer skills.</li><li>
Past experience in telecom operators and service providers is preferred</li><li>
Good knowledge of database systems and BI solutions</li><li>
Good programming and debugging skills.</li><li>
Good problem solving skills.</li><li>
Ability to administer different hardware and operating system solutions.</li></ul></div></div>

Sr. Supervisor Data Engineer

Search 10000+ Big data developer jobs near you this September. New roles added regularly across leading industries. Apply today.