big data architect

CRUTZ LEELA

<p>Big Data Architect required</p><p>Experience: 10-12 years</p><p>Key Responsibilities:</p><ul><li>Data Architecture Design: Designing and implementing scalable and resilient data</li></ul><p>architectures for both batch and streaming data processing.</p><ul><li>Data Modeling: Developing data models and database structures to ensure efficient</li></ul><p>data storage and retrieval.</p><ul><li>Data Security and Governance: Ensuring data security, integrity, and compliance</li></ul><p>with relevant regulations.</p><ul><li>Data Integration: Integrating data from various sources, including legacy systems,</li></ul><p>into the big data infrastructure.</p><ul><li>Performance Optimization: Monitoring and optimizing the performance of the big</li></ul><p>data infrastructure.</p><ul><li>Collaboration: Working with data scientists, engineers, and business stakeholders</li></ul><p>to understand requirements and translate them into technical solutions.</p><ul><li>Technology Selection: Evaluating and recommending new technologies and tools</li></ul><p>for data management and processing.</p><ul><li>Mentorship: Provide guidance and mentorship to junior team members.</li><li>Problem Solving: Identifying and resolving complex data challenges.</li><li>Participate in the pre and post sales process, helping both the sales, professional.</li></ul><p>Skills and Qualifications:</p><ul><li>Bachelor&rsquo;s / Master&rsquo;s degree in computer science, computer engineering, or</li></ul><p>relevant field.</p><ul><li>Overall 10+ years of experience, at least 2 years of experience in Big Data</li></ul><p>Architect.</p><ul><li>Strong understanding of big data technologies: Hadoop, Spark, NoSQL databases,</li></ul><p>cloud-based data services (AWS, Azure, GCP).</p><ul><li>Experience with open-source ecosystem programming languages (i.e., Python,</li></ul><p>Java, Scala, Spark etc.)</p><ul><li>Proficiency in data modeling and database design: SQL, NoSQL.</li><li>Experience with ETL processes: Extracting, transforming, and loading data.</li><li>Strong analytical and problem-solving skills.</li><li>Good communication and collaboration skills.</li><li>Experience with data security and governance principles.</li><li>Knowledge of API design and development.</li><li>Understanding of data visualization techniques.</li><li>Strong understanding of authentication (i.e. LDAP , Active Directory, SAML,</li></ul><p>Kerberos etc.) &amp;amp; authorization configuration for Hadoop based distributed systems.</p><ul><li>Familiarity of DevOps methodology &amp;amp; toolsets and automation experience.</li></ul><p>Job Types: Full-time, Permanent</p><p>Benefits:</p><ul><li>Provident Fund</li></ul><p>Work Location: In person</p>

Big Data Architect

CRUTZ LEELA ENTERPRISES

<p>Position- Big Data Architect</p><p>Experience: 10-12 years</p><p>Key Responsibilities:</p><ul><li>Data Architecture Design: Designing and implementing scalable and resilient data</li></ul><p>architectures for both batch and streaming data processing.</p><ul><li>Data Modeling: Developing data models and database structures to ensure efficient</li></ul><p>data storage and retrieval.</p><ul><li>Data Security and Governance: Ensuring data security, integrity, and compliance</li></ul><p>with relevant regulations.</p><ul><li>Data Integration: Integrating data from various sources, including legacy systems,</li></ul><p>into the big data infrastructure.</p><ul><li>Performance Optimization: Monitoring and optimizing the performance of the big</li></ul><p>data infrastructure.</p><ul><li>Collaboration: Working with data scientists, engineers, and business stakeholders</li></ul><p>to understand requirements and translate them into technical solutions.</p><ul><li>Technology Selection: Evaluating and recommending new technologies and tools</li></ul><p>for data management and processing.</p><ul><li>Mentorship: Provide guidance and mentorship to junior team members.</li></ul><ul><li>Problem Solving: Identifying and resolving complex data challenges.</li></ul><ul><li>Participate in the pre and post sales process, helping both the sales, professional.</li></ul><p>Skills and Qualifications:</p><ul><li>Bachelor&rsquo;s / Master&rsquo;s degree in computer science, computer engineering, or</li></ul><p>relevant field.</p><ul><li>Overall 10+ years of experience, at least 2 years of experience in Big Data</li></ul><p>Architect.</p><ul><li>Strong understanding of big data technologies: Hadoop, Spark, NoSQL databases,</li></ul><p>cloud-based data services (AWS, Azure, GCP).</p><ul><li>Experience with open-source ecosystem programming languages (i.e., Python,</li></ul><p>Java, Scala, Spark etc.)</p><ul><li>Proficiency in data modeling and database design: SQL, NoSQL.</li></ul><ul><li>Experience with ETL processes: Extracting, transforming, and loading data.</li></ul><ul><li>Strong analytical and problem-solving skills.</li></ul><ul><li>Good communication and collaboration skills.</li></ul><ul><li>Experience with data security and governance principles.</li></ul><ul><li>Knowledge of API design and development.</li></ul><ul><li>Understanding of data visualization techniques.</li></ul><ul><li>Strong understanding of authentication (i.e. LDAP , Active Directory, SAML,</li></ul><p>Kerberos etc.) &amp;amp; authorization configuration for Hadoop based distributed systems.</p><ul><li>Familiarity of DevOps methodology &amp;amp; toolsets and automation experience.</li></ul><p>Job Type: Permanent</p><p>Pay: &#8377;500,000.00 - &#8377;1,800,000.00 per year</p><p>Benefits:</p><ul><li>Provident Fund</li></ul><p>Work Location: In person</p>

Emergys

<div><div><div><div><div><div><div><div><div><div><div><div><div><div><div><p><b>Experience</b>: 10-12 years</p>
<p><b>
Location</b>: Pune, India (Work from Office)</p>
<p><b>
Key Responsibilities:</b></p>
<ul>
<li>Data Architecture Design: Designing and implementing scalable and resilient data architectures for both batch and streaming data processing.</li>
<li>Data Modeling: Developing data models and database structures to ensure efficient data storage and retrieval.</li>
<li>Data Security and Governance: Ensuring data security, integrity, and compliance with relevant regulations.</li>
<li>Data Integration: Integrating data from various sources, including legacy systems, into the big data infrastructure.</li>
<li>Performance Optimization: Monitoring and optimizing the performance of the big data infrastructure.</li>
<li>Collaboration: Working with data scientists, engineers, and business stakeholders to understand requirements and translate them into technical solutions.</li>
<li>Technology Selection: Evaluating and recommending new technologies and tools for data management and processing.</li>
<li>Mentorship: Provide guidance and mentorship to junior team members.</li>
<li>Problem Solving: Identifying and resolving complex data challenges.</li>
<li>Participate in the pre and post sales process, helping both the sales, professional.</li>
</ul>
<p><b>Skills and Qualifications:</b></p>
<ul>
<li>Bachelor&rsquo;s / Master&rsquo;s degree in computer science, computer engineering, or relevant field.</li>
<li>Overall 10+ years of experience, at least 2 years of experience in Big Data Architect.</li>
<li>Strong understanding of big data technologies: Hadoop, Spark, NoSQL databases, cloud-based data services (AWS, Azure, GCP).</li>
<li>Experience with open-source ecosystem programming languages (i.e., Python, Java, Scala, Spark etc.)</li>
<li>Proficiency in data modeling and database design: SQL, NoSQL.</li>
<li>Experience with ETL processes: Extracting, transforming, and loading data.</li>
<li>Strong analytical and problem-solving skills.</li>
<li>Good communication and collaboration skills.</li>
<li>Experience with data security and governance principles.</li>
<li>Knowledge of API design and development.</li>
<li>Understanding of data visualization techniques.</li>
<li>Strong understanding of authentication (i.e. LDAP , Active Directory, SAML, Kerberos etc.) &amp; authorization configuration for Hadoop based distributed systems.</li>
<li>Familiarity of DevOps methodology &amp; toolsets and automation experience.</li></ul></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div><p></p>

Apple

<p>The people here at Apple don&rsquo;t just build products - they build the kind of wonder that&rsquo;s revolutionized entire industries. It&rsquo;s the diversity of those people and their ideas that encourages the innovation that runs through everything we do, from amazing technology to industry-leading environmental efforts. Join Apple, and help us leave the world better than we found it.
<br>
<br>
We are looking for passionate and dedicated professionals to join our Data Services&rsquo; Data Analytics and Engineering team. This team manages various state of the art open source technologies in Big Data Analytics and Gen-AI areas, including Hadoop ecosystem, AI/ML and Vector Databases as well as Enterprise MPP systems. The ideal candidate should have experience in architecting/deploying large-scale Analytics systems and services, ready to learn and explore new leading edge technologies.<br>
<br>
<b>Description</b><br>
Are you passionate about building highly scalable, distributed applications that efficiently handle large volumes of data? Do you thrive on mastering various Big Data and MPP technologies and selecting the best ones for each application? If you are excited by the challenge of working in a fast-paced environment where your technical skills are tested daily and you want your work to impact millions of Apple users, we want to hear from you.
<br>
In this role, you will join Apple's Data Services Data Analytics and Engineering team as a highly motivated, detail-oriented, and energetic individual with excellent communication skills.
<br>
Your responsibilities will include:<br>
Design, architect, engineer and mentor in Big Data and MPP technologies, platforms and applications.<br>
Lead the design, development and management of new solutions using the latest technologies and data platforms such as SingleStore, Vertica, Clickhouse, Doris across various environments, including physical &amp; virtual machines, containers, and cloud platforms.<br>
Develop scalable, robust systems that adapt to evolving business requirements.<br>
Measure, Compare, Optimise and Tune the System / Datastore performance<br>
Create proofs of concept for emerging technologies, performance benchmarking and architect scalable applications.<br>
Recommend and implement tools and processes to accelerate the development, deployment and maintenance of Analytics workloads.<br>
Collaborate with application owners, developers, and project managers, leveraging your expertise to provide guidance on architectural decisions and address complex technical challenges related to scaling and performance tuning.<br>
<br>
<b>Preferred Qualifications</b><br>
Experience with data visualisation tools such as Tableau, Looker and Apache Superset.<br>
Knowledge of open-source Vector databases such as Milvus, Weaviate, etc. is a plus.<br>
Proficient in Application development using any programming language, with a focus on designing and building tools and frameworks for managing and scaling Big Data technologies.<br>
<br>
<b>Minimum Qualifications</b><br>
Strong educational background in Computer Science or Software Engineering.<br>
8 to 12 years of professional experience managing complex, distributed and multi-tier Big Data / Analytics environments.<br>
Proven experience as a database SRE, specifically with MPP technologies such as SingleStore, ClickHouse, Vertica, Doris, etc<br>
In-depth knowledge of datastore performance tuning, query optimisation and troubleshooting along with DR setup / Backup and recovery procedures.<br>
Extensive experience working with public cloud platforms (AWS &amp; GCP) and Kubernetes container deployments.<br>
Expertise in automation, including the development of tools and scripts to monitor server-side components and improve operational efficiency.<br>
Familiarity with architectural and design considerations related to scalability, reusability, and flexibility.<br>
Demonstrate strong problem-solving and critical thinking skills, along with a solid ability to quickly evaluate and apply new technologies.<br>
A passion for solving challenging engineering problems and triaging complex operational issues in collaboration with various engineering and customer teams.&quot;,&quot;internalDetails&quot;:null,&quot;eeoContent&quot;:null</p>

Data Analytics / MPP Engineer

Quantiphi

<div><div>While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.<br>
<br>
If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!</div><div>
You will be working as an Associate Data Architect within the domain, designing and delivering big data pipelines for structured and unstructured data that are running across multiple geographies, helping healthcare organizations achieve their business goals with use of data ingestion technologies, cloud services &amp; DevOps. You will be working with Architects from other specialties such as Cloud engineering, Software engineering, ML engineering to create platforms, solutions and applications that cater to latest trends in the healthcare industry such as digital diagnosis, software as a medical product, AI marketplace, amongst others.<br>
Role &amp; Responsibilities:<br>
Work with cloud engineers and customers to solve big data problems by developing utilities for migration, storage and processing on Google Cloud.<br>
Design and build a cloud migration strategy for cloud and on-premise applications.<br>
Diagnose and troubleshoot complex distributed systems problems and develop solutions with a significant impact at massive scale.<br>
Build tools to ingest and jobs to process several terabytes or petabytes per day.<br>
Design and develop next-gen storage and compute solutions for several large customers.<br>
Communicate with a wide set of teams, including Infrastructure, Network, Engineering, DevOps, SiteOps teams, and cloud customers.<br>
Build advanced tooling for automation, testing, monitoring, administration, and data operations across multiple cloud clusters.<br>
Better understanding of Data modeling and governance</div><div><div><div><div><div><div><div><div><i>
If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us</i><i>!</i></div></div></div></div></div></div></div></div></div><div></div>

Associate Tech Architect - Data Engineer

Confiz

<div><p>We are looking for an experienced <b>Data Architect</b> with 7+ years of progressive experience in data engineering, architecture design, and technical leadership. You will lead the architectural strategy and execution for our real-time and batch data platforms, ensuring scalability, reliability, and performance across a modern cloud-native ecosystem. This role is suited for a visionary technologist who can drive enterprise-grade solutions, establish best practices, and mentor high-performing engineering teams.</p>
<p><b>Responsibilities:</b></p>
<ul><li>Design and implement scalable, secure, and efficient data architectures for integrated systems.</li>
<li>Develop and maintain enterprise-wide data models, ensuring alignment with business objectives.</li>
<li>Develop data flow frameworks to ensure seamless integration between applications, databases, and cloud platforms.</li>
<li>Define and enforce data architecture standards, best practices, and governance policies.</li>
<li>Manage data import processes from various source systems/API and oversee data export to other systems/API, ensuring accuracy and reliability.</li>
<li>Collaborate with cross-functional teams to understand data requirements and translate them into technical solutions.</li>
<li>Optimize data storage, retrieval, and processing to enhance performance and reliability.</li>
<li>Ensure data security, compliance, and integrity across all systems.</li>
<li>Conduct code reviews, provide feedback, and ensure best practices are followed.</li>
<li>Identify and resolve data-related issues, bottlenecks, and inefficiencies.</li>
<li>Stay updated with emerging technologies and industry best practices in data architecture and engineering.</li>
<li>Completed documentation of Data Architecture structure.</li>
</ul>
<p><b>Qualifications:</b></p>
<ul><li>Bachelors or equivalent Degree in Computer Science, Information Technology or related field.</li>
<li>8+ years of professional, hands-on experience building complex solution that collect, process, store and analyze large volumes of business-critical data &bull; 3+ years' experience in Big Data Distributed systems such as Microsoft Fabric, Databricks</li>
<li>3+ years of experience developing both multidimensional and tabular models with large and complex datasets;</li>
<li>1+ years working in an Architect role including: designing, modeling and building complex data solutions</li>
<li>Strong experience in developing data pipelines using PySpark and ETL tools like ADF, Talend</li>
<li>Proficiency in Python &amp; SQL</li>
<li>Solid understanding of data modelling, ETL processes, and best practices in data warehousing domain</li>
<li>Must have good knowledge of Data Governance Practices and Data Quality Tools</li>
<li>Must have good knowledge of data security and privacy measures</li>
<li>Experience in managing teams, being a technical coach</li>
<li>Experience driving knowledge transfer and/or training programs</li>
<li>Experience with agile delivery methodologies, proof of concepts and prototyping</li>
<li>Experience with Azure ARM templates, PowerShell and CI/CD using Azure DevOps</li>
<li>Demonstrated strategic thinking through a proven ability to research client inquiries and emerging issues related to regulations, industry practice and new application technologies</li>
<li>Ability to conduct data profiling, cataloguing, and mapping for technical design and construction of technical data flows</li>
<li>Ability to facilitate design discussions and communicate architectural decisions</li>
<li>Experience in Microsoft Fabric and Purview is an additional advantage</li>
</ul>
<p>We have an amazing team of 700+ individuals working on highly innovative enterprise projects &amp; products. Our customer base includes Fortune 100 retail and CPG companies, leading store chains, fast-growth fintech, and multiple Silicon Valley startups.</p>
<p>What makes Confiz stand out is our focus on processes and culture. Confiz is <b>ISO 9001:2015</b> (QMS), <b>ISO 27001:2022</b> (ISMS), <b>ISO 20000-1:2018</b> (ITSM), <b>ISO 14001:2015</b> (EMS), <b>ISO 45001:2018 </b>(OHSMS) Certified. We have a vibrant culture of learning via collaboration and making workplace fun.</p>
<p>People who work with us work with cutting-edge technologies while contributing success to the company as well as to themselves.</p>
<p>To know more about Confiz Limited, visit: https://www.linkedin.com/company/confiz-pakistan/</p></div>

Data Architect

EPAM Systems

<div><p>We are looking for a skilled <b>Big Data DevOps Engineer </b>with profound knowledge of cloud technologies and experience with a variety of data tools and platforms.</p><p>
The ideal candidate will have proven experience in managing and optimizing big data operations across cloud environments.</p><p><b>
Responsibilities</b></p><ul><li>
Cloud Solutions Development: Build, deploy, and maintain scalable and secure big data solutions across major cloud platforms (AWS, Azure, Google Cloud) to support advanced data processing and analytics</li><li>
Data Tools Management: Configure and manage data processing tools such as Apache Spark, Kafka, Airflow, and Databricks to efficiently handle large-scale data workflows</li><li>
Automation and CI/CD: Utilize Terraform for infrastructure as code, automate workflows with Jenkins, and maintain code with GitLab for continuous integration and continuous delivery processes</li><li>
Performance Optimization: Monitor and optimize the performance of big data tools and cloud platforms, ensuring cost-effectiveness and operational efficiency</li><li>
Security and Compliance: Implement and enforce data security measures, ensuring compliance with data protection regulations and best practices in cloud environments</li><li>
Collaborative Development: Work collaboratively with data scientists, data engineers, and other stakeholders to understand requirements and deliver high-quality data solutions</li></ul><p><b>
Requirements</b></p><ul><li>
Strong experience with big data technologies, particularly Apache Spark or Databricks</li><li>
Extensive background in cloud services administration with platforms like AWS, Azure, or Google Cloud</li><li>
In-depth knowledge of infrastructure as code using Terraform</li><li>
Proficiency in implementing CI/CD pipelines with tools such as Jenkins, GitLab or GitHub Actions</li><li>
Solid scripting skills in Python, Scala, or similar languages</li><li>
Experience with SQL and relational/non-relational databases</li><li>
Excellent problem-solving abilities and a team-oriented attitude</li></ul><p><b>
Nice to have</b></p><ul><li>
Certifications in cloud technology (e.g., AWS Certified Solutions Architect, Google Cloud Certified Professional Cloud Architect)</li><li>
Additional experience with real-time data processing tools like Apache Kafka</li></ul><p><b>
We offer</b></p><ul><li>
CONTINUOUS UPSKILLING, LEARNING &amp; DEVELOPMENT<ul><li>
Diversity of tasks and projects</li><li>
Assessment center for objective review of competency level</li><li>
Personal development plan</li><li>
Mentoring programs and leadership development</li><li>
Certification and professional development support</li><li>
Access to learning platforms including more than 2,500 internal courses and the LinkedIn Learning library with 20,000+ courses</li><li>
English courses taught by certified teachers</li></ul></li><li>
CORPORATE BENEFITS<ul><li>
Extra leave days</li><li>
Referral bonuses</li></ul></li><li>
COMPENSATION PACKAGE<ul><li>
Competitive compensation paid in USD</li><li>
Regular salary and performance reviews</li></ul></li><li>
MEDICAL &amp; HEALTHCARE<ul><li>
Private health insurance</li><li>
Well-being events</li></ul></li><li>
WORKING ENVIRONMENT<ul><li>
Recreation areas and kitchens</li><li>
Tea, coffee, and snacks</li><li>
Well-being events</li><li>
Sports equipment and game consoles</li><li>
IT Equipment</li><li>
Microsoft's Software Assurance Home Use Program (HUP)</li></ul></li></ul><p><i>
EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.</i></p></div>

Middle Big Data DevOps Engineer

Search 10000+ Big data architect jobs near you this November. New roles added regularly across leading industries. Apply today.

10000 big data architect jobs NEAR_ME

Big Data Architect

Big Data Architect

Big Data Architect

Data Analytics / MPP Engineer

Associate Tech Architect - Data Engineer

Data Architect

Middle Big Data DevOps Engineer