Lead I - Data Engineering - Data Integration and AI Analytics

5 - 7 Years
2 Openings
Hyderabad

Role Proficiency:

The Senior Data Integration and AI Analytics Specialist is responsible for designing, developing, and maintaining scalable data pipelines and AI analytics workflows to support business intelligence and operational goals. This role requires deep expertise in data integration platforms such as Dataiku, cloud data warehousing with Snowflake and AWS, and scripting with Python for automation and data transformation. The specialist will collaborate closely with data scientists and business stakeholders to deliver actionable insights, automate complex workflows, and ensure data integrity and efficiency across environments. Key Responsibilities: Develop and maintain ETL workflows integrating data from diverse sources including SQL Server, SAP HANA, Oracle, AWS Redshift, and cloud storage platforms. Manage and administer Dataiku environment tasks including license management, credential handling, and environment setup. Automate data pipelines, batch processing, and API integrations for efficient and reliable data flow. Collaborate with data science teams to prepare data for AI model training and experimentation. Implement data validation, monitoring, and reconciliation reporting to reduce manual effort and increase accuracy. Develop Python-based tools for auditing, user activity tracking, and process automation within analytics environments. Create interactive dashboards to visualize KPIs and provide insights for stakeholders supporting decision-making. Document data workflows, requirements, and troubleshooting guides to enhance team knowledge sharing. Support cloud infrastructure provisioning and DevOps CI/CD pipelines relevant to data operations. Required Skills & Qualifications: Strong experience with Dataiku DSS for data pipeline development, maintenance, and administration. Expertise in SQL and Python (including Pandas, PySpark) for data integration, transformation, and scripting. Proficiency in cloud data platforms including Snowflake, AWS (S3, Redshift), and experience with associated DevOps tools. Good understanding of ETL design patterns, batch scheduling, and workflow orchestration. Familiarity with API automation and data validation techniques. Experience working collaboratively in cross-functional teams involving data scientists and business users. Strong analytical, problem-solving, and documentation skills with a focus on operational efficiency. Certifications in Dataiku Core Designer, Dataiku ML Practitioner, AWS Solution Architect, or Microsoft Azure Fundamentals are preferred. Experience: Minimum 5-8 years of experience in data integration, data engineering, or AI/analytics development roles. Hands-on experience in automating complex data workflows and building scalable analytics solutions. Previous exposure to cloud infrastructure provisioning and basic DevOps tooling is an advantage.

Dataiku DSS,SQL & Python,Cloud data platforms,Etl Design