Senior AI/Big Data Engineer

• Design and implement large-scale, fault-tolerant data pipelines on OCI, using services like OCI Data Integration , OCI Data Flow (Apache Spark) , Object Storage , and Autonomous Database .
• Build and manage streaming data architectures using tools such as OCI GoldenGate , Apache Kafka , and Spark/Flink Streaming .
• Enforce standards and automation across the entire data lifecycle , including schema evolution, dataset migration, and deprecation strategies.
• Improve platform resilience, data quality, and observability with advanced monitoring, alerting, and automated data governance.
• Serve as a technical leader , mentoring junior engineers, reviewing designs and code, and promoting engineering best practices.
• Collaborate cross-functionally with ML engineers, platform teams, and data scientists to integrate data services with AI/ML workloads.
• Partner in AI pipeline enablement , ensuring Lakehouse services efficiently support model training, feature engineering, and real-time inference.

Minimum Qualifications

• Bachelor’s or Master’s degree in Computer Science , Engineering , or related technical field.
• 4–6 year’s experience designing and building cloud-based data pipelines and distributed systems .
• Proficiency in at least one core language: Python , Java , or Scala .
• Familiar with lakehouse formats (Iceberg, Delta, Hudi), file formats (Parquet, ORC, Avro), and streaming platforms (Kafka, Kinesis).
• Strong understanding of distributed systems fundamentals: partitioning , replication , idempotency , consensus protocols .

Engineering & Infrastructure

• 5+ years building distributed systems or production-grade data platforms in the cloud.
• Strong coding proficiency in Python , Java , or Scala , with an emphasis on performance and reliability.
• Expertise in SQL and PLSQL , data modeling, and query optimization.
• Proven experience with cloud-native architectures —especially OCI , AWS, Azure, or GCP.

Lakehouse & Streaming Mastery

• Deep knowledge of modern lakehouse/table formats : Apache Iceberg , Delta Lake , or Apache Hudi .
• Production experience with big data compute engines : Spark , Flink , or Trino .
• Skilled in real-time streaming and event-driven architectures using Kafka , Flink , GoldenGate , or Streaming .
• Experience managing data lakes , catalogs, and metadata governance in large-scale environments.

AI/ML Integration

• Hands-on experience enabling ML pipelines : from data ingestion to model training and deployment.
• Familiarity with ML frameworks (e.g., PyTorch , XGBoost , scikit-learn ).
• Understanding of modern ML architectures : including RAG , prompt chaining , and agent-based workflows .
• Awareness of MLOps practices , including model versioning, feature stores, and integration with AI pipelines.

️ DevOps & Operational Excellence

• Deep understanding of CI/CD , infrastructure-as-code (IaC), and release automation using tools like Terraform , GitHub Actions , or CloudFormation .
• Experience with Docker , Kubernetes , and cloud-native container orchestration .
• Strong focus on testing, documentation , and system observability (Prometheus, Grafana, ELK stack).
• Comfortable with cost/performance tuning , incident response, and data security standards (IAM, encryption, auditing).
  

Preferred Qualifications

• Experience with Oracle’s cloud-native tools : OCI Data Integration , Data Flow , Autonomous Database , GoldenGate , OCI Streaming .
• Experience with query engines like Trino or Presto , and tools like dbt or Apache Airflow .
• Familiarity with data cataloging , RBAC/ABAC , and enterprise data governance frameworks.
• Exposure to vector databases and LLM tooling (embeddings, vector search, prompt orchestration).
• Solid understanding of data warehouse design principles , star/snowflake schemas, and ETL optimization.
  

Soft Skills & Team Expectations

• Proven ability to lead technical initiatives independently end-to-end .
• Comfortable working in cross-functional teams and mentoring junior engineers.
• Excellent problem-solving skills , design thinking, and attention to operational excellence.
• Passion for learning emerging data and AI technologies and sharing knowledge across teams.