Data Engineer

Overview:

OmniTrust Technologies, based in Alpharetta, GA, is an IT services firm delivering technology solutions to clients across all 50 states in the United States. We specialize in identifying and connecting top talent with the right organizations, enabling businesses to succeed in today’s competitive market.

Our expertise spans cutting-edge technologies including Artificial Intelligence, Machine Learning, Java, data warehousing, cloud computing, and build engineering. As a global provider of IT staffing solutions, software training and development, career advisory services, and consulting, we offer a comprehensive suite of services to both individuals and enterprises.

Driven by a strong commitment to our clients, we continuously seek exceptional talent to meet evolving business needs. As part of this mission, we are currently offering an exciting job opportunity.

Position Overview :

We are seeking a highly skilled Senior Data Engineer with strong experience in AI-powered systems, real-time data streaming, and multi-cloud architecture (AWS & GCP). The ideal candidate will have expertise in building scalable data pipelines, cloud-native microservices, and analytics platforms that support high-volume workloads and real-time insights.

This role requires hands-on experience with distributed systems, modern data architecture (Medallion Architecture), and DevOps-driven deployments.

Key Responsibilities :

• Design and implement scalable data pipelines for batch and real-time processing using Kafka, Spark Streaming, and Airflow

• Develop AI-powered APIs leveraging LLMs to unify and analyze multi-cloud data sources

• Architect and deploy cloud-native microservices using Spring Boot, Flask, and REST APIs

• Build and optimize data warehouse solutions using Snowflake, Redshift, BigQuery, and PostgreSQL

• Implement Medallion Architecture (Bronze-Silver-Gold) for structured data processing

• Develop streaming architectures using AWS Kinesis and Glue

• Optimize distributed systems for performance, cost-efficiency, and scalability

• Implement CDC pipelines and incremental loading strategies

• Deploy containerized applications using Docker and Kubernetes

• Build CI/CD pipelines using Jenkins and Terraform

• Ensure security through OAuth2, JWT authentication, and RBAC implementations

Required Technical Skills

Programming :

• Python, Java, TypeScript

• Linux Shell scripting

Cloud & Data Platforms

• AWS (Kinesis, Glue, S3, Redshift, RDS)

• GCP (BigQuery)

• Databricks

Databases & Warehousing

• Snowflake, Redshift, PostgreSQL, MySQL

• MongoDB, Cassandra, DynamoDB

Data Engineering & Streaming

• Kafka, Spark Streaming

• Airflow

• Medallion Architecture

• CDC (Change Data Capture)

Infrastructure & DevOps

• Docker, Kubernetes

• CI/CD pipelines (Jenkins)

• Terraform

• Monitoring with Prometheus & Grafana

Web Technologies

• React.js, Next.js, JavaScript, HTML, CSS

Preferred Qualifications

• Experience building AI-powered analytics systems using LLM APIs

• Strong understanding of distributed system design

• Experience working with multi-cloud cost analytics platforms

• Ability to optimize infrastructure costs while improving scalability

• Experience delivering 99.9% uptime production systems

Attributes:

We are in search of a candidate who embodies passion, intelligence, and critical thinking skills. Our ideal candidate is a proactive communicator who can clearly and concisely document their work. They should possess a keen attention to detail, thoughtfulness, and respect for others, all while maintaining a strong focus on teamwork. Strong problem-solving skills and the ability to work both independently and within a team are crucial. Additionally, the candidate should be able to adapt to shifting requirements and maintain a positive attitude in a fast-paced environment.

What we Offer:

The company offers a comprehensive compensation package that includes competitive pay, bonus potential, and a range of benefits such as group health insurance, dental and vision coverage, and paid time off for vacation.