Databricks Certified Data Engineer /AWS Solutions Architect

Job Overview -
We are seeking a dynamic and highly skilled Databricks Certified Data Engineer, combined with AWS Solutions Architect expertise, to join our innovative data team. In this role, you will lead the design, development, and deployment of scalable data solutions leveraging cloud platforms and big data technologies. Your expertise will drive the transformation of complex data into actionable insights, enabling strategic decision-making across the organization. If you thrive in a fast-paced environment and are passionate about building robust data architectures, this is your opportunity to make a significant impact!

Duties

• Design, develop, and optimize large-scale data pipelines using Databricks, Spark, Hadoop, and other big data frameworks to support analytics and reporting needs.
• Architect and implement cloud-based solutions on AWS, including services such as S3, EC2, Lambda, and Redshift, ensuring high availability and security.
• Collaborate with cross-functional teams to gather requirements and translate them into scalable data models and ETL workflows utilizing tools like Informatica, Talend, and custom Python or Bash scripts.
• Develop and maintain data warehouses and data lakes on platforms such as Azure Data Lake and Azure Synapse Analytics for efficient data storage and retrieval.
• Build RESTful APIs for seamless integration between systems while ensuring compliance with security standards.
• Conduct analysis of large datasets to identify trends, anomalies, or opportunities for process improvements; support model training efforts for advanced analytics initiatives.
• Maintain comprehensive documentation of architecture designs, workflows, and best practices aligned with Agile methodologies.

Skills

• Extensive experience with AWS cloud services, including S3, EC2, Lambda, Redshift, and related tools.
• Strong proficiency in Java, Python, Shell Scripting (Bash), VBA, and SQL for developing scalable data solutions.
• Deep understanding of big data ecosystems such as Hadoop, Apache Hive, Spark, and related technologies.
• Proven ability to design and optimize data warehouses using Microsoft SQL Server, Oracle databases, or similar platforms.
• Knowledge of Looker or other BI tools for creating interactive dashboards and reports.
• Familiarity with ETL processes using Talend or Informatica; experience with RESTful API development is a plus.
• Solid understanding of database design principles along with analysis skills to interpret complex datasets effectively.
• Experience working within Agile teams; strong communication skills to collaborate effectively across departments.
• Additional skills include familiarity with Linked Data concepts, analysis techniques for model training, shell scripting in Unix environments, and expertise in analytics-driven decision-making. Join us to leverage your expertise in cloud architecture and big data engineering—helping us unlock the full potential of our data assets while advancing your career in a vibrant environment dedicated to innovation!
• Hybrid 2-3 in office
• US Citizen required, Public Trust

Pay: $75.00 - $85.00 per hour

Work Location: Hybrid remote in Washington, DC 20006