fahad albahar

Clearwater

<div>EXPERIENCE: 5&ndash;10 Years
Core Responsibilities:<ul><li>Pipeline Management: Maintain high-throughput streaming pipelines to ingest logs from various sources (Firewalls, Cloud, Endpoints) to a central destination.</li></ul>
&bull; Log Normalization: Write parsers to convert raw, messy logs into standard schemas (e.g., OCSF or ECS) for consistent querying. &bull; Cost Optimization: Implement routing logic to send &quot;high-value&quot; data to the SIEM and &quot;bulk&quot; data to low-cost Object Storage (Data Lake).<ul><li>Data Preparation: Clean and structure data to enable AI/ML detection models and advanced analytics.</li></ul>
Must-Have Skills:<ul><li>Data Engineering: Proficiency in Python (for ETL) and SQL (for complex querying).</li><li>Streaming Tech: Experience with Message Queues (e.g., Kafka, Pub/Sub) and stream processing concepts.</li><li>Log Handling: Mastery of Regex and log parsing strategies for standard formats (Syslog, CEF, JSON).</li><li>Storage Architecture: Understanding of Data Lake principles (Parquet/Avro formats) vs. Data Warehouses.</li></ul>
Preferred / Nice to Have:<ul><li>Experience with Vector Databases for storing embeddings.</li><li>Knowledge of Log Observability/Routing tools (middleware that routes logs).</li><li>Familiarity with Big Data frameworks (e.g., Spark, Flink).</li></ul></div>

Senior Security Data Engineer

Join Clearwater and explore 1   exciting career opportunities. They're hiring for various roles with options for remote or on-site work. Apply today and become part of their innovative team.

Clearwater

About Company

Senior Security Data Engineer