Big Data DevOps / Data Platform Engineer

We are a global algorithmic trading firm with an advanced technology stack, driving the full trading cycle — from market research and strategy design to algorithm development and software engineering. Our strength lies in deep market expertise and the continuous evolution of our infrastructure to stay ahead of the industry.

We are looking for a BigData DevOps Engineer to strengthen our Data Engineering team. This role is focused on building and maintaining reliable and scalable data infrastructure that powers our trading research and production systems.

Responsibilities

• Build and operate our Kubernetes (EKS) data platform: cluster provisioning with Terraform, node autoscaling with Karpenter.
• Operate and scale Airflow on Kubernetes and Spark on EKS.
• Build and manage monitoring and observability (Prometheus, Grafana, Loki) for data quality (DBT), pipelines, and storage systems.
• Design and maintain AWS data infrastructure: S3 lifecycle and layout, VPC networking, storage cost optimization.
• Work closely with Data Engineers to ensure smooth deployment, versioning, and scaling of data jobs and pipelines.
• Automate deployment and CI/CD processes with GitLab CI/CD.
• Ensure security best practices for sensitive data handling.
• Integrate and support Project Nessie as a transactional catalog for Data Lakes with Git-like semantics.
• Write infrastructure automation using Python/Bash/Ansible.
• Document systems, pipelines, and DevOps processes.

Qualifications:

• 5+ years of experience in DevOps/IT Ops/DataOps roles, with hands-on experience in data-related infrastructure.
• Experience with real-time data streaming, DBT, Project Nessie, Apache Spark.
• Strong hands-on Kubernetes administration experience: provisioning, upgrades, troubleshooting, etc.
• Hands-on experience operating Amazon EKS in production.
• Hands-on experience with ClickHouse.
• Solid knowledge of cloud services (AWS/Azure/GCP).
• Proficiency in scripting and automation (Python, Bash, Ansible).
• Experience with GitOps and ArgoCD: Helm, app-of-apps, debugging OutOfSync/drift.
• Experience with Karpenter (or Cluster Autoscaler).
• Strong knowledge of Linux (networking, kernel tuning, performance optimization).
• Understanding of data governance concepts.
• Strong communication skills and a proactive, ownership-driven mindset.
• English level B2 or higher.

Nice to have:

• Strong knowledge of AWS services (S3, EC2, IAM, networking).
• Understanding of security in data engineering (IAM, encryption, access control).
• Experience with high-performance systems or in trading environments.
• Experience with CloudNativePG (Postgres Kubernetes operator).
• Experience with Trino or other distributed SQL query engines over data lakes.
• Familiarity with DBT and data quality tooling/monitoring.
• Keycloak or similar identity and access management systems.

What we offer:

• Remote-first: work from anywhere.
• Tech-driven environment: no bureaucracy, legacy systems, or tech debt.
• Growth & development: work with top talent in a fast-paced, innovation-driven culture.
• Perks: medical insurance, sports benefits, home office setup, and educational support.