Senior Data Engineer - Product
- Anywhere in the World
Headquarters: Portugal Remote
The Engineering (Tech) Team is responsible for all Feedzai product development. Together with Product Management and Data Science, we build the next generation of tools to catch fraud in real-time with a machine learning first approach. Formed by engineers and managed by engineers, at Feedzai, you will find one of the most talented teams out there, from junior to senior engineers.We are fast-paced and provide a safe, open, and collaborative environment that encourages us to lean in, try new things and discover our potential with continuous learning for everyone.While building the best value for our customers, you will work with a wide range of technical challenges. Such as building distributed systems that need to operate 24/7 and ultra-low latencies, solving UI/UX problems to help fraud analysts to fight fraud more efficiently. In addition, designing extensive databases from relational, NoSQL and graphs, validate and develop new data science techniques and algorithms.Feedzai’s Pulse Engineering organization powers the Risk Engine that evaluates transactions in real-time based on strategies configured by data scientists and fraud analysts. At the same time, the Datascience Framework (DSF) provides a platform for clients to design, test, and promote these strategies into production. DSF runs complex, high-volume data workloads - Spark jobs on EMR or Kubernetes, Hadoop ecosystem components (HDFS/YARN), data ingestion pipelines via Firehose and Glue into S3, and interactive workflows through JupyterLabs and the DS API.You:As a Senior Data Engineer in Pulse Engineering, you will ensure the stability, scalability, and performance of DSF, working at the intersection of distributed systems, big data engineering, and developer experience. Your work will ensure that DS workloads run reliably, efficiently, and at scale, directly supporting the Risk Engine in production.You are a Big Data specialist with deep hands-on expertise in Apache Spark and distributed data systems. You know how to tune jobs, troubleshoot cluster behaviour, and design scalable data workflows. At the same time, you’re a software engineer at heart — someone who writes clean, maintainable code, understands APIs, and can build platform components in Java with first-principles engineering discipline. You take pride in operating what you build, debugging complex distributed systems, and enabling data scientists and analysts with a platform that is reliable, predictable, and performant.Our philosophy across Engineering includes the following key ideas, which you will have a key role promoting in your team: Teams operate with a "you build it, you run it" DevOps mindset, taking end-to-end ownership of development, deployment, and operations. You'll drive a culture focused on automation, observability, and operational excellence, enabling continuous delivery with confidence.Architecture evolves towards a decoupled, microservice-based architecture, positioning it to scale efficiently in a multi-tenant, cloud-native environment. This effort is central to Feedzai’s long-term vision and product evolution Your Day-to-DayRe-architect and scale existing big data processing components powering DSF.Analyse workload patterns (Spark jobs, notebook activity, DS API usage) and drive performance, reliability, and cost improvements.Ensure stability of Spark jobs running on EMR or Kubernetes clusters.Operate and evolve Hadoop ecosystem components (HDFS, YARN) and Spark runtimes.Maintain and improve ingestion pipelines between Runtime and DSF (Firehose, Glue → S3).Improve the developer and power-user experience across JupyterLabs and the DS API.Collaborate with product engineers, data scientists, and platform teams on DSF roadmap execution.Own services throughout their lifecycle following DevOps practices ("you build it, you run it"). You Have & You Know-How5+ years of experience building and operating distributed big data systemsStrong experience with Apache Spark - tuning, debugging, orchestrationStrong programming fundamentals (Java required; Scala or Python a plus)Solid knowledge of the Hadoop ecosystem (HDFS, YARN)Experience operating Linux-based systems in cloud environmentsFamiliarity with JupyterHub/JupyterLabs workflowsExperience designing and operating ETL/ELT pipelinesComfort with continuous delivery, monitoring, and on-call responsibilitiesAbility to work autonomously on complex technical challengesPreferred/Valued Qualifications and Skills: Experience running Spark on EMR or Kubernetes.Experience with Kubernetes, AWS S3, AWS Glue, Nessie, Trino, Spark, Kafka, Iceberg and AirflowExperience building data pipelines or automation for data-heavy workflowsContributions to OSS, especially in the Big Data ecosystemExperience developing or maintaining DS/ML engineering platforms.Feedzai is an engineering company committed to growing with high standards in building its products. As an engineering manager, you will help to guarantee that this engineering culture persists at management levels combining both technical leadership and people leadership.The Product Team builds our product to disrupt the financial crime industry from a data-led approach. We partner with our clients using a holistic lens and have result-driven solutions to manage financial risk with a cloud-first platform and a world-class UX interface. Being part of this team, you have a voice in planning, strategizing, and challenging the status quo. Your thoughts and ideas are valued. Our fast-paced and open environment encourages us to lean in, try new things, and discover our potential. We define and act on what could be in tomorrow's world, not on what is today. Join Us! #LI-Remote #LI-MG3To apply: https://weworkremotely.com/remote-jobs/feedzai-senior-data-engineer-product