Data Engineer

Sonatype is the software supply chain security company. We provide the world's best end-to-end software supply chain security solution, combining the only proactive protection against malicious open source, the only enterprise grade SBOM management and the leading open source dependency management platform. This empowers enterprises to create and maintain secure, quality, and innovative software at scale.

As founders of Nexus Repository and stewards of Maven Central, the world's largest repository of Java open-source software, we are software pioneers and our open source expertise is unmatched. We empower innovation with an unparalleled commitment to build faster, safer software and harness AI and data intelligence to mitigate risk, maximize efficiencies, and drive powerful software development.

More than 2,000 organizations, including 70% of the Fortune 100 and 15 million software developers, rely on Sonatype to optimize their software supply chains.

• At Sonatype, we help organizations build better, more secure software by enabling them to understand and control their software supply chains. Our products are trusted by thousands of engineering teams globally, providing critical insights into dependency health, license risk, and software security. We're passionate about empowering developers—and we back it with data.

• We're looking for a Data Engineer with full stack expertise to join our growing Data Platform team.
• This role blends data engineering, microservices, and fullstack development to deliver end-to-end services that power analytics, machine learning, and advanced search across Sonatype.
• You will design and build data-driven microservices and workflows using Java, Python, and Spring Batch, implement frontends for data workflows, and deploy everything through CI/CD pipelines into AWS ECS/Fargate. Y
• ou'll also ensure services are monitorable, debuggable, and reliable at scale, while clearly documenting designs with Mermaid-based sequence and dataflow diagrams.
• This is a hands-on engineering role for someone who thrives at the intersection of data systems, fullstack development, ML, and cloud-native platforms.

• Design, build, and maintain data pipelines, ETL/ELT workflows, and scalable microservices.
• Development of complex web scraping (playwright) and realtime pipelines (Kafka/Queues/Flink)Develop end-to-end microservices with backend (Java 5+, Python 5+, Spring Batch 2+) and frontend (React or any).
• Deploy, publish, and operate services in AWS ECS/Fargate using CI/CD pipelines (Jenkins, GitOps).
• Architect and optimize data storage models in SQL (MySQL, PostgreSQL) and NoSQL stores.
• Implement web scraping and external data ingestion pipelines.
• Enable Databricks and PySpark-based workflows for large-scale analytics.
• Build advanced data search capabilities (fuzzy matching, vector similarity search, semantic retrieval).
• Apply ML techniques (scikit-learn, classification algorithms, predictive modeling) to data-driven solutions.
• Implement observability, debugging, monitoring, and alerting for deployed services.
• Create Mermaid sequence diagrams, flowcharts, and dataflow diagrams to document system architecture and workflows.
• Drive best practices in fullstack data service development, including architecture, testing, and documentation.

• 2+ years of experience as a Data Engineer or a Software Backend engineering role.
• Strong programming skills in Python, Scala, or JavaHands-on experience with HBase or similar NoSQL columnar stores.
• Hands-on experience with distributed data systems like Spark, Kafka, or Flink.
• Proficient in writing complex SQL and optimizing queries for performance.
• Experience building and maintaining robust ETL/ELT pipelines in production.
• Familiarity with workflow orchestration tools (Airflow, Dagster, or similar).
• Understanding of data modeling techniques (star schema, dimensional modeling, etc.).
• Familiarity with CI/CD pipelines (Jenkins or similar)Ability to visualize and communicate architectures using Mermaid diagrams.

• Experience working with Databricks, dbt, Terraform, or Kubernetes.
• Familiarity with streaming data pipelines or real-time processing.
• Exposure to data governance frameworks and tools.
• Experience supporting data products or ML pipelines in production.
• Strong understanding of data privacy, security, and compliance best practices

• Data with purpose: Work on problems that directly impact how the world builds secure software
• Modern tooling: Leverage the best of open-source and cloud-native technologies
• Collaborative culture: Join a passionate team that values learning, autonomy, and impact

At Sonatype, we value diversity and inclusivity. We offer perks such as parental leave, diversity and inclusion working groups, and flexible working practices to allow our employees to show up as their whole selves. We are an equal-opportunity employer, and we do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. If you have a disability or special need that requires accommodation, please do not hesitate to let us know.