Astuto Technologies
2 weeks ago
We are seeking a highly skilled and experienced Staff Engineer - Data Pipelines to join our data engineering team. The ideal candidate will have a strong background in designing, building, and maintaining data pipelines, with expertise in setting up and optimizing data processing frameworks such as Apache Spark or similar, Kafka/Pulsar, and Data Lake.
Responsibilities:
- Design and develop robust, scalable, and efficient data pipelines to collect, process, and store large volumes of data from various sources.
- Collaborate with data scientists, data analysts, and other teams to understand data requirements and supply data solutions.
- Set up and configure Apache Spark or similar for stream processing, batch processing, and real-time analytics.
- Manage and perfect data streaming and messaging platforms such as Kafka and Pulsar for real-time data ingestion.
- Work on data quality monitoring and ensure data accuracy, consistency, and reliability.
- Collaborate with cross-functional teams to troubleshoot and resolve data pipeline issues.
- Keep abreast of industry trends and emerging technologies related to data pipelines and recommend improvements.
- Bachelor's or higher degree in Computer Science, Data Engineering, or a related field.
- Proven experience (2+ years) in designing and implementing data pipelines in a production environment.
- Strong ability in setting up and configuring Apache Spark, and Kafka/Pulsar for data streaming, transformation, and processing.
- Hands-on experience with AWS services, including S3 Lambda and EMR.
- Solid knowledge of data warehousing concepts and technologies.
- Strong programming skills in languages such as Python or Scala.
- Familiarity with service-oriented architecture and gRPC or similar.
- Familiarity with data modeling, database systems, and SQL.
- Experience with Clickhouse or similar.
- Excellent problem-solving and communication skills.
- Strong team player and ability to collaborate with cross-functional teams.
- Results-oriented and able to work independently with minimal supervision.
- Proactive in staying up to date with industry trends and best practices.
(ref:hirist.tech)
-
Astuto Technologies
3 weeks ago
bangalore, India Astuto Technologies Private Limited Full timeAbout Astuto :We aim to build an AI-first Cloud Efficiency Platform that helps businesses maximize the ROI on public cloud investments. Given the increase in complexity and cost of cloud infrastructure and services, this space has an enormous opportunity. Over $600 billion is being spent on the public cloud, and an estimated $180 billion is being wasted. In...
-
Astuto Technologies
3 weeks ago
Bangalore, India Astuto Technologies Private Limited Full timeAbout Astuto :We aim to build an AI-first Cloud Efficiency Platform that helps businesses maximize the ROI on public cloud investments. Given the increase in complexity and cost of cloud infrastructure and services, this space has an enormous opportunity. Over $600 billion is being spent on the public cloud, and an estimated $180 billion is being wasted. In...
-
Astuto Technologies
2 weeks ago
Bangalore, India Astuto Technologies Private Limited Full timeAbout Astuto :We aim to build an AI-first Cloud Efficiency Platform that helps businesses maximize the ROI on public cloud investments. Given the increase in complexity and cost of cloud infrastructure and services, this space has an enormous opportunity. Over $600 billion is being spent on the public cloud, and an estimated $180 billion is being wasted. In...
-
Astuto Technologies
2 months ago
Bangalore, India Astuto Technologies Private Limited Full timeAbout Astuto :We are a Cloud Management and Governance startup focused on helping businesses run an Efficient, Secure, Compliant, Reliable, and Well-Managed Cloud. We are helping customers in two areas : a) Cloud Governance : Helping businesses with Cloud FinOps, Security, and Reliability Governance.b) Modernization : Helping businesses improve the...
-
Astuto Technologies
3 weeks ago
bangalore, India Astuto Technologies Private Limited Full timeWe are seeking a highly skilled and experienced Staff Engineer - Data Pipelines to join our data engineering team. The ideal candidate will have a strong background in designing, building, and maintaining data pipelines, with expertise in setting up and optimizing data processing frameworks such as Apache Spark or similar, Kafka/Pulsar, and Data...
-
Astuto Technologies
2 weeks ago
Bangalore, Karnataka, India Astuto Technologies Private Limited Full timeWe are seeking a highly skilled and experienced Staff Engineer - Data Pipelines to join our data engineering team. The ideal candidate will have a strong background in designing, building, and maintaining data pipelines, with expertise in setting up and optimizing data processing frameworks such as Apache Spark or similar, Kafka/Pulsar, and Data...
-
Astuto Technologies
2 weeks ago
Bangalore, Karnataka, India Astuto Technologies Private Limited Full timeWe are seeking a highly skilled and experienced Staff Engineer - Data Pipelines to join our data engineering team. The ideal candidate will have a strong background in designing, building, and maintaining data pipelines, with expertise in setting up and optimizing data processing frameworks such as Apache Spark or similar, Kafka/Pulsar, and Data...
