DevOps with Airflow and Apache

4 months ago


Pune Maharashtra, India Serendipity Corporate Services Full time

At least 6+ years in web services development and solid understanding of web technologies in JAVA
- Good understanding of Apache Airflow Workflow Management Platform - Must have experience in managing, monitoring and troubleshooting Airflow in production environments.
- Strong knowledge of Python
- Strong expertise with Docker/Kubernetes
- Strong Expertise of writing JUnits & How to configure it through maven
- Working knowledge of building Micro Services, RESTful web Services using any framework (Spring Boot, JaxRS, Jersey)
- Good understanding of NoSQL databases and have worked upon any one of them (HBase, Cassandra, Big query, Mongo)
- Good understanding of Message Queues and have worked upon any one of them (Azure Service Bus, Kafka, RabbitMQ, Pub Sub)
- Good understanding of Maven, Git
- Good understanding of Jenkins/ Azure DevOps for CI/CD architecture
- Good understanding of Programming Algorithms and Data Structures
- Experience with BDD & Cucumber
- **Good to Have**:_
- Monitoring experience - Stack driver or Prometheus or Azure equivalent
- Operational readiness - SLI/SLO, DevOps experience
- Service mesh - ex: Istio
- Knowledge on Graph technology
- Have worked upon any of the big data technologies
- Have worked upon tools like white source, Veracode...
- Knowledge of Angular
- Good understanding of Streaming technologies and processing engines (Dataflow, Flink, Spark)
- Knowledge of Azure DevOps

**Salary**: Up to ₹2,095,535.58 per year

**Benefits**:

- Health insurance

Schedule:

- Day shift

Supplemental pay types:

- Performance bonus

Work Location: In person

**Speak with the employer**
+91 7350971253


  • Apache Spark Expert

    4 months ago


    Pune, India Info Origin Full time

    Position Title: - Apache Spark Expert**Location**: - Pune, MH, OnsitePosition Type: - Full-TimeInterview Process: - Webcam/SkypeReq Id: - 3314Salary/Rate: - MarketPosition Duration: - Permanent**Key Responsibilities**: - Provide architectural consultancy to clients and develop implementation roadmaps for Apache Spark and Kafka solutions. - Translate...


  • Pune, India Attentions AI Full time

    Greetings From Attentions.aiJob Description for Senior DevOps Engineer.Position Title:DevOps EngineerLocation:Pune, IndiaDepartment:Software DevelopmentEmployment Type:Full-timeCompany Overview:Attentions is a cutting-edge generative AI company dedicated to developing innovative products across all modalities, including text, vision, audio, and more. Our...


  • pune, India EXL Full time

    Position - Senior DevOps EngineerLocation - PuneExperience - 5+ YearsRoles & Responsibilities:Responsible for building and managing our cloud infrastructure (AWS) with code first mindset.Design and architect ETL/ELT pipelines running on AWS EMR and SnowflakePerformance tune data pipelines using technologies such as Apache Spark, Hive and TrinoWrite...


  • Pune H.O, India EXL Full time

    Position - Senior DevOps EngineerLocation - PuneExperience - 5+ Years Roles & Responsibilities:Responsible for building and managing our cloud infrastructure (AWS) with code first mindset.Design and architect ETL/ELT pipelines running on AWS EMR and SnowflakePerformance tune data pipelines using technologies such as Apache Spark, Hive and TrinoWrite and...


  • Pune, India EXL Full time

    Position - Senior DevOps EngineerLocation - PuneExperience - 5+ YearsRoles & Responsibilities:Responsible for building and managing our cloud infrastructure (AWS) with code first mindset.Design and architect ETL/ELT pipelines running on AWS EMR and SnowflakePerformance tune data pipelines using technologies such as Apache Spark, Hive and TrinoWrite...

  • Senior Data Engineer

    2 months ago


    Pune, India MeedenLabs Full time

    Job Description- Write and optimize complex database queries and views to ensure efficient data retrieval and processing.- Develop and maintain Java APIs for data integration and processing tasks.- Utilize Apache Beam for building data processing pipelines.Big Data and Stream Processing:- Gain exposure to Apache Flink for stream processing and real-time data...

  • DevOps

    2 weeks ago


    Pune, Maharashtra, India Capgemini Full time

    **Migration exp from Apache Kafka to Confluent Kafka**: - **Devops and Kubernets**: - **Design, deploy, and maintain Kafka clusters across multiple environments (development, testing, production).**: - **Configure Kafka brokers, topics, partitions, and ACLs (Access Control Lists) as per security requirements.**: - **Monitor Kafka infrastructure...


  • Pune, India EXL Full time

    Position - Senior DevOps EngineerLocation - PuneExperience - 5+ YearsRoles & Responsibilities:Responsible for building and managing our cloud infrastructure(AWS)with code first mindset.Design and architectETL/ELT pipelinesrunning onAWS EMRandSnowflakePerformance tune data pipelines using technologies such asApache Spark, HiveandTrinoWrite and maintain...

  • DevOps

    1 month ago


    Pune, Maharashtra, India Capgemini Full time

    Migration exp from Apache Kafka to Confluent Kafka - 5+years - Devops, Kubernetes - Design, deploy, and maintain Kafka clusters across multiple environments (development, testing, production). - Configure Kafka brokers, topics, partitions, and ACLs (Access Control Lists) as per security requirements. - Monitor Kafka infrastructure performance and ensure high...


  • pune, India Attentions AI Full time

    Greetings From Attentions.aiJob Description for Senior DevOps Engineer.Position Title: DevOps EngineerLocation: Pune, IndiaDepartment: Software DevelopmentEmployment Type: Full-timeCompany Overview: Attentions is a cutting-edge generative AI company dedicated to developing innovative products across all modalities, including text, vision, audio, and more....


  • Pune, India Attentions AI Full time

    Greetings From Attentions.aiJob Description for Senior DevOps Engineer.Position Title: DevOps EngineerLocation: Pune, IndiaDepartment: Software DevelopmentEmployment Type: Full-timeCompany Overview: Attentions is a cutting-edge generative AI company dedicated to developing innovative products across all modalities, including text, vision, audio, and more....


  • pune, India Attentions AI Full time

    Greetings From Attentions.ai Job Description for Senior DevOps Engineer. Position Title: DevOps Engineer Location: Pune, India Department: Software Development Employment Type: Full-time Company Overview: Attentions is a cutting-edge generative AI company dedicated to developing innovative products across all modalities, including text, vision,...


  • Pune, India Attentions AI Full time

    Greetings From Attentions.aiJob Description for Senior DevOps Engineer.Position Title: DevOps EngineerLocation: Pune, IndiaDepartment: Software DevelopmentEmployment Type: Full-timeCompany Overview: Attentions is a cutting-edge generative AI company dedicated to developing innovative products across all modalities, including text, vision, audio, and more....


  • Pune, India Natobotics Technologies Private Limited Full time

    Loc : Pune/banglore/hyderabad exp:4+ years, The NiFi Engineer will be responsible for the implementation, support, documentation, and maintenance of all aspects of the project's Apache NiFi infrastructure. RESPONSIBILITIES: Designing and building data flows in Apache NiFi (including data movement, tagging, routing, and some...


  • Pune, India Natobotics Technologies Private Limited Full time

    Loc : Pune/banglore exp:4+ years, The NiFi Engineer will be responsible for the implementation, support, documentation, and maintenance of all aspects of the project's Apache NiFi infrastructure. RESPONSIBILITIES: Designing and building data flows in Apache NiFi (including data movement, tagging, routing, and some transformation/conversion). Recurring...

  • Data Engineer

    4 months ago


    Pune, India HyrEzy Talent Solutions Full time

    Job Description and Responsibilities :Solicit business, functional, non-functional and technical requirements through interviewing and requirements gathering process.Analyze and document above requirements and data definitions, perform data analysis, assist in change management, training and testing efforts.Works with stakeholders to gather requirements on...


  • pune, India Capco Full time

    Data EngineerExperience ; 6+Location : Pune/Hyderabad (Hybrid)Notice Period : Immediate to 30 DaysMust Haves:Hands-on experience with Data Engineering (Spark, Scala, Hadoop and scheduling services like Apache Airflow).Hands on experience or understanding on DevOps related to Jenkins CI/CD.Hands on experience in building microservices (Java/SpringBoot...

  • Senior Data Engineer

    3 months ago


    Pune, India Capco Full time

    Data EngineerExperience ; 6+Location : Pune/Hyderabad (Hybrid)Notice Period : Immediate to 30 DaysMust Haves:Hands-on experience with Data Engineering (Spark, Scala, Hadoop and scheduling services like Apache Airflow).Hands on experience or understanding on DevOps related to Jenkins CI/CD.Hands on experience in building microservices (Java/SpringBoot...


  • Pune, India EXL Full time

    Position - Senior DevOps Engineer Location - Pune Experience - 5+ Years Roles & Responsibilities: Responsible for building and managing our cloud infrastructure (AWS) with code first mindset. Design and architect ETL/ELT pipelines running on AWS EMR and Snowflake Performance tune data pipelines using technologies such as Apache Spark, Hive and...


  • pune, India EXL Full time

    Position - Senior DevOps EngineerLocation - PuneExperience - 5+ Years Roles & Responsibilities:Responsible for building and managing our cloud infrastructure (AWS) with code first mindset.Design and architect ETL/ELT pipelines running on AWS EMR and SnowflakePerformance tune data pipelines using technologies such as Apache Spark, Hive and TrinoWrite and...