Senior Data Engineer

2 weeks ago

Vijayawada, India Josys Full time

Location: Bengaluru, Karnataka, India


About The Role

We're looking for an experienced Senior Data Engineer (6-8 years) to join our data team. You'll be key in building and maintaining our data systems on AWS. You'll use your strong skills in big data tools and cloud technology to help our analytics team get valuable insights from our data. You'll be in charge of the whole process of our data pipelines, making sure the data is good, reliable, and fast.


What You'll Do

  • Design and build efficient data pipelines using Spark / PySpark / Scala.
  • Manage complex data processes with Airflow, creating and fixing any issues with the workflows (DAGs).
  • Clean, transform, and prepare data for analysis.
  • Use Python for data tasks, automation, and building tools.
  • Work with AWS services like S3, Redshift, EMR, Glue, and Athena to manage our data infrastructure.
  • Collaborate closely with the Analytics team to understand what data they need and provide solutions.
  • Help develop and maintain our Node.js backend, using Typescript, for data services.
  • Use YAML to manage the settings for our data tools.
  • Set up and manage automated deployment processes (CI/CD) using GitHub Actions.
  • Monitor and fix problems in our data pipelines to keep them running smoothly.
  • Implement checks to ensure our data is accurate and consistent.
  • Help design and build data warehouses and data lakes.
  • Use SQL extensively to query and work with data in different systems.
  • Work with streaming data using technologies like Kafka for real-time data processing.
  • Stay updated on the latest data engineering technologies.
  • Guide and mentor junior data engineers.
  • Help create data management rules and procedures.


What You'll Need

  • Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
  • 6-8 years of experience as a Data Engineer.
  • Strong skills in Spark and Scala for handling large amounts of data.
  • Good experience with Airflow for managing data workflows and understanding DAGs.
  • Solid understanding of how to transform and prepare data.
  • Strong programming skills in Python for data tasks and automation..
  • Proven experience working with AWS cloud services (S3, Redshift, EMR, Glue, IAM, EC2, and Athena).
  • Experience building data solutions for Analytics teams.
  • Familiarity with Node.js for backend development.
  • Experience with Typescript for backend development is a plus.
  • Experience using YAML for configuration management.
  • Hands-on experience with GitHub Actions for automated deployment (CI/CD).
  • Good understanding of data warehousing concepts.
  • Strong database skills - OLAP/OLTP
  • Excellent command of SQL for data querying and manipulation.
  • Experience with stream processing using Kafka or similar technologies.
  • Excellent problem-solving, analytical, and communication skills.
  • Ability to work well independently and as part of a team.


Bonus Points

  • Familiarity with data lake technologies (e.g., Delta Lake, Apache Iceberg).
  • Experience with other stream processing technologies (e.g., Flink, Kinesis).
  • Knowledge of data management, data quality, statistics and data governance frameworks.
  • Experience with tools for managing infrastructure as code (e.g., Terraform).
  • Familiarity with container technologies (e.g., Docker, Kubernetes).
  • Experience with monitoring and logging tools (e.g., Prometheus, Grafana).

  • Senior Big Data Engineer

    3 weeks ago

    Vijayawada, Andhra Pradesh, India Veltris Full time

    Veltris is a Digital Product Engineering Services partner committed to driving technology-enabled transformation across enterprises, businesses, and industries. We specialize in delivering next-generation solutions for sectors including healthcare, technology, communications, manufacturing, and finance. With a focus on innovation and acceleration, Veltris...

  • Digital Marketing Associate

    3 weeks ago

    Vijayawada, India Political Engineer Full time

    Company Description Technical Avengers Pvt. Ltd. specializes in precision-driven strategies, visually-striking branding, and high-performance digital campaigns. Focused on delivering measurable ROI, we ensure clarity, speed, trust, and result-oriented execution. Every solution is tailored, data-backed, and aimed at amplifying reach, boosting engagement, and...

  • Junior AI

    2 weeks ago

    Vijayawada, India Omni Reach Full time

    Job DescriptionThis is a remote position.Role OverviewWe are hiring smart, analytical freshers and early-career engineers passionate about data, mathematics, problem-solving, and building systems. This is a developmental role aimed at nurturing strong fundamentals in Data Science, Machine Learning, and Data Engineering over time. You will work on real-world...

  • Senior Planning Engineer

    6 days ago

    Vijayawada, Andhra Pradesh, India Origin Build Group Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Role & responsibilitiesThis is a full-time on-site role for a Senior Planning Engineer based in Vijayawada. The Senior Planning Engineer will be responsible for planning and coordinating project activities, managing construction planning, and overseeing project control. The role involves analyzing project specifications, ensuring resource allocation, and...

  • Senior Test Engineer

    2 weeks ago

    Vijayawada, India HCLTech Full time

    Job Title/Role - Senior Test Engineer Experience Required - 5-10 + years Mandatory Skills - Railroad Tester, API Testing,AI Force,Qtest, Selenium Job description Proficiency in test automation using tools like Selenium, JMeter, Postman, and PEGA Unit Testing, integrated with CI/CD pipelines Experience in API and microservices testing, including contract...

  • Senior Test Engineer

    2 weeks ago

    Vijayawada, India HCLTech Full time

    Job Title/Role - Senior Test EngineerExperience Required - 5-10 + yearsMandatory Skills - Railroad Tester, API Testing,AI Force,Qtest, SeleniumJob descriptionProficiency in test automation using tools like Selenium, JMeter, Postman, and PEGA Unit Testing, integrated with CI/CD pipelinesExperience in API and microservices testing, including contract...

  • Senior Test Engineer

    2 weeks ago

    Vijayawada, India HCLTech Full time

    Job Title/Role - Senior Test EngineerExperience Required - 5-10 + yearsMandatory Skills - Railroad Tester, API Testing,AI Force,Qtest, Selenium Job descriptionProficiency in test automation using tools like Selenium, JMeter, Postman, and PEGA Unit Testing, integrated with CI/CD pipelinesExperience in API and microservices testing, including contract...

  • Senior Test Engineer

    2 weeks ago

    Vijayawada, India HCLTech Full time

    Job Title/Role - Senior Test Engineer Experience Required - 5-10 + years Mandatory Skills - Railroad Tester, API Testing,AI Force,Qtest, Selenium Job description Proficiency in test automation using tools like Selenium, JMeter, Postman, and PEGA Unit Testing, integrated with CI/CD pipelines Experience in API and microservices testing, including contract...

  • Senior Test Engineer

    2 weeks ago

    Vijayawada, India HCLTech Full time

    Job Title/Role - Senior Test EngineerExperience Required - 5-10 + yearsMandatory Skills - Railroad Tester, API Testing,AI Force,Qtest, Selenium Job descriptionProficiency in test automation using tools like Selenium, JMeter, Postman, and PEGA Unit Testing, integrated with CI/CD pipelinesExperience in API and microservices testing, including contract...

  • Senior Test Engineer

    2 weeks ago

    Vijayawada, India HCLTech Full time

    Job Title/Role - Senior Test EngineerExperience Required - 5-10 + yearsMandatory Skills - Railroad Tester, API Testing,AI Force,Qtest, Selenium Job descriptionProficiency in test automation using tools like Selenium, JMeter, Postman, and PEGA Unit Testing, integrated with CI/CD pipelinesExperience in API and microservices testing, including contract...