Senior Data Engineer

5 days ago

bangalore district, India Josys Full time

Location: Bengaluru, Karnataka, India

About The Role

We're looking for an experienced Senior Data Engineer (6-8 years) to join our data team. You'll be key in building and maintaining our data systems on AWS. You'll use your strong skills in big data tools and cloud technology to help our analytics team get valuable insights from our data. You'll be in charge of the whole process of our data pipelines, making sure the data is good, reliable, and fast.

What You'll Do

  • Design and build efficient data pipelines using Spark / PySpark / Scala.
  • Manage complex data processes with Airflow, creating and fixing any issues with the workflows (DAGs).
  • Clean, transform, and prepare data for analysis.
  • Use Python for data tasks, automation, and building tools.
  • Work with AWS services like S3, Redshift, EMR, Glue, and Athena to manage our data infrastructure.
  • Collaborate closely with the Analytics team to understand what data they need and provide solutions.
  • Help develop and maintain our Node.js backend, using Typescript, for data services.
  • Use YAML to manage the settings for our data tools.
  • Set up and manage automated deployment processes (CI/CD) using GitHub Actions.
  • Monitor and fix problems in our data pipelines to keep them running smoothly.
  • Implement checks to ensure our data is accurate and consistent.
  • Help design and build data warehouses and data lakes.
  • Use SQL extensively to query and work with data in different systems.
  • Work with streaming data using technologies like Kafka for real-time data processing.
  • Stay updated on the latest data engineering technologies.
  • Guide and mentor junior data engineers.
  • Help create data management rules and procedures.

What You'll Need

  • Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
  • 6-8 years of experience as a Data Engineer.
  • Strong skills in Spark and Scala for handling large amounts of data.
  • Good experience with Airflow for managing data workflows and understanding DAGs.
  • Solid understanding of how to transform and prepare data.
  • Strong programming skills in Python for data tasks and automation..
  • Proven experience working with AWS cloud services (S3, Redshift, EMR, Glue, IAM, EC2, and Athena).
  • Experience building data solutions for Analytics teams.
  • Familiarity with Node.js for backend development.
  • Experience with Typescript for backend development is a plus.
  • Experience using YAML for configuration management.
  • Hands-on experience with GitHub Actions for automated deployment (CI/CD).
  • Good understanding of data warehousing concepts.
  • Strong database skills - OLAP/OLTP
  • Excellent command of SQL for data querying and manipulation.
  • Experience with stream processing using Kafka or similar technologies.
  • Excellent problem-solving, analytical, and communication skills.
  • Ability to work well independently and as part of a team.

Bonus Points

  • Familiarity with data lake technologies (e.g., Delta Lake, Apache Iceberg).
  • Experience with other stream processing technologies (e.g., Flink, Kinesis).
  • Knowledge of data management, data quality, statistics and data governance frameworks.
  • Experience with tools for managing infrastructure as code (e.g., Terraform).
  • Familiarity with container technologies (e.g., Docker, Kubernetes).
  • Experience with monitoring and logging tools (e.g., Prometheus, Grafana).

  • Senior Data Engineer

    2 weeks ago

    bangalore district, India ITC Infotech Full time

    Senior Data Engineer - Azure Stack Experience - 3-12 years Location : Bangalore, Kolkata, Gurgaon, Pune We are looking for a highly skilled Senior Data Engineer with deep expertise in Azure Stack to join our growing team. As a Senior Data Engineer, you will be instrumental in designing and developing robust data pipelines, data lakes, and cloud-based...

  • Senior/Lead Data Engineer

    4 days ago

    bangalore, India Eucloid Data Solutions Full time

    About The Role:Eucloid is looking for a senior/ lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure on Databricks for our clients by participating in activities which may includestarting from up- stream and down-stream technology selection to designing...

  • Senior Data Engineer

    4 days ago

    bangalore district, India USEReady Full time

    Job Title: Senior Databricks Engineer Experience Level: 5-8 Years Job Summary As a Senior Databricks Engineer, you will be responsible for designing, developing, and optimizing our data architecture and pipelines on the Databricks Lakehouse Platform. You will leverage your deep expertise in Spark, Delta Lake, and cloud technologies to build scalable and...

  • Senior Data Engineer

    2 weeks ago

    bangalore district, India Persistent Systems Full time

    About Position: We are seeking a highly skilled and experienced Senior Data Engineer to join our growing data team. The ideal candidate will have deep expertise in SQL/PLSQL development , AWS cloud services , DBT (Data Build Tool) , Python , and Snowflake . You will be responsible for designing, building, and maintaining scalable data pipelines...

  • Senior Data Engineer

    4 days ago

    bangalore district, India Syncarp Full time

    Looking for Lead Engineer / Technical Lead for a Global IT Service Provider based out of Chennai/Bangalore/Hyderabad/Pune Locations. Job Description: Lead Engineer: Exp: 5-8 Years Design, develop, and maintain simulation services and tools for ML feature, model, and rule evaluation. Build and optimize data pipelines for point-in-time historical data...

  • Senior Data Engineer

    1 week ago

    bangalore district, India Grid Dynamics Full time

    About Company Grid Dynamics (Nasdaq:GDYN) is a digital-native technology services provider that accelerates growth and bolsters competitive advantage for Fortune 1000 companies. Grid Dynamics provides digital transformation consulting and implementation services in omnichannel customer experience, big data analytics, search, artificial intelligence, cloud...

  • Senior Data Engineer

    2 weeks ago

    bangalore district, India JRD Systems Full time

    We are seeking a Senior Data Engineer with deep expertise in AWS Cloud Development . This role is critical to both the development of new data products and the modernization of existing platforms. The ideal candidate is a seasoned data engineer with hands-on experience designing, building, and optimizing large-scale data pipelines and architectures ...

  • Data Engineer

    4 days ago

    bangalore, India Data-Hat AI Full time

    Department: Data Engineering & AI Solutions Reports To: Lead Data Solutions Architect Travel: International travel required (up to 30–40%)  Position Summary: We are hiring a senior-level Data Engineer to lead the design, development, and optimization of high-performance data infrastructure that underpins mission-critical AI systems. With 12+ years of...

  • Senior Data Engineer

    2 weeks ago

    chennai district, India Eucloid Data Solutions Full time

    Job Description Eucloid is looking for a skilled Data Engineer to join our team and contribute to the design, development, and optimization of data frameworks pipelines using Python, Elasticsearch, SQL DB and Queue System. The ideal candidate will have a strong technical background, a passion for solving complex problems, and experience in building...

  • Senior Data Engineer

    7 days ago

    bangalore district, India Quantiphi Full time

    Role: Senior Data Engineer Experience Level: 4 to 7 Years Work location: Mumbai/Bangalore (Hybrid) Role & Responsibilities: Work with business users and other stakeholders to understand business processes. Ability to design and implement Dimensional and Fact tables Identify and implement data transformation/cleansing requirements Develop a highly...