Senior Data Engineer

2 weeks ago

Bengaluru, India Josys Full time

Location: Bengaluru, Karnataka, IndiaAbout The RoleWe're looking for an experienced Senior Data Engineer (6-8 years) to join our data team. You'll be key in building and maintaining our data systems on AWS. You'll use your strong skills in big data tools and cloud technology to help our analytics team get valuable insights from our data. You'll be in charge of the whole process of our data pipelines, making sure the data is good, reliable, and fast.What You'll Do- Design and build efficient data pipelines using Spark / PySpark / Scala.- Manage complex data processes with Airflow, creating and fixing any issues with the workflows (DAGs).- Clean, transform, and prepare data for analysis.- Use Python for data tasks, automation, and building tools.- Work with AWS services like S3, Redshift, EMR, Glue, and Athena to manage our data infrastructure.- Collaborate closely with the Analytics team to understand what data they need and provide solutions.- Help develop and maintain our Node.js backend, using Typescript, for data services.- Use YAML to manage the settings for our data tools.- Set up and manage automated deployment processes (CI/CD) using GitHub Actions.- Monitor and fix problems in our data pipelines to keep them running smoothly.- Implement checks to ensure our data is accurate and consistent.- Help design and build data warehouses and data lakes.- Use SQL extensively to query and work with data in different systems.- Work with streaming data using technologies like Kafka for real-time data processing.- Stay updated on the latest data engineering technologies.- Guide and mentor junior data engineers.- Help create data management rules and procedures.What You'll Need- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.- 6-8 years of experience as a Data Engineer.- Strong skills in Spark and Scala for handling large amounts of data.- Good experience with Airflow for managing data workflows and understanding DAGs.- Solid understanding of how to transform and prepare data.- Strong programming skills in Python for data tasks and automation..- Proven experience working with AWS cloud services (S3, Redshift, EMR, Glue, IAM, EC2, and Athena).- Experience building data solutions for Analytics teams.- Familiarity with Node.js for backend development.- Experience with Typescript for backend development is a plus.- Experience using YAML for configuration management.- Hands-on experience with GitHub Actions for automated deployment (CI/CD).- Good understanding of data warehousing concepts.- Strong database skills - OLAP/OLTP- Excellent command of SQL for data querying and manipulation.- Experience with stream processing using Kafka or similar technologies.- Excellent problem-solving, analytical, and communication skills.- Ability to work well independently and as part of a team.Bonus Points- Familiarity with data lake technologies (e.g., Delta Lake, Apache Iceberg).- Experience with other stream processing technologies (e.g., Flink, Kinesis).- Knowledge of data management, data quality, statistics and data governance frameworks.- Experience with tools for managing infrastructure as code (e.g., Terraform).- Familiarity with container technologies (e.g., Docker, Kubernetes).- Experience with monitoring and logging tools (e.g., Prometheus, Grafana).

  • Senior Data Engineer cum Data Scientist

    7 days ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    About the RoleAs a Senior Data Engineer cum Data Scientist, you will be a key contributor to designing, developing, and operating advanced data architectures, pipelines, and analytics solutions within the Palantir ecosystemYou will leverage your deep expertise in Palantir Foundry and Gotham platforms, combined with strong AI capabilities, to solve complex...

  • Senior Azure Data Engineer

    1 week ago

    Bengaluru, Karnataka, India NTT DATA Global Delivery Services Ltd Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    ERP Senior Specialist Job Title: Azure Senior Data Engineer We are looking a highly skilled and experienced Senior Azure Data Engineer to join our team. This role requires deep expertise in Azure Data Factory and Databricks, including CI/CD setup for Databricks, and strong proficiency in ETL, SQL, and modern data engineering practices. You will be the...

  • Data Engineer Senior Consultant

    5 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 1,20,000 - ₹ 3,00,000 per year

    Req ID: 341131NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer Senior Consultant to join our team in Bangalore, Karnātaka (IN-KA), India (IN). Location: | Experience:...

  • Senior Data Scientist

    4 weeks ago

    Bengaluru, India Enable Data Incorporated Full time

    Enable Data Incorporated is looking for a talented and experienced Senior Data Scientist to join our innovative team. In this role, you will be responsible for driving data science initiatives, developing predictive models, and providing insights that guide key business decisions. The ideal candidate has a strong background in statistical analysis, machine...

  • Senior Data Scientist

    2 weeks ago

    Bengaluru, Karnataka, India Enable Data Incorporated Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Enable Data Incorporated is looking for a talented and experienced Senior Data Scientist to join our innovative team. In this role, you will be responsible for driving data science initiatives, developing predictive models, and providing insights that guide key business decisions. The ideal candidate has a strong background in statistical analysis, machine...

  • Senior Data Scientist

    7 days ago

    Bengaluru, Karnataka, India Enable Data Incorporated Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Enable Data Incorporated is looking for a talented and experienced Senior Data Scientist to join our innovative team. In this role, you will be responsible for driving data science initiatives, developing predictive models, and providing insights that guide key business decisions. The ideal candidate has a strong background in statistical analysis, machine...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    As a Data Engineer, your main responsibilities will involve: The project is a migration of a SAS code to DBT code, so it's important that the candidate has experience in this type of projects. Analyze data models and derive logical conclusions. Processes modeling. Hands on development and monitoring of the Azure cloud Platform and various associated...

  • Data Engineer

    7 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Design and implement tailored data solutions to meet customer needs and use cases, spanning from streaming to data lakes, analytics, and beyond within a dynamically evolving technical stack. Provide thought leadership by recommending the most appropriate technologies and solutions for a given use case, covering the entire spectrum from the application layer...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 8,00,000 - ₹ 16,00,000 per year

    Build scalable data pipelines from structured and unstructured sources Design ontologies that model business entities, relationships, and process flows Collaborate with domain SMEs and ML engineers to structure agent inputs and outputs Implement transformation, validation, and enrichment logic across diverse datasets Maintain and optimize cloud data...

  • Senior Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Capco Technologies Pvt Ltd Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Data Engineer -SCALA Job Title: Senior Data Engineer/Developer Number of Positions: 2 Job Description: The Senior Data Engineer will be responsible for designing, developing, and maintaining scalable data pipelines and building out new API integrations to support continuing increases in data volume and complexity. They will collaborate with analytics...