Senior Data Engineer

2 days ago

Yelahanka, India Josys Full time

Location: Bengaluru, Karnataka, India About The Role We're looking for an experienced Senior Data Engineer (6-8 years) to join our data team. You'll be key in building and maintaining our data systems on AWS. You'll use your strong skills in big data tools and cloud technology to help our analytics team get valuable insights from our data. You'll be in charge of the whole process of our data pipelines, making sure the data is good, reliable, and fast. What You'll Do - Design and build efficient data pipelines using Spark / PySpark / Scala. - Manage complex data processes with Airflow, creating and fixing any issues with the workflows (DAGs). - Clean, transform, and prepare data for analysis. - Use Python for data tasks, automation, and building tools. - Work with AWS services like S3, Redshift, EMR, Glue, and Athena to manage our data infrastructure. - Collaborate closely with the Analytics team to understand what data they need and provide solutions. - Help develop and maintain our Node.Js backend, using Typescript, for data services. - Use YAML to manage the settings for our data tools. - Set up and manage automated deployment processes (CI/CD) using GitHub Actions. - Monitor and fix problems in our data pipelines to keep them running smoothly. - Implement checks to ensure our data is accurate and consistent. - Help design and build data warehouses and data lakes. - Use SQL extensively to query and work with data in different systems. - Work with streaming data using technologies like Kafka for real-time data processing. - Stay updated on the latest data engineering technologies. - Guide and mentor junior data engineers. - Help create data management rules and procedures. What You'll Need - Bachelor's or Master's degree in Computer Science, Engineering, or a related field. - 6-8 years of experience as a Data Engineer. - Strong skills in Spark and Scala for handling large amounts of data. - Good experience with Airflow for managing data workflows and understanding DAGs. - Solid understanding of how to transform and prepare data. - Strong programming skills in Python for data tasks and automation.. - Proven experience working with AWS cloud services (S3, Redshift, EMR, Glue, IAM, EC2, and Athena). - Experience building data solutions for Analytics teams. - Familiarity with Node.Js for backend development. - Experience with Typescript for backend development is a plus. - Experience using YAML for configuration management. - Hands-on experience with GitHub Actions for automated deployment (CI/CD). - Good understanding of data warehousing concepts. - Strong database skills - OLAP/OLTP - Excellent command of SQL for data querying and manipulation. - Experience with stream processing using Kafka or similar technologies. - Excellent problem-solving, analytical, and communication skills. - Ability to work well independently and as part of a team. Bonus Points - Familiarity with data lake technologies (e.G., Delta Lake, Apache Iceberg). - Experience with other stream processing technologies (e.G., Flink, Kinesis). - Knowledge of data management, data quality, statistics and data governance frameworks. - Experience with tools for managing infrastructure as code (e.G., Terraform). - Familiarity with container technologies (e.G., Docker, Kubernetes). - Experience with monitoring and logging tools (e.G., Prometheus, Grafana).

  • Senior Data Engineer

    3 weeks ago

    Yelahanka, India Cito Techno Full time

    Company Description CITO Techno Services Pvt. Ltd. is a forward-thinking technology company specializing in IT services, business consulting, and software development. Our mission is to empower businesses with cutting-edge digital solutions that drive efficiency, innovation, and scalable growth. We combine deep industry expertise with agile methodologies to...

  • Senior Data Engineer

    4 days ago

    Yelahanka, India Persistent Systems Full time

    About Position: We are seeking a highly skilled and experienced Senior Data Engineer to join our growing data team. The ideal candidate will have deep expertise in SQL/PLSQL development, AWS cloud services, DBT (Data Build Tool), Python, and Snowflake. You will be responsible for designing, building, and maintaining scalable data pipelines and infrastructure...

  • Senior Data Engineer

    2 weeks ago

    Yelahanka, Karnataka, India AuxoAI Engineering Pvt. Ltd. Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    AuxoAI is seeking a skilled and experienced Data Engineer to join our dynamic team. The ideal candidate will have 7-10 years of prior experience in data engineering, with a strong background in Databricks. This role offers an exciting opportunity to work on diverse projects, collaborating with cross-functional teams to design, build, and optimize data...

  • Senior Data Architect

    2 days ago

    Yelahanka, India Rakuten Symphony Full time

    Job Title: Senior Data Architect – OSS Minimum - 10+ years of hands-on experience in data architecture, data modelling, and designing large-scale data platforms. Location: Bangalore (Onsite) Why should you choose us? Rakuten Symphony is a Rakuten Group company, that provides global B2B services for the mobile telco industry and enables next-generation,...

  • Data Engineer

    4 days ago

    Yelahanka, India CentricaSoft Full time

    Title: Data Engineer - PySparkAbout CentricaSoft:CentricaSoft is a data-driven technology partner delivering end-to-end data solutions for our clients. We design, build, and scale modern data platforms that empower business decisions. We’re growing our data engineering team and seeking a hands-on PySpark Developer who thrives in a fast-paced, collaborative...

  • Data Engineer

    1 week ago

    Yelahanka, India CentricaSoft Full time

    Title: Data Engineer - PySpark About CentricaSoft: CentricaSoft is a data-driven technology partner delivering end-to-end data solutions for our clients. We design, build, and scale modern data platforms that empower business decisions. We’re growing our data engineering team and seeking a hands-on PySpark Developer who thrives in a fast-paced,...

  • Data Engineer

    1 week ago

    Yelahanka, India CentricaSoft Full time

    Title: Data Engineer - PySparkAbout CentricaSoft:CentricaSoft is a data-driven technology partner delivering end-to-end data solutions for our clients. We design, build, and scale modern data platforms that empower business decisions. We’re growing our data engineering team and seeking a hands-on PySpark Developer who thrives in a fast-paced, collaborative...

  • Data engineer

    2 days ago

    Yelahanka, India CentricaSoft Full time

    Title: Data Engineer - Py SparkAbout Centrica Soft:Centrica Soft is a data-driven technology partner delivering end-to-end data solutions for our clients. We design, build, and scale modern data platforms that empower business decisions. We’re growing our data engineering team and seeking a hands-on Py Spark Developer who thrives in a fast-paced,...

  • Data Engineer

    1 week ago

    Yelahanka, India CentricaSoft Full time

    Title: Data Engineer - PySparkAbout CentricaSoft:CentricaSoft is a data-driven technology partner delivering end-to-end data solutions for our clients. We design, build, and scale modern data platforms that empower business decisions. We’re growing our data engineering team and seeking a hands-on PySpark Developer who thrives in a fast-paced, collaborative...

  • Data Engineer

    4 days ago

    Yelahanka, India Ascendion Full time

    Location: Bengaluru, Chennai, Hyderabad and Pune (Hybrid Work model) Job Summary: We are looking for a talented GCP BigQuery Developer with strong SQL skills and basic proficiency in Python to join our data engineering team. The ideal candidate should have hands-on experience working with Google Cloud Platform (GCP) services, particularly BigQuery, and be...