Data Engineer

3 days ago

Kanpur, Uttar Pradesh, India iXceed Solutions Full time

Role - Data Engineer

Location - Chennai (Hybrid)

Type - Permanent

Job Description:

Key Responsibilities:

  • Lead the design, development, and deployment of scalable data pipelines and ETL processes using Scala and Apache Spark (Databricks).
  • Develop and maintain data processing workflows using PySpark and Python.
  • Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver robust data solutions.
  • Architect and implement data ingestion, transformation, and integration solutions for structured and unstructured data.
  • Optimize Spark jobs for performance and cost efficiency on cloud platforms.
  • Lead, mentor, and guide a team of data engineers, ensuring best practices in coding, testing, and deployment.
  • Establish and enforce data engineering standards, code reviews, and documentation.
  • Troubleshoot and resolve complex data pipeline issues.
  • Stay current with emerging technologies and recommend improvements to existing data infrastructure.
  • Work closely with DevOps and cloud teams to ensure smooth deployment and monitoring of data pipelines.

Required Qualifications:

  • Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
  • 10+ years of professional experience in data engineering.
  • Strong hands-on programming experience in Scala, with a deep understanding of functional programming concepts.
  • Extensive experience with Apache Spark, including Spark SQL, Spark Streaming, and optimization techniques.
  • Proficiency in Python and PySpark for data processing tasks.
  • Experience working with Databricks or similar cloud-based Spark platforms.
  • Solid understanding of distributed computing, big data architectures, and data modeling.
  • Experience with cloud platforms such as AWS, Azure, or GCP.
  • Familiarity with data storage technologies like HDFS, S3, Delta Lake, and relational/non-relational databases.
  • Strong leadership skills with experience leading and mentoring engineering teams.
  • Excellent problem-solving, communication, and collaboration skills.

  • Senior Azure Stack Engineer

    4 days ago

    Kanpur, Uttar Pradesh, India NTT DATA Full time US$ 90,000 - US$ 1,20,000 per year

    Req ID: 336844NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Senior Azure Stack Engineer to join our team in Remote, Uttar Pradesh (IN-UP), India (IN). Job Title: Senior Azure...

  • Professional Services Engineer

    1 day ago

    Kanpur, Uttar Pradesh, India Data Dynamics Full time

    Company DescriptionData Dynamics is a global leader in enterprise data management, focusing on Digital Trust and Data Democratisation. With flagship AI-powered self-service data management software, the company empowers individuals across organisations to manage, secure, and optimise their data. Trusted by over 300 organisations, including 25% of the Fortune...

  • Data Engineer

    3 weeks ago

    Kanpur, Uttar Pradesh, India Natlov Technologies Pvt Ltd Full time

    We are looking for a skilled Data Engineer with expertise in AWS services and strong programming skills.Key Responsibilities:Build and maintain data pipelines and workflows using AWS services (Lambda, S3, DynamoDB, Glue, SageMaker).Develop and deploy solutions leveraging Python and Docker.Work on image processing tasks using libraries such as OpenCV (CV2),...

  • Data Engineer

    2 weeks ago

    Kanpur, Uttar Pradesh, India Denave Full time

    Job ResponsibilitiesDesign, build & maintain scalable data pipelines for ingestion, processing & storage.Collaborate with data scientists, analysts, and product teams to deliver high-quality data solutions.Optimize data systems for performance, reliability, scalability, and cost-efficiency.Implement data quality checks ensuring accuracy, completeness, and...

  • Data Engineering Leader

    2 days ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineering Full time ₹ 23,24,000

    Data Engineering LeadershipWe are seeking a seasoned data engineering professional to lead the development of a scalable data ingestion framework, ensuring high data quality and validation.

  • Data Engineering Lead

    2 weeks ago

    Kanpur, Uttar Pradesh, India beBeeDatabaseEngineering Full time ₹ 25,00,000 - ₹ 40,00,000

    We are seeking an experienced Data Engineering Lead to drive the development and implementation of our data infrastructure.Job DescriptionThe Data Engineering Lead will be responsible for designing, building, and maintaining scalable data systems that support our AI and ML initiatives. This includes:Designing and implementing database schemas, structures,...

  • Data Architect and Engineer

    1 week ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Data Engineer NeededWe're looking for a highly skilled data engineer to join our team.Hands-on expertise in Python and PySpark is essential.5+ years of experience in data engineering with a focus on big data, telemetry, and streaming analytics.The ideal candidate will have strong business acumen, excellent communication skills, and the ability to work...

  • Data Engineering Leader

    1 week ago

    Kanpur, Uttar Pradesh, India beBeeSnowflake Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job DescriptionWe are seeking a Technical Lead-Data Engineer to lead the design and implementation of data engineering solutions.Design and implement data engineering solutions using OLAP design, star/snowflake schema, ETL/ELT processes, normalization/denormalization, and data modeling.Develop complex SQL queries, including joins, window functions, and...

  • Data Engineer

    6 days ago

    Kanpur, Uttar Pradesh, India Comptech Associates Full time

    About the Job Job title: Data Engineer(Immediate Joiner with Healthcare OR Pharma Client experience) Location: Noida, India Onsite Duration: Fulltime Note : Need an immediate Joiner with strong experience in Healthcare/Pharma Job description:Work closely with data scientists, analysts, and business stakeholders to understand data needsEnsure data quality,...

  • Data Engineering Specialist

    4 days ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineering Full time US$ 9,00,000 - US$ 12,00,000

    Job Title: Data Engineering SpecialistAbout the Role:We are seeking a highly skilled Data Engineer to join our team. The ideal candidate will have a strong background in distributed data processing and storage, as well as experience with cloud-based technologies.Key Responsibilities:- Design, develop, and maintain large-scale data engineering systems-...