Lead Data Engineer

4 days ago

Chennai, Tamil Nadu, India Cynosure Corporate Solutions Full time ₹ 15,00,000 - ₹ 20,00,000 per year

Roles and Responsibilities:

  • Working with clients to understand their data.
  • Based on the understanding you will be building the data structures and pipelines.
  • You will be working on the application from end to end collaborating with UI and other development teams.
  • You will be working with various cloud providers such as Azure & AWS.
  • You will be engineering data using the Hadoop/Spark ecosystem.
  • You will be responsible for designing, building, optimizing and supporting new and existing data pipelines.
  • Orchestrating jobs using various tools such Oozie, Airflow, etc.
  • Developing programs for cleaning and processing data.
  • You will be responsible for building the data pipelines to migrate and load the data into the HDFS either on-prem or in the cloud.
  • Developing Data ingestion/process/integration pipelines effectively.
  • Creating Hive data structures, metadata and loading the data into data lakes / Bigdata warehouse environments.
  • Optimized (Performance tuning) many data pipelines effectively to minimize cost.
  • Code versioning control and git repository is up to date.
  • You should be able to explain the data pipeline to internal and external stakeholders.
  • You will be responsible for building and maintaining CI/CD of the data pipelines.

Preferred Qualifications:

  • Bachelors degree in computer science or related field.
  • Minimum of 5+ years working experience with Spark, Hadoop eco systems.
  • Minimum of 4+ years working experience on designing data streaming pipelines
  • Minimum experience of 3+ years on NoSQL and Spark Streaming.
  • Proven experience with big data ecosystem tools such as Sqoop, Spark, SQL, API, Hive, Oozie, Airflow, etc..
  • Solid experience in all phases of SDLC with 10+ years of experience (plan, design, develop, test, release, maintain and support)
  • Hands-on experience using Azures data engineering stack.
  • Should have implemented projects using programming languages such as Scala or Python.
  • Working experience on SQL complex data merging techniques such as windowing functions etc..
  • Hands-on experience with on-prem distribution tools such as Cloudera/Horton Works/MapR.

  • Lead Data Engineer

    7 days ago

    Chennai, Tamil Nadu, India AABM Cloud Data Solutions Full time ₹ 12,00,000 - ₹ 18,00,000 per year

    About the RoleWe are seeking a Lead Data Engineer to join our team. In this role, you will be crucial in designing, building, and maintaining our data infrastructure. This is a hands-on position requiring a strong foundation in data warehousing, ETL processes, and database management. The ideal candidate will have a keen eye for detail and the ability to...

  • Data Engineer

    4 days ago

    Chennai, Tamil Nadu, India NTT DATA Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Req ID: 336026NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer - Azure to join our team in Chennai, Tamil Nādu (IN-TN), India (IN). "Job Duties: Key Responsibilities:...

  • Data Engineer

    12 hours ago

    Chennai, Tamil Nadu, India CX Data Labs Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    HiRole - Azure Data engineerLocation - Chennai, Gurugram (5days Onsite)Full timeImmediate joiner Only (15days Notice period only)Technical Skills:o Proficiency in SQL for complex queries, database design, and optimization.o Strong experience with PySpark for data transformation and processing.o Hands-on experience with Databricks for building and managing...

  • Lead Data Engineer

    2 days ago

    Chennai, Tamil Nadu, India ResourceTree Global Services Pvt Ltd Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Company DescriptionResourceTree Global Services Pvt Ltd. (ResourceTree) is a Talent Builder dedicated to offering careers to professionals across various industries, including Information & Technology, Manufacturing, Services, Banking, Agri-tech, and Edu-tech. ResourceTree has earned a reputation for helping industry leaders transform their talent needs into...

  • Lead Data Engineer

    4 days ago

    Chennai, Tamil Nadu, India Ignitho Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Position: Lead Data EngineerLocation: ChennaiWork Hours: 12:00 PM to 9:00 PM IST (US Business Hours)Availability: ImmediateExperience: 6 + YearsAbout the Company:Ignitho Inc. is a leading AI and data engineering company with a global presence, including offices in the US, UK, India, and Costa Rica.Visit our website to learn more about our work and culture:...

  • Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India NTT DATA North America Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer - Azure to join our team in Chennai, Tamil Nādu (IN-TN), India (IN)."Job Duties: Key Responsibilities:•  ...

  • Data Engineer

    4 days ago

    Chennai, Tamil Nadu, India NTT DATA North America Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Req ID:336025NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer - Azure to join our team in Chennai, Tamil Nādu (IN-TN), India (IN)."Job Duties: Key...

  • data engineering lead

    4 weeks ago

    Chennai, Tamil Nadu, India Randstad Full time

    At least 5 years' experience as a Senior or Lead Data Engineer Proven and demonstratable experience of Python, AWS, and SQL/ Oracle Bachelor's honours degree preferably in computer science, or engineering field, or demonstrable technical experience in a related engineering discipline At least 3 years of ETL experience using flat files to transposed...

  • Lead Data Engineer

    7 hours ago

    Chennai, Tamil Nadu, India ResourceTree Global Services Pvt Ltd Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Description : We are seeking a highly skilled and motivated Lead/Senior Data Engineer to join our. Data Integration and Analytics team. The ideal candidate will bring deep expertise in AWS Redshift, with strong knowledge of Redshift architecture, query performance tuning, and workload optimization You will be responsible for leading the design,...

  • Data Analyst

    2 days ago

    Chennai, Tamil Nadu, India Lead Hr Services Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role & responsibilitiesCollect, process, and analyze production, quality, maintenance, and supply chain data.Develop dashboards, reports, and visualizations to monitor KPIs (Production, OEE, Rejection %, Downtime, Inventory, etc.).Identify trends, patterns, and insights to support process improvements and lean manufacturing initiatives.Collaborate with...